verikun 0.20.0 → 0.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/skills/verikun/SKILL.md +25 -4
- package/CHANGELOG.md +143 -0
- package/dist/capture.js +29 -0
- package/dist/cli.js +83 -14
- package/dist/companion/manager.js +397 -0
- package/dist/companion/protocol.js +111 -0
- package/dist/companion/sock-client.js +22 -0
- package/dist/drivers/adb.js +105 -1
- package/dist/errors.js +23 -1
- package/dist/image.js +40 -0
- package/dist/run.js +4 -1
- package/dist/version.js +1 -1
- package/package.json +3 -2
- package/tools/verikun-companion/prebuilt/verikun-companion.jar +0 -0
|
@@ -251,10 +251,31 @@ outweigh dozens of `vk ui` calls. When you do, `vk` already downscales the PNG
|
|
|
251
251
|
**Two uses of a screenshot — keep them apart.** The cost above is about *reading a
|
|
252
252
|
screenshot back into context* to decide your next move; that is what to avoid (perceive
|
|
253
253
|
and verify with the hierarchy instead). A screenshot taken purely as **report evidence
|
|
254
|
-
and never read back** costs
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
254
|
+
and never read back** costs no tokens. So when you drive a flow to produce a report, **do**
|
|
255
|
+
`vk screenshot` around each significant transition (and before a risky or verification step)
|
|
256
|
+
— then leave it in the report, don't read the PNG back. A visual trail makes post-run review
|
|
257
|
+
far easier, and a failing step already auto-captures its own screen.
|
|
258
|
+
|
|
259
|
+
It is not free in wall clock, though: a capture is ~1.1s on a physical Android phone. That is
|
|
260
|
+
cheap next to a hierarchy read (~2.4s — see below) but it is not zero, so screenshot the
|
|
261
|
+
transitions worth reviewing rather than every step.
|
|
262
|
+
|
|
263
|
+
**Most of a run's time is reading the UI hierarchy.** Every selector command (`tap`, `text`,
|
|
264
|
+
`find`, `assert`, `swipe --on`) costs one read; measured on a physical mid-range Android
|
|
265
|
+
phone, one read is ~2.4s, nearly all of it fixed per-invocation cost inside `uiautomator`
|
|
266
|
+
that does not depend on how complex the screen is. iOS is ~10x cheaper. So prefer one
|
|
267
|
+
`vk assert` over a `vk ui` you have to scan, batch a known flow with `vk batch` rather than
|
|
268
|
+
re-checking between every step, and don't add a redundant `vk ui` just to confirm what an
|
|
269
|
+
`assert` already proved.
|
|
270
|
+
|
|
271
|
+
On Android this is handled for you: verikun keeps an accessibility connection alive on the
|
|
272
|
+
device (the *companion*), which cuts a read to ~0.2s. The first read on a device costs ~5.8s
|
|
273
|
+
to set it up, then every read after is fast. Nothing to enable.
|
|
274
|
+
|
|
275
|
+
It holds the device's single `UiAutomation` connection while it runs, so Appium and Layout
|
|
276
|
+
Inspector cannot attach. If the user needs those, tell them `VERIKUN_COMPANION=0` or
|
|
277
|
+
`vk companion stop` — don't disable it pre-emptively. A failure never breaks a run: verikun
|
|
278
|
+
falls back to the slower stock read on its own.
|
|
258
279
|
|
|
259
280
|
**Remember identifiers across runs.** After a flow succeeds, save the selectors
|
|
260
281
|
you found to memory — the mapping from human intent to selector, plus the screen
|
package/CHANGELOG.md
CHANGED
|
@@ -6,7 +6,72 @@ All notable changes to this project are documented here. The format is based on
|
|
|
6
6
|
|
|
7
7
|
## [Unreleased]
|
|
8
8
|
|
|
9
|
+
## [0.21.0] - 2026-08-13
|
|
10
|
+
|
|
9
11
|
### Added
|
|
12
|
+
- **An on-device companion that makes an Android UI-hierarchy read ~12x faster** — `vk ui`
|
|
13
|
+
goes from **2.44s to 0.18s** end to end on a physical SM-A415F, with **identical** output.
|
|
14
|
+
Reading the hierarchy is the dominant cost of any suite, so this moves the whole runtime,
|
|
15
|
+
not a rounding error. Every selector command benefits, not just `vk ui`: `find` 12.3x,
|
|
16
|
+
`assert` 12.0x, `wait` 11.5x, `tap` 8.1x, `text` 3.6x (the remainder is `adb shell input`,
|
|
17
|
+
which does not go through the companion yet).
|
|
18
|
+
|
|
19
|
+
**On by default; `VERIKUN_COMPANION=0` opts out.** It shipped opt-in first and that was
|
|
20
|
+
wrong — nobody discovers an environment variable they were never told about, and the
|
|
21
|
+
people who most need the speedup are the least likely to go looking for it.
|
|
22
|
+
|
|
23
|
+
It also makes auto-wait honour its window: a 5s wait for an element that never appears takes
|
|
24
|
+
~6.4s rather than ~10.8s, because the stock path's final 2.4s read starts just before the
|
|
25
|
+
deadline and overshoots.
|
|
26
|
+
|
|
27
|
+
The first read on a device costs ~6s to push, start and calibrate. The verdict is then
|
|
28
|
+
remembered **on the device** (`/data/local/tmp/verikun-companion.note`, keyed by verikun
|
|
29
|
+
version), so a restart after the 15-minute idle shutdown reuses it and costs ~2.1s. The
|
|
30
|
+
same note records a device the companion could not start on at all, so that phone falls
|
|
31
|
+
straight through to the stock read instead of paying a doomed startup on every command —
|
|
32
|
+
which is what makes being on by default safe rather than merely fast.
|
|
33
|
+
|
|
34
|
+
Profiling `adb shell uiautomator dump` showed only ~0.10s of its ~2.4s is work. The rest is
|
|
35
|
+
paid fresh every call: ~1.22s starting ART and loading `uiautomator.jar`, and ~1.00s in
|
|
36
|
+
`waitForIdle(1000, 10000)`. That second one is not a flat sleep — it waits for the
|
|
37
|
+
accessibility event stream to have been *quiet* for a second, and a freshly connected
|
|
38
|
+
bridge has no history of quiet, so it must observe one. A long-lived connection already
|
|
39
|
+
does. The companion therefore keeps the full idle semantics and still returns in
|
|
40
|
+
milliseconds; it is not trading safety for speed.
|
|
41
|
+
|
|
42
|
+
It is Java compiled to dex (4KB), pushed to `/data/local/tmp` and run by the phone's own
|
|
43
|
+
runtime via `app_process` — scrcpy's approach. **Nothing is installed**: no APK, no root,
|
|
44
|
+
package list untouched. It shuts itself down after 15 minutes idle. It does **not** cache
|
|
45
|
+
the hierarchy — every read walks the live tree; only the *connection* is kept, so the
|
|
46
|
+
"re-capture fresh every command" rule is intact. It borrows the platform's own serialiser,
|
|
47
|
+
so its XML is byte-identical to `uiautomator dump`'s.
|
|
48
|
+
|
|
49
|
+
**A device has exactly one `UiAutomation` connection and the companion holds it** — while
|
|
50
|
+
it runs, `uiautomator dump` is SIGKILLed, and Appium, Layout Inspector and TalkBack cannot
|
|
51
|
+
attach. That is the one real cost, and the reason for the opt-out: `VERIKUN_COMPANION=0`,
|
|
52
|
+
or `vk companion stop` to hand it back once. `vk companion status` reports it.
|
|
53
|
+
|
|
54
|
+
It cannot fail a test. Every failure route releases the connection *before* falling back,
|
|
55
|
+
because the stock path is not merely slower while the companion holds it — it is
|
|
56
|
+
unavailable. A dead companion frees the connection by dying; one that cannot be asked
|
|
57
|
+
nicely is killed. A fallback read costs ~3.4s against 2.4s if the companion had never
|
|
58
|
+
existed.
|
|
59
|
+
|
|
60
|
+
Before trusting it, verikun takes **one** real `uiautomator dump` and checks the companion
|
|
61
|
+
reproduces it byte for byte, then remembers the answer on the device. The dumper clips node
|
|
62
|
+
bounds to a display size, and **which size the platform uses changed between Android
|
|
63
|
+
versions** — measured on three devices: a Samsung SM-A415F and a Google Pixel 3a, both on
|
|
64
|
+
Android 12, clip to the app window; a Pixel 6 emulator on Android 14 clips to the physical
|
|
65
|
+
display. The boundary is in AOSP itself — `DumpCommand` reads `getSize()` on the
|
|
66
|
+
`android12-release` and `android13-release` branches and `getRealSize()` from
|
|
67
|
+
`android14-release` onward — so it is the platform version, not the vendor, and a
|
|
68
|
+
hard-coded choice is wrong on one side of it whichever side you pick. The gap (44–254px)
|
|
69
|
+
would not fail loudly: it would shift every element near the bottom of the screen and land
|
|
70
|
+
taps elsewhere while still reporting success. If neither candidate
|
|
71
|
+
matches, the companion is declined and the stock path is used.
|
|
72
|
+
|
|
73
|
+
New: `vk companion <status|stop>`, `VERIKUN_COMPANION`, `tools/` for on-device programs,
|
|
74
|
+
and a [companion guide](https://ddikman.github.io/verikun/guides/companion/).
|
|
10
75
|
- **A `Self-healing in CI` docs page, and a plan-cache step in the reference workflow.** The
|
|
11
76
|
question it answers came from a team running verikun in CI: *should it be allowed to heal a
|
|
12
77
|
drifted step, or should that just fail the build?* The answer was spread across five pages,
|
|
@@ -44,7 +109,85 @@ All notable changes to this project are documented here. The format is based on
|
|
|
44
109
|
|
|
45
110
|
No version bump: no CLI behaviour changed.
|
|
46
111
|
|
|
112
|
+
### Changed
|
|
113
|
+
- **Android screenshots are about twice as fast**, and byte-for-byte identical. `screencap -p`
|
|
114
|
+
makes the *phone* PNG-encode a full-resolution image, which we then immediately decode and
|
|
115
|
+
shrink — so the encode was pure waste. Captures now come off the device as raw pixels
|
|
116
|
+
(`screencap`, no `-p`) and are encoded here, at the size we actually keep. Measured on a
|
|
117
|
+
physical SM-A415F: **`vk screenshot` 2.60s → 1.12s**, and `--full` 2.60s → 1.19s. The larger
|
|
118
|
+
transfer is not the problem it looks like — 10MB of RGBA box-downscales in ~10ms and deflates
|
|
119
|
+
in ~110ms on the host, against ~1.4s of on-device encoding avoided. Verified on-device: the
|
|
120
|
+
PNG the old and new paths produce is identical byte-for-byte.
|
|
121
|
+
|
|
122
|
+
This is a new **optional** `Driver.screenshotRaw()`, and `null` is a first-class answer: a
|
|
123
|
+
backend without a raw path (iOS — `simctl` is already ~0.2s, so there is nothing to win) or a
|
|
124
|
+
capture whose header we do not recognise falls back to `screenshot()`. Failure-evidence
|
|
125
|
+
captures take the same route and stay full-resolution. An OEM laying the framebuffer out
|
|
126
|
+
differently gets the old speed, never a wrong image.
|
|
127
|
+
|
|
47
128
|
### Fixed
|
|
129
|
+
- **Several `vk` processes starting on one device at once could fail outright.** Calibration
|
|
130
|
+
works by *releasing* the device's single `UiAutomation` connection to take a real
|
|
131
|
+
`uiautomator dump` — so two processes calibrating at once SIGKILL each other's dump.
|
|
132
|
+
MEASURED: five concurrent first-ever reads on one device gave exit codes `[3,3,0,3,3]`,
|
|
133
|
+
each reporting `Killed`.
|
|
134
|
+
|
|
135
|
+
Calibration is now claimed, and the claim is granted by the **companion itself**, whose
|
|
136
|
+
single-threaded accept loop makes it genuinely atomic. A process that does not get the
|
|
137
|
+
claim waits for the verdict and never touches the connection meanwhile — waiting processes
|
|
138
|
+
used to "helpfully" acquire it during the window the holder had deliberately released it,
|
|
139
|
+
killing the very dump they were waiting for. A claim from a process that then dies goes
|
|
140
|
+
stale after 45s, so nothing can wedge a device permanently.
|
|
141
|
+
|
|
142
|
+
The obvious host-side lock does **not** work and is worth recording: Android's toybox
|
|
143
|
+
`mkdir` **succeeds on an existing directory** (exit `0`, unlike POSIX), so a `mkdir` mutex
|
|
144
|
+
silently grants itself to every caller. Five concurrent attempts all reported success.
|
|
145
|
+
After the fix, five concurrent first-ever reads are `[0,0,0,0,0]`.
|
|
146
|
+
|
|
147
|
+
- **A transient or local problem could permanently mark a device as unable to run the
|
|
148
|
+
companion.** That verdict is sticky — every later command on that phone would skip to the
|
|
149
|
+
2.4s path — and two things could write it wrongly: a checkout that had not built the jar
|
|
150
|
+
(a fault of *this* working copy, not of the device), and a startup collision (transient,
|
|
151
|
+
and usually leaving a perfectly good companion running). It is now only recorded when the
|
|
152
|
+
jar exists *and* a second look still finds nothing. `start()` also re-probes before
|
|
153
|
+
stopping anything, so two racing processes can no longer kill each other's companion.
|
|
154
|
+
|
|
155
|
+
- **A command with a 120-second wait budget could abort after 20, because the app had not
|
|
156
|
+
drawn yet.** `launch` force-stops the app before starting it (and `--clear` wipes its data
|
|
157
|
+
too), leaving a gap of a second or two with no window at all. The platform correctly
|
|
158
|
+
reports a null root for that gap, and the capture layer escalated it to a fatal environment
|
|
159
|
+
error (exit `3`) after three attempts — throwing away a budget the caller had explicitly
|
|
160
|
+
asked for. Measured before the fix: `wait --timeout 120000` aborted at ~20s on roughly half
|
|
161
|
+
of the runs, with ~100 seconds unspent.
|
|
162
|
+
|
|
163
|
+
A null root is now `NoWindowError`: an observation about the screen, not a broken machine.
|
|
164
|
+
Every polling caller — `wait`, `find`, `assert`, `tap`, `text`, and the `vk ai` engine's
|
|
165
|
+
guards — treats it as "nothing on screen yet" and keeps polling to its own deadline. Every
|
|
166
|
+
*other* capture failure still surfaces immediately, because a missing adb or a wedged
|
|
167
|
+
dumper is a machine to fix and polling it for two minutes helps nobody. A caller with no
|
|
168
|
+
wait budget (a bare `vk ui`) still gets exit `3`, unchanged.
|
|
169
|
+
|
|
170
|
+
The companion made this visible rather than causing it: with the stock dump's 2.4s latency
|
|
171
|
+
the three attempts spanned 7-14s and usually outlasted the gap by accident, so the bug was
|
|
172
|
+
being masked by being slow. It also no longer stands the companion *down* for a null root —
|
|
173
|
+
that is the device's state, not a companion fault, and releasing the connection dropped the
|
|
174
|
+
whole process onto the slow path for the rest of its life. After the fix: 0/5 aborts, and
|
|
175
|
+
the companion stays connected across `launch --clear`.
|
|
176
|
+
|
|
177
|
+
- **A failed `uiautomator dump` could silently return the PREVIOUS screen.** `dumpXml` ran
|
|
178
|
+
`uiautomator dump <path>` and then `cat <path>`, accepting anything containing
|
|
179
|
+
`<hierarchy>` — but it never checked that *this* dump had written the file. The dump writes
|
|
180
|
+
to a fixed path and leaves the last successful result there whenever it fails, and it fails
|
|
181
|
+
without saying so in its exit code: AOSP's `DumpCommand` prints `ERROR: could not get idle
|
|
182
|
+
state` and returns normally when `waitForIdle` times out on an animating screen. The stale
|
|
183
|
+
XML is perfectly well-formed, so every check passed and the caller resolved selectors —
|
|
184
|
+
and tapped coordinates — from a screen that could be minutes old. Measured: a dump killed
|
|
185
|
+
at 18:46 happily served the 18:44 hierarchy.
|
|
186
|
+
|
|
187
|
+
The file is now removed in the same device shell immediately before the dump, so `cat` can
|
|
188
|
+
only succeed if this dump produced it; a genuinely failed capture now fails loudly (exit
|
|
189
|
+
`3`) instead of quietly lying. Pre-existing — the companion below made it easy to hit, and
|
|
190
|
+
is how it was found.
|
|
48
191
|
- **The release gate stopped understanding `npm pack --json`, which blocked the 0.20.0
|
|
49
192
|
publish.** `scripts/check-package-contents.mjs` read the pack result as
|
|
50
193
|
`JSON.parse(raw)[0]`, but **npm 12 returns an object keyed by package name** where npm ≤ 11
|
package/dist/capture.js
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
// Screen capture: the one place that decides HOW a screenshot is obtained.
|
|
3
|
+
//
|
|
4
|
+
// Sits between the driver (device I/O) and image.ts (pure image math) because it
|
|
5
|
+
// needs both, and lives apart from cli.ts so run.ts's failure-evidence capture can
|
|
6
|
+
// share it without an import cycle.
|
|
7
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
8
|
+
exports.capturePng = capturePng;
|
|
9
|
+
const image_1 = require("./image");
|
|
10
|
+
/**
|
|
11
|
+
* Capture the screen as a PNG whose longest edge is at most `maxEdge` px
|
|
12
|
+
* (`null` = full size). `buf` is always the image to write.
|
|
13
|
+
*
|
|
14
|
+
* Prefers the driver's raw path — pixels straight off the device, PNG-encoded here —
|
|
15
|
+
* which avoids an on-device encode of an image we are about to shrink anyway, and is
|
|
16
|
+
* roughly 2x faster end to end on Android. A backend without a raw path, or one whose
|
|
17
|
+
* capture came back in a shape we do not recognize, falls back to `screenshot()`, so
|
|
18
|
+
* the only observable difference is how long it took.
|
|
19
|
+
*/
|
|
20
|
+
function capturePng(driver, maxEdge) {
|
|
21
|
+
const raw = driver.screenshotRaw?.() ?? null;
|
|
22
|
+
if (raw)
|
|
23
|
+
return (0, image_1.pngFromRaw)(raw, maxEdge);
|
|
24
|
+
const png = driver.screenshot();
|
|
25
|
+
if (maxEdge === null) {
|
|
26
|
+
return { buf: png, width: 0, height: 0, scaled: false, origWidth: 0, origHeight: 0, reason: 'full size requested' };
|
|
27
|
+
}
|
|
28
|
+
return (0, image_1.downscalePng)(png, maxEdge);
|
|
29
|
+
}
|
package/dist/cli.js
CHANGED
|
@@ -67,7 +67,8 @@ const format_1 = require("./ui/format");
|
|
|
67
67
|
const viewport_1 = require("./ui/viewport");
|
|
68
68
|
const output_1 = require("./output");
|
|
69
69
|
const run_1 = require("./run");
|
|
70
|
-
const
|
|
70
|
+
const capture_1 = require("./capture");
|
|
71
|
+
const manager_1 = require("./companion/manager");
|
|
71
72
|
const engine_1 = require("./agent/engine");
|
|
72
73
|
const lint_1 = require("./agent/lint");
|
|
73
74
|
const claude_1 = require("./agent/claude");
|
|
@@ -182,6 +183,29 @@ function pollStep(flags, deadline) {
|
|
|
182
183
|
const interval = (0, args_1.flagNum)(flags, 'interval') ?? DEFAULT_POLL_MS;
|
|
183
184
|
return Math.min(interval, Math.max(0, deadline - Date.now()));
|
|
184
185
|
}
|
|
186
|
+
/**
|
|
187
|
+
* Read the hierarchy for a caller that is polling, treating "no window yet" as "nothing on
|
|
188
|
+
* screen yet" rather than a fatal environment error.
|
|
189
|
+
*
|
|
190
|
+
* A `NoWindowError` means the device genuinely had nothing to show — `launch --clear` and
|
|
191
|
+
* `launch` both leave a gap where the app has been stopped and has not drawn. That clears in
|
|
192
|
+
* a second or two, so a caller that has a wait budget should keep polling; escalating to
|
|
193
|
+
* exit 3 throws away the budget it was explicitly given. MEASURED: a `wait --timeout 120000`
|
|
194
|
+
* used to abort at ~20s with 100 seconds unspent.
|
|
195
|
+
*
|
|
196
|
+
* Every OTHER capture failure still propagates untouched — a missing adb, an unauthorised
|
|
197
|
+
* device or a wedged dumper is a machine to fix, and polling it for two minutes helps nobody.
|
|
198
|
+
*/
|
|
199
|
+
function readForPoll(ctx, opts = {}) {
|
|
200
|
+
try {
|
|
201
|
+
return ctx.driver.getElements(opts);
|
|
202
|
+
}
|
|
203
|
+
catch (e) {
|
|
204
|
+
if (e instanceof errors_1.NoWindowError)
|
|
205
|
+
return [];
|
|
206
|
+
throw e;
|
|
207
|
+
}
|
|
208
|
+
}
|
|
185
209
|
/**
|
|
186
210
|
* matchElements with auto-wait: re-capture + re-match until at least one element
|
|
187
211
|
* matches or the window elapses. Returns the final result either way (empty on miss).
|
|
@@ -189,7 +213,7 @@ function pollStep(flags, deadline) {
|
|
|
189
213
|
async function matchWaiting(ctx, sel, opts = {}) {
|
|
190
214
|
const deadline = Date.now() + waitWindowMs(ctx.flags);
|
|
191
215
|
for (;;) {
|
|
192
|
-
const res = (0, selector_1.matchElements)(ctx
|
|
216
|
+
const res = (0, selector_1.matchElements)(readForPoll(ctx, opts), sel);
|
|
193
217
|
if (res.matches.length > 0 || Date.now() >= deadline)
|
|
194
218
|
return res;
|
|
195
219
|
await sleep(pollStep(ctx.flags, deadline));
|
|
@@ -205,7 +229,7 @@ async function resolveOneWaiting(ctx, sel, opts = {}) {
|
|
|
205
229
|
const start = Date.now();
|
|
206
230
|
const deadline = start + windowMs;
|
|
207
231
|
for (;;) {
|
|
208
|
-
const els = ctx
|
|
232
|
+
const els = readForPoll(ctx, opts);
|
|
209
233
|
if ((0, selector_1.matchElements)(els, sel).matches.length >= 1) {
|
|
210
234
|
const { element, tier } = (0, selector_1.resolveOne)(els, sel); // 1 → resolved; >1 → throws ambiguity
|
|
211
235
|
// The snapshot rides along: scroll-into-view needs the scrollable containers
|
|
@@ -731,32 +755,31 @@ function assertSafeAppId(appId) {
|
|
|
731
755
|
return appId;
|
|
732
756
|
}
|
|
733
757
|
function cmdScreenshot(ctx) {
|
|
734
|
-
const raw = ctx.driver.screenshot();
|
|
735
758
|
// Precedence: --full (original) > --max <px> (explicit) > --more (preset) > default.
|
|
736
759
|
const maxEdge = (0, args_1.flagNum)(ctx.flags, 'max') ?? ((0, args_1.flagBool)(ctx.flags, 'more') ? MORE_SHOT_MAX_EDGE : shotMaxEdge());
|
|
737
|
-
const res = (0, args_1.flagBool)(ctx.flags, 'full') ? null :
|
|
738
|
-
const buf = res
|
|
760
|
+
const res = (0, capture_1.capturePng)(ctx.driver, (0, args_1.flagBool)(ctx.flags, 'full') ? null : maxEdge);
|
|
761
|
+
const buf = res.buf;
|
|
739
762
|
const outFlag = (0, args_1.flagStr)(ctx.flags, 'out');
|
|
740
763
|
const path = outFlag ? confineToCwd(outFlag) : (0, output_1.defaultScreenshotPath)();
|
|
741
764
|
(0, node_fs_1.writeFileSync)(path, buf);
|
|
742
765
|
ctx.record?.attachImage(buf);
|
|
743
|
-
ctx.record?.note({ message: res
|
|
766
|
+
ctx.record?.note({ message: res.scaled ? `${path} (${res.width}×${res.height})` : path });
|
|
744
767
|
// Surface the one case worth knowing about: we wanted to shrink but couldn't.
|
|
745
|
-
if (
|
|
768
|
+
if (!res.scaled && res.reason?.startsWith('unsupported')) {
|
|
746
769
|
(0, output_1.err)(`screenshot not downscaled: ${res.reason}`);
|
|
747
770
|
}
|
|
748
771
|
if ((0, args_1.flagBool)(ctx.flags, 'json')) {
|
|
749
772
|
(0, output_1.json)({
|
|
750
773
|
path,
|
|
751
774
|
bytes: buf.length,
|
|
752
|
-
...(res
|
|
775
|
+
...(res.scaled
|
|
753
776
|
? { width: res.width, height: res.height, scaledFrom: { width: res.origWidth, height: res.origHeight } }
|
|
754
777
|
: {}),
|
|
755
778
|
});
|
|
756
779
|
}
|
|
757
780
|
else {
|
|
758
781
|
(0, output_1.out)(path);
|
|
759
|
-
if (res
|
|
782
|
+
if (res.scaled)
|
|
760
783
|
(0, output_1.err)(`scaled ${res.origWidth}×${res.origHeight} -> ${res.width}×${res.height} (max edge ${maxEdge}px; --more for detail, --full for original)`);
|
|
761
784
|
}
|
|
762
785
|
return 0;
|
|
@@ -819,7 +842,7 @@ async function cmdWait(ctx) {
|
|
|
819
842
|
const interval = (0, args_1.flagNum)(ctx.flags, 'interval') ?? 400;
|
|
820
843
|
const deadline = Date.now() + timeout;
|
|
821
844
|
while (Date.now() < deadline) {
|
|
822
|
-
const { matches, tier } = (0, selector_1.matchElements)(ctx
|
|
845
|
+
const { matches, tier } = (0, selector_1.matchElements)(readForPoll(ctx), sel);
|
|
823
846
|
if (gone ? matches.length === 0 : matches.length > 0) {
|
|
824
847
|
ctx.record?.note({ selector: sel, tier, element: matches[0], message: gone ? 'gone' : `${matches.length} match(es)` });
|
|
825
848
|
if (gone)
|
|
@@ -877,10 +900,10 @@ async function cmdAssert(ctx) {
|
|
|
877
900
|
// Auto-wait subsumes the common "wait then assert": poll until the assertion
|
|
878
901
|
// passes or the window elapses. `--gone` therefore waits for disappearance.
|
|
879
902
|
const deadline = Date.now() + waitWindowMs(ctx.flags);
|
|
880
|
-
let result = evalAssert(ctx
|
|
903
|
+
let result = evalAssert(readForPoll(ctx), sel, ctx.flags);
|
|
881
904
|
while (!result.pass && Date.now() < deadline) {
|
|
882
905
|
await sleep(pollStep(ctx.flags, deadline));
|
|
883
|
-
result = evalAssert(ctx
|
|
906
|
+
result = evalAssert(readForPoll(ctx), sel, ctx.flags);
|
|
884
907
|
}
|
|
885
908
|
const { pass, reason, matches } = result;
|
|
886
909
|
ctx.record?.note({ selector: sel, element: matches[0], message: `${pass ? 'PASS' : 'FAIL'} — ${reason}` });
|
|
@@ -1550,7 +1573,7 @@ async function resolveBackend(platform, device, flags) {
|
|
|
1550
1573
|
// vice versa), and neither is allowed to derail recording the failure.
|
|
1551
1574
|
const out = {};
|
|
1552
1575
|
try {
|
|
1553
|
-
out.png =
|
|
1576
|
+
out.png = (0, capture_1.capturePng)(driver, null).buf;
|
|
1554
1577
|
}
|
|
1555
1578
|
catch {
|
|
1556
1579
|
/* device may be gone — that may be why we failed */
|
|
@@ -1933,6 +1956,48 @@ async function cmdSuiteEntry(positionals, flags) {
|
|
|
1933
1956
|
}
|
|
1934
1957
|
// ---------------------------------------------------------------------------
|
|
1935
1958
|
// Dispatch
|
|
1959
|
+
/**
|
|
1960
|
+
* Inspect or stop the on-device companion (`tools/verikun-companion`).
|
|
1961
|
+
*
|
|
1962
|
+
* `stop` exists because the companion holds the device's ONE UiAutomation connection for as
|
|
1963
|
+
* long as it runs, which locks out Appium, Layout Inspector and a second verikun. Without a
|
|
1964
|
+
* way to hand that back, the only recourse would be `adb shell pkill`.
|
|
1965
|
+
*/
|
|
1966
|
+
function cmdCompanion(ctx) {
|
|
1967
|
+
const action = ctx.positionals[0] ?? 'status';
|
|
1968
|
+
if (action !== 'status' && action !== 'stop') {
|
|
1969
|
+
throw new errors_1.CliError(`Usage: verikun companion <status|stop>`, 2);
|
|
1970
|
+
}
|
|
1971
|
+
if (ctx.platform !== 'android') {
|
|
1972
|
+
throw new errors_1.CliError('The companion is Android-only; iOS reads the hierarchy through idb, which is already fast.', 3);
|
|
1973
|
+
}
|
|
1974
|
+
const companion = new manager_1.Companion({
|
|
1975
|
+
adb: process.env.ADB || 'adb',
|
|
1976
|
+
serial: ctx.driver.resolvedSerial(),
|
|
1977
|
+
// `status`/`stop` never calibrate, so this is unreachable — but a throwing stub is
|
|
1978
|
+
// honest about that, where a silent no-op would hide a future miswiring.
|
|
1979
|
+
stockDump: () => {
|
|
1980
|
+
throw new errors_1.CliError('the companion command never calibrates', 3);
|
|
1981
|
+
},
|
|
1982
|
+
});
|
|
1983
|
+
if (action === 'stop') {
|
|
1984
|
+
companion.stop();
|
|
1985
|
+
if ((0, args_1.flagBool)(ctx.flags, 'json'))
|
|
1986
|
+
(0, output_1.json)({ companion: 'stopped' });
|
|
1987
|
+
else
|
|
1988
|
+
(0, output_1.out)('companion stopped');
|
|
1989
|
+
return 0;
|
|
1990
|
+
}
|
|
1991
|
+
const state = companion.describe();
|
|
1992
|
+
if ((0, args_1.flagBool)(ctx.flags, 'json'))
|
|
1993
|
+
(0, output_1.json)({ companion: state, enabled: (0, manager_1.companionEnabled)() });
|
|
1994
|
+
else {
|
|
1995
|
+
(0, output_1.out)(state);
|
|
1996
|
+
if (!(0, manager_1.companionEnabled)())
|
|
1997
|
+
(0, output_1.err)('note: disabled by VERIKUN_COMPANION — hierarchy reads use the slower stock dump');
|
|
1998
|
+
}
|
|
1999
|
+
return 0;
|
|
2000
|
+
}
|
|
1936
2001
|
// ---------------------------------------------------------------------------
|
|
1937
2002
|
async function executeCommand(command, ctx) {
|
|
1938
2003
|
switch (command) {
|
|
@@ -1940,6 +2005,8 @@ async function executeCommand(command, ctx) {
|
|
|
1940
2005
|
return cmdDevices(ctx);
|
|
1941
2006
|
case 'doctor':
|
|
1942
2007
|
return cmdDoctor(ctx);
|
|
2008
|
+
case 'companion':
|
|
2009
|
+
return cmdCompanion(ctx);
|
|
1943
2010
|
case 'ui':
|
|
1944
2011
|
case 'dump':
|
|
1945
2012
|
return cmdUi(ctx);
|
|
@@ -2280,6 +2347,8 @@ ENVIRONMENT
|
|
|
2280
2347
|
Claude Code plugin is out of date (a warning only —
|
|
2281
2348
|
it never changes the exit code). --fix disables
|
|
2282
2349
|
animations. VERIKUN_NO_UPDATE_CHECK skips the check
|
|
2350
|
+
companion <status|stop> [--json] On-device hierarchy reader (Android, on by default;
|
|
2351
|
+
VERIKUN_COMPANION=0 opts out)
|
|
2283
2352
|
|
|
2284
2353
|
TEST RUNS (actions are recorded; a run auto-starts on first action)
|
|
2285
2354
|
run start [name] [--force] Begin a named run (else one starts implicitly)
|
|
@@ -0,0 +1,397 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
// Lifecycle for the on-device companion: get one running, prove it agrees with the
|
|
3
|
+
// platform, use it, and get out of its way the moment anything goes wrong.
|
|
4
|
+
//
|
|
5
|
+
// The whole design is shaped by one constraint, measured on-device: only ONE UiAutomation
|
|
6
|
+
// may be connected per device, and the newcomer is SIGKILLed. So unlike `screenshotRaw()`,
|
|
7
|
+
// this cannot be a per-call try/catch — while the companion holds the connection the stock
|
|
8
|
+
// path is not merely slower, it is *unavailable* (exit 137). Every failure route here
|
|
9
|
+
// therefore gets the companion off the connection BEFORE the caller falls back.
|
|
10
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
11
|
+
exports.Companion = void 0;
|
|
12
|
+
exports.companionJarPath = companionJarPath;
|
|
13
|
+
exports.companionEnabled = companionEnabled;
|
|
14
|
+
exports.releaseCompanionOn = releaseCompanionOn;
|
|
15
|
+
const node_fs_1 = require("node:fs");
|
|
16
|
+
const node_path_1 = require("node:path");
|
|
17
|
+
const exec_1 = require("../exec");
|
|
18
|
+
const output_1 = require("../output");
|
|
19
|
+
const errors_1 = require("../errors");
|
|
20
|
+
const version_1 = require("../version");
|
|
21
|
+
const protocol_1 = require("./protocol");
|
|
22
|
+
const DEVICE_JAR = '/data/local/tmp/verikun-companion.jar';
|
|
23
|
+
/** What we learned about this device last time, kept ON the device because that is what the
|
|
24
|
+
* knowledge is about — it survives every `vk` process, every working directory, and a
|
|
25
|
+
* disposable CI runner. See readDeviceNote/writeDeviceNote. */
|
|
26
|
+
const DEVICE_NOTE = '/data/local/tmp/verikun-companion.note';
|
|
27
|
+
/** How long to let the process that claimed calibration finish before giving up on it.
|
|
28
|
+
* Generous: calibration is ~5s, and being patient costs one slow read while being impatient
|
|
29
|
+
* costs the mutual-SIGKILL contention the claim exists to prevent. */
|
|
30
|
+
const CALIBRATION_WAIT_MS = 30000;
|
|
31
|
+
const SOCKET = 'verikun-companion';
|
|
32
|
+
const MAIN_CLASS = 'dev.verikun.companion.CompanionApp';
|
|
33
|
+
/** uiautomator.jar supplies UiAutomationShellWrapper + AccessibilityNodeInfoDumper, which the
|
|
34
|
+
* companion borrows so its XML is byte-identical to `uiautomator dump`'s. */
|
|
35
|
+
const DEVICE_CLASSPATH = [
|
|
36
|
+
'/system/framework/android.test.runner.jar',
|
|
37
|
+
'/system/framework/uiautomator.jar',
|
|
38
|
+
DEVICE_JAR,
|
|
39
|
+
].join(':');
|
|
40
|
+
/** Cold start measured at ~1.5s on a physical SM-A415F; the margin is for slower devices. */
|
|
41
|
+
const START_TIMEOUT_MS = 12000;
|
|
42
|
+
/** The companion's way of saying getRootInActiveWindow() returned null. */
|
|
43
|
+
const NULL_ROOT_REPLY = /null root node/i;
|
|
44
|
+
const START_POLL_MS = 150;
|
|
45
|
+
/** The packaged companion jar: shipped in the npm tarball, and present in a source checkout
|
|
46
|
+
* once `tools/verikun-companion/build.sh` has run. Absent means "no companion available",
|
|
47
|
+
* which is a normal state, not an error. */
|
|
48
|
+
function companionJarPath() {
|
|
49
|
+
// dist/companion/manager.js → repo root is two levels up.
|
|
50
|
+
const candidate = (0, node_path_1.resolve)(__dirname, '..', '..', 'tools', 'verikun-companion', 'prebuilt', 'verikun-companion.jar');
|
|
51
|
+
return (0, node_fs_1.existsSync)(candidate) ? candidate : null;
|
|
52
|
+
}
|
|
53
|
+
/**
|
|
54
|
+
* ON by default; `VERIKUN_COMPANION=0` opts out.
|
|
55
|
+
*
|
|
56
|
+
* It started opt-in, and that was wrong: a hierarchy read is the dominant cost of every
|
|
57
|
+
* Android run, nobody discovers an environment variable they have not been told about, and
|
|
58
|
+
* the people who most need the speedup are the least likely to go looking for it. Being
|
|
59
|
+
* fast by default is the whole point.
|
|
60
|
+
*
|
|
61
|
+
* What makes that safe is that a failure cannot cost a test: the stock path is always there
|
|
62
|
+
* and every failure route hands the UiAutomation connection back before using it. What
|
|
63
|
+
* makes it not *slow* is the on-device note — a device where the companion cannot run says
|
|
64
|
+
* so once, and is never probed again.
|
|
65
|
+
*
|
|
66
|
+
* The real cost is that the companion holds the device's single UiAutomation connection
|
|
67
|
+
* while it runs, so Appium, Layout Inspector and TalkBack cannot attach. `VERIKUN_COMPANION=0`
|
|
68
|
+
* or `vk companion stop` hands it back.
|
|
69
|
+
*/
|
|
70
|
+
function companionEnabled() {
|
|
71
|
+
const v = (process.env.VERIKUN_COMPANION ?? '').trim().toLowerCase();
|
|
72
|
+
return !(v === '0' || v === 'false' || v === 'off' || v === 'no');
|
|
73
|
+
}
|
|
74
|
+
/**
|
|
75
|
+
* Ask any companion on this device to drop the UiAutomation connection, so a stock
|
|
76
|
+
* `uiautomator dump` can run. Best-effort and silent: no companion is the normal case.
|
|
77
|
+
*
|
|
78
|
+
* Called from the STOCK dump path when it fails, including when the caller never opted in.
|
|
79
|
+
* A companion started by an earlier `VERIKUN_COMPANION=1` command outlives that process, so
|
|
80
|
+
* a plain `vk tap` afterwards would otherwise be SIGKILLed for as long as the companion
|
|
81
|
+
* lives. verikun's own helper must never be the reason verikun cannot read the screen.
|
|
82
|
+
*
|
|
83
|
+
* `release`, not `quit`: the process stays warm, so the next opted-in command re-acquires in
|
|
84
|
+
* ~1s instead of paying a full cold start.
|
|
85
|
+
*/
|
|
86
|
+
function releaseCompanionOn(serial) {
|
|
87
|
+
try {
|
|
88
|
+
(0, protocol_1.requestSync)((0, protocol_1.portForSerial)(serial), 'release', 4000);
|
|
89
|
+
return true;
|
|
90
|
+
}
|
|
91
|
+
catch {
|
|
92
|
+
return false;
|
|
93
|
+
}
|
|
94
|
+
}
|
|
95
|
+
class Companion {
|
|
96
|
+
deps;
|
|
97
|
+
port;
|
|
98
|
+
/** Latched off for the rest of this process once anything goes wrong: a companion that
|
|
99
|
+
* has already failed is not worth re-probing per read, since the retry would cost more
|
|
100
|
+
* than the stock dump it is trying to avoid. */
|
|
101
|
+
unusable = false;
|
|
102
|
+
dims;
|
|
103
|
+
constructor(deps) {
|
|
104
|
+
this.deps = deps;
|
|
105
|
+
this.port = (0, protocol_1.portForSerial)(deps.serial);
|
|
106
|
+
}
|
|
107
|
+
adb(args, timeout = 15000) {
|
|
108
|
+
return (0, exec_1.runText)(this.deps.adb, ['-s', this.deps.serial, ...args], { timeout });
|
|
109
|
+
}
|
|
110
|
+
/**
|
|
111
|
+
* What we concluded about this device last time: `<verikun version>|<app|real|unsupported>`.
|
|
112
|
+
*
|
|
113
|
+
* Kept on the DEVICE rather than in `.verikun/`, because it is a fact about the phone, not
|
|
114
|
+
* about a working directory — it survives a `cd`, and a disposable CI runner inherits it
|
|
115
|
+
* instead of starting cold. Keyed by verikun version so an upgrade re-tries a device that
|
|
116
|
+
* an older build could not use.
|
|
117
|
+
*/
|
|
118
|
+
readDeviceNote() {
|
|
119
|
+
const r = this.adb(['shell', `cat ${DEVICE_NOTE} 2>/dev/null`], 5000);
|
|
120
|
+
const [version, verdict] = r.stdout.trim().split('|');
|
|
121
|
+
if (version !== version_1.VERSION)
|
|
122
|
+
return undefined;
|
|
123
|
+
if (verdict === 'unsupported' || verdict === 'app' || verdict === 'real')
|
|
124
|
+
return verdict;
|
|
125
|
+
return undefined;
|
|
126
|
+
}
|
|
127
|
+
writeDeviceNote(verdict) {
|
|
128
|
+
// Single-quoted, and every value here comes from a closed set — nothing caller-supplied
|
|
129
|
+
// reaches the device shell.
|
|
130
|
+
this.adb(['shell', `echo '${version_1.VERSION}|${verdict}' > ${DEVICE_NOTE}`], 5000);
|
|
131
|
+
}
|
|
132
|
+
/**
|
|
133
|
+
* The hierarchy XML, or null when the companion cannot serve it — in which case the
|
|
134
|
+
* UiAutomation connection has already been handed back, so the caller's stock dump will
|
|
135
|
+
* work rather than being SIGKILLed.
|
|
136
|
+
*/
|
|
137
|
+
dump(idleMs) {
|
|
138
|
+
if (this.unusable)
|
|
139
|
+
return null;
|
|
140
|
+
try {
|
|
141
|
+
if (!this.dims) {
|
|
142
|
+
this.dims = this.ensureReady();
|
|
143
|
+
if (!this.dims)
|
|
144
|
+
return null;
|
|
145
|
+
}
|
|
146
|
+
const reply = (0, protocol_1.requestSync)(this.port, (0, protocol_1.dumpCommand)(idleMs, this.dims));
|
|
147
|
+
if (!(0, protocol_1.isHierarchy)(reply)) {
|
|
148
|
+
// The companion reports its own failures as plain text (`ERROR …`, `released —
|
|
149
|
+
// call acquire first`). Those must never reach the XML parser as though they were
|
|
150
|
+
// a screen: an unparseable "hierarchy" reads as zero elements, which is the
|
|
151
|
+
// "absent" lie that silently skips a guard.
|
|
152
|
+
const detail = reply.toString('utf8').trim().slice(0, 200) || 'empty reply';
|
|
153
|
+
// A null root is the DEVICE having no window, not the companion malfunctioning.
|
|
154
|
+
// Standing down for it would release a perfectly healthy connection and drop the
|
|
155
|
+
// whole process onto the 2.4s path — measured after every `launch --clear`, which
|
|
156
|
+
// leaves exactly this gap.
|
|
157
|
+
if (NULL_ROOT_REPLY.test(detail)) {
|
|
158
|
+
throw new errors_1.NoWindowError('No window to read: the app has not drawn yet (force-stopped, or mid-launch).');
|
|
159
|
+
}
|
|
160
|
+
throw new Error(detail);
|
|
161
|
+
}
|
|
162
|
+
return reply.toString('utf8');
|
|
163
|
+
}
|
|
164
|
+
catch (e) {
|
|
165
|
+
if (e instanceof errors_1.NoWindowError)
|
|
166
|
+
throw e; // transient screen state — the companion is fine
|
|
167
|
+
this.standDown(`companion dump failed (${e.message})`);
|
|
168
|
+
return null;
|
|
169
|
+
}
|
|
170
|
+
}
|
|
171
|
+
/** Get a calibrated companion running, or give up for this process. */
|
|
172
|
+
ensureReady() {
|
|
173
|
+
// FAST PATH, and the only one that needs no coordination: a companion that is already
|
|
174
|
+
// running AND calibrated. Note the `dims` check — an uncalibrated companion is NOT ready,
|
|
175
|
+
// because calibrating it needs the UiAutomation connection exclusively, and that has to
|
|
176
|
+
// happen under the lock below like every other exclusive use.
|
|
177
|
+
const live = this.probeState();
|
|
178
|
+
if (live.usable && live.dims) {
|
|
179
|
+
// Alive but NOT holding the connection: something asked it to let go — the stock dump
|
|
180
|
+
// path does exactly that when it fails, and `vk companion stop` is not the only route.
|
|
181
|
+
// Take the connection back before using it, or every read here answers "released" and
|
|
182
|
+
// silently falls through to the 2.4s path for the rest of the run.
|
|
183
|
+
if (!live.held && !this.acquire())
|
|
184
|
+
return undefined;
|
|
185
|
+
return live.dims;
|
|
186
|
+
}
|
|
187
|
+
// Nothing listening. Before paying a start + calibration, ask the device what happened
|
|
188
|
+
// last time — this is what stops a phone the companion cannot run on from costing every
|
|
189
|
+
// single command a doomed start attempt now that this is on by default.
|
|
190
|
+
const note = this.readDeviceNote();
|
|
191
|
+
if (note === 'unsupported') {
|
|
192
|
+
this.unusable = true;
|
|
193
|
+
return undefined;
|
|
194
|
+
}
|
|
195
|
+
return this.startAndCalibrate(note, live);
|
|
196
|
+
}
|
|
197
|
+
/** Wait for the process that claimed calibration to publish its verdict. */
|
|
198
|
+
waitForCalibration() {
|
|
199
|
+
const deadline = Date.now() + CALIBRATION_WAIT_MS;
|
|
200
|
+
while (Date.now() < deadline) {
|
|
201
|
+
(0, exec_1.sleepSync)(START_POLL_MS);
|
|
202
|
+
const state = this.probeState();
|
|
203
|
+
if (!state.usable)
|
|
204
|
+
return undefined; // it died; our caller falls back
|
|
205
|
+
if (!state.dims)
|
|
206
|
+
continue; // still working — do NOT touch the connection
|
|
207
|
+
if (!state.held && !this.acquire())
|
|
208
|
+
return undefined;
|
|
209
|
+
return state.dims;
|
|
210
|
+
}
|
|
211
|
+
return undefined;
|
|
212
|
+
}
|
|
213
|
+
startAndCalibrate(note, state) {
|
|
214
|
+
if (!state.usable && !this.start()) {
|
|
215
|
+
// Only record "this phone cannot run it" when the phone is genuinely the problem, and
|
|
216
|
+
// only after looking once more. Two ways this verdict could otherwise be wrong, and it
|
|
217
|
+
// is STICKY — every later command on this device would skip straight to the 2.4s path:
|
|
218
|
+
// - the jar is missing, which is a fault of THIS checkout, not of the device;
|
|
219
|
+
// - several `vk` processes cold-started at once and collided, which is transient and
|
|
220
|
+
// usually leaves a perfectly good companion running (started by whoever won).
|
|
221
|
+
if (companionJarPath() && !this.probeState().usable)
|
|
222
|
+
this.writeDeviceNote('unsupported');
|
|
223
|
+
this.unusable = true;
|
|
224
|
+
return undefined;
|
|
225
|
+
}
|
|
226
|
+
// A remembered verdict skips calibration entirely — worth ~4.7s of the cold start, and
|
|
227
|
+
// the companion idle-shuts-down every 15 minutes, so restarts are routine.
|
|
228
|
+
if (note === 'app' || note === 'real') {
|
|
229
|
+
try {
|
|
230
|
+
(0, protocol_1.requestSync)(this.port, `calibrated ${note}`, 4000);
|
|
231
|
+
return note;
|
|
232
|
+
}
|
|
233
|
+
catch {
|
|
234
|
+
/* fall through and calibrate properly */
|
|
235
|
+
}
|
|
236
|
+
}
|
|
237
|
+
// ONLY ONE PROCESS MAY CALIBRATE. It works by releasing the UiAutomation connection to
|
|
238
|
+
// take a real `uiautomator dump`, so a second process doing it concurrently SIGKILLs the
|
|
239
|
+
// first one's dump — MEASURED as five concurrent first reads exiting [3,3,0,3,3]. The
|
|
240
|
+
// claim is granted by the companion itself, whose single-threaded accept loop makes it
|
|
241
|
+
// genuinely atomic; the obvious host-side lock does not work, because Android's toybox
|
|
242
|
+
// `mkdir` SUCCEEDS on an existing directory and so grants itself to every caller.
|
|
243
|
+
const already = this.probeState();
|
|
244
|
+
if (already.dims)
|
|
245
|
+
return already.dims;
|
|
246
|
+
if (!this.claimCalibration())
|
|
247
|
+
return this.waitForCalibration();
|
|
248
|
+
return this.calibrate();
|
|
249
|
+
}
|
|
250
|
+
claimCalibration() {
|
|
251
|
+
try {
|
|
252
|
+
return (0, protocol_1.requestSync)(this.port, 'claim-calibration', 8000).toString('utf8').trim() === 'granted';
|
|
253
|
+
}
|
|
254
|
+
catch {
|
|
255
|
+
return false;
|
|
256
|
+
}
|
|
257
|
+
}
|
|
258
|
+
/** Retake the UiAutomation connection. ~1.05s: the cold-bridge idle wait, paid once. */
|
|
259
|
+
acquire() {
|
|
260
|
+
try {
|
|
261
|
+
(0, protocol_1.requestSync)(this.port, 'acquire', 20000);
|
|
262
|
+
return true;
|
|
263
|
+
}
|
|
264
|
+
catch (e) {
|
|
265
|
+
this.standDown(`companion could not reacquire the connection (${e.message})`);
|
|
266
|
+
return false;
|
|
267
|
+
}
|
|
268
|
+
}
|
|
269
|
+
probeState() {
|
|
270
|
+
try {
|
|
271
|
+
return (0, protocol_1.parseState)((0, protocol_1.requestSync)(this.port, 'state', 4000));
|
|
272
|
+
}
|
|
273
|
+
catch {
|
|
274
|
+
return { usable: false, held: false };
|
|
275
|
+
}
|
|
276
|
+
}
|
|
277
|
+
/** Push the jar, forward the socket, spawn detached, wait for it to answer. */
|
|
278
|
+
start() {
|
|
279
|
+
const jar = companionJarPath();
|
|
280
|
+
if (!jar) {
|
|
281
|
+
// Nothing to push. A source checkout that has not run tools/verikun-companion/build.sh
|
|
282
|
+
// lands here, and so would a broken install — neither is a fact about the device.
|
|
283
|
+
return false;
|
|
284
|
+
}
|
|
285
|
+
// Look once more before tearing anything down. Between our probe and now, another `vk`
|
|
286
|
+
// process may have started a perfectly good companion — and stopStale() below would kill
|
|
287
|
+
// it, so two processes racing could ping-pong indefinitely, each killing the other's.
|
|
288
|
+
if (this.probeState().usable)
|
|
289
|
+
return true;
|
|
290
|
+
// A companion left by an older verikun answers with a different protocol number. It is
|
|
291
|
+
// not merely useless — it is still holding the connection, so it must be stopped.
|
|
292
|
+
this.stopStale();
|
|
293
|
+
if (this.adb(['push', jar, DEVICE_JAR], 30000).code !== 0)
|
|
294
|
+
return false;
|
|
295
|
+
if (this.adb(['forward', `tcp:${this.port}`, `localabstract:${SOCKET}`]).code !== 0)
|
|
296
|
+
return false;
|
|
297
|
+
// Detached, and stdio fully redirected: every `vk` call is its own process, so a
|
|
298
|
+
// companion tied to this adb shell would die before the next command could reuse it —
|
|
299
|
+
// which is the entire point of it existing.
|
|
300
|
+
this.adb(['shell', `nohup env CLASSPATH=${DEVICE_CLASSPATH} app_process / ${MAIN_CLASS} >/dev/null 2>&1 &`]);
|
|
301
|
+
const deadline = Date.now() + START_TIMEOUT_MS;
|
|
302
|
+
while (Date.now() < deadline) {
|
|
303
|
+
if (this.probeState().usable)
|
|
304
|
+
return true;
|
|
305
|
+
(0, exec_1.sleepSync)(START_POLL_MS);
|
|
306
|
+
}
|
|
307
|
+
(0, output_1.err)('[verikun] companion did not start; using the stock hierarchy dump');
|
|
308
|
+
return false;
|
|
309
|
+
}
|
|
310
|
+
stopStale() {
|
|
311
|
+
try {
|
|
312
|
+
(0, protocol_1.requestSync)(this.port, 'quit', 3000);
|
|
313
|
+
}
|
|
314
|
+
catch {
|
|
315
|
+
/* nothing listening, or it is wedged — killByName is the backstop */
|
|
316
|
+
}
|
|
317
|
+
this.killByName();
|
|
318
|
+
}
|
|
319
|
+
/**
|
|
320
|
+
* Prove the companion's dump matches the platform's own before trusting it.
|
|
321
|
+
*
|
|
322
|
+
* The dumper clips every node's bounds to a display size, and which size the platform
|
|
323
|
+
* uses varies by build: AOSP's `DumpCommand` reads `getRealSize()`, while a physical
|
|
324
|
+
* SM-A415F's stock dump matches `getSize()`. Guessing wrong does not fail loudly — it
|
|
325
|
+
* shifts every element near the bottom of the screen, and the resulting tap lands
|
|
326
|
+
* somewhere else while still reporting success, which is the worst failure a testing
|
|
327
|
+
* tool has. So do not guess: take one real `uiautomator dump` and adopt whichever
|
|
328
|
+
* source reproduces it byte for byte.
|
|
329
|
+
*
|
|
330
|
+
* Costs one stock dump (~2.4s) plus a reconnect (~1.05s), once per companion — not once
|
|
331
|
+
* per process, because the verdict is stored in the companion itself.
|
|
332
|
+
*/
|
|
333
|
+
calibrate() {
|
|
334
|
+
try {
|
|
335
|
+
// The stock dump cannot run while we hold the connection — it would be SIGKILLed.
|
|
336
|
+
(0, protocol_1.requestSync)(this.port, 'release', 5000);
|
|
337
|
+
const stock = this.deps.stockDump().trim();
|
|
338
|
+
(0, protocol_1.requestSync)(this.port, 'acquire', 20000);
|
|
339
|
+
for (const dims of ['app', 'real']) {
|
|
340
|
+
const reply = (0, protocol_1.requestSync)(this.port, (0, protocol_1.dumpCommand)(0, dims));
|
|
341
|
+
if ((0, protocol_1.isHierarchy)(reply) && reply.toString('utf8').trim() === stock) {
|
|
342
|
+
(0, protocol_1.requestSync)(this.port, `calibrated ${dims}`, 4000);
|
|
343
|
+
this.writeDeviceNote(dims);
|
|
344
|
+
return dims;
|
|
345
|
+
}
|
|
346
|
+
}
|
|
347
|
+
// Neither matched. Usually the screen simply moved between the two dumps, but it
|
|
348
|
+
// could equally be a device whose bounds we would get wrong — and being slow is
|
|
349
|
+
// strictly better than tapping the wrong pixel, so decline rather than pick one.
|
|
350
|
+
this.standDown('companion output did not match the platform dump; using the stock path');
|
|
351
|
+
return undefined;
|
|
352
|
+
}
|
|
353
|
+
catch (e) {
|
|
354
|
+
this.standDown(`companion calibration failed (${e.message})`);
|
|
355
|
+
return undefined;
|
|
356
|
+
}
|
|
357
|
+
}
|
|
358
|
+
/** Hand the UiAutomation connection back, and stop using the companion in this process. */
|
|
359
|
+
standDown(reason) {
|
|
360
|
+
this.unusable = true;
|
|
361
|
+
(0, output_1.err)(`[verikun] ${reason}`);
|
|
362
|
+
try {
|
|
363
|
+
(0, protocol_1.requestSync)(this.port, 'release', 4000);
|
|
364
|
+
}
|
|
365
|
+
catch {
|
|
366
|
+
// It cannot be asked, so it has to be taken: while it holds the connection the stock
|
|
367
|
+
// dump is SIGKILLed, and the caller is about to depend on the stock dump.
|
|
368
|
+
this.killByName();
|
|
369
|
+
}
|
|
370
|
+
}
|
|
371
|
+
/** Kill by command line rather than a remembered pid — the companion outlives the process
|
|
372
|
+
* that started it, so whoever needs it gone usually never had the pid. */
|
|
373
|
+
killByName() {
|
|
374
|
+
this.adb(['shell', `pkill -f ${MAIN_CLASS}`], 5000);
|
|
375
|
+
}
|
|
376
|
+
/** Explicit teardown: `vk companion stop`. */
|
|
377
|
+
stop() {
|
|
378
|
+
try {
|
|
379
|
+
(0, protocol_1.requestSync)(this.port, 'quit', 4000);
|
|
380
|
+
}
|
|
381
|
+
catch {
|
|
382
|
+
this.killByName();
|
|
383
|
+
}
|
|
384
|
+
this.adb(['forward', '--remove', `tcp:${this.port}`]);
|
|
385
|
+
}
|
|
386
|
+
/** For `vk companion status`. */
|
|
387
|
+
describe() {
|
|
388
|
+
try {
|
|
389
|
+
const state = (0, protocol_1.requestSync)(this.port, 'state', 4000).toString('utf8').trim();
|
|
390
|
+
return `running on port ${this.port} (${state})`;
|
|
391
|
+
}
|
|
392
|
+
catch {
|
|
393
|
+
return 'not running';
|
|
394
|
+
}
|
|
395
|
+
}
|
|
396
|
+
}
|
|
397
|
+
exports.Companion = Companion;
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
// Wire protocol for the on-device companion (tools/verikun-companion).
|
|
3
|
+
//
|
|
4
|
+
// One request per connection: write a command line, read until EOF. Kept separate from
|
|
5
|
+
// the lifecycle manager so the framing rules — which are where the sharp edges are —
|
|
6
|
+
// can be unit-tested without a device.
|
|
7
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
8
|
+
exports.CompanionUnavailableError = exports.COMPANION_PROTOCOL = void 0;
|
|
9
|
+
exports.isLiveReply = isLiveReply;
|
|
10
|
+
exports.pingMatches = pingMatches;
|
|
11
|
+
exports.parseState = parseState;
|
|
12
|
+
exports.isHierarchy = isHierarchy;
|
|
13
|
+
exports.dumpCommand = dumpCommand;
|
|
14
|
+
exports.portForSerial = portForSerial;
|
|
15
|
+
exports.requestSync = requestSync;
|
|
16
|
+
const node_child_process_1 = require("node:child_process");
|
|
17
|
+
const node_path_1 = require("node:path");
|
|
18
|
+
/** Bumped in lockstep with PROTOCOL_VERSION in CompanionApp.java. A companion left running
|
|
19
|
+
* by an older verikun answers `ping` with a different number and is restarted rather than
|
|
20
|
+
* talked to, so a changed dump format can never be replayed by a stale process. */
|
|
21
|
+
exports.COMPANION_PROTOCOL = '1';
|
|
22
|
+
class CompanionUnavailableError extends Error {
|
|
23
|
+
}
|
|
24
|
+
exports.CompanionUnavailableError = CompanionUnavailableError;
|
|
25
|
+
/**
|
|
26
|
+
* Is this reply from a live companion, or from nothing at all?
|
|
27
|
+
*
|
|
28
|
+
* Load-bearing: `adb forward` keeps the host port open whether or not anything is
|
|
29
|
+
* listening on the device end, so a companion that died answers with a successful
|
|
30
|
+
* connection and ZERO bytes rather than ECONNREFUSED. A client that only handles connect
|
|
31
|
+
* errors reads that as a valid empty hierarchy — which is exactly the "absent" lie that
|
|
32
|
+
* silently skips a guard. Treat an empty reply as unavailable, always.
|
|
33
|
+
*/
|
|
34
|
+
function isLiveReply(reply) {
|
|
35
|
+
return reply.length > 0;
|
|
36
|
+
}
|
|
37
|
+
/** Does this `ping` reply come from a companion we can speak to? */
|
|
38
|
+
function pingMatches(reply) {
|
|
39
|
+
if (!isLiveReply(reply))
|
|
40
|
+
return false;
|
|
41
|
+
const [name, version] = reply.toString('utf8').trim().split(/\s+/);
|
|
42
|
+
return name === 'verikun-companion' && version === exports.COMPANION_PROTOCOL;
|
|
43
|
+
}
|
|
44
|
+
/**
|
|
45
|
+
* Parse a `state` reply: `verikun-companion <proto> (ready <dims>|uncalibrated) (held|released)`.
|
|
46
|
+
*
|
|
47
|
+
* One round trip answers all three questions, because each one costs a spawned client
|
|
48
|
+
* process on the host — and this runs before every read that has not already resolved.
|
|
49
|
+
*/
|
|
50
|
+
function parseState(reply) {
|
|
51
|
+
const unusable = { usable: false, held: false };
|
|
52
|
+
if (!isLiveReply(reply))
|
|
53
|
+
return unusable;
|
|
54
|
+
const parts = reply.toString('utf8').trim().split(/\s+/);
|
|
55
|
+
if (parts[0] !== 'verikun-companion' || parts[1] !== exports.COMPANION_PROTOCOL)
|
|
56
|
+
return unusable;
|
|
57
|
+
const dims = parts[2] === 'ready' ? parts[3] : undefined;
|
|
58
|
+
return { usable: true, dims, held: parts.includes('held') };
|
|
59
|
+
}
|
|
60
|
+
/** A dump reply is only usable if it is actually a hierarchy — the companion reports its
|
|
61
|
+
* own failures as plain text (`ERROR …`, `released — call acquire first`), and those must
|
|
62
|
+
* never reach the XML parser as if they were a screen. */
|
|
63
|
+
function isHierarchy(reply) {
|
|
64
|
+
return isLiveReply(reply) && reply.toString('utf8', 0, 512).includes('<hierarchy');
|
|
65
|
+
}
|
|
66
|
+
/** The command line for a dump, given the idle window and the calibrated dimension source. */
|
|
67
|
+
function dumpCommand(idleMs, dims) {
|
|
68
|
+
return `dump ${Math.max(0, Math.round(idleMs))} ${dims}`;
|
|
69
|
+
}
|
|
70
|
+
/**
|
|
71
|
+
* Local port for a device's companion. One companion per device (only one UiAutomation may
|
|
72
|
+
* be connected at a time), so ports are derived from the serial rather than allocated — two
|
|
73
|
+
* verikun processes driving the SAME device must land on the same forward, and two driving
|
|
74
|
+
* DIFFERENT devices must not collide.
|
|
75
|
+
*/
|
|
76
|
+
function portForSerial(serial, base = 8299, span = 200) {
|
|
77
|
+
let hash = 0;
|
|
78
|
+
for (let i = 0; i < serial.length; i++)
|
|
79
|
+
hash = (hash * 31 + serial.charCodeAt(i)) >>> 0;
|
|
80
|
+
return base + (hash % span);
|
|
81
|
+
}
|
|
82
|
+
/**
|
|
83
|
+
* Send one command and read the whole reply, synchronously.
|
|
84
|
+
*
|
|
85
|
+
* Synchronous because `Driver` is: every device call in verikun is a `spawnSync`, and
|
|
86
|
+
* making the hierarchy read async would ripple through cli.ts, run.ts and the engine for
|
|
87
|
+
* no behavioural gain. Node has no sync socket, so the socket work happens in a child
|
|
88
|
+
* process (`sock-client.js`) and we read its stdout — the same shape as shelling to `adb`.
|
|
89
|
+
* Measured cost of that indirection: 39ms per round trip, against 2400ms for the stock
|
|
90
|
+
* dump it replaces.
|
|
91
|
+
*
|
|
92
|
+
* Throws CompanionUnavailableError for every failure mode, so callers have exactly one
|
|
93
|
+
* thing to catch before handing the connection back and using the stock path.
|
|
94
|
+
*/
|
|
95
|
+
function requestSync(port, command, timeoutMs = 10000) {
|
|
96
|
+
const client = (0, node_path_1.join)(__dirname, 'sock-client.js');
|
|
97
|
+
const r = (0, node_child_process_1.spawnSync)(process.execPath, [client, String(port), ...command.split(' ')], {
|
|
98
|
+
timeout: timeoutMs,
|
|
99
|
+
maxBuffer: 64 * 1024 * 1024,
|
|
100
|
+
});
|
|
101
|
+
if (r.error)
|
|
102
|
+
throw new CompanionUnavailableError(r.error.message);
|
|
103
|
+
if (r.status !== 0)
|
|
104
|
+
throw new CompanionUnavailableError(`companion unreachable on port ${port}`);
|
|
105
|
+
const out = r.stdout ?? Buffer.alloc(0);
|
|
106
|
+
// `adb forward` keeps the host port open whether or not anything is listening on the
|
|
107
|
+
// device end, so a dead companion connects fine and returns nothing. See isLiveReply.
|
|
108
|
+
if (!isLiveReply(out))
|
|
109
|
+
throw new CompanionUnavailableError('companion is not running');
|
|
110
|
+
return out;
|
|
111
|
+
}
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
// One-shot socket client, run as its own process.
|
|
3
|
+
//
|
|
4
|
+
// It exists because `Driver` is entirely synchronous — every device call is a spawnSync —
|
|
5
|
+
// and Node has no synchronous socket. Rather than make getElements() async and ripple that
|
|
6
|
+
// through cli.ts, run.ts and the engine, the companion is reached the same way every other
|
|
7
|
+
// device call is: by spawning something and reading its stdout.
|
|
8
|
+
//
|
|
9
|
+
// Measured on a physical SM-A415F: 39ms per round trip including this process's own
|
|
10
|
+
// startup, against 2400ms for `uiautomator dump`. The process spawn is most of that 39ms
|
|
11
|
+
// and is the price of staying synchronous — worth it at ~60x.
|
|
12
|
+
//
|
|
13
|
+
// Usage: node sock-client.js <port> <command...> → reply on stdout, exit 3 if unreachable.
|
|
14
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
15
|
+
const node_net_1 = require("node:net");
|
|
16
|
+
const [portArg, ...command] = process.argv.slice(2);
|
|
17
|
+
const chunks = [];
|
|
18
|
+
const sock = (0, node_net_1.connect)(Number(portArg), '127.0.0.1');
|
|
19
|
+
sock.on('connect', () => sock.write(command.join(' ') + '\n'));
|
|
20
|
+
sock.on('data', (c) => chunks.push(c));
|
|
21
|
+
sock.on('end', () => process.stdout.write(Buffer.concat(chunks)));
|
|
22
|
+
sock.on('error', () => process.exit(3));
|
package/dist/drivers/adb.js
CHANGED
|
@@ -5,6 +5,7 @@ exports.probeAdb = probeAdb;
|
|
|
5
5
|
exports.escapeText = escapeText;
|
|
6
6
|
exports.adbTransport = adbTransport;
|
|
7
7
|
exports.severanceRisk = severanceRisk;
|
|
8
|
+
const manager_1 = require("../companion/manager");
|
|
8
9
|
const errors_1 = require("../errors");
|
|
9
10
|
const exec_1 = require("../exec");
|
|
10
11
|
const android_parse_1 = require("../ui/android-parse");
|
|
@@ -68,6 +69,22 @@ const KEYCODES = {
|
|
|
68
69
|
page_down: 93,
|
|
69
70
|
};
|
|
70
71
|
const DUMP_PATHS = ['/sdcard/window_dump.xml', '/data/local/tmp/window_dump.xml'];
|
|
72
|
+
/** Idle window the companion waits for before dumping. Kept at the stock command's own
|
|
73
|
+
* 1000ms so behaviour is unchanged — and it is free: on a warm accessibility bridge the
|
|
74
|
+
* screen has already been quiet, so waitForIdle returns at once (measured: `dump 1000`
|
|
75
|
+
* costs the same ~10ms as `dump 0`). The stock path pays a full second for it only
|
|
76
|
+
* because a freshly connected bridge has no history of quiet to draw on. */
|
|
77
|
+
const COMPANION_IDLE_MS = 1000;
|
|
78
|
+
/** What both capture paths say when there is no window to read: AOSP's DumpCommand prints
|
|
79
|
+
* "ERROR: null root node returned by UiTestAutomationBridge." and writes no file, and the
|
|
80
|
+
* companion reports the same condition from getRootInActiveWindow(). Transient — see
|
|
81
|
+
* NoWindowError. */
|
|
82
|
+
const NULL_ROOT = /null root node/i;
|
|
83
|
+
/** Header sizes `screencap` writes before the pixels: width/height/format, plus a
|
|
84
|
+
* colorspace word since Android 9. Newest first — see `screenshotRaw`. */
|
|
85
|
+
const RAW_HEADER_SIZES = [16, 12];
|
|
86
|
+
/** android.graphics.PixelFormat.RGBA_8888 */
|
|
87
|
+
const PIXEL_FORMAT_RGBA_8888 = 1;
|
|
71
88
|
const DEFAULT_LOG_LINES = 200;
|
|
72
89
|
/**
|
|
73
90
|
* Escape a string for `adb shell input text <arg>`. The argument is parsed twice
|
|
@@ -139,6 +156,8 @@ class AdbDriver {
|
|
|
139
156
|
cachedScreen;
|
|
140
157
|
/** Rotation of the most recent dump — see viewport(). */
|
|
141
158
|
lastRotation;
|
|
159
|
+
/** undefined = not built yet, null = opted out. See companionOrNull(). */
|
|
160
|
+
companion;
|
|
142
161
|
constructor(serial) {
|
|
143
162
|
this.requested = serial;
|
|
144
163
|
}
|
|
@@ -244,16 +263,69 @@ class AdbDriver {
|
|
|
244
263
|
}
|
|
245
264
|
return this.cachedScreen;
|
|
246
265
|
}
|
|
266
|
+
/** The resident companion, when opted in. Built lazily so a run that never reads the
|
|
267
|
+
* hierarchy never touches the device with it. */
|
|
268
|
+
companionOrNull() {
|
|
269
|
+
if (!(0, manager_1.companionEnabled)())
|
|
270
|
+
return null;
|
|
271
|
+
if (this.companion === undefined) {
|
|
272
|
+
this.companion = new manager_1.Companion({
|
|
273
|
+
adb: ADB,
|
|
274
|
+
serial: this.resolvedSerial(),
|
|
275
|
+
// Calibration compares the companion against exactly the dump the rest of verikun
|
|
276
|
+
// would otherwise have used, so pass the stock path itself.
|
|
277
|
+
stockDump: () => this.stockDumpXml(),
|
|
278
|
+
});
|
|
279
|
+
}
|
|
280
|
+
return this.companion;
|
|
281
|
+
}
|
|
247
282
|
dumpXml() {
|
|
283
|
+
// The companion answers in ~40ms against ~2400ms for the stock path. A null here means
|
|
284
|
+
// it could not serve the read AND has already handed the UiAutomation connection back —
|
|
285
|
+
// which matters, because the stock dump below is SIGKILLed while it is held. A
|
|
286
|
+
// NoWindowError propagates instead: there is nothing for the stock path to read either,
|
|
287
|
+
// and re-asking it would only burn the caller's wait budget more slowly.
|
|
288
|
+
const fast = this.companionOrNull()?.dump(COMPANION_IDLE_MS);
|
|
289
|
+
if (fast)
|
|
290
|
+
return fast;
|
|
291
|
+
return this.stockDumpXml();
|
|
292
|
+
}
|
|
293
|
+
stockDumpXml() {
|
|
248
294
|
let lastErr = '';
|
|
249
295
|
for (let attempt = 0; attempt < 3; attempt++) {
|
|
250
296
|
const path = DUMP_PATHS[Math.min(attempt, DUMP_PATHS.length - 1)];
|
|
251
|
-
|
|
297
|
+
// DELETE THE FILE FIRST, in the same device shell. `uiautomator dump` writes to a
|
|
298
|
+
// fixed path and leaves the PREVIOUS dump there whenever it fails — and it fails
|
|
299
|
+
// without saying so in the exit code: AOSP's DumpCommand prints "ERROR: could not get
|
|
300
|
+
// idle state" and returns normally when waitForIdle times out (an animating screen),
|
|
301
|
+
// and the whole process is SIGKILLed if something else holds the device's single
|
|
302
|
+
// UiAutomation connection. Reading the file back then returns a stale screen that is
|
|
303
|
+
// perfectly well-formed, so every check below passes and the caller taps coordinates
|
|
304
|
+
// from a screen that is minutes old. MEASURED: a dump killed at 18:46 happily served
|
|
305
|
+
// the 18:44 hierarchy. Removing it first makes that impossible — `cat` can only
|
|
306
|
+
// succeed if THIS dump wrote it.
|
|
307
|
+
const dump = (0, exec_1.runText)(ADB, this.withSerial(['shell', `rm -f ${path}; uiautomator dump ${path}`]), {
|
|
308
|
+
timeout: 15000,
|
|
309
|
+
});
|
|
252
310
|
const cat = (0, exec_1.runBinary)(ADB, this.withSerial(['exec-out', 'cat', path]));
|
|
253
311
|
const xml = cat.stdout.toString('utf8');
|
|
254
312
|
if (xml.includes('<hierarchy'))
|
|
255
313
|
return xml;
|
|
256
314
|
lastErr = `${dump.stdout} ${dump.stderr} ${cat.stderr}`.replace(/\s+/g, ' ').trim();
|
|
315
|
+
// No window is not a failed capture — it is a successful reading of a screen that has
|
|
316
|
+
// nothing on it yet. Retrying it here just spends someone else's wait budget three
|
|
317
|
+
// times as fast; hand it up to whoever knows how long they are willing to wait.
|
|
318
|
+
if (NULL_ROOT.test(lastErr)) {
|
|
319
|
+
throw new errors_1.NoWindowError('No window to read: the app has not drawn yet (force-stopped, or mid-launch). ' +
|
|
320
|
+
'Retry, or use a command that waits (`vk wait`, or any selector lookup).');
|
|
321
|
+
}
|
|
322
|
+
// A companion holds the device's ONE UiAutomation connection and SIGKILLs anything
|
|
323
|
+
// else that wants it — including this dump. It outlives the process that started it,
|
|
324
|
+
// so a later command that never opted in would fail for as long as it lives. Ask it
|
|
325
|
+
// to let go and try again: verikun's own helper must not be why verikun cannot read
|
|
326
|
+
// the screen. No companion running is the normal case and costs one refused connect.
|
|
327
|
+
if (attempt === 0)
|
|
328
|
+
(0, manager_1.releaseCompanionOn)(this.resolvedSerial());
|
|
257
329
|
}
|
|
258
330
|
throw new errors_1.CliError(`Failed to capture UI hierarchy after 3 attempts. ${lastErr}\n` +
|
|
259
331
|
'Tip: disable animations (`verikun doctor --fix`) and ensure the screen is idle.', 3);
|
|
@@ -265,6 +337,38 @@ class AdbDriver {
|
|
|
265
337
|
}
|
|
266
338
|
return r.stdout;
|
|
267
339
|
}
|
|
340
|
+
/**
|
|
341
|
+
* `screencap` without `-p`: the framebuffer as-is, skipping the on-device PNG
|
|
342
|
+
* encode that dominates a capture (MEASURED on an SM-A415F: 2.50s with `-p`,
|
|
343
|
+
* 1.04s without — the bigger transfer is far cheaper than the deflate it avoids).
|
|
344
|
+
*
|
|
345
|
+
* Returns null rather than throwing on anything unexpected, so an OEM or Android
|
|
346
|
+
* version that lays the buffer out differently silently falls back to the PNG
|
|
347
|
+
* path. Getting a wrong-but-plausible image would be far worse than being slow.
|
|
348
|
+
*/
|
|
349
|
+
screenshotRaw() {
|
|
350
|
+
const r = (0, exec_1.runBinary)(ADB, this.withSerial(['exec-out', 'screencap']));
|
|
351
|
+
const buf = r.stdout;
|
|
352
|
+
if (buf.length < RAW_HEADER_SIZES[0])
|
|
353
|
+
return null;
|
|
354
|
+
const width = buf.readUInt32LE(0);
|
|
355
|
+
const height = buf.readUInt32LE(4);
|
|
356
|
+
const format = buf.readUInt32LE(8);
|
|
357
|
+
// Only RGBA_8888 — the one format every `screencap` we have seen emits, and the
|
|
358
|
+
// only one whose channel order we can assume without guessing.
|
|
359
|
+
if (format !== PIXEL_FORMAT_RGBA_8888)
|
|
360
|
+
return null;
|
|
361
|
+
if (width < 1 || height < 1)
|
|
362
|
+
return null;
|
|
363
|
+
const pixelBytes = width * height * 4;
|
|
364
|
+
// Android 9 added a colorspace word, so the header is 16 bytes on anything
|
|
365
|
+
// modern and 12 before that. Pick whichever the payload length agrees with
|
|
366
|
+
// rather than branching on an OS version we would have to go and ask for.
|
|
367
|
+
const header = RAW_HEADER_SIZES.find((size) => buf.length - size === pixelBytes);
|
|
368
|
+
if (header === undefined)
|
|
369
|
+
return null;
|
|
370
|
+
return { width, height, ch: 4, pixels: buf.subarray(header) };
|
|
371
|
+
}
|
|
268
372
|
screenSize() {
|
|
269
373
|
const out = this.shell(['wm', 'size']);
|
|
270
374
|
const lines = out.split('\n');
|
package/dist/errors.js
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
// 2 usage error or ambiguous selector (caller must refine)
|
|
7
7
|
// 3 environment error (adb/simctl missing, no/multiple devices, dump failed)
|
|
8
8
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
9
|
-
exports.AmbiguousSelectorError = exports.SelectorNotFoundError = exports.probeFailure = exports.envError = exports.notFound = exports.usageError = exports.CliError = void 0;
|
|
9
|
+
exports.AmbiguousSelectorError = exports.NoWindowError = exports.SelectorNotFoundError = exports.probeFailure = exports.envError = exports.notFound = exports.usageError = exports.CliError = void 0;
|
|
10
10
|
exports.isEnvError = isEnvError;
|
|
11
11
|
class CliError extends Error {
|
|
12
12
|
exitCode;
|
|
@@ -53,6 +53,28 @@ class SelectorNotFoundError extends CliError {
|
|
|
53
53
|
}
|
|
54
54
|
}
|
|
55
55
|
exports.SelectorNotFoundError = SelectorNotFoundError;
|
|
56
|
+
/**
|
|
57
|
+
* There is no window to read right now — the app was just force-stopped, or is mid-launch
|
|
58
|
+
* and has not drawn yet. `getRootInActiveWindow()` returns null and the platform says so.
|
|
59
|
+
*
|
|
60
|
+
* This is an OBSERVATION about the screen, not a broken machine, and the difference matters:
|
|
61
|
+
* it clears on its own within a second or two. Every caller that has a wait budget absorbs it
|
|
62
|
+
* and polls again; only a caller with no budget lets it surface (exit 3, unchanged).
|
|
63
|
+
*
|
|
64
|
+
* MEASURED, and this class exists because of it: `launch --clear` leaves a gap with no
|
|
65
|
+
* window, and the old code escalated that to a fatal environment error after three capture
|
|
66
|
+
* attempts. With the slow stock dump those three attempts spanned 7-14s and usually outlasted
|
|
67
|
+
* the gap by accident; once the companion made a read ~0.2s they were spent in under a second,
|
|
68
|
+
* and a `wait --timeout 120000` would abort at ~20s with 100 seconds of its budget unspent.
|
|
69
|
+
* The retry belongs to the caller that knows how long it is willing to wait.
|
|
70
|
+
*/
|
|
71
|
+
class NoWindowError extends CliError {
|
|
72
|
+
constructor(message) {
|
|
73
|
+
super(message, 3);
|
|
74
|
+
this.name = 'NoWindowError';
|
|
75
|
+
}
|
|
76
|
+
}
|
|
77
|
+
exports.NoWindowError = NoWindowError;
|
|
56
78
|
/** Selector matched >1 element. Exit 2. Carries the candidates so the agent runner
|
|
57
79
|
* can ask the model to disambiguate (a heal trigger) instead of aborting. */
|
|
58
80
|
class AmbiguousSelectorError extends CliError {
|
package/dist/image.js
CHANGED
|
@@ -12,9 +12,14 @@
|
|
|
12
12
|
// (palette, 16-bit, interlaced) is left untouched and reported via `reason`,
|
|
13
13
|
// so a screenshot is never corrupted, only (sometimes) not shrunk.
|
|
14
14
|
Object.defineProperty(exports, "__esModule", { value: true });
|
|
15
|
+
exports.pngFromRaw = pngFromRaw;
|
|
15
16
|
exports.downscalePng = downscalePng;
|
|
16
17
|
const node_zlib_1 = require("node:zlib");
|
|
17
18
|
const PNG_SIG = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
|
19
|
+
/** PNG color type for a channel count, mirroring `channelsFor`. */
|
|
20
|
+
function colorTypeFor(ch) {
|
|
21
|
+
return ch === 1 ? 0 : ch === 2 ? 4 : ch === 3 ? 2 : 6;
|
|
22
|
+
}
|
|
18
23
|
/** Channel count for a supported PNG color type, or 0 if unsupported (e.g. palette). */
|
|
19
24
|
function channelsFor(colorType) {
|
|
20
25
|
switch (colorType) {
|
|
@@ -142,6 +147,41 @@ function buildPng(w, h, bitDepth, colorType, idat) {
|
|
|
142
147
|
// compression, filter, interlace methods are all 0 (the only standard values)
|
|
143
148
|
return Buffer.concat([PNG_SIG, chunk('IHDR', ihdr), chunk('IDAT', idat), chunk('IEND', Buffer.alloc(0))]);
|
|
144
149
|
}
|
|
150
|
+
/**
|
|
151
|
+
* Encode raw pixels to a PNG, box-downscaling to `maxEdge` first when the image is
|
|
152
|
+
* bigger (`null` = encode at full size). The counterpart to `downscalePng` for a
|
|
153
|
+
* backend that can hand over pixels directly.
|
|
154
|
+
*
|
|
155
|
+
* Skipping the device-side PNG encode is the point, and it is worth much more than
|
|
156
|
+
* the compression: MEASURED on an SM-A415F, `screencap -p` costs 2.50s while raw
|
|
157
|
+
* `screencap` costs 1.04s — the phone spends ~1.4s deflating an image we then
|
|
158
|
+
* immediately re-encode smaller anyway. Moving that work to the host costs almost
|
|
159
|
+
* nothing (10MB of RGBA box-downscales in ~10ms and deflates in ~110ms), so both
|
|
160
|
+
* the default and `--full` paths come out well ahead.
|
|
161
|
+
*/
|
|
162
|
+
function pngFromRaw(img, maxEdge) {
|
|
163
|
+
const { width, height, ch, pixels } = img;
|
|
164
|
+
const colorType = colorTypeFor(ch);
|
|
165
|
+
const encode = (w, h, px) => buildPng(w, h, 8, colorType, (0, node_zlib_1.deflateSync)(applyNoneFilter(px, w, h, ch)));
|
|
166
|
+
const full = (reason) => ({
|
|
167
|
+
buf: encode(width, height, pixels),
|
|
168
|
+
width, height, scaled: false, origWidth: width, origHeight: height, reason,
|
|
169
|
+
});
|
|
170
|
+
if (maxEdge === null)
|
|
171
|
+
return full();
|
|
172
|
+
if (!(maxEdge >= 1))
|
|
173
|
+
return full('no target size');
|
|
174
|
+
if (Math.max(width, height) <= maxEdge)
|
|
175
|
+
return full('already within target');
|
|
176
|
+
const scale = maxEdge / Math.max(width, height);
|
|
177
|
+
const tw = Math.max(1, Math.round(width * scale));
|
|
178
|
+
const th = Math.max(1, Math.round(height * scale));
|
|
179
|
+
const small = boxDownscale(pixels, width, height, tw, th, ch);
|
|
180
|
+
return {
|
|
181
|
+
buf: encode(tw, th, small),
|
|
182
|
+
width: tw, height: th, scaled: true, origWidth: width, origHeight: height,
|
|
183
|
+
};
|
|
184
|
+
}
|
|
145
185
|
/**
|
|
146
186
|
* Downscale a PNG so its longest edge is at most `maxEdge` px (never upscales).
|
|
147
187
|
* Returns the original buffer unchanged when it is already small enough or is in
|
package/dist/run.js
CHANGED
|
@@ -16,6 +16,7 @@ const format_1 = require("./ui/format");
|
|
|
16
16
|
const errors_1 = require("./errors");
|
|
17
17
|
const output_1 = require("./output");
|
|
18
18
|
const report_1 = require("./report");
|
|
19
|
+
const capture_1 = require("./capture");
|
|
19
20
|
// Commands that become a recorded step (a JUnit testcase). Inspection commands
|
|
20
21
|
// (ui, find, devices, doctor, current) are deliberately excluded — they are how
|
|
21
22
|
// an agent decides what to do, not assertions about the app. `log` is the one
|
|
@@ -477,7 +478,9 @@ class Recorder {
|
|
|
477
478
|
if (!driver)
|
|
478
479
|
return;
|
|
479
480
|
try {
|
|
480
|
-
this
|
|
481
|
+
// Full resolution on purpose — a human reads this in the report — but via the
|
|
482
|
+
// raw path, which reaches the same PNG without the device-side encode.
|
|
483
|
+
this.writeArtifact(failImagePath(this.step.index), (0, capture_1.capturePng)(driver, null).buf);
|
|
481
484
|
this.step.failImage = failImagePath(this.step.index);
|
|
482
485
|
}
|
|
483
486
|
catch (e) {
|
package/dist/version.js
CHANGED
|
@@ -3,4 +3,4 @@ Object.defineProperty(exports, "__esModule", { value: true });
|
|
|
3
3
|
exports.VERSION = void 0;
|
|
4
4
|
// GENERATED by scripts/gen-version.mjs from package.json's "version" at build time
|
|
5
5
|
// (the `prebuild` script). Do NOT edit by hand; bump package.json instead.
|
|
6
|
-
exports.VERSION = '0.
|
|
6
|
+
exports.VERSION = '0.21.0';
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "verikun",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.21.0",
|
|
4
4
|
"description": "Drive Android emulators/devices and iOS simulators for AI agents: tap, type, swipe, screenshot, and inspect the UI hierarchy by semantic identifiers — like Puppeteer for native apps.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"android",
|
|
@@ -30,7 +30,8 @@
|
|
|
30
30
|
"dist",
|
|
31
31
|
"CHANGELOG.md",
|
|
32
32
|
".claude/skills/verikun/SKILL.md",
|
|
33
|
-
"example/*.md"
|
|
33
|
+
"example/*.md",
|
|
34
|
+
"tools/verikun-companion/prebuilt/*.jar"
|
|
34
35
|
],
|
|
35
36
|
"scripts": {
|
|
36
37
|
"prebuild": "node scripts/gen-version.mjs",
|
|
Binary file
|