@petukhovart/agent-view 0.15.0 → 0.16.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/README.md +30 -9
- package/dist/cdp/png.d.ts +15 -0
- package/dist/cdp/png.d.ts.map +1 -0
- package/dist/cdp/png.js +147 -0
- package/dist/cdp/png.js.map +1 -0
- package/dist/cdp/transport.d.ts.map +1 -1
- package/dist/cdp/transport.js +67 -43
- package/dist/cdp/transport.js.map +1 -1
- package/dist/cdp/types.d.ts +28 -5
- package/dist/cdp/types.d.ts.map +1 -1
- package/dist/cdp/types.js.map +1 -1
- package/dist/cli/commands/drag.d.ts +4 -0
- package/dist/cli/commands/drag.d.ts.map +1 -1
- package/dist/cli/commands/drag.js +18 -0
- package/dist/cli/commands/drag.js.map +1 -1
- package/dist/cli/index.js +5 -1
- package/dist/cli/index.js.map +1 -1
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +13 -7
- package/dist/server/server.js.map +1 -1
- package/package.json +1 -1
- package/skills/verify/references/commands.md +9 -2
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "agent-view",
|
|
3
3
|
"description": "Visual verification CLI for Electron/Tauri/Chromium apps via Chrome DevTools Protocol. Ships with verify skill for easy and cheap checks against a live app, including heap-snapshot memory-leak debugging.",
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.16.1",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"cdp",
|
|
7
7
|
"electron",
|
package/README.md
CHANGED
|
@@ -34,7 +34,7 @@ Works with any agent that can run shell commands. There's a Claude Code plugin i
|
|
|
34
34
|
- Electron, Tauri on Windows (WebView2), and plain Chromium. One CLI, same commands. Tauri on macOS/Linux embeds WebKit, which has no Chrome DevTools Protocol, so it is out of reach.
|
|
35
35
|
- `click`, `fill`, and `drag` fire real CDP input events. Vue `v-model`, React controlled inputs, and native fields actually accept the value; synthetic DOM events fail silently there.
|
|
36
36
|
- `watch` emits RFC-6902 JSON-patches of any JS expression between two events. Answers "what mutated after the click?" without parsing screenshots.
|
|
37
|
-
- `dom` returns the accessibility tree with `[ref=N]` handles. `--compact` cuts deep trees by 40–60%; `--diff`, `--count`, and `--max-lines` keep output bounded. `screenshot --crop` and
|
|
37
|
+
- `dom` returns the accessibility tree with `[ref=N]` handles. `--compact` cuts deep trees by 40–60%; `--diff`, `--count`, and `--max-lines` keep output bounded. `screenshot --crop` and `--scale` do the same for vision tokens.
|
|
38
38
|
- Lazy CDP daemon, one persistent socket, 300 ms AX-tree cache. `dom → click → dom` in about 17 ms.
|
|
39
39
|
|
|
40
40
|
---
|
|
@@ -147,8 +147,8 @@ The daemon is why `dom → click → dom` runs in ~17ms total: one persistent CD
|
|
|
147
147
|
| Command | What it gives the agent |
|
|
148
148
|
|---------------|-----------------------------------------------------------------------------------------|
|
|
149
149
|
| `dom` | Accessibility tree with `[ref=N]` handles. Flags: `--filter`, `--compact`, `--count`, `--max-lines`, `--diff`. |
|
|
150
|
-
| `screenshot` | PNG,
|
|
151
|
-
| `click` / `fill` / `drag` | Real CDP input events. Works with Vue/React/native and
|
|
150
|
+
| `screenshot` | PNG, scaled with `--scale`, or `--crop <element>` for one element only. Cuts vision tokens. |
|
|
151
|
+
| `click` / `fill` / `drag` | Real CDP input events. Works with Vue/React/native; `drag` does HTML5 DnD through `Input.dragIntercepted` and pointer DnD through mouse events. |
|
|
152
152
|
| `eval` | Run JS in the page's main world. Read store/state directly instead of scraping DOM. |
|
|
153
153
|
| `watch` | Stream JSON-patch diffs of any expression. Answers "what changed between click and final state?". |
|
|
154
154
|
| `console` | `console.log` + `Log.entryAdded` per page **and per worker**, with `--follow --until <pattern>`. |
|
|
@@ -391,16 +391,37 @@ agent-view fill 3 "hello@example.com"
|
|
|
391
391
|
|
|
392
392
|
### `drag`
|
|
393
393
|
|
|
394
|
-
|
|
394
|
+
Drag-and-drop over CDP. Two paths, picked automatically:
|
|
395
|
+
|
|
396
|
+
- **HTML5** (`draggable=true`, `dragstart`/`dragover`/`drop`). Plain mouse events never start one in Chromium: `dragstart` does not fire, `dataTransfer` stays empty, `drop` never arrives. So `drag` enables `Input.setInterceptDrags`, presses and moves, and when Chromium reports `Input.dragIntercepted` it finishes the gesture with `Input.dispatchDragEvent` (`dragEnter` → N × `dragOver` → `drop`), carrying the app's real `dataTransfer` payload. The same mechanism Puppeteer uses.
|
|
397
|
+
- **Pointer** (`mousedown`/`mousemove`/`mouseup`, pointer events). Used when no `dragIntercepted` arrives within 1 s: `vue-draggable-resizable`, `react-grid-layout`, gridstack, canvas, resize handles.
|
|
395
398
|
|
|
396
399
|
```bash
|
|
397
400
|
agent-view drag --from 42 --to 88 # ref → ref
|
|
398
401
|
agent-view drag --from-pos 86,792 --to-pos 640,200 # coord → coord (canvas, custom DnD)
|
|
399
402
|
agent-view drag --from 42 --to-pos 640,200 # mixed
|
|
400
403
|
agent-view drag --from 5 --to 9 --steps 20 --hold-ms 150
|
|
404
|
+
agent-view drag --from-pos 385,303 --to-pos 1250,589 --cancel # HTML5: dragCancel instead of drop (Esc mid-drag)
|
|
405
|
+
agent-view drag --from-pos 385,303 --to-pos 1250,589 --html5 # fail instead of falling back to pointer
|
|
406
|
+
agent-view drag --from-pos 385,303 --to-pos 1250,589 --pointer # never intercept
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
The output names the path and, for HTML5, the intercepted MIME types and data — check that the payload is the app's (`application/json`, a custom type), not a text selection:
|
|
410
|
+
|
|
411
|
+
```
|
|
412
|
+
Dragged (385, 303) → (1250, 589) via html5 drop (dragOperationsMask=3)
|
|
413
|
+
text/plain: "1:hr:4100@UInt16"
|
|
414
|
+
application/json: "[{\"id\":\"1:hr:4100@UInt16\",\"name\":\"Float HiLo\",...}]"
|
|
401
415
|
```
|
|
402
416
|
|
|
403
|
-
|
|
417
|
+
```
|
|
418
|
+
Dragged (485, 140) → (1250, 589) via pointer
|
|
419
|
+
warning: no HTML5 drag started (Input.dragIntercepted not fired within 1 s): the element at --from is not draggable, or a control (input/select/button) under the cursor swallowed dragstart
|
|
420
|
+
```
|
|
421
|
+
|
|
422
|
+
Pitfalls of the HTML5 path: start on a cell without a control — an `<input>`, `<select>` or `<button>` under the cursor swallows `dragstart` even inside a `draggable` row; Chromium needs several `mouseMoved` past its drag threshold, so `--steps 0` will not start one. `--mask <n>` overrides `dragOperationsMask` on the dispatched events — diagnostics for `effectAllowed` / `dropEffect` mismatches, which the browser enforces only on a real drag.
|
|
423
|
+
|
|
424
|
+
`--steps` (default 10) controls intermediate `mouseMoved` / `dragOver` events so libraries that throttle on movement deltas still see continuous motion. `--hold-ms` inserts a pause between press and the first move (some libs require >100ms for touch-style activation). `--button` accepts `left|right|middle`.
|
|
404
425
|
|
|
405
426
|
### `upload`
|
|
406
427
|
|
|
@@ -442,18 +463,18 @@ Out of reach: `showOpenFilePicker()` (File System Access API) exposes no input t
|
|
|
442
463
|
|
|
443
464
|
### `screenshot`
|
|
444
465
|
|
|
445
|
-
Captures a screenshot, saves to temp dir, prints the file path. PNG
|
|
466
|
+
Captures a screenshot, saves to temp dir, prints the file path. Always PNG.
|
|
446
467
|
|
|
447
468
|
```bash
|
|
448
469
|
agent-view screenshot
|
|
449
|
-
agent-view screenshot --scale 0.5 # Half-res
|
|
450
|
-
agent-view screenshot --scale 0.25 # Quarter-res
|
|
470
|
+
agent-view screenshot --scale 0.5 # Half-res (~3× fewer vision tokens)
|
|
471
|
+
agent-view screenshot --scale 0.25 # Quarter-res (~12× fewer, 1 tile)
|
|
451
472
|
agent-view screenshot --crop "Sidebar" # Crop to element bounding box (~12× fewer in best case)
|
|
452
473
|
agent-view screenshot --crop "Chart" --scale 0.5 # Crop + scale (stacks)
|
|
453
474
|
agent-view screenshot --crop "Active bookings" --crop-up 1 # Crop the card, not its heading
|
|
454
475
|
```
|
|
455
476
|
|
|
456
|
-
`--scale` accepts a factor in `(0, 1]
|
|
477
|
+
`--scale` accepts a factor in `(0, 1]`; recommended for agent loops where vision tokens dominate cost. Crop and scale run inside the server on a plain full-viewport capture: a CDP `clip` resizes the app's render view until the capture completes, and on a hidden or minimised window it never completes, leaving the app drawn in a corner.
|
|
457
478
|
|
|
458
479
|
`--crop <filter>` resolves a DOM element by the same filter syntax as `dom --filter`, then crops the screenshot to its bounding box before encoding. One tile (~1.6k vision tokens) instead of twelve (~19k) in the best case. If the filter matches nothing a warning is emitted to stderr and the full window is captured instead. Combines naturally with `--scale`.
|
|
459
480
|
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
export type PixelRect = {
|
|
2
|
+
x: number;
|
|
3
|
+
y: number;
|
|
4
|
+
width: number;
|
|
5
|
+
height: number;
|
|
6
|
+
};
|
|
7
|
+
/**
|
|
8
|
+
* Crops an 8-bit RGB/RGBA PNG to `rect` (image pixels, clamped to the image)
|
|
9
|
+
* and downscales it by `scale` with an area average. Chromium can do both
|
|
10
|
+
* through `Page.captureScreenshot({ clip })`, but a clip resizes the render
|
|
11
|
+
* view for the capture and restores it only if the capture completes — a
|
|
12
|
+
* hidden window never completes it, so the app stays shrunk.
|
|
13
|
+
*/
|
|
14
|
+
export declare function cropScalePng(png: Buffer, rect: PixelRect | undefined, scale: number): Buffer;
|
|
15
|
+
//# sourceMappingURL=png.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"png.d.ts","sourceRoot":"","sources":["../../src/cdp/png.ts"],"names":[],"mappings":"AAEA,MAAM,MAAM,SAAS,GAAG;IAAE,CAAC,EAAE,MAAM,CAAC;IAAC,CAAC,EAAE,MAAM,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,MAAM,EAAE,MAAM,CAAA;CAAE,CAAA;AAM/E;;;;;;GAMG;AACH,wBAAgB,YAAY,CAAC,GAAG,EAAE,MAAM,EAAE,IAAI,EAAE,SAAS,GAAG,SAAS,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM,CA6B5F"}
|
package/dist/cdp/png.js
ADDED
|
@@ -0,0 +1,147 @@
|
|
|
1
|
+
import { deflateSync, inflateSync } from 'node:zlib';
|
|
2
|
+
const SIGNATURE = Buffer.from([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a]);
|
|
3
|
+
/**
|
|
4
|
+
* Crops an 8-bit RGB/RGBA PNG to `rect` (image pixels, clamped to the image)
|
|
5
|
+
* and downscales it by `scale` with an area average. Chromium can do both
|
|
6
|
+
* through `Page.captureScreenshot({ clip })`, but a clip resizes the render
|
|
7
|
+
* view for the capture and restores it only if the capture completes — a
|
|
8
|
+
* hidden window never completes it, so the app stays shrunk.
|
|
9
|
+
*/
|
|
10
|
+
export function cropScalePng(png, rect, scale) {
|
|
11
|
+
const src = decode(png);
|
|
12
|
+
const x0 = clamp(Math.round(rect?.x ?? 0), 0, src.width - 1);
|
|
13
|
+
const y0 = clamp(Math.round(rect?.y ?? 0), 0, src.height - 1);
|
|
14
|
+
const x1 = clamp(Math.round((rect?.x ?? 0) + (rect?.width ?? src.width)), x0 + 1, src.width);
|
|
15
|
+
const y1 = clamp(Math.round((rect?.y ?? 0) + (rect?.height ?? src.height)), y0 + 1, src.height);
|
|
16
|
+
const width = Math.max(1, Math.round((x1 - x0) * scale));
|
|
17
|
+
const height = Math.max(1, Math.round((y1 - y0) * scale));
|
|
18
|
+
const rgba = Buffer.alloc(width * height * 4);
|
|
19
|
+
for (let y = 0; y < height; y++) {
|
|
20
|
+
const sy0 = y0 + Math.floor((y * (y1 - y0)) / height);
|
|
21
|
+
const sy1 = Math.max(sy0 + 1, y0 + Math.floor(((y + 1) * (y1 - y0)) / height));
|
|
22
|
+
for (let x = 0; x < width; x++) {
|
|
23
|
+
const sx0 = x0 + Math.floor((x * (x1 - x0)) / width);
|
|
24
|
+
const sx1 = Math.max(sx0 + 1, x0 + Math.floor(((x + 1) * (x1 - x0)) / width));
|
|
25
|
+
const sum = [0, 0, 0, 0];
|
|
26
|
+
for (let sy = sy0; sy < sy1; sy++) {
|
|
27
|
+
for (let sx = sx0; sx < sx1; sx++) {
|
|
28
|
+
const i = (sy * src.width + sx) * 4;
|
|
29
|
+
for (let c = 0; c < 4; c++)
|
|
30
|
+
sum[c] += src.rgba[i + c];
|
|
31
|
+
}
|
|
32
|
+
}
|
|
33
|
+
const n = (sy1 - sy0) * (sx1 - sx0);
|
|
34
|
+
const o = (y * width + x) * 4;
|
|
35
|
+
for (let c = 0; c < 4; c++)
|
|
36
|
+
rgba[o + c] = Math.round(sum[c] / n);
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
return encode({ width, height, rgba });
|
|
40
|
+
}
|
|
41
|
+
function clamp(v, min, max) {
|
|
42
|
+
return Math.min(Math.max(v, min), max);
|
|
43
|
+
}
|
|
44
|
+
function decode(png) {
|
|
45
|
+
if (!png.subarray(0, 8).equals(SIGNATURE))
|
|
46
|
+
throw new Error('Not a PNG');
|
|
47
|
+
let width = 0;
|
|
48
|
+
let height = 0;
|
|
49
|
+
let channels = 0;
|
|
50
|
+
const idat = [];
|
|
51
|
+
for (let off = 8; off < png.length;) {
|
|
52
|
+
const len = png.readUInt32BE(off);
|
|
53
|
+
const type = png.toString('latin1', off + 4, off + 8);
|
|
54
|
+
const data = png.subarray(off + 8, off + 8 + len);
|
|
55
|
+
if (type === 'IHDR') {
|
|
56
|
+
width = data.readUInt32BE(0);
|
|
57
|
+
height = data.readUInt32BE(4);
|
|
58
|
+
const colorType = data[9];
|
|
59
|
+
if (data[8] !== 8 || (colorType !== 2 && colorType !== 6) || data[12] !== 0) {
|
|
60
|
+
throw new Error(`Unsupported PNG: bit depth ${data[8]}, color type ${colorType}, interlace ${data[12]}`);
|
|
61
|
+
}
|
|
62
|
+
channels = colorType === 6 ? 4 : 3;
|
|
63
|
+
}
|
|
64
|
+
else if (type === 'IDAT') {
|
|
65
|
+
idat.push(data);
|
|
66
|
+
}
|
|
67
|
+
else if (type === 'IEND') {
|
|
68
|
+
break;
|
|
69
|
+
}
|
|
70
|
+
off += 12 + len;
|
|
71
|
+
}
|
|
72
|
+
const raw = inflateSync(Buffer.concat(idat));
|
|
73
|
+
const stride = width * channels;
|
|
74
|
+
const pixels = Buffer.alloc(stride * height);
|
|
75
|
+
for (let y = 0; y < height; y++) {
|
|
76
|
+
const filter = raw[y * (stride + 1)];
|
|
77
|
+
const line = raw.subarray(y * (stride + 1) + 1, (y + 1) * (stride + 1));
|
|
78
|
+
const row = y * stride;
|
|
79
|
+
for (let i = 0; i < stride; i++) {
|
|
80
|
+
const a = i >= channels ? pixels[row + i - channels] : 0;
|
|
81
|
+
const b = y > 0 ? pixels[row - stride + i] : 0;
|
|
82
|
+
const c = i >= channels && y > 0 ? pixels[row - stride + i - channels] : 0;
|
|
83
|
+
pixels[row + i] = (line[i] + unfilter(filter, a, b, c)) & 0xff;
|
|
84
|
+
}
|
|
85
|
+
}
|
|
86
|
+
if (channels === 4)
|
|
87
|
+
return { width, height, rgba: pixels };
|
|
88
|
+
const rgba = Buffer.alloc(width * height * 4, 0xff);
|
|
89
|
+
for (let p = 0; p < width * height; p++)
|
|
90
|
+
pixels.copy(rgba, p * 4, p * 3, p * 3 + 3);
|
|
91
|
+
return { width, height, rgba };
|
|
92
|
+
}
|
|
93
|
+
function unfilter(filter, a, b, c) {
|
|
94
|
+
switch (filter) {
|
|
95
|
+
case 0: return 0;
|
|
96
|
+
case 1: return a;
|
|
97
|
+
case 2: return b;
|
|
98
|
+
case 3: return (a + b) >> 1;
|
|
99
|
+
case 4: {
|
|
100
|
+
const p = a + b - c;
|
|
101
|
+
const pa = Math.abs(p - a);
|
|
102
|
+
const pb = Math.abs(p - b);
|
|
103
|
+
const pc = Math.abs(p - c);
|
|
104
|
+
return pa <= pb && pa <= pc ? a : pb <= pc ? b : c;
|
|
105
|
+
}
|
|
106
|
+
default: throw new Error(`Unknown PNG filter ${filter}`);
|
|
107
|
+
}
|
|
108
|
+
}
|
|
109
|
+
function encode({ width, height, rgba }) {
|
|
110
|
+
const stride = width * 4;
|
|
111
|
+
const raw = Buffer.alloc((stride + 1) * height);
|
|
112
|
+
for (let y = 0; y < height; y++) {
|
|
113
|
+
// Filter 1 (Sub) — flat UI areas become runs of zeros and deflate well.
|
|
114
|
+
raw[y * (stride + 1)] = 1;
|
|
115
|
+
for (let i = 0; i < stride; i++) {
|
|
116
|
+
const left = i >= 4 ? rgba[y * stride + i - 4] : 0;
|
|
117
|
+
raw[y * (stride + 1) + 1 + i] = (rgba[y * stride + i] - left) & 0xff;
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
const ihdr = Buffer.alloc(13);
|
|
121
|
+
ihdr.writeUInt32BE(width, 0);
|
|
122
|
+
ihdr.writeUInt32BE(height, 4);
|
|
123
|
+
ihdr[8] = 8;
|
|
124
|
+
ihdr[9] = 6;
|
|
125
|
+
return Buffer.concat([SIGNATURE, chunk('IHDR', ihdr), chunk('IDAT', deflateSync(raw)), chunk('IEND', Buffer.alloc(0))]);
|
|
126
|
+
}
|
|
127
|
+
function chunk(type, data) {
|
|
128
|
+
const head = Buffer.alloc(8);
|
|
129
|
+
head.writeUInt32BE(data.length, 0);
|
|
130
|
+
head.write(type, 4, 'latin1');
|
|
131
|
+
const crc = Buffer.alloc(4);
|
|
132
|
+
crc.writeUInt32BE(crc32(Buffer.concat([head.subarray(4), data])), 0);
|
|
133
|
+
return Buffer.concat([head, data, crc]);
|
|
134
|
+
}
|
|
135
|
+
const CRC_TABLE = Array.from({ length: 256 }, (_, n) => {
|
|
136
|
+
let c = n;
|
|
137
|
+
for (let k = 0; k < 8; k++)
|
|
138
|
+
c = c & 1 ? 0xedb88320 ^ (c >>> 1) : c >>> 1;
|
|
139
|
+
return c >>> 0;
|
|
140
|
+
});
|
|
141
|
+
function crc32(buf) {
|
|
142
|
+
let c = 0xffffffff;
|
|
143
|
+
for (const byte of buf)
|
|
144
|
+
c = CRC_TABLE[(c ^ byte) & 0xff] ^ (c >>> 8);
|
|
145
|
+
return (c ^ 0xffffffff) >>> 0;
|
|
146
|
+
}
|
|
147
|
+
//# sourceMappingURL=png.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"png.js","sourceRoot":"","sources":["../../src/cdp/png.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,WAAW,EAAE,MAAM,WAAW,CAAA;AAMpD,MAAM,SAAS,GAAG,MAAM,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC,CAAA;AAE/E;;;;;;GAMG;AACH,MAAM,UAAU,YAAY,CAAC,GAAW,EAAE,IAA2B,EAAE,KAAa;IAClF,MAAM,GAAG,GAAG,MAAM,CAAC,GAAG,CAAC,CAAA;IACvB,MAAM,EAAE,GAAG,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC,EAAE,GAAG,CAAC,KAAK,GAAG,CAAC,CAAC,CAAA;IAC5D,MAAM,EAAE,GAAG,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC,EAAE,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,CAAA;IAC7D,MAAM,EAAE,GAAG,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,EAAE,GAAG,CAAC,KAAK,CAAC,CAAA;IAC5F,MAAM,EAAE,GAAG,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,IAAI,EAAE,MAAM,IAAI,GAAG,CAAC,MAAM,CAAC,CAAC,EAAE,EAAE,GAAG,CAAC,EAAE,GAAG,CAAC,MAAM,CAAC,CAAA;IAC/F,MAAM,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,EAAE,CAAC,GAAG,KAAK,CAAC,CAAC,CAAA;IACxD,MAAM,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,EAAE,CAAC,GAAG,KAAK,CAAC,CAAC,CAAA;IACzD,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,KAAK,GAAG,MAAM,GAAG,CAAC,CAAC,CAAA;IAE7C,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QAChC,MAAM,GAAG,GAAG,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,MAAM,CAAC,CAAA;QACrD,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,GAAG,CAAC,EAAE,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,CAAA;QAC9E,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,EAAE,CAAC,EAAE,EAAE,CAAC;YAC/B,MAAM,GAAG,GAAG,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,KAAK,CAAC,CAAA;YACpD,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,GAAG,CAAC,EAAE,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,KAAK,CAAC,CAAC,CAAA;YAC7E,MAAM,GAAG,GAAG,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAA;YACxB,KAAK,IAAI,EAAE,GAAG,GAAG,EAAE,EAAE,GAAG,GAAG,EAAE,EAAE,EAAE,EAAE,CAAC;gBAClC,KAAK,IAAI,EAAE,GAAG,GAAG,EAAE,EAAE,GAAG,GAAG,EAAE,EAAE,EAAE,EAAE,CAAC;oBAClC,MAAM,CAAC,GAAG,CAAC,EAAE,GAAG,GAAG,CAAC,KAAK,GAAG,EAAE,CAAC,GAAG,CAAC,CAAA;oBACnC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE;wBAAE,GAAG,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAA;gBACvD,CAAC;YACH,CAAC;YACD,MAAM,CAAC,GAAG,CAAC,GAAG,GAAG,GAAG,CAAC,GAAG,CAAC,GAAG,GAAG,GAAG,CAAC,CAAA;YACnC,MAAM,CAAC,GAAG,CAAC,CAAC,GAAG,KAAK,GAAG,CAAC,CAAC,GAAG,CAAC,CAAA;YAC7B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE;gBAAE,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAA;QAClE,CAAC;IACH,CAAC;IACD,OAAO,MAAM,CAAC,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAA;AACxC,CAAC;AAED,SAAS,KAAK,CAAC,CAAS,EAAE,GAAW,EAAE,GAAW;IAChD,OAAO,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,GAAG,CAAC,CAAA;AACxC,CAAC;AAED,SAAS,MAAM,CAAC,GAAW;IACzB,IAAI,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,SAAS,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,WAAW,CAAC,CAAA;IACvE,IAAI,KAAK,GAAG,CAAC,CAAA;IACb,IAAI,MAAM,GAAG,CAAC,CAAA;IACd,IAAI,QAAQ,GAAG,CAAC,CAAA;IAChB,MAAM,IAAI,GAAa,EAAE,CAAA;IACzB,KAAK,IAAI,GAAG,GAAG,CAAC,EAAE,GAAG,GAAG,GAAG,CAAC,MAAM,GAAG,CAAC;QACpC,MAAM,GAAG,GAAG,GAAG,CAAC,YAAY,CAAC,GAAG,CAAC,CAAA;QACjC,MAAM,IAAI,GAAG,GAAG,CAAC,QAAQ,CAAC,QAAQ,EAAE,GAAG,GAAG,CAAC,EAAE,GAAG,GAAG,CAAC,CAAC,CAAA;QACrD,MAAM,IAAI,GAAG,GAAG,CAAC,QAAQ,CAAC,GAAG,GAAG,CAAC,EAAE,GAAG,GAAG,CAAC,GAAG,GAAG,CAAC,CAAA;QACjD,IAAI,IAAI,KAAK,MAAM,EAAE,CAAC;YACpB,KAAK,GAAG,IAAI,CAAC,YAAY,CAAC,CAAC,CAAC,CAAA;YAC5B,MAAM,GAAG,IAAI,CAAC,YAAY,CAAC,CAAC,CAAC,CAAA;YAC7B,MAAM,SAAS,GAAG,IAAI,CAAC,CAAC,CAAC,CAAA;YACzB,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,SAAS,KAAK,CAAC,IAAI,SAAS,KAAK,CAAC,CAAC,IAAI,IAAI,CAAC,EAAE,CAAC,KAAK,CAAC,EAAE,CAAC;gBAC5E,MAAM,IAAI,KAAK,CAAC,8BAA8B,IAAI,CAAC,CAAC,CAAC,gBAAgB,SAAS,eAAe,IAAI,CAAC,EAAE,CAAC,EAAE,CAAC,CAAA;YAC1G,CAAC;YACD,QAAQ,GAAG,SAAS,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;QACpC,CAAC;aAAM,IAAI,IAAI,KAAK,MAAM,EAAE,CAAC;YAC3B,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;QACjB,CAAC;aAAM,IAAI,IAAI,KAAK,MAAM,EAAE,CAAC;YAC3B,MAAK;QACP,CAAC;QACD,GAAG,IAAI,EAAE,GAAG,GAAG,CAAA;IACjB,CAAC;IAED,MAAM,GAAG,GAAG,WAAW,CAAC,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAA;IAC5C,MAAM,MAAM,GAAG,KAAK,GAAG,QAAQ,CAAA;IAC/B,MAAM,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,MAAM,GAAG,MAAM,CAAC,CAAA;IAC5C,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QAChC,MAAM,MAAM,GAAG,GAAG,CAAC,CAAC,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAA;QACpC,MAAM,IAAI,GAAG,GAAG,CAAC,QAAQ,CAAC,CAAC,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAA;QACvE,MAAM,GAAG,GAAG,CAAC,GAAG,MAAM,CAAA;QACtB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YAChC,MAAM,CAAC,GAAG,CAAC,IAAI,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,GAAG,CAAC,GAAG,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;YACxD,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,GAAG,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;YAC9C,MAAM,CAAC,GAAG,CAAC,IAAI,QAAQ,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,GAAG,GAAG,MAAM,GAAG,CAAC,GAAG,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;YAC1E,MAAM,CAAC,GAAG,GAAG,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,QAAQ,CAAC,MAAM,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC,GAAG,IAAI,CAAA;QAChE,CAAC;IACH,CAAC;IAED,IAAI,QAAQ,KAAK,CAAC;QAAE,OAAO,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,EAAE,CAAA;IAC1D,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,KAAK,GAAG,MAAM,GAAG,CAAC,EAAE,IAAI,CAAC,CAAA;IACnD,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,KAAK,GAAG,MAAM,EAAE,CAAC,EAAE;QAAE,MAAM,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAA;IACnF,OAAO,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,CAAA;AAChC,CAAC;AAED,SAAS,QAAQ,CAAC,MAAc,EAAE,CAAS,EAAE,CAAS,EAAE,CAAS;IAC/D,QAAQ,MAAM,EAAE,CAAC;QACf,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,CAAA;QAChB,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,CAAA;QAChB,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,CAAA;QAChB,KAAK,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,CAAA;QAC3B,KAAK,CAAC,CAAC,CAAC,CAAC;YACP,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,GAAG,CAAC,CAAA;YACnB,MAAM,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAA;YAC1B,MAAM,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAA;YAC1B,MAAM,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC,CAAA;YAC1B,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;QACpD,CAAC;QACD,OAAO,CAAC,CAAC,MAAM,IAAI,KAAK,CAAC,sBAAsB,MAAM,EAAE,CAAC,CAAA;IAC1D,CAAC;AACH,CAAC;AAED,SAAS,MAAM,CAAC,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAS;IAC5C,MAAM,MAAM,GAAG,KAAK,GAAG,CAAC,CAAA;IACxB,MAAM,GAAG,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAA;IAC/C,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QAChC,wEAAwE;QACxE,GAAG,CAAC,CAAC,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAA;QACzB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YAChC,MAAM,IAAI,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,GAAG,MAAM,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;YAClD,GAAG,CAAC,CAAC,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,GAAG,MAAM,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,GAAG,IAAI,CAAA;QACtE,CAAC;IACH,CAAC;IACD,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC,CAAA;IAC7B,IAAI,CAAC,aAAa,CAAC,KAAK,EAAE,CAAC,CAAC,CAAA;IAC5B,IAAI,CAAC,aAAa,CAAC,MAAM,EAAE,CAAC,CAAC,CAAA;IAC7B,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,CAAA;IACX,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,CAAA;IACX,OAAO,MAAM,CAAC,MAAM,CAAC,CAAC,SAAS,EAAE,KAAK,CAAC,MAAM,EAAE,IAAI,CAAC,EAAE,KAAK,CAAC,MAAM,EAAE,WAAW,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,CAAC,MAAM,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAA;AACzH,CAAC;AAED,SAAS,KAAK,CAAC,IAAY,EAAE,IAAY;IACvC,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;IAC5B,IAAI,CAAC,aAAa,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC,CAAC,CAAA;IAClC,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,EAAE,QAAQ,CAAC,CAAA;IAC7B,MAAM,GAAG,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAA;IAC3B,GAAG,CAAC,aAAa,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAA;IACpE,OAAO,MAAM,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,IAAI,EAAE,GAAG,CAAC,CAAC,CAAA;AACzC,CAAC;AAED,MAAM,SAAS,GAAG,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,GAAG,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;IACrD,IAAI,CAAC,GAAG,CAAC,CAAA;IACT,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,EAAE,CAAC,EAAE;QAAE,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,UAAU,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAA;IACxE,OAAO,CAAC,KAAK,CAAC,CAAA;AAChB,CAAC,CAAC,CAAA;AAEF,SAAS,KAAK,CAAC,GAAW;IACxB,IAAI,CAAC,GAAG,UAAU,CAAA;IAClB,KAAK,MAAM,IAAI,IAAI,GAAG;QAAE,CAAC,GAAG,SAAS,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAA;IACpE,OAAO,CAAC,CAAC,GAAG,UAAU,CAAC,KAAK,CAAC,CAAA;AAC/B,CAAC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"transport.d.ts","sourceRoot":"","sources":["../../src/cdp/transport.ts"],"names":[],"mappings":"AAEA,OAAO,EAaL,KAAK,SAAS,
|
|
1
|
+
{"version":3,"file":"transport.d.ts","sourceRoot":"","sources":["../../src/cdp/transport.ts"],"names":[],"mappings":"AAEA,OAAO,EAaL,KAAK,SAAS,EAKd,KAAK,UAAU,EACf,KAAK,cAAc,EACnB,KAAK,WAAW,EAgBjB,MAAM,YAAY,CAAA;AACnB,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,eAAe,CAAA;AA6BhD,qBAAa,eAAgB,SAAQ,KAAK;gBAC5B,OAAO,EAAE,MAAM;CAI5B;AAmCD,wBAAsB,WAAW,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,SAAS,EAAE,CAAC,CAkBpE;AAED,wBAAsB,oBAAoB,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,UAAU,EAAE,CAAC,CAQ9E;AAgzBD,wBAAsB,gBAAgB,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,UAAU,GAAG,OAAO,CAAC,cAAc,CAAC,CA+BhG;AAMD,wBAAsB,aAAa,CACjC,IAAI,EAAE,MAAM,EACZ,MAAM,EAAE,UAAU,EAClB,KAAK,EAAE,WAAW,GACjB,OAAO,CAAC,WAAW,CAAC,CAoWtB"}
|
package/dist/cdp/transport.js
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
// @ts-expect-error no types available for chrome-remote-interface
|
|
2
2
|
import CDP from 'chrome-remote-interface';
|
|
3
3
|
import { TargetType, ConsoleLevel, ConsoleSource, EvaluationError, MouseButton, NetworkEventKind, NetworkResourceType, JsDialogType, FileChooserMode, } from './types.js';
|
|
4
|
+
import { cropScalePng } from './png.js';
|
|
4
5
|
// CDP hosts to try: IPv4 first, then IPv6 (WebView2/Tauri often listens on ::1)
|
|
5
6
|
const CDP_HOSTS = ['127.0.0.1', '::1'];
|
|
6
7
|
// Maps targetId → host for connection routing
|
|
@@ -752,6 +753,8 @@ export async function connectToRuntime(port, target) {
|
|
|
752
753
|
},
|
|
753
754
|
};
|
|
754
755
|
}
|
|
756
|
+
const DRAG_INTERCEPT_TIMEOUT_MS = 1000;
|
|
757
|
+
const DRAG_NOT_INTERCEPTED = 'no HTML5 drag started (Input.dragIntercepted not fired within 1 s): the element at --from is not draggable, or a control (input/select/button) under the cursor swallowed dragstart';
|
|
755
758
|
export async function connectToPage(port, target, cache) {
|
|
756
759
|
if (target.type !== TargetType.Page && target.type !== TargetType.Iframe) {
|
|
757
760
|
throw new Error(`connectToPage requires a page/iframe target, got: ${target.type}`);
|
|
@@ -791,8 +794,6 @@ export async function connectToPage(port, target, cache) {
|
|
|
791
794
|
});
|
|
792
795
|
// null = not yet tested; true = available; false = unavailable (API not supported)
|
|
793
796
|
let queryAXTreeAvailable = null;
|
|
794
|
-
// null = not yet tested; true = webp supported; false = not supported (old Chrome/Electron)
|
|
795
|
-
let webpSupported = null;
|
|
796
797
|
Page.frameNavigated(async () => {
|
|
797
798
|
cache.invalidate(cacheKey);
|
|
798
799
|
fileChooserSub.forget();
|
|
@@ -816,21 +817,64 @@ export async function connectToPage(port, target, cache) {
|
|
|
816
817
|
await Input.dispatchMouseEvent({ type: 'mouseReleased', x, y, button, clickCount: i });
|
|
817
818
|
}
|
|
818
819
|
}
|
|
820
|
+
const sleep = (ms) => new Promise(r => setTimeout(r, ms));
|
|
819
821
|
async function dispatchDrag(from, to, opts) {
|
|
820
822
|
const steps = Math.max(0, opts?.steps ?? 10);
|
|
821
823
|
const button = opts?.button ?? MouseButton.Left;
|
|
822
824
|
const holdMs = Math.max(0, opts?.holdMs ?? 0);
|
|
823
|
-
|
|
824
|
-
|
|
825
|
-
await new Promise(r => setTimeout(r, holdMs));
|
|
826
|
-
for (let i = 1; i <= steps; i++) {
|
|
825
|
+
const mode = opts?.mode ?? 'auto';
|
|
826
|
+
const at = (i) => {
|
|
827
827
|
const t = i / (steps + 1);
|
|
828
|
-
|
|
829
|
-
|
|
830
|
-
|
|
828
|
+
return { x: from.x + (to.x - from.x) * t, y: from.y + (to.y - from.y) * t };
|
|
829
|
+
};
|
|
830
|
+
// With interception on, Chromium reports a starting HTML5 drag as `Input.dragIntercepted`
|
|
831
|
+
// instead of showing the native drag image; after that only `dispatchDragEvent` reaches the page.
|
|
832
|
+
let intercepted;
|
|
833
|
+
const unsub = mode === 'pointer' ? undefined : Input.dragIntercepted(({ data }) => { intercepted = data; });
|
|
834
|
+
if (unsub)
|
|
835
|
+
await Input.setInterceptDrags({ enabled: true });
|
|
836
|
+
try {
|
|
837
|
+
await Input.dispatchMouseEvent({ type: 'mouseMoved', x: from.x, y: from.y });
|
|
838
|
+
await Input.dispatchMouseEvent({ type: 'mousePressed', x: from.x, y: from.y, button, clickCount: 1 });
|
|
839
|
+
if (holdMs > 0)
|
|
840
|
+
await sleep(holdMs);
|
|
841
|
+
// Chromium starts an HTML5 drag only after several mouseMoved past its threshold; the
|
|
842
|
+
// pause between them is what lets the renderer process each one.
|
|
843
|
+
let i = 1;
|
|
844
|
+
for (; i <= steps + 1 && !intercepted; i++) {
|
|
845
|
+
const p = at(i);
|
|
846
|
+
await Input.dispatchMouseEvent({ type: 'mouseMoved', x: p.x, y: p.y, button });
|
|
847
|
+
if (unsub)
|
|
848
|
+
await sleep(20);
|
|
849
|
+
}
|
|
850
|
+
if (unsub && !intercepted) {
|
|
851
|
+
const deadline = Date.now() + DRAG_INTERCEPT_TIMEOUT_MS;
|
|
852
|
+
while (!intercepted && Date.now() < deadline)
|
|
853
|
+
await sleep(50);
|
|
854
|
+
}
|
|
855
|
+
if (intercepted) {
|
|
856
|
+
const data = opts?.mask === undefined ? intercepted : { ...intercepted, dragOperationsMask: opts.mask };
|
|
857
|
+
const ended = opts?.cancel ? 'dragCancel' : 'drop';
|
|
858
|
+
await Input.dispatchDragEvent({ type: 'dragEnter', x: at(i - 1).x, y: at(i - 1).y, data });
|
|
859
|
+
for (; i <= steps + 1; i++) {
|
|
860
|
+
const p = at(i);
|
|
861
|
+
await Input.dispatchDragEvent({ type: 'dragOver', x: p.x, y: p.y, data });
|
|
862
|
+
await sleep(20);
|
|
863
|
+
}
|
|
864
|
+
await Input.dispatchDragEvent({ type: ended, x: to.x, y: to.y, data });
|
|
865
|
+
await Input.dispatchMouseEvent({ type: 'mouseReleased', x: to.x, y: to.y, button, clickCount: 1 });
|
|
866
|
+
return { path: 'html5', ended, data };
|
|
867
|
+
}
|
|
868
|
+
await Input.dispatchMouseEvent({ type: 'mouseReleased', x: to.x, y: to.y, button, clickCount: 1 });
|
|
869
|
+
if (mode === 'html5')
|
|
870
|
+
throw new Error(DRAG_NOT_INTERCEPTED);
|
|
871
|
+
return { path: 'pointer', warning: unsub ? DRAG_NOT_INTERCEPTED : undefined };
|
|
872
|
+
}
|
|
873
|
+
finally {
|
|
874
|
+
unsub?.();
|
|
875
|
+
if (unsub)
|
|
876
|
+
await Input.setInterceptDrags({ enabled: false });
|
|
831
877
|
}
|
|
832
|
-
await Input.dispatchMouseEvent({ type: 'mouseMoved', x: to.x, y: to.y, button });
|
|
833
|
-
await Input.dispatchMouseEvent({ type: 'mouseReleased', x: to.x, y: to.y, button, clickCount: 1 });
|
|
834
878
|
}
|
|
835
879
|
async function scrollNodeIntoView(backendNodeId) {
|
|
836
880
|
const { object } = await DOM.resolveNode({ backendNodeId });
|
|
@@ -968,38 +1012,18 @@ export async function connectToPage(port, target, cache) {
|
|
|
968
1012
|
}
|
|
969
1013
|
},
|
|
970
1014
|
async captureScreenshot(opts) {
|
|
1015
|
+
// Never pass `clip` to Chromium: see cropScalePng for why.
|
|
1016
|
+
const { data } = await Page.captureScreenshot({ format: 'png' });
|
|
1017
|
+
const png = Buffer.from(data, 'base64');
|
|
971
1018
|
const scale = opts?.scale ?? 1;
|
|
972
|
-
|
|
973
|
-
|
|
974
|
-
|
|
975
|
-
|
|
976
|
-
|
|
977
|
-
|
|
978
|
-
|
|
979
|
-
|
|
980
|
-
}
|
|
981
|
-
if (scale >= 1) {
|
|
982
|
-
const { data } = await Page.captureScreenshot({ format: 'png' });
|
|
983
|
-
return { buffer: Buffer.from(data, 'base64'), format: 'png' };
|
|
984
|
-
}
|
|
985
|
-
const { cssLayoutViewport } = await Page.getLayoutMetrics();
|
|
986
|
-
const clip = { x: 0, y: 0, width: cssLayoutViewport.clientWidth, height: cssLayoutViewport.clientHeight, scale };
|
|
987
|
-
if (webpSupported !== false) {
|
|
988
|
-
try {
|
|
989
|
-
const { data } = await Page.captureScreenshot({ format: 'webp', quality: 80, clip });
|
|
990
|
-
webpSupported = true;
|
|
991
|
-
return { buffer: Buffer.from(data, 'base64'), format: 'webp' };
|
|
992
|
-
}
|
|
993
|
-
catch {
|
|
994
|
-
if (webpSupported === null) {
|
|
995
|
-
// eslint-disable-next-line no-console
|
|
996
|
-
console.error('[agent-view] WebP not supported by this Chrome/Electron version, falling back to JPEG');
|
|
997
|
-
webpSupported = false;
|
|
998
|
-
}
|
|
999
|
-
}
|
|
1000
|
-
}
|
|
1001
|
-
const { data } = await Page.captureScreenshot({ format: 'jpeg', quality: 80, clip });
|
|
1002
|
-
return { buffer: Buffer.from(data, 'base64'), format: 'jpeg' };
|
|
1019
|
+
if (opts?.clip === undefined && scale >= 1)
|
|
1020
|
+
return { buffer: png, format: 'png' };
|
|
1021
|
+
// The capture is in device pixels, clip rects are in CSS pixels.
|
|
1022
|
+
const { layoutViewport, cssLayoutViewport } = await Page.getLayoutMetrics();
|
|
1023
|
+
const dpr = layoutViewport.clientWidth / cssLayoutViewport.clientWidth;
|
|
1024
|
+
const clip = opts?.clip;
|
|
1025
|
+
const rect = clip && { x: clip.x * dpr, y: clip.y * dpr, width: clip.width * dpr, height: clip.height * dpr };
|
|
1026
|
+
return { buffer: cropScalePng(png, rect, Math.min(scale, 1)), format: 'png' };
|
|
1003
1027
|
},
|
|
1004
1028
|
async clickByNodeId(backendNodeId, opts) {
|
|
1005
1029
|
const { x, y } = await resolveBoxCenter(backendNodeId, true);
|
|
@@ -1017,7 +1041,7 @@ export async function connectToPage(port, target, cache) {
|
|
|
1017
1041
|
return resolveBoxRect(nodeId, opts?.scrollIntoView ?? true);
|
|
1018
1042
|
},
|
|
1019
1043
|
async dragBetweenPositions(from, to, opts) {
|
|
1020
|
-
|
|
1044
|
+
return dispatchDrag(from, to, opts);
|
|
1021
1045
|
},
|
|
1022
1046
|
async uploadByNodeId(backendNodeId, files) {
|
|
1023
1047
|
await DOM.setFileInputFiles({ backendNodeId, files });
|