@angelky7/dsh-browser-control 1.7.2 → 1.7.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +5 -4
- package/README.md +6 -5
- package/lib/index.js +339 -0
- package/lib/types/index.js +274 -0
- package/llms.txt +1 -1
- package/package.json +2 -2
package/README.en.md
CHANGED
|
@@ -64,8 +64,8 @@ In one line: **for "use MY browser" tasks (logged-in Bilibili, Zhihu, admin pane
|
|
|
64
64
|
|
|
65
65
|
| File | Purpose |
|
|
66
66
|
|---|---|
|
|
67
|
-
| [DSH-Browser-Control-v1.7.2.zip](https://github.com/angelky7/dsh-browser-control/releases/download/v1.7.2/DSH-Browser-Control-v1.7.2.zip) | Chrome extension (unzip and load;
|
|
68
|
-
| [dsh-browser-control-plugin-v1.7.
|
|
67
|
+
| [DSH-Browser-Control-v1.7.2.zip](https://github.com/angelky7/dsh-browser-control/releases/download/v1.7.2/DSH-Browser-Control-v1.7.2.zip) | Chrome extension (unzip and load; **the extension no longer changes with feature updates from v1.7.2 on**: v1.7.3 does not touch the extension — skip this if v1.7.2 is already loaded) |
|
|
68
|
+
| [dsh-browser-control-plugin-v1.7.3.zip](https://github.com/angelky7/dsh-browser-control/releases/download/v1.7.3/dsh-browser-control-plugin-v1.7.3.zip) | dsh plugin (offline fallback; online installs use Option A/B) |
|
|
69
69
|
|
|
70
70
|
## Install the Chrome extension (30 seconds)
|
|
71
71
|
|
|
@@ -104,7 +104,7 @@ npm install @angelky7/dsh-browser-control
|
|
|
104
104
|
|
|
105
105
|
```bash
|
|
106
106
|
# Straight from GitHub
|
|
107
|
-
dsh plugin --profile web add "github:angelky7/dsh-browser-control#v1.7.
|
|
107
|
+
dsh plugin --profile web add "github:angelky7/dsh-browser-control#v1.7.3"
|
|
108
108
|
|
|
109
109
|
# Local checkout for debugging (note: the explicit file: prefix is required)
|
|
110
110
|
dsh plugin --profile web add "file:D:\path\to\dsh-browser-control"
|
|
@@ -267,10 +267,11 @@ curl -X POST http://127.0.0.1:9777/api/command \
|
|
|
267
267
|
| `browser_windows` | Window control: list / open / update state & geometry / close a whole window / move tabs across windows (the only tool that may touch window geometry, v1.6.0+) |
|
|
268
268
|
| `browser_emulate` | Switch to a device viewport (mobile / desktop / custom, v1.0.7+); v1.7.1+ adds throttling: `network` slow3g/fast3g/offline/reset + `cpuThrottlingRate` (1–20), the receipt states the applied numbers, `device:"reset"` clears them too |
|
|
269
269
|
| `browser_metrics` | Read performance counters (DOM nodes / JS heap / layout & script durations, Performance.getMetrics, read-only; puts numbers behind "this page is heavy", v1.7.1+) |
|
|
270
|
+
| `browser_drag` | Drag from/to a selector or viewport point: `auto` synthesizes html5 dragstart→dragover→drop for `draggable="true"` sources (no drop unless accepted), otherwise sends real mouse input (press, interpolated moves, release — sliders/canvas/sortable widgets). **Plugin-side only** over the cdp channel embedded in extension v1.7.2, v1.7.3+ |
|
|
270
271
|
| `browser_cleanup` | Clean up temp files |
|
|
271
272
|
| `browser_info` | Which browser and extension build is attached (run it first when debugging, v1.0.9+) |
|
|
272
273
|
|
|
273
|
-
|
|
274
|
+
26 tools in total. `browser_wait`, `browser_find`, `browser_dialog` and `browser_info` existed inside the extension up to v1.0.8 but were never registered as tools, so the agent could not reach them; `browser_network_clear` existed in code and was simply missing from this table. `verify/verify_surface.mjs` now reconciles the extension command table, the commands the plugin calls, this table and the built bundle, so any drift fails `npm run verify`.
|
|
274
275
|
|
|
275
276
|
## Architecture
|
|
276
277
|
|
package/README.md
CHANGED
|
@@ -64,8 +64,8 @@ Chrome 扩展(CDP 驱动)
|
|
|
64
64
|
|
|
65
65
|
| 文件 | 说明 |
|
|
66
66
|
|---|---|
|
|
67
|
-
| [DSH-Browser-Control-v1.7.2.zip](https://github.com/angelky7/dsh-browser-control/releases/download/v1.7.2/DSH-Browser-Control-v1.7.2.zip) | Chrome
|
|
68
|
-
| [dsh-browser-control-plugin-v1.7.
|
|
67
|
+
| [DSH-Browser-Control-v1.7.2.zip](https://github.com/angelky7/dsh-browser-control/releases/download/v1.7.2/DSH-Browser-Control-v1.7.2.zip) | Chrome 扩展(解压后加载;**扩展从 v1.7.2 起不再随功能更新**:v1.7.3 没有动扩展,装过 v1.7.2 的不用重装) |
|
|
68
|
+
| [dsh-browser-control-plugin-v1.7.3.zip](https://github.com/angelky7/dsh-browser-control/releases/download/v1.7.3/dsh-browser-control-plugin-v1.7.3.zip) | dsh 插件(离线兜底,在线装直接用方式 A/B) |
|
|
69
69
|
|
|
70
70
|
## 安装 Chrome 扩展(30 秒)
|
|
71
71
|
|
|
@@ -104,7 +104,7 @@ npm install @angelky7/dsh-browser-control
|
|
|
104
104
|
|
|
105
105
|
```bash
|
|
106
106
|
# 直接从 GitHub 安装
|
|
107
|
-
dsh plugin --profile web add "github:angelky7/dsh-browser-control#v1.7.
|
|
107
|
+
dsh plugin --profile web add "github:angelky7/dsh-browser-control#v1.7.3"
|
|
108
108
|
|
|
109
109
|
# 本地目录调试(注意:必须显式 file: 前缀)
|
|
110
110
|
dsh plugin --profile web add "file:D:\path\to\dsh-browser-control"
|
|
@@ -260,11 +260,12 @@ curl -X POST http://127.0.0.1:9777/api/command \
|
|
|
260
260
|
| `browser_archive` | 整页存成单文件 MHTML(正文/图片/样式全内联,可当文本 grep 的离线证据;是快照,内嵌脚本不会跑,v1.5.0+) |
|
|
261
261
|
| `browser_emulate` | 切设备视口(移动 / 桌面 / 自定义,v1.0.7+);v1.7.1 起还能限速:`network` slow3g/fast3g/offline/reset + `cpuThrottlingRate`(1–20),回执写明生效数值,`device:"reset"` 一并还原 |
|
|
262
262
|
| `browser_metrics` | 读性能计数器(DOM 节点数 / JS 堆 / 布局与脚本耗时,Performance.getMetrics,只读;让「这页面很重」有数字说话,v1.7.1+) |
|
|
263
|
+
| `browser_drag` | 拖拽:from/to 用选择器或视口坐标;`auto` 对 `draggable="true"` 的源走 html5 事件合成(dragstart→dragover→drop,未接受就不 drop),其余走真实鼠标输入(按下→插值移动→抬起,滑杆/画布/排序组件用这个)。**纯插件实现**(经 v1.7.2 预埋的 cdp 通道),v1.7.3+ |
|
|
263
264
|
| `browser_windows` | 窗口控制:清单/开窗/改状态与几何/关整窗/跨窗挪标签(唯一能碰窗口几何的工具,v1.6.0+) |
|
|
264
265
|
| `browser_cleanup` | 清理临时文件 |
|
|
265
266
|
| `browser_info` | 报告实际连上的浏览器与扩展版本(排障先跑它,v1.0.9+) |
|
|
266
267
|
|
|
267
|
-
共
|
|
268
|
+
共 26 个工具。`browser_wait` / `browser_find` / `browser_dialog` / `browser_info` 在 v1.0.8 及以前是「扩展里已实现、但没注册成工具」,agent 调不到;`browser_network_clear` 则是代码里有、这张表漏了。这些现在由 `verify/verify_surface.mjs` 做三方对账(扩展命令表 ↔ 插件调用 ↔ 文档表 ↔ 构建产物),漂移会直接让 `npm run verify` 失败。
|
|
268
269
|
|
|
269
270
|
## 架构
|
|
270
271
|
|
|
@@ -296,7 +297,7 @@ Chrome 浏览器
|
|
|
296
297
|
| 实测台 `verify_extension_e2e`:真桥 + 真扩展代码 + 真 WebSocket 端到端(帧契约、命令可达、错误分类) | ✅ 2026-10-11,只有 `chrome.*` 是假的;视觉与真实 CDP 行为仍需实机确认 |
|
|
297
298
|
| 实测台 `verify_cdp_retry`:CDP 会话被抢走后的恢复 / 重放不许把副作用做两遍 / 重试有界(6 场景) | ✅ 2026-10-11,真实 `background.js` + 有状态假 `chrome.debugger`;另用合并预览文件交叉验证过 |
|
|
298
299
|
| 实测台 `verify_render_contract`:每条工具的真实返回值同时喂给宿主的输出校验器与它自己的 `render`,断言「模型可见文本里有这次调用真正要的数据」 | ✅ 2026-10-12,真桥 + 假扩展 + 宿主 `validateJsonSchemaValue`;这一层此前**零覆盖**,所以两类缺陷活了 12 / 5 个版本 |
|
|
299
|
-
| 实测台 `verify_compat_matrix`:兼容矩阵——从 git 标签物化**真实历史扩展代码**(v1.2.0–v1.7.
|
|
300
|
+
| 实测台 `verify_compat_matrix`:兼容矩阵——从 git 标签物化**真实历史扩展代码**(v1.2.0–v1.7.2),当前插件对每个面跑 28 个探针逐格验收「≥ 当年组合」;新工具报带重装指引的 extension-outdated、绝不许 unknown command 冒充;命令面 append-only(扩展升级对旧插件也纯增益) | ✅ 2026-11-04,6 面 × 186 项;老 hello 无 capabilities ⇒ 版本映射兜底路径的实测,v1.7.2 面实测 capabilities 精确握手 |
|
|
300
301
|
| **真实宿主 × 真实浏览器**:本机 dsh `0.2.0-rc.2` 用隔离 `DSH_HOME` 挂载本插件,用户的真实 Chrome 连上桥,9 条只读工具全部走真实代码路径 | ✅ 2026-10-12,Chrome 153 + 扩展 1.0.10:`browser_read` 交出 1341 字符真实正文、`browser_snapshot` 交出 14 条带 ref 的清单、输出 schema 零违规;**也正是这一跑发现了旧扩展会静默忽略 `mode:"ax"`**(本机扩展是 1.0.10,早于 AX 快照的 1.0.11)——已改成插件侧显式回报 `axFallback` |
|
|
301
302
|
| `tsc --noEmit` + `publint` | ✅ 2026-10-10 |
|
|
302
303
|
|
package/lib/index.js
CHANGED
|
@@ -1061,6 +1061,17 @@ var BridgeController = class {
|
|
|
1061
1061
|
get shotsDir() {
|
|
1062
1062
|
return this.current === void 0 ? void 0 : path.resolve(this.current.shotsDir);
|
|
1063
1063
|
}
|
|
1064
|
+
/**
|
|
1065
|
+
* Current policy flags. Tools that compose several commands (browser_drag
|
|
1066
|
+
* runs `cdp`/`eval`) read this to refuse with a tool-level message; the
|
|
1067
|
+
* per-command gate in execute() stays as the enforcement backstop.
|
|
1068
|
+
*/
|
|
1069
|
+
get policy() {
|
|
1070
|
+
return {
|
|
1071
|
+
readOnly: this.current?.readOnly === true,
|
|
1072
|
+
allowScript: this.current?.allowScript !== false
|
|
1073
|
+
};
|
|
1074
|
+
}
|
|
1064
1075
|
/** Version the connected extension reported in its hello frame, if any. */
|
|
1065
1076
|
extensionVersion() {
|
|
1066
1077
|
return this.server?.status.hello?.version;
|
|
@@ -3135,6 +3146,334 @@ function applyBrowserTools(ctx, controller) {
|
|
|
3135
3146
|
return await controller.execute("metrics", params, exec.signal);
|
|
3136
3147
|
}
|
|
3137
3148
|
}));
|
|
3149
|
+
registerDragTool(ctx, controller);
|
|
3150
|
+
}
|
|
3151
|
+
/** One CDP round-trip over the embedded `cdp` passthrough (extension v1.7.2+). */
|
|
3152
|
+
async function cdpCall(ctrl, tabId, method, params, signal) {
|
|
3153
|
+
return await ctrl.execute("cdp", {
|
|
3154
|
+
method,
|
|
3155
|
+
params,
|
|
3156
|
+
tabId
|
|
3157
|
+
}, signal);
|
|
3158
|
+
}
|
|
3159
|
+
function dragTargetArgs(selector, x, y, side) {
|
|
3160
|
+
const hasSel = typeof selector === "string" && selector.length > 0;
|
|
3161
|
+
const hasX = typeof x === "number" && Number.isFinite(x);
|
|
3162
|
+
const hasY = typeof y === "number" && Number.isFinite(y);
|
|
3163
|
+
if (hasSel && (hasX || hasY)) throw new Error(`${side}: give either "${side}" (CSS selector) or "${side}X"/"${side}Y" coordinates, not both`);
|
|
3164
|
+
if (!hasSel && !(hasX && hasY)) throw new Error(`${side}: needs a CSS selector ("${side}") or both "${side}X" and "${side}Y" coordinates`);
|
|
3165
|
+
if (hasSel) return { selector };
|
|
3166
|
+
return {
|
|
3167
|
+
x,
|
|
3168
|
+
y
|
|
3169
|
+
};
|
|
3170
|
+
}
|
|
3171
|
+
/**
|
|
3172
|
+
* Resolve a drag endpoint. Selector endpoints become viewport coordinates via
|
|
3173
|
+
* the CDP DOM domain — no page JavaScript, so mouse mode also works with
|
|
3174
|
+
* `allowScript: false`. Top document only; cross-origin iframes are a later
|
|
3175
|
+
* iteration.
|
|
3176
|
+
*/
|
|
3177
|
+
async function resolveDragPoint(ctrl, tabId, target, signal) {
|
|
3178
|
+
if (target.selector === void 0) return {
|
|
3179
|
+
x: target.x,
|
|
3180
|
+
y: target.y
|
|
3181
|
+
};
|
|
3182
|
+
const selector = target.selector;
|
|
3183
|
+
const rootId = (await cdpCall(ctrl, tabId, "DOM.getDocument", { depth: 0 }, signal))?.root?.nodeId;
|
|
3184
|
+
if (typeof rootId !== "number") throw new Error("DOM.getDocument returned no root node");
|
|
3185
|
+
const nodeId = (await cdpCall(ctrl, tabId, "DOM.querySelector", {
|
|
3186
|
+
nodeId: rootId,
|
|
3187
|
+
selector
|
|
3188
|
+
}, signal))?.nodeId ?? 0;
|
|
3189
|
+
if (nodeId === 0) throw new Error(`no element matches selector ${JSON.stringify(selector)} (top document only)`);
|
|
3190
|
+
await cdpCall(ctrl, tabId, "DOM.scrollIntoViewIfNeeded", { nodeId }, signal);
|
|
3191
|
+
let box;
|
|
3192
|
+
try {
|
|
3193
|
+
box = await cdpCall(ctrl, tabId, "DOM.getBoxModel", { nodeId }, signal);
|
|
3194
|
+
} catch (error) {
|
|
3195
|
+
throw new Error(`cannot measure ${JSON.stringify(selector)} — hidden, zero-size or detached? (${error instanceof Error ? error.message : String(error)})`);
|
|
3196
|
+
}
|
|
3197
|
+
const q = box?.model?.content;
|
|
3198
|
+
if (!Array.isArray(q) || q.length < 8) throw new Error(`DOM.getBoxModel returned no quad for ${JSON.stringify(selector)}`);
|
|
3199
|
+
return {
|
|
3200
|
+
selector,
|
|
3201
|
+
x: (q[0] + q[2] + q[4] + q[6]) / 4,
|
|
3202
|
+
y: (q[1] + q[3] + q[5] + q[7]) / 4,
|
|
3203
|
+
width: typeof box?.model?.width === "number" ? box.model.width : Math.hypot(q[2] - q[0], q[3] - q[1]),
|
|
3204
|
+
height: typeof box?.model?.height === "number" ? box.model.height : Math.hypot(q[4] - q[2], q[5] - q[3]),
|
|
3205
|
+
nodeId
|
|
3206
|
+
};
|
|
3207
|
+
}
|
|
3208
|
+
/** Whether the source declares the native HTML5 `draggable` attribute. */
|
|
3209
|
+
async function sourceIsDraggable(ctrl, tabId, nodeId, signal) {
|
|
3210
|
+
const attrs = await cdpCall(ctrl, tabId, "DOM.getAttributes", { nodeId }, signal);
|
|
3211
|
+
const list = Array.isArray(attrs?.attributes) ? attrs.attributes.map(String) : [];
|
|
3212
|
+
const at = list.indexOf("draggable");
|
|
3213
|
+
return at >= 0 && list[at + 1] === "true";
|
|
3214
|
+
}
|
|
3215
|
+
/**
|
|
3216
|
+
* Synthesize a native-order HTML5 drag in the page: dragstart on the source,
|
|
3217
|
+
* dragenter/dragover on the target, drop only when dragover was prevented
|
|
3218
|
+
* (the browser's own rule), dragend on the source, all sharing one DataTransfer.
|
|
3219
|
+
* Untrusted events — pages that gate on `event.isTrusted` will ignore them,
|
|
3220
|
+
* which the tool description states instead of pretending otherwise.
|
|
3221
|
+
*/
|
|
3222
|
+
function html5DragExpression(fromSelector, to) {
|
|
3223
|
+
const toExpr = to.selector !== void 0 ? `document.querySelector(${JSON.stringify(to.selector)})` : `document.elementFromPoint(${to.x}, ${to.y})`;
|
|
3224
|
+
return `(() => {
|
|
3225
|
+
const src = document.querySelector(${JSON.stringify(fromSelector)});
|
|
3226
|
+
if (!src) return { error: 'no element matches from selector ' + ${JSON.stringify(fromSelector)} };
|
|
3227
|
+
const dst = ${toExpr};
|
|
3228
|
+
if (!dst) return { error: 'target element not found' };
|
|
3229
|
+
const dt = new DataTransfer();
|
|
3230
|
+
const log = [];
|
|
3231
|
+
const sr = src.getBoundingClientRect();
|
|
3232
|
+
const dr = dst.getBoundingClientRect();
|
|
3233
|
+
const sx = sr.left + sr.width / 2, sy = sr.top + sr.height / 2;
|
|
3234
|
+
const tx = dr.left + dr.width / 2, ty = dr.top + dr.height / 2;
|
|
3235
|
+
const fire = (type, target, cx, cy) => {
|
|
3236
|
+
const ev = new DragEvent(type, { bubbles: true, cancelable: true, composed: true, dataTransfer: dt, clientX: cx, clientY: cy });
|
|
3237
|
+
target.dispatchEvent(ev);
|
|
3238
|
+
log.push({ type, defaultPrevented: ev.defaultPrevented });
|
|
3239
|
+
return ev.defaultPrevented;
|
|
3240
|
+
};
|
|
3241
|
+
const startCancelled = fire('dragstart', src, sx, sy);
|
|
3242
|
+
let dragenterDefaultPrevented = false, overPrevented = false, dropFired = false, dropPrevented = false;
|
|
3243
|
+
if (!startCancelled) {
|
|
3244
|
+
dragenterDefaultPrevented = fire('dragenter', dst, tx, ty);
|
|
3245
|
+
overPrevented = fire('dragover', dst, tx, ty);
|
|
3246
|
+
if (overPrevented) {
|
|
3247
|
+
dropFired = true;
|
|
3248
|
+
dropPrevented = fire('drop', dst, tx, ty);
|
|
3249
|
+
}
|
|
3250
|
+
}
|
|
3251
|
+
fire('dragend', src, sx, sy);
|
|
3252
|
+
return { startCancelled, dragenterDefaultPrevented, overPrevented, dropFired, dropPrevented, events: log.length, log };
|
|
3253
|
+
})()`;
|
|
3254
|
+
}
|
|
3255
|
+
function roundDragPoint(p) {
|
|
3256
|
+
const out = {
|
|
3257
|
+
x: Math.round(p.x * 100) / 100,
|
|
3258
|
+
y: Math.round(p.y * 100) / 100
|
|
3259
|
+
};
|
|
3260
|
+
if (p.selector !== void 0) out.selector = p.selector;
|
|
3261
|
+
if (p.width !== void 0) out.width = Math.round(p.width * 100) / 100;
|
|
3262
|
+
if (p.height !== void 0) out.height = Math.round(p.height * 100) / 100;
|
|
3263
|
+
return out;
|
|
3264
|
+
}
|
|
3265
|
+
function registerDragTool(ctx, controller) {
|
|
3266
|
+
ctx.tools.register(defineTool({
|
|
3267
|
+
name: "browser_drag",
|
|
3268
|
+
description: "Drag a page element or point onto another with real input. mode \"mouse\" sends trusted press→moves→release input via CDP (canvas, sliders, sortable widgets); mode \"html5\" synthesizes dragstart/dragover/drop in the page for [draggable] elements and needs page scripting; \"auto\" (default) picks html5 only when the source declares draggable=\"true\". Endpoints are CSS selectors (top document) or fromX/fromY + toX/toY viewport coordinates. Needs browser extension v1.7.2+ (embedded cdp channel).",
|
|
3269
|
+
parameters: {
|
|
3270
|
+
from: {
|
|
3271
|
+
type: "string",
|
|
3272
|
+
description: "CSS selector of the drag source; use fromX/fromY instead for raw coordinates."
|
|
3273
|
+
},
|
|
3274
|
+
to: {
|
|
3275
|
+
type: "string",
|
|
3276
|
+
description: "CSS selector of the drop target; use toX/toY instead for raw coordinates."
|
|
3277
|
+
},
|
|
3278
|
+
fromX: {
|
|
3279
|
+
type: "number",
|
|
3280
|
+
description: "Viewport x of the drag start point (with fromY)."
|
|
3281
|
+
},
|
|
3282
|
+
fromY: {
|
|
3283
|
+
type: "number",
|
|
3284
|
+
description: "Viewport y of the drag start point (with fromX)."
|
|
3285
|
+
},
|
|
3286
|
+
toX: {
|
|
3287
|
+
type: "number",
|
|
3288
|
+
description: "Viewport x of the drop point (with toY)."
|
|
3289
|
+
},
|
|
3290
|
+
toY: {
|
|
3291
|
+
type: "number",
|
|
3292
|
+
description: "Viewport y of the drop point (with toX)."
|
|
3293
|
+
},
|
|
3294
|
+
tabId: {
|
|
3295
|
+
type: "number",
|
|
3296
|
+
description: "Target tab; defaults to the active tab."
|
|
3297
|
+
},
|
|
3298
|
+
mode: {
|
|
3299
|
+
type: "string",
|
|
3300
|
+
description: "\"auto\" (default), \"mouse\" or \"html5\" — see the tool description."
|
|
3301
|
+
},
|
|
3302
|
+
steps: {
|
|
3303
|
+
type: "number",
|
|
3304
|
+
description: "Intermediate mouse moves in mouse mode, 1–50 (default 12)."
|
|
3305
|
+
}
|
|
3306
|
+
},
|
|
3307
|
+
output: {
|
|
3308
|
+
schema: {
|
|
3309
|
+
type: "object",
|
|
3310
|
+
additionalProperties: false,
|
|
3311
|
+
properties: {
|
|
3312
|
+
tabId: {
|
|
3313
|
+
type: "number",
|
|
3314
|
+
required: true
|
|
3315
|
+
},
|
|
3316
|
+
mode: {
|
|
3317
|
+
type: "string",
|
|
3318
|
+
required: true
|
|
3319
|
+
},
|
|
3320
|
+
from: {
|
|
3321
|
+
type: "object",
|
|
3322
|
+
additionalProperties: false,
|
|
3323
|
+
properties: {
|
|
3324
|
+
selector: { type: "string" },
|
|
3325
|
+
x: { type: "number" },
|
|
3326
|
+
y: { type: "number" },
|
|
3327
|
+
width: { type: "number" },
|
|
3328
|
+
height: { type: "number" }
|
|
3329
|
+
}
|
|
3330
|
+
},
|
|
3331
|
+
to: {
|
|
3332
|
+
type: "object",
|
|
3333
|
+
additionalProperties: false,
|
|
3334
|
+
properties: {
|
|
3335
|
+
selector: { type: "string" },
|
|
3336
|
+
x: { type: "number" },
|
|
3337
|
+
y: { type: "number" },
|
|
3338
|
+
width: { type: "number" },
|
|
3339
|
+
height: { type: "number" }
|
|
3340
|
+
}
|
|
3341
|
+
},
|
|
3342
|
+
steps: { type: "number" },
|
|
3343
|
+
dispatched: { type: "number" },
|
|
3344
|
+
startCancelled: { type: "boolean" },
|
|
3345
|
+
dragenterDefaultPrevented: { type: "boolean" },
|
|
3346
|
+
overPrevented: { type: "boolean" },
|
|
3347
|
+
dropFired: { type: "boolean" },
|
|
3348
|
+
dropPrevented: { type: "boolean" },
|
|
3349
|
+
events: { type: "number" },
|
|
3350
|
+
elapsedMs: {
|
|
3351
|
+
type: "number",
|
|
3352
|
+
required: true
|
|
3353
|
+
}
|
|
3354
|
+
}
|
|
3355
|
+
},
|
|
3356
|
+
render: (_args, value) => {
|
|
3357
|
+
const v = value;
|
|
3358
|
+
const d = (p) => p?.selector !== void 0 ? p.selector : `(${p?.x ?? "?"},${p?.y ?? "?"})`;
|
|
3359
|
+
if (v.mode === "mouse") return [{
|
|
3360
|
+
type: "text",
|
|
3361
|
+
text: `Tab ${v.tabId}: dragged ${d(v.from)} → ${d(v.to)} with mouse input (${v.steps} steps, ${v.dispatched} dispatches, ${v.elapsedMs}ms)`
|
|
3362
|
+
}];
|
|
3363
|
+
const verdict = v.startCancelled === true ? "dragstart was cancelled by the page — nothing dropped" : v.overPrevented === true ? v.dropPrevented === true ? "drop fired and was handled" : "drop fired but no drop handler ran" : "target did not accept the drop (no dragover preventDefault)";
|
|
3364
|
+
return [{
|
|
3365
|
+
type: "text",
|
|
3366
|
+
text: `Tab ${v.tabId}: dragged ${d(v.from)} → ${d(v.to)} with html5 events (${v.events} events, ${v.elapsedMs}ms) — ${verdict}`
|
|
3367
|
+
}];
|
|
3368
|
+
}
|
|
3369
|
+
},
|
|
3370
|
+
presentCall: (args) => ({
|
|
3371
|
+
card: "generic",
|
|
3372
|
+
title: `Drag ${args.from ?? `(${args.fromX},${args.fromY})`} → ${args.to ?? `(${args.toX},${args.toY})`}`,
|
|
3373
|
+
kind: "other"
|
|
3374
|
+
}),
|
|
3375
|
+
async execute(args, exec) {
|
|
3376
|
+
const mode = args.mode === void 0 ? "auto" : args.mode;
|
|
3377
|
+
if (mode !== "auto" && mode !== "mouse" && mode !== "html5") throw new Error(`mode must be "auto", "mouse" or "html5" — got ${JSON.stringify(mode)}`);
|
|
3378
|
+
const from = dragTargetArgs(args.from, args.fromX, args.fromY, "from");
|
|
3379
|
+
const to = dragTargetArgs(args.to, args.toX, args.toY, "to");
|
|
3380
|
+
controller.requireExtensionCommand("cdp", "1.7.2");
|
|
3381
|
+
if (controller.policy.readOnly) throw new Error("browser_drag changes page state and read-only mode is on — turn readOnly off in the browser-bridge config to drag");
|
|
3382
|
+
let tabId = args.tabId;
|
|
3383
|
+
if (tabId === void 0) {
|
|
3384
|
+
const listing = await controller.execute("tabs.list", {}, exec.signal);
|
|
3385
|
+
if (typeof listing?.activeTabId !== "number") throw new Error("could not resolve the active tab — pass tabId explicitly");
|
|
3386
|
+
tabId = listing.activeTabId;
|
|
3387
|
+
}
|
|
3388
|
+
const started = Date.now();
|
|
3389
|
+
let resolvedMode = mode === "auto" ? "mouse" : mode;
|
|
3390
|
+
let source;
|
|
3391
|
+
if (mode !== "html5") source = await resolveDragPoint(controller, tabId, from, exec.signal);
|
|
3392
|
+
if (mode === "auto" && source?.nodeId !== void 0 && await sourceIsDraggable(controller, tabId, source.nodeId, exec.signal)) resolvedMode = "html5";
|
|
3393
|
+
if (resolvedMode === "mouse") {
|
|
3394
|
+
const target = await resolveDragPoint(controller, tabId, to, exec.signal);
|
|
3395
|
+
const steps = Math.min(50, Math.max(1, Math.trunc(typeof args.steps === "number" ? args.steps : 12)));
|
|
3396
|
+
const sx = source.x, sy = source.y, tx = target.x, ty = target.y;
|
|
3397
|
+
const dispatch = async (params) => {
|
|
3398
|
+
await cdpCall(controller, tabId, "Input.dispatchMouseEvent", params, exec.signal);
|
|
3399
|
+
};
|
|
3400
|
+
await dispatch({
|
|
3401
|
+
type: "mouseMoved",
|
|
3402
|
+
x: sx,
|
|
3403
|
+
y: sy,
|
|
3404
|
+
button: "none",
|
|
3405
|
+
buttons: 0
|
|
3406
|
+
});
|
|
3407
|
+
await dispatch({
|
|
3408
|
+
type: "mousePressed",
|
|
3409
|
+
x: sx,
|
|
3410
|
+
y: sy,
|
|
3411
|
+
button: "left",
|
|
3412
|
+
buttons: 1,
|
|
3413
|
+
clickCount: 1
|
|
3414
|
+
});
|
|
3415
|
+
for (let i = 1; i <= steps; i++) {
|
|
3416
|
+
const f = i / steps;
|
|
3417
|
+
await dispatch({
|
|
3418
|
+
type: "mouseMoved",
|
|
3419
|
+
x: sx + (tx - sx) * f,
|
|
3420
|
+
y: sy + (ty - sy) * f,
|
|
3421
|
+
button: "left",
|
|
3422
|
+
buttons: 1
|
|
3423
|
+
});
|
|
3424
|
+
}
|
|
3425
|
+
await dispatch({
|
|
3426
|
+
type: "mouseReleased",
|
|
3427
|
+
x: tx,
|
|
3428
|
+
y: ty,
|
|
3429
|
+
button: "left",
|
|
3430
|
+
buttons: 0,
|
|
3431
|
+
clickCount: 1
|
|
3432
|
+
});
|
|
3433
|
+
await dispatch({
|
|
3434
|
+
type: "mouseMoved",
|
|
3435
|
+
x: tx,
|
|
3436
|
+
y: ty,
|
|
3437
|
+
button: "none",
|
|
3438
|
+
buttons: 0
|
|
3439
|
+
});
|
|
3440
|
+
return {
|
|
3441
|
+
tabId,
|
|
3442
|
+
mode: "mouse",
|
|
3443
|
+
from: roundDragPoint(source),
|
|
3444
|
+
to: roundDragPoint(target),
|
|
3445
|
+
steps,
|
|
3446
|
+
dispatched: steps + 4,
|
|
3447
|
+
elapsedMs: Date.now() - started
|
|
3448
|
+
};
|
|
3449
|
+
}
|
|
3450
|
+
if (from.selector === void 0) throw new Error("html5 mode needs a CSS selector for \"from\" (a point has no draggable element)");
|
|
3451
|
+
if (!controller.policy.allowScript) throw new Error("html5 drag synthesizes DOM events and needs page scripting — allowScript is off; use mode:\"mouse\" or re-enable the script switch");
|
|
3452
|
+
const raw = await controller.execute("eval", {
|
|
3453
|
+
expression: html5DragExpression(from.selector, to),
|
|
3454
|
+
tabId
|
|
3455
|
+
}, exec.signal);
|
|
3456
|
+
const outcome = raw?.value;
|
|
3457
|
+
if (outcome === void 0 || typeof outcome !== "object") throw new Error("html5 drag expression returned no outcome — page may have navigated mid-drag");
|
|
3458
|
+
if (typeof outcome.error === "string") throw new Error(outcome.error);
|
|
3459
|
+
return {
|
|
3460
|
+
tabId: typeof raw?.tabId === "number" ? raw.tabId : tabId,
|
|
3461
|
+
mode: "html5",
|
|
3462
|
+
from: from.selector !== void 0 ? { selector: from.selector } : roundDragPoint(from),
|
|
3463
|
+
to: to.selector !== void 0 ? { selector: to.selector } : {
|
|
3464
|
+
x: to.x,
|
|
3465
|
+
y: to.y
|
|
3466
|
+
},
|
|
3467
|
+
startCancelled: outcome.startCancelled === true,
|
|
3468
|
+
dragenterDefaultPrevented: outcome.dragenterDefaultPrevented === true,
|
|
3469
|
+
overPrevented: outcome.overPrevented === true,
|
|
3470
|
+
dropFired: outcome.dropFired === true,
|
|
3471
|
+
dropPrevented: outcome.dropPrevented === true,
|
|
3472
|
+
events: typeof outcome.events === "number" ? outcome.events : 0,
|
|
3473
|
+
elapsedMs: Date.now() - started
|
|
3474
|
+
};
|
|
3475
|
+
}
|
|
3476
|
+
}));
|
|
3138
3477
|
}
|
|
3139
3478
|
/** Cordis plugin entry: wire the settings-driven lifecycle plus the model-facing tools. */
|
|
3140
3479
|
function apply(ctx, config) {
|
package/lib/types/index.js
CHANGED
|
@@ -123,6 +123,14 @@ class BridgeController {
|
|
|
123
123
|
get shotsDir() {
|
|
124
124
|
return this.current === undefined ? undefined : path.resolve(this.current.shotsDir);
|
|
125
125
|
}
|
|
126
|
+
/**
|
|
127
|
+
* Current policy flags. Tools that compose several commands (browser_drag
|
|
128
|
+
* runs `cdp`/`eval`) read this to refuse with a tool-level message; the
|
|
129
|
+
* per-command gate in execute() stays as the enforcement backstop.
|
|
130
|
+
*/
|
|
131
|
+
get policy() {
|
|
132
|
+
return { readOnly: this.current?.readOnly === true, allowScript: this.current?.allowScript !== false };
|
|
133
|
+
}
|
|
126
134
|
/** Version the connected extension reported in its hello frame, if any. */
|
|
127
135
|
extensionVersion() {
|
|
128
136
|
return this.server?.status.hello?.version;
|
|
@@ -1683,6 +1691,272 @@ function applyBrowserTools(ctx, controller) {
|
|
|
1683
1691
|
return await controller.execute('metrics', params, exec.signal);
|
|
1684
1692
|
},
|
|
1685
1693
|
}));
|
|
1694
|
+
registerDragTool(ctx, controller);
|
|
1695
|
+
}
|
|
1696
|
+
/* ------------------- browser_drag: composed plugin-side over the cdp channel ------------------- */
|
|
1697
|
+
/** One CDP round-trip over the embedded `cdp` passthrough (extension v1.7.2+). */
|
|
1698
|
+
async function cdpCall(ctrl, tabId, method, params, signal) {
|
|
1699
|
+
return await ctrl.execute('cdp', { method, params, tabId }, signal);
|
|
1700
|
+
}
|
|
1701
|
+
function dragTargetArgs(selector, x, y, side) {
|
|
1702
|
+
const hasSel = typeof selector === 'string' && selector.length > 0;
|
|
1703
|
+
const hasX = typeof x === 'number' && Number.isFinite(x);
|
|
1704
|
+
const hasY = typeof y === 'number' && Number.isFinite(y);
|
|
1705
|
+
if (hasSel && (hasX || hasY))
|
|
1706
|
+
throw new Error(`${side}: give either "${side}" (CSS selector) or "${side}X"/"${side}Y" coordinates, not both`);
|
|
1707
|
+
if (!hasSel && !(hasX && hasY))
|
|
1708
|
+
throw new Error(`${side}: needs a CSS selector ("${side}") or both "${side}X" and "${side}Y" coordinates`);
|
|
1709
|
+
if (hasSel)
|
|
1710
|
+
return { selector };
|
|
1711
|
+
return { x, y };
|
|
1712
|
+
}
|
|
1713
|
+
/**
|
|
1714
|
+
* Resolve a drag endpoint. Selector endpoints become viewport coordinates via
|
|
1715
|
+
* the CDP DOM domain — no page JavaScript, so mouse mode also works with
|
|
1716
|
+
* `allowScript: false`. Top document only; cross-origin iframes are a later
|
|
1717
|
+
* iteration.
|
|
1718
|
+
*/
|
|
1719
|
+
async function resolveDragPoint(ctrl, tabId, target, signal) {
|
|
1720
|
+
if (target.selector === undefined)
|
|
1721
|
+
return { x: target.x, y: target.y };
|
|
1722
|
+
const selector = target.selector;
|
|
1723
|
+
const doc = await cdpCall(ctrl, tabId, 'DOM.getDocument', { depth: 0 }, signal);
|
|
1724
|
+
const rootId = doc?.root?.nodeId;
|
|
1725
|
+
if (typeof rootId !== 'number')
|
|
1726
|
+
throw new Error('DOM.getDocument returned no root node');
|
|
1727
|
+
const found = await cdpCall(ctrl, tabId, 'DOM.querySelector', { nodeId: rootId, selector }, signal);
|
|
1728
|
+
const nodeId = found?.nodeId ?? 0;
|
|
1729
|
+
if (nodeId === 0)
|
|
1730
|
+
throw new Error(`no element matches selector ${JSON.stringify(selector)} (top document only)`);
|
|
1731
|
+
await cdpCall(ctrl, tabId, 'DOM.scrollIntoViewIfNeeded', { nodeId }, signal);
|
|
1732
|
+
let box;
|
|
1733
|
+
try {
|
|
1734
|
+
box = await cdpCall(ctrl, tabId, 'DOM.getBoxModel', { nodeId }, signal);
|
|
1735
|
+
}
|
|
1736
|
+
catch (error) {
|
|
1737
|
+
throw new Error(`cannot measure ${JSON.stringify(selector)} — hidden, zero-size or detached? (${error instanceof Error ? error.message : String(error)})`);
|
|
1738
|
+
}
|
|
1739
|
+
const q = box?.model?.content;
|
|
1740
|
+
if (!Array.isArray(q) || q.length < 8)
|
|
1741
|
+
throw new Error(`DOM.getBoxModel returned no quad for ${JSON.stringify(selector)}`);
|
|
1742
|
+
const x = (q[0] + q[2] + q[4] + q[6]) / 4;
|
|
1743
|
+
const y = (q[1] + q[3] + q[5] + q[7]) / 4;
|
|
1744
|
+
const width = typeof box?.model?.width === 'number' ? box.model.width : Math.hypot(q[2] - q[0], q[3] - q[1]);
|
|
1745
|
+
const height = typeof box?.model?.height === 'number' ? box.model.height : Math.hypot(q[4] - q[2], q[5] - q[3]);
|
|
1746
|
+
return { selector, x, y, width, height, nodeId };
|
|
1747
|
+
}
|
|
1748
|
+
/** Whether the source declares the native HTML5 `draggable` attribute. */
|
|
1749
|
+
async function sourceIsDraggable(ctrl, tabId, nodeId, signal) {
|
|
1750
|
+
const attrs = await cdpCall(ctrl, tabId, 'DOM.getAttributes', { nodeId }, signal);
|
|
1751
|
+
const list = Array.isArray(attrs?.attributes) ? attrs.attributes.map(String) : [];
|
|
1752
|
+
const at = list.indexOf('draggable');
|
|
1753
|
+
return at >= 0 && list[at + 1] === 'true';
|
|
1754
|
+
}
|
|
1755
|
+
/**
|
|
1756
|
+
* Synthesize a native-order HTML5 drag in the page: dragstart on the source,
|
|
1757
|
+
* dragenter/dragover on the target, drop only when dragover was prevented
|
|
1758
|
+
* (the browser's own rule), dragend on the source, all sharing one DataTransfer.
|
|
1759
|
+
* Untrusted events — pages that gate on `event.isTrusted` will ignore them,
|
|
1760
|
+
* which the tool description states instead of pretending otherwise.
|
|
1761
|
+
*/
|
|
1762
|
+
function html5DragExpression(fromSelector, to) {
|
|
1763
|
+
const toExpr = to.selector !== undefined
|
|
1764
|
+
? `document.querySelector(${JSON.stringify(to.selector)})`
|
|
1765
|
+
: `document.elementFromPoint(${to.x}, ${to.y})`;
|
|
1766
|
+
return `(() => {
|
|
1767
|
+
const src = document.querySelector(${JSON.stringify(fromSelector)});
|
|
1768
|
+
if (!src) return { error: 'no element matches from selector ' + ${JSON.stringify(fromSelector)} };
|
|
1769
|
+
const dst = ${toExpr};
|
|
1770
|
+
if (!dst) return { error: 'target element not found' };
|
|
1771
|
+
const dt = new DataTransfer();
|
|
1772
|
+
const log = [];
|
|
1773
|
+
const sr = src.getBoundingClientRect();
|
|
1774
|
+
const dr = dst.getBoundingClientRect();
|
|
1775
|
+
const sx = sr.left + sr.width / 2, sy = sr.top + sr.height / 2;
|
|
1776
|
+
const tx = dr.left + dr.width / 2, ty = dr.top + dr.height / 2;
|
|
1777
|
+
const fire = (type, target, cx, cy) => {
|
|
1778
|
+
const ev = new DragEvent(type, { bubbles: true, cancelable: true, composed: true, dataTransfer: dt, clientX: cx, clientY: cy });
|
|
1779
|
+
target.dispatchEvent(ev);
|
|
1780
|
+
log.push({ type, defaultPrevented: ev.defaultPrevented });
|
|
1781
|
+
return ev.defaultPrevented;
|
|
1782
|
+
};
|
|
1783
|
+
const startCancelled = fire('dragstart', src, sx, sy);
|
|
1784
|
+
let dragenterDefaultPrevented = false, overPrevented = false, dropFired = false, dropPrevented = false;
|
|
1785
|
+
if (!startCancelled) {
|
|
1786
|
+
dragenterDefaultPrevented = fire('dragenter', dst, tx, ty);
|
|
1787
|
+
overPrevented = fire('dragover', dst, tx, ty);
|
|
1788
|
+
if (overPrevented) {
|
|
1789
|
+
dropFired = true;
|
|
1790
|
+
dropPrevented = fire('drop', dst, tx, ty);
|
|
1791
|
+
}
|
|
1792
|
+
}
|
|
1793
|
+
fire('dragend', src, sx, sy);
|
|
1794
|
+
return { startCancelled, dragenterDefaultPrevented, overPrevented, dropFired, dropPrevented, events: log.length, log };
|
|
1795
|
+
})()`;
|
|
1796
|
+
}
|
|
1797
|
+
function roundDragPoint(p) {
|
|
1798
|
+
const out = {
|
|
1799
|
+
x: Math.round(p.x * 100) / 100,
|
|
1800
|
+
y: Math.round(p.y * 100) / 100,
|
|
1801
|
+
};
|
|
1802
|
+
if (p.selector !== undefined)
|
|
1803
|
+
out.selector = p.selector;
|
|
1804
|
+
if (p.width !== undefined)
|
|
1805
|
+
out.width = Math.round(p.width * 100) / 100;
|
|
1806
|
+
if (p.height !== undefined)
|
|
1807
|
+
out.height = Math.round(p.height * 100) / 100;
|
|
1808
|
+
return out;
|
|
1809
|
+
}
|
|
1810
|
+
function registerDragTool(ctx, controller) {
|
|
1811
|
+
ctx.tools.register(defineTool({
|
|
1812
|
+
name: 'browser_drag',
|
|
1813
|
+
description: 'Drag a page element or point onto another with real input. mode "mouse" sends trusted press→moves→release input via CDP (canvas, sliders, sortable widgets); mode "html5" synthesizes dragstart/dragover/drop in the page for [draggable] elements and needs page scripting; "auto" (default) picks html5 only when the source declares draggable="true". Endpoints are CSS selectors (top document) or fromX/fromY + toX/toY viewport coordinates. Needs browser extension v1.7.2+ (embedded cdp channel).',
|
|
1814
|
+
parameters: {
|
|
1815
|
+
from: { type: 'string', description: 'CSS selector of the drag source; use fromX/fromY instead for raw coordinates.' },
|
|
1816
|
+
to: { type: 'string', description: 'CSS selector of the drop target; use toX/toY instead for raw coordinates.' },
|
|
1817
|
+
fromX: { type: 'number', description: 'Viewport x of the drag start point (with fromY).' },
|
|
1818
|
+
fromY: { type: 'number', description: 'Viewport y of the drag start point (with fromX).' },
|
|
1819
|
+
toX: { type: 'number', description: 'Viewport x of the drop point (with toY).' },
|
|
1820
|
+
toY: { type: 'number', description: 'Viewport y of the drop point (with toX).' },
|
|
1821
|
+
tabId: { type: 'number', description: 'Target tab; defaults to the active tab.' },
|
|
1822
|
+
mode: { type: 'string', description: '"auto" (default), "mouse" or "html5" — see the tool description.' },
|
|
1823
|
+
steps: { type: 'number', description: 'Intermediate mouse moves in mouse mode, 1–50 (default 12).' },
|
|
1824
|
+
},
|
|
1825
|
+
output: {
|
|
1826
|
+
schema: {
|
|
1827
|
+
type: 'object',
|
|
1828
|
+
additionalProperties: false,
|
|
1829
|
+
properties: {
|
|
1830
|
+
tabId: { type: 'number', required: true },
|
|
1831
|
+
mode: { type: 'string', required: true },
|
|
1832
|
+
from: {
|
|
1833
|
+
type: 'object',
|
|
1834
|
+
additionalProperties: false,
|
|
1835
|
+
properties: {
|
|
1836
|
+
selector: { type: 'string' },
|
|
1837
|
+
x: { type: 'number' },
|
|
1838
|
+
y: { type: 'number' },
|
|
1839
|
+
width: { type: 'number' },
|
|
1840
|
+
height: { type: 'number' },
|
|
1841
|
+
},
|
|
1842
|
+
},
|
|
1843
|
+
to: {
|
|
1844
|
+
type: 'object',
|
|
1845
|
+
additionalProperties: false,
|
|
1846
|
+
properties: {
|
|
1847
|
+
selector: { type: 'string' },
|
|
1848
|
+
x: { type: 'number' },
|
|
1849
|
+
y: { type: 'number' },
|
|
1850
|
+
width: { type: 'number' },
|
|
1851
|
+
height: { type: 'number' },
|
|
1852
|
+
},
|
|
1853
|
+
},
|
|
1854
|
+
steps: { type: 'number' },
|
|
1855
|
+
dispatched: { type: 'number' },
|
|
1856
|
+
startCancelled: { type: 'boolean' },
|
|
1857
|
+
dragenterDefaultPrevented: { type: 'boolean' },
|
|
1858
|
+
overPrevented: { type: 'boolean' },
|
|
1859
|
+
dropFired: { type: 'boolean' },
|
|
1860
|
+
dropPrevented: { type: 'boolean' },
|
|
1861
|
+
events: { type: 'number' },
|
|
1862
|
+
elapsedMs: { type: 'number', required: true },
|
|
1863
|
+
},
|
|
1864
|
+
},
|
|
1865
|
+
render: (_args, value) => {
|
|
1866
|
+
const v = value;
|
|
1867
|
+
const d = (p) => p?.selector !== undefined ? p.selector : `(${p?.x ?? '?'},${p?.y ?? '?'})`;
|
|
1868
|
+
if (v.mode === 'mouse') {
|
|
1869
|
+
return [{ type: 'text', text: `Tab ${v.tabId}: dragged ${d(v.from)} → ${d(v.to)} with mouse input (${v.steps} steps, ${v.dispatched} dispatches, ${v.elapsedMs}ms)` }];
|
|
1870
|
+
}
|
|
1871
|
+
const verdict = v.startCancelled === true
|
|
1872
|
+
? 'dragstart was cancelled by the page — nothing dropped'
|
|
1873
|
+
: v.overPrevented === true
|
|
1874
|
+
? (v.dropPrevented === true ? 'drop fired and was handled' : 'drop fired but no drop handler ran')
|
|
1875
|
+
: 'target did not accept the drop (no dragover preventDefault)';
|
|
1876
|
+
return [{ type: 'text', text: `Tab ${v.tabId}: dragged ${d(v.from)} → ${d(v.to)} with html5 events (${v.events} events, ${v.elapsedMs}ms) — ${verdict}` }];
|
|
1877
|
+
},
|
|
1878
|
+
},
|
|
1879
|
+
presentCall: args => ({
|
|
1880
|
+
card: 'generic',
|
|
1881
|
+
title: `Drag ${args.from ?? `(${args.fromX},${args.fromY})`} → ${args.to ?? `(${args.toX},${args.toY})`}`,
|
|
1882
|
+
kind: 'other',
|
|
1883
|
+
}),
|
|
1884
|
+
async execute(args, exec) {
|
|
1885
|
+
const mode = args.mode === undefined ? 'auto' : args.mode;
|
|
1886
|
+
if (mode !== 'auto' && mode !== 'mouse' && mode !== 'html5')
|
|
1887
|
+
throw new Error(`mode must be "auto", "mouse" or "html5" — got ${JSON.stringify(mode)}`);
|
|
1888
|
+
const from = dragTargetArgs(args.from, args.fromX, args.fromY, 'from');
|
|
1889
|
+
const to = dragTargetArgs(args.to, args.toX, args.toY, 'to');
|
|
1890
|
+
controller.requireExtensionCommand('cdp', '1.7.2');
|
|
1891
|
+
if (controller.policy.readOnly)
|
|
1892
|
+
throw new Error('browser_drag changes page state and read-only mode is on — turn readOnly off in the browser-bridge config to drag');
|
|
1893
|
+
let tabId = args.tabId;
|
|
1894
|
+
if (tabId === undefined) {
|
|
1895
|
+
const listing = await controller.execute('tabs.list', {}, exec.signal);
|
|
1896
|
+
if (typeof listing?.activeTabId !== 'number')
|
|
1897
|
+
throw new Error('could not resolve the active tab — pass tabId explicitly');
|
|
1898
|
+
tabId = listing.activeTabId;
|
|
1899
|
+
}
|
|
1900
|
+
const started = Date.now();
|
|
1901
|
+
let resolvedMode = mode === 'auto' ? 'mouse' : mode;
|
|
1902
|
+
let source;
|
|
1903
|
+
if (mode !== 'html5')
|
|
1904
|
+
source = await resolveDragPoint(controller, tabId, from, exec.signal);
|
|
1905
|
+
if (mode === 'auto' && source?.nodeId !== undefined && await sourceIsDraggable(controller, tabId, source.nodeId, exec.signal)) {
|
|
1906
|
+
resolvedMode = 'html5';
|
|
1907
|
+
}
|
|
1908
|
+
if (resolvedMode === 'mouse') {
|
|
1909
|
+
const target = await resolveDragPoint(controller, tabId, to, exec.signal);
|
|
1910
|
+
const steps = Math.min(50, Math.max(1, Math.trunc(typeof args.steps === 'number' ? args.steps : 12)));
|
|
1911
|
+
const sx = source.x, sy = source.y, tx = target.x, ty = target.y;
|
|
1912
|
+
const dispatch = async (params) => {
|
|
1913
|
+
await cdpCall(controller, tabId, 'Input.dispatchMouseEvent', params, exec.signal);
|
|
1914
|
+
};
|
|
1915
|
+
await dispatch({ type: 'mouseMoved', x: sx, y: sy, button: 'none', buttons: 0 });
|
|
1916
|
+
await dispatch({ type: 'mousePressed', x: sx, y: sy, button: 'left', buttons: 1, clickCount: 1 });
|
|
1917
|
+
for (let i = 1; i <= steps; i++) {
|
|
1918
|
+
const f = i / steps;
|
|
1919
|
+
await dispatch({ type: 'mouseMoved', x: sx + (tx - sx) * f, y: sy + (ty - sy) * f, button: 'left', buttons: 1 });
|
|
1920
|
+
}
|
|
1921
|
+
await dispatch({ type: 'mouseReleased', x: tx, y: ty, button: 'left', buttons: 0, clickCount: 1 });
|
|
1922
|
+
await dispatch({ type: 'mouseMoved', x: tx, y: ty, button: 'none', buttons: 0 });
|
|
1923
|
+
const receipt = {
|
|
1924
|
+
tabId,
|
|
1925
|
+
mode: 'mouse',
|
|
1926
|
+
from: roundDragPoint(source),
|
|
1927
|
+
to: roundDragPoint(target),
|
|
1928
|
+
steps,
|
|
1929
|
+
dispatched: steps + 4,
|
|
1930
|
+
elapsedMs: Date.now() - started,
|
|
1931
|
+
};
|
|
1932
|
+
return receipt;
|
|
1933
|
+
}
|
|
1934
|
+
if (from.selector === undefined)
|
|
1935
|
+
throw new Error('html5 mode needs a CSS selector for "from" (a point has no draggable element)');
|
|
1936
|
+
if (!controller.policy.allowScript)
|
|
1937
|
+
throw new Error('html5 drag synthesizes DOM events and needs page scripting — allowScript is off; use mode:"mouse" or re-enable the script switch');
|
|
1938
|
+
const raw = await controller.execute('eval', { expression: html5DragExpression(from.selector, to), tabId }, exec.signal);
|
|
1939
|
+
const outcome = raw?.value;
|
|
1940
|
+
if (outcome === undefined || typeof outcome !== 'object')
|
|
1941
|
+
throw new Error('html5 drag expression returned no outcome — page may have navigated mid-drag');
|
|
1942
|
+
if (typeof outcome.error === 'string')
|
|
1943
|
+
throw new Error(outcome.error);
|
|
1944
|
+
const receipt = {
|
|
1945
|
+
tabId: typeof raw?.tabId === 'number' ? raw.tabId : tabId,
|
|
1946
|
+
mode: 'html5',
|
|
1947
|
+
from: from.selector !== undefined ? { selector: from.selector } : roundDragPoint(from),
|
|
1948
|
+
to: to.selector !== undefined ? { selector: to.selector } : { x: to.x, y: to.y },
|
|
1949
|
+
startCancelled: outcome.startCancelled === true,
|
|
1950
|
+
dragenterDefaultPrevented: outcome.dragenterDefaultPrevented === true,
|
|
1951
|
+
overPrevented: outcome.overPrevented === true,
|
|
1952
|
+
dropFired: outcome.dropFired === true,
|
|
1953
|
+
dropPrevented: outcome.dropPrevented === true,
|
|
1954
|
+
events: typeof outcome.events === 'number' ? outcome.events : 0,
|
|
1955
|
+
elapsedMs: Date.now() - started,
|
|
1956
|
+
};
|
|
1957
|
+
return receipt;
|
|
1958
|
+
},
|
|
1959
|
+
}));
|
|
1686
1960
|
}
|
|
1687
1961
|
/** Cordis plugin entry: wire the settings-driven lifecycle plus the model-facing tools. */
|
|
1688
1962
|
export function apply(ctx, config) {
|
package/llms.txt
CHANGED
|
@@ -42,7 +42,7 @@ Chrome 浏览器扩展 + DeepSeek Harness 插件,让 AI Agent 通过 CDP 驱
|
|
|
42
42
|
|
|
43
43
|
## 关键文件
|
|
44
44
|
|
|
45
|
-
- `src/index.ts` -
|
|
45
|
+
- `src/index.ts` - 26 个 `browser_*` 工具注册 + settings 接线的版本兼容分支
|
|
46
46
|
- `src/policy.ts` - 三张互不相同的表:只读白名单、脚本命令、插件本地破坏性命令(`browser_cleanup` 不经过 `execute()`,必须自己站闸门)
|
|
47
47
|
- `src/server.ts` - WebSocket 服务 + HTTP 面(鉴权、来源校验、状态页)
|
|
48
48
|
- `src/ws.ts` - RFC 6455 帧编解码
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@angelky7/dsh-browser-control",
|
|
3
3
|
"description": "Local WebSocket bridge between dsh and the DSH Browser Control browser extension, with model-facing browser_* tools and a Settings-managed on/off switch",
|
|
4
|
-
"version": "1.7.
|
|
4
|
+
"version": "1.7.3",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
|
7
7
|
"types": "lib/types/index.d.ts",
|
|
@@ -37,7 +37,7 @@
|
|
|
37
37
|
"build": "tsc -p tsconfig.json && tsdown",
|
|
38
38
|
"typecheck": "tsc -p tsconfig.json --noEmit",
|
|
39
39
|
"check": "npm run typecheck && npm run build && publint",
|
|
40
|
-
"verify": "node verify/verify_config_patch.mjs && node verify/verify_peers.mjs && node verify/verify_surface.mjs && node verify/verify_tab_groups.mjs && node verify/verify_tabs_close_many.mjs && node verify/verify_cdp_retry.mjs && node verify/verify_snapshot_ax.mjs && node verify/verify_hard_lock.mjs && node verify/verify_policy.mjs && node verify/verify_cookies.mjs && node verify/verify_extension_e2e.mjs && node verify/verify_plugin.mjs legacy && node verify/verify_plugin.mjs conflict && node verify/verify_plugin.mjs new && node verify/verify_http.mjs && node verify/verify_errors.mjs && node verify/verify_hold.mjs && node verify/verify_protocol.mjs && node verify/verify_bundle_http.mjs && node verify/verify_nav_report.mjs && node verify/verify_nav_landing.mjs && node verify/verify_render_contract.mjs && node verify/verify_press_events.mjs && node verify/verify_nav_timeout.mjs && node verify/verify_nav_waituntil.mjs && node verify/verify_archive.mjs && node verify/verify_windows.mjs && node verify/verify_history_nav.mjs && node verify/verify_metrics_throttle.mjs && node verify/verify_compat_matrix.mjs"
|
|
40
|
+
"verify": "node verify/verify_config_patch.mjs && node verify/verify_peers.mjs && node verify/verify_surface.mjs && node verify/verify_tab_groups.mjs && node verify/verify_tabs_close_many.mjs && node verify/verify_cdp_retry.mjs && node verify/verify_snapshot_ax.mjs && node verify/verify_hard_lock.mjs && node verify/verify_policy.mjs && node verify/verify_cookies.mjs && node verify/verify_extension_e2e.mjs && node verify/verify_plugin.mjs legacy && node verify/verify_plugin.mjs conflict && node verify/verify_plugin.mjs new && node verify/verify_http.mjs && node verify/verify_errors.mjs && node verify/verify_hold.mjs && node verify/verify_protocol.mjs && node verify/verify_bundle_http.mjs && node verify/verify_nav_report.mjs && node verify/verify_nav_landing.mjs && node verify/verify_render_contract.mjs && node verify/verify_press_events.mjs && node verify/verify_nav_timeout.mjs && node verify/verify_nav_waituntil.mjs && node verify/verify_archive.mjs && node verify/verify_windows.mjs && node verify/verify_history_nav.mjs && node verify/verify_metrics_throttle.mjs && node verify/verify_drag.mjs && node verify/verify_compat_matrix.mjs"
|
|
41
41
|
},
|
|
42
42
|
"publishConfig": {
|
|
43
43
|
"access": "public",
|