ghost-bridge 0.3.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +124 -53
- package/dist/server.js +71 -1
- package/extension/background.js +295 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,18 +1,20 @@
|
|
|
1
|
-
# Ghost Bridge
|
|
1
|
+
# Ghost Bridge
|
|
2
2
|
|
|
3
|
-
>
|
|
3
|
+
> 无侵入 Chrome AI 副驾 —— 通过 MCP 让 AI 无缝接管你正在使用的浏览器,实时调试、观察页面、操控交互,无需启动新浏览器实例。
|
|
4
4
|
|
|
5
|
-
##
|
|
6
|
-
- `server.js`:MCP server,通过 stdio 被 Claude CLI 拉起,与扩展用 WebSocket 通信
|
|
7
|
-
- `extension/`:Chrome MV3 扩展,使用 `chrome.debugger` 附加当前激活标签
|
|
5
|
+
## ✨ 特性
|
|
8
6
|
|
|
9
|
-
|
|
7
|
+
- 🔌 **零配置附加** — 不依赖 `--remote-debugging-port`,通过 Chrome 扩展直接获取 CDP
|
|
8
|
+
- 🔍 **无 sourcemap 调试** — 片段截取、字符串搜索、覆盖率分析,在压缩代码中定位问题
|
|
9
|
+
- 🌐 **网络请求分析** — 完整记录请求/响应,支持多维度过滤和响应体查看
|
|
10
|
+
- 📸 **页面截图与内容提取** — 视觉分析 + 结构化数据提取
|
|
11
|
+
- 🎯 **DOM 交互操控** — AI 可直接点击按钮、填写表单、按键提交,使用 CDP 物理级模拟,兼容 React/Vue/Angular
|
|
12
|
+
- 📊 **性能诊断** — JS 堆内存、DOM 规模、Layout 开销、Web Vitals、资源加载分析
|
|
13
|
+
- 🔄 **多实例支持** — 自动单例管理,多个 MCP 客户端共享同一 Chrome 连接
|
|
10
14
|
|
|
11
|
-
|
|
15
|
+
## 快速开始
|
|
12
16
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
### 1. 安装與初始化
|
|
17
|
+
### 1. 安装与初始化
|
|
16
18
|
|
|
17
19
|
```bash
|
|
18
20
|
# 全局安装
|
|
@@ -25,61 +27,130 @@ ghost-bridge init
|
|
|
25
27
|
### 2. 加载 Chrome 扩展
|
|
26
28
|
|
|
27
29
|
1. 打开 Chrome,访问 `chrome://extensions`
|
|
28
|
-
2.
|
|
29
|
-
3.
|
|
30
|
+
2. 开启右上角的 **开发者模式**
|
|
31
|
+
3. 点击 **加载已解压的扩展程序**
|
|
30
32
|
4. 选择目录:`~/.ghost-bridge/extension`
|
|
31
|
-
> 提示:运行 `ghost-bridge extension --open` 可直接打开该目录
|
|
32
33
|
|
|
33
|
-
|
|
34
|
+
> 💡 运行 `ghost-bridge extension --open` 可直接打开该目录
|
|
35
|
+
|
|
36
|
+
### 3. 连接并使用
|
|
37
|
+
|
|
38
|
+
1. 点击浏览器工具栏中的 Ghost Bridge 图标
|
|
39
|
+
2. 点击 **连接**,等待状态变为 ✅ 已连接
|
|
40
|
+
3. 打开 Claude Desktop 或 Claude CLI,即可使用所有调试工具
|
|
41
|
+
|
|
42
|
+
## CLI 命令
|
|
43
|
+
|
|
44
|
+
| 命令 | 说明 |
|
|
45
|
+
|------|------|
|
|
46
|
+
| `ghost-bridge init` | 配置 MCP 并复制扩展文件 |
|
|
47
|
+
| `ghost-bridge status` | 检查配置状态 |
|
|
48
|
+
| `ghost-bridge extension` | 显示扩展安装路径(`--open` 打开目录) |
|
|
49
|
+
| `ghost-bridge start` | 手动启动 MCP 服务(通常不需要) |
|
|
50
|
+
|
|
51
|
+
## 工具一览
|
|
52
|
+
|
|
53
|
+
### 🔍 基础调试
|
|
54
|
+
|
|
55
|
+
| 工具 | 说明 |
|
|
56
|
+
|------|------|
|
|
57
|
+
| `get_server_info` | 获取服务器状态(端口、连接状态、角色) |
|
|
58
|
+
| `get_last_error` | 汇总最近的异常 / console 错误 / 网络报错,附行列与脚本标识 |
|
|
59
|
+
| `get_script_source` | 抓取目标脚本源码,支持按 URL 片段筛选、指定行列定位、beautify |
|
|
60
|
+
| `coverage_snapshot` | 启动执行覆盖率采集(默认 1.5s),返回最活跃的脚本列表 |
|
|
61
|
+
| `find_by_string` | 在页面脚本源码中按关键词搜索,返回 200 字符上下文窗口 |
|
|
62
|
+
| `symbolic_hints` | 采集资源列表、全局变量 key、localStorage key、UA 与 URL |
|
|
63
|
+
| `eval_script` | 在页面执行 JS 表达式(谨慎使用) |
|
|
64
|
+
|
|
65
|
+
### � 网络请求分析
|
|
66
|
+
|
|
67
|
+
| 工具 | 说明 |
|
|
68
|
+
|------|------|
|
|
69
|
+
| `list_network_requests` | 列出捕获的网络请求,支持按 URL / 方法 / 状态 / 资源类型过滤 |
|
|
70
|
+
| `get_network_detail` | 获取单个请求的详细信息(请求头、响应头、timing),可选获取响应体 |
|
|
71
|
+
| `clear_network_requests` | 清空已捕获的网络请求记录 |
|
|
72
|
+
|
|
73
|
+
### 📸 页面内容
|
|
74
|
+
|
|
75
|
+
| 工具 | 说明 |
|
|
76
|
+
|------|------|
|
|
77
|
+
| `capture_screenshot` | 截取页面截图(支持完整长截图、指定区域、JPEG/PNG 格式) |
|
|
78
|
+
| `get_page_content` | 提取页面内容:纯文本 / HTML / 结构化数据(标题、链接、按钮、表单) |
|
|
79
|
+
|
|
80
|
+
### 🎯 页面交互(DOM 操作)
|
|
34
81
|
|
|
35
|
-
|
|
82
|
+
| 工具 | 说明 |
|
|
83
|
+
|------|------|
|
|
84
|
+
| `get_interactive_snapshot` | 扫描页面所有可见可交互元素,返回带 ref 短标识(e1, e2...)的精简列表,支持 Shadow DOM 穿透 |
|
|
85
|
+
| `dispatch_action` | 对目标元素执行动作(click / fill / press / scroll / select / hover / focus),使用 CDP 物理级模拟 |
|
|
36
86
|
|
|
37
|
-
|
|
87
|
+
**交互工作流示例:**
|
|
38
88
|
|
|
39
|
-
|
|
89
|
+
```
|
|
90
|
+
1. AI 调用 get_interactive_snapshot
|
|
91
|
+
→ 返回: [{ref:"e1", tag:"input", placeholder:"Search..."}, {ref:"e2", tag:"button", text:"Login"}]
|
|
92
|
+
|
|
93
|
+
2. AI 调用 dispatch_action({ref: "e1", action: "fill", value: "hello"})
|
|
94
|
+
→ 在搜索框中输入 "hello"
|
|
40
95
|
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
96
|
+
3. AI 调用 dispatch_action({ref: "e2", action: "click"})
|
|
97
|
+
→ 点击 Login 按钮
|
|
98
|
+
|
|
99
|
+
4. AI 调用 capture_screenshot 验证操作结果
|
|
100
|
+
```
|
|
45
101
|
|
|
46
|
-
|
|
47
|
-
- 端口:`3301`(`server.js` / `extension/background.js`)
|
|
48
|
-
- token:`1`(仅用于本机 WS 校验,如需修改请保持两端一致)
|
|
102
|
+
### 📊 性能分析
|
|
49
103
|
|
|
50
|
-
|
|
104
|
+
| 工具 | 说明 |
|
|
105
|
+
|------|------|
|
|
106
|
+
| `perf_metrics` | 获取页面性能指标,包含三层数据:|
|
|
51
107
|
|
|
52
|
-
|
|
53
|
-
- **get_last_error**:汇总最近异常/console/网络报错,附带行列与脚本标识
|
|
54
|
-
- **get_script_source**:支持 `scriptUrlContains`、`line`、`column`,返回源码片段(无 sourcemap 仍可用)
|
|
55
|
-
- **coverage_snapshot**:默认 1.5s,输出调用次数最高的脚本
|
|
56
|
-
- **find_by_string**:在脚本源码里按关键词搜索,返回上下文 200 字符窗口
|
|
57
|
-
- **symbolic_hints**:采集资源列表、全局变量 key、localStorage key、UA 与 URL
|
|
58
|
-
- **eval_script**:只读表达式执行;谨慎使用,避免改写页面状态
|
|
108
|
+
**`perf_metrics` 返回的数据:**
|
|
59
109
|
|
|
60
|
-
|
|
61
|
-
- **
|
|
62
|
-
|
|
63
|
-
- 支持按请求方法过滤(`method`: GET/POST/PUT/DELETE)
|
|
64
|
-
- 支持按状态过滤(`status`: success/error/failed/pending)
|
|
65
|
-
- 支持按资源类型过滤(`resourceType`: XHR/Fetch/Script/Image)
|
|
66
|
-
- 返回:URL、方法、状态码、耗时、大小等摘要信息
|
|
110
|
+
- **引擎级指标** — JS 堆内存(使用量/总量/占比)、DOM 节点数、事件监听器数、Layout 重排次数与耗时、脚本执行时间
|
|
111
|
+
- **Web Vitals** — Navigation Timing 各阶段(DNS / TTFB / DOM Interactive / Load)、FP、FCP、Long Tasks 统计
|
|
112
|
+
- **资源加载摘要** — 按类型分组统计(count / size / avgDuration)、最慢资源识别
|
|
67
113
|
|
|
68
|
-
|
|
69
|
-
- 请求头和响应头
|
|
70
|
-
- 请求方法、状态码、MIME 类型
|
|
71
|
-
- 耗时分析(timing)
|
|
72
|
-
- 可选获取响应体(`includeBody: true`)
|
|
114
|
+
## 配置
|
|
73
115
|
|
|
74
|
-
|
|
116
|
+
| 项目 | 默认值 | 说明 |
|
|
117
|
+
|------|--------|------|
|
|
118
|
+
| 端口 | `33333` | WebSocket 服务端口,自动递增寻找可用端口 |
|
|
119
|
+
| Token | 当月自动生成 | 本机 WS 校验,基于当月 1 号时间戳 |
|
|
120
|
+
| 自动 Detach | `false` | 保持附加,便于持续捕获异常和网络请求 |
|
|
75
121
|
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
-
|
|
79
|
-
-
|
|
80
|
-
|
|
122
|
+
环境变量:
|
|
123
|
+
|
|
124
|
+
- `GHOST_BRIDGE_PORT` — 自定义基础端口
|
|
125
|
+
- `GHOST_BRIDGE_TOKEN` — 自定义 token
|
|
126
|
+
|
|
127
|
+
## 架构
|
|
128
|
+
|
|
129
|
+
```
|
|
130
|
+
┌──────────────┐ stdio ┌──────────────┐ WebSocket ┌──────────────┐
|
|
131
|
+
│ Claude CLI │ ◄────────────► │ MCP Server │ ◄──────────────►│Chrome Extension│
|
|
132
|
+
│ / Desktop │ │ (server.js) │ │(background.js)│
|
|
133
|
+
└──────────────┘ └──────────────┘ └──────┬───────┘
|
|
134
|
+
│ CDP
|
|
135
|
+
┌─────▼──────┐
|
|
136
|
+
│ 浏览器页面 │
|
|
137
|
+
└────────────┘
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
- **MCP Server** — 通过 stdio 被 Claude 拉起,与扩展通过 WebSocket 通信
|
|
141
|
+
- **Chrome Extension (MV3)** — 使用 `chrome.debugger` API 附加页面,Offscreen Document 维持 WebSocket 长连接
|
|
142
|
+
- **单例模式** — 多个 MCP 客户端自动协调,首个实例为主服务,后续实例作为客户端连接
|
|
81
143
|
|
|
82
144
|
## 已知限制
|
|
83
|
-
|
|
84
|
-
-
|
|
85
|
-
-
|
|
145
|
+
|
|
146
|
+
- 扩展 Service Worker 可能被挂起,已内置重连策略;若长时间无流量需重新唤醒
|
|
147
|
+
- 若目标页面已打开 DevTools,`chrome.debugger.attach` 可能失败,请关闭后重试
|
|
148
|
+
- 大体积单行 bundle beautify 可能耗时,服务端对超长源码会截取片段
|
|
149
|
+
- 跨月时 token 会自动更新,扩展和服务端需在同月内启动
|
|
150
|
+
- DOM 交互:SPA 路由变化后 ref 标识会失效,需重新调用 `get_interactive_snapshot`
|
|
151
|
+
- DOM 交互:跨域 iframe 内的元素暂不支持细粒度操作
|
|
152
|
+
- DOM 交互:Shadow DOM 内的元素可以被扫描到,但部分封闭模式的 Shadow Root 可能无法穿透
|
|
153
|
+
|
|
154
|
+
## License
|
|
155
|
+
|
|
156
|
+
MIT
|
package/dist/server.js
CHANGED
|
@@ -28896,7 +28896,7 @@ function buildSnippet(source, line, column, { beautifyEnabled = true, contextLin
|
|
|
28896
28896
|
return result;
|
|
28897
28897
|
}
|
|
28898
28898
|
var server = new Server(
|
|
28899
|
-
{ name: "ghost-bridge", version: "0.
|
|
28899
|
+
{ name: "ghost-bridge", version: "0.4.0" },
|
|
28900
28900
|
{ capabilities: { tools: {} } }
|
|
28901
28901
|
);
|
|
28902
28902
|
server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
|
@@ -29067,6 +29067,66 @@ server.setRequestHandler(ListToolsRequestSchema, async () => ({
|
|
|
29067
29067
|
}
|
|
29068
29068
|
}
|
|
29069
29069
|
}
|
|
29070
|
+
},
|
|
29071
|
+
{
|
|
29072
|
+
name: "get_interactive_snapshot",
|
|
29073
|
+
description: "\u3010\u64CD\u4F5C\u9875\u9762\u524D\u5FC5\u987B\u5148\u8C03\u7528\u3011\u626B\u63CF\u5F53\u524D\u9875\u9762\u6240\u6709\u53EF\u89C1\u7684\u53EF\u4EA4\u4E92\u5143\u7D20\uFF08\u6309\u94AE/\u94FE\u63A5/\u8F93\u5165\u6846/\u4E0B\u62C9\u6846\u7B49\uFF09\uFF0C\u8FD4\u56DE\u5E26\u6709 ref \u77ED\u6807\u8BC6\uFF08\u5982 e1, e2, e3\uFF09\u7684\u7CBE\u7B80\u5217\u8868\uFF0C\u5305\u542B\u5143\u7D20\u7C7B\u578B\u3001\u6587\u672C\u548C\u4F4D\u7F6E\u3002Token \u6781\u7701\uFF08\u901A\u5E38 < 1000 tokens\uFF09\uFF0C\u4E13\u4E3A AI \u64CD\u4F5C\u9875\u9762\u800C\u8BBE\u8BA1\u3002\u83B7\u53D6\u540E\u53EF\u901A\u8FC7 dispatch_action \u5DE5\u5177\u4F7F\u7528 ref \u6807\u8BC6\u6765\u70B9\u51FB\u3001\u586B\u5199\u3001\u6309\u952E\u7B49\u3002\u652F\u6301 Shadow DOM \u7A7F\u900F\u3002\n\u26A0\uFE0F \u4EC5\u7528\u4E8E\u4EA4\u4E92\u64CD\u4F5C\u524D\u7684\u5143\u7D20\u5B9A\u4F4D\u3002\u5982\u9700\u6392\u67E5 UI/CSS \u5E03\u5C40\u95EE\u9898\uFF0C\u8BF7\u4F7F\u7528 capture_screenshot \u6216 get_page_content\u3002",
|
|
29074
|
+
inputSchema: {
|
|
29075
|
+
type: "object",
|
|
29076
|
+
properties: {
|
|
29077
|
+
selector: {
|
|
29078
|
+
type: "string",
|
|
29079
|
+
description: "CSS \u9009\u62E9\u5668\uFF0C\u9650\u5B9A\u626B\u63CF\u8303\u56F4\u3002\u4E0D\u6307\u5B9A\u5219\u626B\u63CF\u6574\u4E2A\u9875\u9762"
|
|
29080
|
+
},
|
|
29081
|
+
includeText: {
|
|
29082
|
+
type: "boolean",
|
|
29083
|
+
description: "\u662F\u5426\u5305\u542B\u5143\u7D20\u7684\u6587\u672C/\u5360\u4F4D\u7B26\u7B49\u4FE1\u606F\uFF0C\u9ED8\u8BA4 true"
|
|
29084
|
+
},
|
|
29085
|
+
maxElements: {
|
|
29086
|
+
type: "number",
|
|
29087
|
+
description: "\u6700\u5927\u8FD4\u56DE\u5143\u7D20\u6570\u91CF\uFF0C\u9ED8\u8BA4 100"
|
|
29088
|
+
}
|
|
29089
|
+
}
|
|
29090
|
+
}
|
|
29091
|
+
},
|
|
29092
|
+
{
|
|
29093
|
+
name: "dispatch_action",
|
|
29094
|
+
description: "\u3010\u64CD\u4F5C\u9875\u9762\u5143\u7D20\u3011\u5BF9 get_interactive_snapshot \u8FD4\u56DE\u7684\u5143\u7D20\u6267\u884C\u52A8\u4F5C\u3002\u901A\u8FC7 ref \u6807\u8BC6\uFF08\u5982 e1, e5\uFF09\u7CBE\u51C6\u5B9A\u4F4D\u5143\u7D20\uFF0C\u4F7F\u7528 CDP \u7269\u7406\u7EA7\u6A21\u62DF\u6267\u884C\u64CD\u4F5C\uFF0C\u517C\u5BB9\u6240\u6709\u524D\u7AEF\u6846\u67B6\uFF08React/Vue/Angular\uFF09\uFF0C\u6210\u529F\u7387\u6781\u9AD8\u3002\n\u652F\u6301\u7684\u52A8\u4F5C\uFF1Aclick\uFF08\u70B9\u51FB\uFF09\u3001fill\uFF08\u586B\u5199\u8F93\u5165\u6846\uFF09\u3001press\uFF08\u6309\u952E\u5982 Enter\uFF09\u3001scroll\uFF08\u6EDA\u52A8\uFF09\u3001select\uFF08\u4E0B\u62C9\u9009\u62E9\uFF09\u3001hover\uFF08\u60AC\u505C\uFF09\u3001focus\uFF08\u805A\u7126\uFF09\u3002\n\u26A0\uFE0F \u4F7F\u7528\u524D\u5FC5\u987B\u5148\u8C03\u7528 get_interactive_snapshot \u83B7\u53D6\u5143\u7D20\u5217\u8868\u3002\u64CD\u4F5C\u540E\u5EFA\u8BAE\u7528 capture_screenshot \u6216\u518D\u6B21 get_interactive_snapshot \u9A8C\u8BC1\u7ED3\u679C\u3002",
|
|
29095
|
+
inputSchema: {
|
|
29096
|
+
type: "object",
|
|
29097
|
+
properties: {
|
|
29098
|
+
ref: {
|
|
29099
|
+
type: "string",
|
|
29100
|
+
description: "\u76EE\u6807\u5143\u7D20\u7684 ref \u6807\u8BC6\uFF0C\u5982 'e1'\u3001'e5'\uFF08\u4ECE get_interactive_snapshot \u83B7\u53D6\uFF09"
|
|
29101
|
+
},
|
|
29102
|
+
action: {
|
|
29103
|
+
type: "string",
|
|
29104
|
+
enum: ["click", "fill", "press", "scroll", "select", "hover", "focus"],
|
|
29105
|
+
description: "\u8981\u6267\u884C\u7684\u52A8\u4F5C\u7C7B\u578B"
|
|
29106
|
+
},
|
|
29107
|
+
value: {
|
|
29108
|
+
type: "string",
|
|
29109
|
+
description: "fill \u65F6\u4E3A\u8981\u8F93\u5165\u7684\u6587\u672C\uFF1Bselect \u65F6\u4E3A\u8981\u9009\u62E9\u7684 option value\uFF1Bpress \u65F6\u4E3A\u6309\u952E\u540D\uFF08\u53EF\u9009\uFF09"
|
|
29110
|
+
},
|
|
29111
|
+
key: {
|
|
29112
|
+
type: "string",
|
|
29113
|
+
description: "press \u52A8\u4F5C\u7684\u6309\u952E\u540D\uFF0C\u5982 'Enter'\u3001'Escape'\u3001'Tab'\u3001'Backspace'\u3002\u9ED8\u8BA4 'Enter'"
|
|
29114
|
+
},
|
|
29115
|
+
deltaX: {
|
|
29116
|
+
type: "number",
|
|
29117
|
+
description: "scroll \u52A8\u4F5C\u7684\u6C34\u5E73\u6EDA\u52A8\u91CF\uFF08\u50CF\u7D20\uFF09\uFF0C\u9ED8\u8BA4 0"
|
|
29118
|
+
},
|
|
29119
|
+
deltaY: {
|
|
29120
|
+
type: "number",
|
|
29121
|
+
description: "scroll \u52A8\u4F5C\u7684\u5782\u76F4\u6EDA\u52A8\u91CF\uFF08\u50CF\u7D20\uFF09\uFF0C\u9ED8\u8BA4 300\uFF08\u6B63\u6570\u5411\u4E0B\uFF0C\u8D1F\u6570\u5411\u4E0A\uFF09"
|
|
29122
|
+
},
|
|
29123
|
+
waitMs: {
|
|
29124
|
+
type: "number",
|
|
29125
|
+
description: "\u64CD\u4F5C\u540E\u7B49\u5F85\u9875\u9762\u54CD\u5E94\u7684\u65F6\u95F4\uFF08\u6BEB\u79D2\uFF09\uFF0C\u9ED8\u8BA4 500\uFF0C\u6700\u5927 3000"
|
|
29126
|
+
}
|
|
29127
|
+
},
|
|
29128
|
+
required: ["ref", "action"]
|
|
29129
|
+
}
|
|
29070
29130
|
}
|
|
29071
29131
|
]
|
|
29072
29132
|
}));
|
|
@@ -29223,6 +29283,16 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
|
|
|
29223
29283
|
const res = await askChrome("getPageContent", { mode, selector, maxLength, includeMetadata });
|
|
29224
29284
|
return { content: [{ type: "text", text: jsonText(res) }] };
|
|
29225
29285
|
}
|
|
29286
|
+
if (name === "get_interactive_snapshot") {
|
|
29287
|
+
const { selector, includeText, maxElements } = args;
|
|
29288
|
+
const res = await askChrome("getInteractiveSnapshot", { selector, includeText, maxElements });
|
|
29289
|
+
return { content: [{ type: "text", text: jsonText(res) }] };
|
|
29290
|
+
}
|
|
29291
|
+
if (name === "dispatch_action") {
|
|
29292
|
+
const { ref, action, value, key, deltaX, deltaY, waitMs } = args;
|
|
29293
|
+
const res = await askChrome("dispatchAction", { ref, action, value, key, deltaX, deltaY, waitMs }, { timeoutMs: 1e4 });
|
|
29294
|
+
return { content: [{ type: "text", text: jsonText(res) }] };
|
|
29295
|
+
}
|
|
29226
29296
|
return { content: [{ type: "text", text: `\u672A\u77E5\u5DE5\u5177\uFF1A${name}` }] };
|
|
29227
29297
|
} catch (e) {
|
|
29228
29298
|
return { content: [{ type: "text", text: `Error: ${e.message}` }] };
|
package/extension/background.js
CHANGED
|
@@ -912,6 +912,299 @@ async function handleGetPageContent(params = {}) {
|
|
|
912
912
|
return result?.value
|
|
913
913
|
}
|
|
914
914
|
|
|
915
|
+
// ========== DOM 交互:可交互元素快照 ==========
|
|
916
|
+
|
|
917
|
+
async function handleGetInteractiveSnapshot(params = {}) {
|
|
918
|
+
const target = await ensureAttached()
|
|
919
|
+
const { selector, includeText = true, maxElements = 100 } = params
|
|
920
|
+
|
|
921
|
+
const selectorStr = selector ? JSON.stringify(selector) : 'null'
|
|
922
|
+
|
|
923
|
+
const expression = `(function() {
|
|
924
|
+
try {
|
|
925
|
+
let refCounter = 0;
|
|
926
|
+
const elements = [];
|
|
927
|
+
|
|
928
|
+
// 判断元素是否可见
|
|
929
|
+
function isVisible(el) {
|
|
930
|
+
if (!el.offsetParent && el.tagName !== 'HTML' && el.tagName !== 'BODY' &&
|
|
931
|
+
window.getComputedStyle(el).position !== 'fixed' &&
|
|
932
|
+
window.getComputedStyle(el).position !== 'sticky') return false;
|
|
933
|
+
const style = window.getComputedStyle(el);
|
|
934
|
+
if (style.display === 'none' || style.visibility === 'hidden' || parseFloat(style.opacity) === 0) return false;
|
|
935
|
+
const rect = el.getBoundingClientRect();
|
|
936
|
+
if (rect.width === 0 && rect.height === 0) return false;
|
|
937
|
+
return true;
|
|
938
|
+
}
|
|
939
|
+
|
|
940
|
+
// 判断元素是否可交互
|
|
941
|
+
function isInteractive(el) {
|
|
942
|
+
const tag = el.tagName.toLowerCase();
|
|
943
|
+
if (['a', 'button', 'input', 'select', 'textarea'].includes(tag)) return true;
|
|
944
|
+
if (el.getAttribute('role') === 'button' || el.getAttribute('role') === 'link' ||
|
|
945
|
+
el.getAttribute('role') === 'tab' || el.getAttribute('role') === 'menuitem' ||
|
|
946
|
+
el.getAttribute('role') === 'checkbox' || el.getAttribute('role') === 'radio' ||
|
|
947
|
+
el.getAttribute('role') === 'switch' || el.getAttribute('role') === 'combobox') return true;
|
|
948
|
+
if (el.getAttribute('tabindex') && parseInt(el.getAttribute('tabindex')) >= 0) return true;
|
|
949
|
+
if (el.getAttribute('contenteditable') === 'true') return true;
|
|
950
|
+
if (el.onclick || el.getAttribute('onclick')) return true;
|
|
951
|
+
return false;
|
|
952
|
+
}
|
|
953
|
+
|
|
954
|
+
// 递归遍历 DOM 树(含 Shadow DOM)
|
|
955
|
+
function walkDOM(node, root) {
|
|
956
|
+
if (elements.length >= ${maxElements}) return;
|
|
957
|
+
const children = node.children || [];
|
|
958
|
+
for (let i = 0; i < children.length; i++) {
|
|
959
|
+
if (elements.length >= ${maxElements}) return;
|
|
960
|
+
const el = children[i];
|
|
961
|
+
if (isInteractive(el) && isVisible(el)) {
|
|
962
|
+
refCounter++;
|
|
963
|
+
const ref = 'e' + refCounter;
|
|
964
|
+
el.setAttribute('data-ghost-ref', ref);
|
|
965
|
+
const tag = el.tagName.toLowerCase();
|
|
966
|
+
const rect = el.getBoundingClientRect();
|
|
967
|
+
const entry = {
|
|
968
|
+
ref: ref,
|
|
969
|
+
tag: tag,
|
|
970
|
+
cx: Math.round(rect.left + rect.width / 2),
|
|
971
|
+
cy: Math.round(rect.top + rect.height / 2),
|
|
972
|
+
};
|
|
973
|
+
// 类型信息
|
|
974
|
+
if (el.type) entry.type = el.type;
|
|
975
|
+
if (el.name) entry.name = el.name;
|
|
976
|
+
if (el.getAttribute('role')) entry.role = el.getAttribute('role');
|
|
977
|
+
// 文本信息
|
|
978
|
+
if (${includeText}) {
|
|
979
|
+
if (el.placeholder) entry.placeholder = el.placeholder.slice(0, 80);
|
|
980
|
+
if (el.value && tag !== 'textarea') entry.value = el.value.slice(0, 80);
|
|
981
|
+
if (tag === 'a') entry.href = (el.href || '').slice(0, 150);
|
|
982
|
+
if (tag === 'select') {
|
|
983
|
+
entry.options = Array.from(el.options).slice(0, 10).map(o => ({
|
|
984
|
+
value: o.value, text: o.text.slice(0, 50), selected: o.selected
|
|
985
|
+
}));
|
|
986
|
+
}
|
|
987
|
+
const text = (el.innerText || el.textContent || el.getAttribute('aria-label') || '').trim();
|
|
988
|
+
if (text && text.length <= 100) entry.text = text;
|
|
989
|
+
else if (text) entry.text = text.slice(0, 97) + '...';
|
|
990
|
+
}
|
|
991
|
+
// disabled 状态
|
|
992
|
+
if (el.disabled) entry.disabled = true;
|
|
993
|
+
elements.push(entry);
|
|
994
|
+
}
|
|
995
|
+
// 递归子节点
|
|
996
|
+
walkDOM(el, root);
|
|
997
|
+
// 穿透 Shadow DOM
|
|
998
|
+
if (el.shadowRoot) {
|
|
999
|
+
walkDOM(el.shadowRoot, root);
|
|
1000
|
+
}
|
|
1001
|
+
}
|
|
1002
|
+
}
|
|
1003
|
+
|
|
1004
|
+
// 清理旧的 ref 标记
|
|
1005
|
+
document.querySelectorAll('[data-ghost-ref]').forEach(el => el.removeAttribute('data-ghost-ref'));
|
|
1006
|
+
|
|
1007
|
+
// 确定扫描根节点
|
|
1008
|
+
let rootEl = document.body;
|
|
1009
|
+
const sel = ${selectorStr};
|
|
1010
|
+
if (sel) {
|
|
1011
|
+
rootEl = document.querySelector(sel);
|
|
1012
|
+
if (!rootEl) return { error: '选择器未匹配到任何元素', selector: sel };
|
|
1013
|
+
}
|
|
1014
|
+
|
|
1015
|
+
walkDOM(rootEl, rootEl);
|
|
1016
|
+
|
|
1017
|
+
return {
|
|
1018
|
+
url: window.location.href,
|
|
1019
|
+
title: document.title,
|
|
1020
|
+
elementCount: elements.length,
|
|
1021
|
+
viewport: {
|
|
1022
|
+
width: window.innerWidth,
|
|
1023
|
+
height: window.innerHeight,
|
|
1024
|
+
scrollX: Math.round(window.scrollX),
|
|
1025
|
+
scrollY: Math.round(window.scrollY),
|
|
1026
|
+
},
|
|
1027
|
+
elements: elements,
|
|
1028
|
+
};
|
|
1029
|
+
} catch (e) { return { error: e.message }; }
|
|
1030
|
+
})()`
|
|
1031
|
+
|
|
1032
|
+
const { result } = await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
|
|
1033
|
+
expression,
|
|
1034
|
+
returnByValue: true,
|
|
1035
|
+
})
|
|
1036
|
+
|
|
1037
|
+
if (result?.value?.error) throw new Error(result.value.error)
|
|
1038
|
+
return result?.value
|
|
1039
|
+
}
|
|
1040
|
+
|
|
1041
|
+
// ========== DOM 交互:动作分发器 ==========
|
|
1042
|
+
|
|
1043
|
+
async function handleDispatchAction(params = {}) {
|
|
1044
|
+
const target = await ensureAttached()
|
|
1045
|
+
const { ref, action, value, key, deltaX, deltaY, waitMs = 500 } = params
|
|
1046
|
+
|
|
1047
|
+
if (!ref) throw new Error("需要提供 ref(元素标识,如 'e1')")
|
|
1048
|
+
if (!action) throw new Error("需要提供 action(动作类型:click/fill/press/scroll/select/hover/focus)")
|
|
1049
|
+
|
|
1050
|
+
// Step 1: 实时获取目标元素的最新坐标和状态
|
|
1051
|
+
const locateExpression = `(function() {
|
|
1052
|
+
try {
|
|
1053
|
+
const el = document.querySelector('[data-ghost-ref="${ref}"]');
|
|
1054
|
+
if (!el) return { error: '元素未找到,ref 可能已失效,请重新获取快照' };
|
|
1055
|
+
const rect = el.getBoundingClientRect();
|
|
1056
|
+
if (rect.width === 0 && rect.height === 0) return { error: '元素不可见(宽高为 0)' };
|
|
1057
|
+
return {
|
|
1058
|
+
found: true,
|
|
1059
|
+
tag: el.tagName.toLowerCase(),
|
|
1060
|
+
type: el.type || '',
|
|
1061
|
+
cx: Math.round(rect.left + rect.width / 2),
|
|
1062
|
+
cy: Math.round(rect.top + rect.height / 2),
|
|
1063
|
+
disabled: el.disabled || false,
|
|
1064
|
+
value: (el.value || '').slice(0, 100),
|
|
1065
|
+
};
|
|
1066
|
+
} catch (e) { return { error: e.message }; }
|
|
1067
|
+
})()`
|
|
1068
|
+
|
|
1069
|
+
const { result: locResult } = await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
|
|
1070
|
+
expression: locateExpression,
|
|
1071
|
+
returnByValue: true,
|
|
1072
|
+
})
|
|
1073
|
+
|
|
1074
|
+
const loc = locResult?.value
|
|
1075
|
+
if (!loc || loc.error) throw new Error(loc?.error || "无法定位元素")
|
|
1076
|
+
if (loc.disabled) throw new Error(`元素 ${ref} 已被禁用 (disabled)`)
|
|
1077
|
+
|
|
1078
|
+
const cx = loc.cx
|
|
1079
|
+
const cy = loc.cy
|
|
1080
|
+
|
|
1081
|
+
let actionResult = { ref, action, success: true }
|
|
1082
|
+
|
|
1083
|
+
// Step 2: 根据动作类型执行 CDP 命令
|
|
1084
|
+
if (action === "click") {
|
|
1085
|
+
// 物理级 CDP 鼠标点击
|
|
1086
|
+
await chrome.debugger.sendCommand(target, "Input.dispatchMouseEvent", {
|
|
1087
|
+
type: "mousePressed", x: cx, y: cy, button: "left", clickCount: 1,
|
|
1088
|
+
})
|
|
1089
|
+
await chrome.debugger.sendCommand(target, "Input.dispatchMouseEvent", {
|
|
1090
|
+
type: "mouseReleased", x: cx, y: cy, button: "left", clickCount: 1,
|
|
1091
|
+
})
|
|
1092
|
+
actionResult.detail = `已点击 ${ref} (${loc.tag}) 坐标 (${cx}, ${cy})`
|
|
1093
|
+
|
|
1094
|
+
} else if (action === "fill") {
|
|
1095
|
+
if (value === undefined || value === null) throw new Error("fill 动作需要提供 value 参数")
|
|
1096
|
+
// 先点击聚焦
|
|
1097
|
+
await chrome.debugger.sendCommand(target, "Input.dispatchMouseEvent", {
|
|
1098
|
+
type: "mousePressed", x: cx, y: cy, button: "left", clickCount: 1,
|
|
1099
|
+
})
|
|
1100
|
+
await chrome.debugger.sendCommand(target, "Input.dispatchMouseEvent", {
|
|
1101
|
+
type: "mouseReleased", x: cx, y: cy, button: "left", clickCount: 1,
|
|
1102
|
+
})
|
|
1103
|
+
// 全选并清空已有内容
|
|
1104
|
+
await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
|
|
1105
|
+
expression: `(function() {
|
|
1106
|
+
const el = document.querySelector('[data-ghost-ref="${ref}"]');
|
|
1107
|
+
if (el) { el.focus(); el.select && el.select(); }
|
|
1108
|
+
})()`,
|
|
1109
|
+
})
|
|
1110
|
+
// 用 CDP 模拟键盘输入
|
|
1111
|
+
await chrome.debugger.sendCommand(target, "Input.insertText", {
|
|
1112
|
+
text: String(value),
|
|
1113
|
+
})
|
|
1114
|
+
// 强制触发 input/change 事件(兼容 React/Vue)
|
|
1115
|
+
await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
|
|
1116
|
+
expression: `(function() {
|
|
1117
|
+
const el = document.querySelector('[data-ghost-ref="${ref}"]');
|
|
1118
|
+
if (el) {
|
|
1119
|
+
el.dispatchEvent(new Event('input', { bubbles: true }));
|
|
1120
|
+
el.dispatchEvent(new Event('change', { bubbles: true }));
|
|
1121
|
+
}
|
|
1122
|
+
})()`,
|
|
1123
|
+
})
|
|
1124
|
+
actionResult.detail = `已在 ${ref} (${loc.tag}) 中填入 "${String(value).slice(0, 50)}"`
|
|
1125
|
+
|
|
1126
|
+
} else if (action === "press") {
|
|
1127
|
+
// 模拟键盘按键
|
|
1128
|
+
const keyName = key || value || "Enter"
|
|
1129
|
+
// 先确保元素聚焦
|
|
1130
|
+
await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
|
|
1131
|
+
expression: `(function() {
|
|
1132
|
+
const el = document.querySelector('[data-ghost-ref="${ref}"]');
|
|
1133
|
+
if (el) el.focus();
|
|
1134
|
+
})()`,
|
|
1135
|
+
})
|
|
1136
|
+
await chrome.debugger.sendCommand(target, "Input.dispatchKeyEvent", {
|
|
1137
|
+
type: "keyDown", key: keyName,
|
|
1138
|
+
})
|
|
1139
|
+
await chrome.debugger.sendCommand(target, "Input.dispatchKeyEvent", {
|
|
1140
|
+
type: "keyUp", key: keyName,
|
|
1141
|
+
})
|
|
1142
|
+
actionResult.detail = `已在 ${ref} 上按下 ${keyName}`
|
|
1143
|
+
|
|
1144
|
+
} else if (action === "scroll") {
|
|
1145
|
+
const dx = deltaX || 0
|
|
1146
|
+
const dy = deltaY || 300
|
|
1147
|
+
await chrome.debugger.sendCommand(target, "Input.dispatchMouseEvent", {
|
|
1148
|
+
type: "mouseWheel", x: cx, y: cy, deltaX: dx, deltaY: dy,
|
|
1149
|
+
})
|
|
1150
|
+
actionResult.detail = `已在 ${ref} 位置滚动 (${dx}, ${dy})`
|
|
1151
|
+
|
|
1152
|
+
} else if (action === "select") {
|
|
1153
|
+
// 下拉框选择
|
|
1154
|
+
if (value === undefined) throw new Error("select 动作需要提供 value 参数")
|
|
1155
|
+
await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
|
|
1156
|
+
expression: `(function() {
|
|
1157
|
+
const el = document.querySelector('[data-ghost-ref="${ref}"]');
|
|
1158
|
+
if (el && el.tagName === 'SELECT') {
|
|
1159
|
+
el.value = ${JSON.stringify(String(value))};
|
|
1160
|
+
el.dispatchEvent(new Event('change', { bubbles: true }));
|
|
1161
|
+
}
|
|
1162
|
+
})()`,
|
|
1163
|
+
})
|
|
1164
|
+
actionResult.detail = `已在 ${ref} 选择值 "${value}"`
|
|
1165
|
+
|
|
1166
|
+
} else if (action === "hover") {
|
|
1167
|
+
await chrome.debugger.sendCommand(target, "Input.dispatchMouseEvent", {
|
|
1168
|
+
type: "mouseMoved", x: cx, y: cy,
|
|
1169
|
+
})
|
|
1170
|
+
actionResult.detail = `已将鼠标悬停到 ${ref} (${cx}, ${cy})`
|
|
1171
|
+
|
|
1172
|
+
} else if (action === "focus") {
|
|
1173
|
+
await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
|
|
1174
|
+
expression: `(function() {
|
|
1175
|
+
const el = document.querySelector('[data-ghost-ref="${ref}"]');
|
|
1176
|
+
if (el) el.focus();
|
|
1177
|
+
})()`,
|
|
1178
|
+
})
|
|
1179
|
+
actionResult.detail = `已聚焦到 ${ref}`
|
|
1180
|
+
|
|
1181
|
+
} else {
|
|
1182
|
+
throw new Error(`不支持的动作类型: ${action},可选: click/fill/press/scroll/select/hover/focus`)
|
|
1183
|
+
}
|
|
1184
|
+
|
|
1185
|
+
// Step 3: 等待页面响应
|
|
1186
|
+
if (waitMs > 0) {
|
|
1187
|
+
await sleep(Math.min(waitMs, 3000))
|
|
1188
|
+
}
|
|
1189
|
+
|
|
1190
|
+
// Step 4: 获取操作后状态摘要
|
|
1191
|
+
const { result: afterResult } = await chrome.debugger.sendCommand(target, "Runtime.evaluate", {
|
|
1192
|
+
expression: `(function() {
|
|
1193
|
+
return {
|
|
1194
|
+
url: window.location.href,
|
|
1195
|
+
title: document.title,
|
|
1196
|
+
readyState: document.readyState,
|
|
1197
|
+
};
|
|
1198
|
+
})()`,
|
|
1199
|
+
returnByValue: true,
|
|
1200
|
+
})
|
|
1201
|
+
if (afterResult?.value) {
|
|
1202
|
+
actionResult.pageAfter = afterResult.value
|
|
1203
|
+
}
|
|
1204
|
+
|
|
1205
|
+
return actionResult
|
|
1206
|
+
}
|
|
1207
|
+
|
|
915
1208
|
// 处理来自服务器的命令
|
|
916
1209
|
async function handleCommand(message) {
|
|
917
1210
|
const { id, command, params, token } = message
|
|
@@ -938,6 +1231,8 @@ async function handleCommand(message) {
|
|
|
938
1231
|
else if (command === "perfMetrics") result = await handlePerfMetrics(params)
|
|
939
1232
|
else if (command === "captureScreenshot") result = await handleCaptureScreenshot(params)
|
|
940
1233
|
else if (command === "getPageContent") result = await handleGetPageContent(params)
|
|
1234
|
+
else if (command === "getInteractiveSnapshot") result = await handleGetInteractiveSnapshot(params)
|
|
1235
|
+
else if (command === "dispatchAction") result = await handleDispatchAction(params)
|
|
941
1236
|
else throw new Error(`未知指令 ${command}`)
|
|
942
1237
|
|
|
943
1238
|
sendToServer({ id, result })
|