dsh-live-browser 0.0.0-stage → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.en.md +298 -0
- package/README.md +311 -2
- package/client.js +835 -0
- package/cordis.patch.yml +3 -0
- package/lib/agent-script.js +723 -0
- package/lib/bridge.js +336 -0
- package/lib/config.js +84 -0
- package/lib/index.js +98 -0
- package/lib/output.js +12 -0
- package/lib/rpc.js +398 -0
- package/lib/schema.js +52 -0
- package/lib/skills.js +60 -0
- package/lib/tools.js +756 -0
- package/package.json +72 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 dsh-live-browser contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.en.md
ADDED
|
@@ -0,0 +1,298 @@
|
|
|
1
|
+
# dsh-live-browser
|
|
2
|
+
|
|
3
|
+
[](https://dsh-plugin.org/plugins/c347087870/dsh-live-browser)
|
|
4
|
+
|
|
5
|
+
Let the agent drive the browser DSH already ships — inside DSH, with nothing
|
|
6
|
+
leaving the window.
|
|
7
|
+
|
|
8
|
+
[中文](README.md) | English
|
|
9
|
+
|
|
10
|
+
DSH's own `@deepseek-ai/dsh-client-ui-sidebar-browser` renders a real Electron
|
|
11
|
+
`<webview>` in the right sidebar, with an address bar, history and an independent
|
|
12
|
+
browser session. It exposes navigation only, so the agent can open a page but
|
|
13
|
+
cannot click, type, read or scroll it.
|
|
14
|
+
|
|
15
|
+
This plugin closes that gap **without modifying DSH**: it finds the guests that
|
|
16
|
+
plugin mounted, injects a command surface into each document, and gives the model
|
|
17
|
+
25 `browser_*` tools that operate the page the human is looking at. It also fills
|
|
18
|
+
in the tab's right-click menu: **close other tabs / close all tabs / close tabs to
|
|
19
|
+
the right**.
|
|
20
|
+
|
|
21
|
+
## Install
|
|
22
|
+
|
|
23
|
+
Requires DSH 0.2.x and Node.js 20 or newer. The plugin declares its DSH bundle in
|
|
24
|
+
`package.json`, so DSH registers it on install.
|
|
25
|
+
|
|
26
|
+
### From GitHub
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
# Web / CLI profile
|
|
30
|
+
dsh plugin --profile web add github:c347087870/dsh-live-browser
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
The desktop profile is owned by the Electron application, so install it from the
|
|
34
|
+
UI instead: **Plugins → Add plugin**, paste
|
|
35
|
+
`https://github.com/c347087870/dsh-live-browser`, then restart DSH.
|
|
36
|
+
|
|
37
|
+
### From npm
|
|
38
|
+
|
|
39
|
+
No npm package yet; once published the command is
|
|
40
|
+
`dsh plugin --profile web add dsh-live-browser`.
|
|
41
|
+
|
|
42
|
+
### Verify the install
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
node -e "const p=require('./node_modules/dsh-live-browser/package.json');console.log(p.version, p.dsh)"
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
`dsh.bundle.patch` must be present, and `lib/index.js` must exist. After a
|
|
49
|
+
restart the plugin logs
|
|
50
|
+
`[dsh-live-browser] loaded: 25 tools, driving the DSH sidebar browser`.
|
|
51
|
+
|
|
52
|
+
> A junction (`mklink /J`, `ln -s`) does **not** work for local development: Node
|
|
53
|
+
> resolves the real path, so `schemastery` is looked up where the source lives
|
|
54
|
+
> rather than in the profile. Copy the package directory, or install it with
|
|
55
|
+
> `dsh plugin`.
|
|
56
|
+
|
|
57
|
+
## Settings
|
|
58
|
+
|
|
59
|
+
DSH renders these on the plugin's settings card. Every one has a default, so the
|
|
60
|
+
plugin works without touching any of them.
|
|
61
|
+
|
|
62
|
+
| Setting | Default | Meaning |
|
|
63
|
+
|---|---|---|
|
|
64
|
+
| `confirmSensitive` | `true` | Ask the user before an action marked `sensitive` |
|
|
65
|
+
| `allowEval` | `true` | Allow `browser_eval` to run arbitrary JavaScript in the page |
|
|
66
|
+
| `autoOpenPanel` | `true` | Open the sidebar browser when a command arrives with no page |
|
|
67
|
+
| `commandTimeoutMs` | `30000` | How long one command may wait for the panel |
|
|
68
|
+
| `openPanelWaitMs` | `8000` | How long to wait for a new panel to become ready |
|
|
69
|
+
| `maxSnapshotNodes` | `400` | Default node cap for `browser_snapshot` |
|
|
70
|
+
| `maxTextChars` | `12000` | Default character cap for `browser_read_text` |
|
|
71
|
+
| `waitTimeoutMs` | `10000` | Default timeout for `browser_wait` |
|
|
72
|
+
| `uploadMaxBytes` | `16777216` | Largest file `browser_upload` will hand to a page |
|
|
73
|
+
| `imageDir` | empty | Where `browser_screenshot` writes PNGs; empty uses `~/.dsh/storages/dsh-live-browser/images` |
|
|
74
|
+
| `downloadsDir` | empty | Directory `browser_downloads` lists; empty uses the system Downloads folder |
|
|
75
|
+
|
|
76
|
+
`allowEval: false` is the switch to reach for if you want the agent to read pages
|
|
77
|
+
without running code in them. `browser_eval` then returns `executed: false` and
|
|
78
|
+
points at `browser_snapshot` / `browser_read_text` instead.
|
|
79
|
+
|
|
80
|
+
## Opening the browser
|
|
81
|
+
|
|
82
|
+
The agent never asks you to open a browser first, and it never reaches for an
|
|
83
|
+
external window:
|
|
84
|
+
|
|
85
|
+
- Tools queue a command on the host, tagged with the **conversation** that asked.
|
|
86
|
+
- The plugin's client bundle polls continuously, even when no browser tab exists.
|
|
87
|
+
- If a command arrives with no page on screen, the client opens the **DSH sidebar
|
|
88
|
+
browser** itself (the same `openTabIn(sessionId, 'browser', …)` call the sidebar
|
|
89
|
+
uses internally), waits for the guest to mount, then runs the command.
|
|
90
|
+
|
|
91
|
+
So `browser_navigate` on a cold start shows up as a sidebar tab that opens by
|
|
92
|
+
itself and loads the requested page.
|
|
93
|
+
|
|
94
|
+
### One browser per conversation
|
|
95
|
+
|
|
96
|
+
DSH keeps a separate sidebar per Session, and this plugin respects that:
|
|
97
|
+
|
|
98
|
+
- Every command carries the Session id of the chat that issued it
|
|
99
|
+
(`exec.agent.session.id` on the host).
|
|
100
|
+
- A panel only receives commands belonging to its own Session, so a second chat's
|
|
101
|
+
sidebar polling at the same time cannot steal them.
|
|
102
|
+
- A panel only targets the sidebar of the chat **on screen**: background chats keep
|
|
103
|
+
theirs mounted (`keepMounted`), but those are marked `hidden` and are never
|
|
104
|
+
mistaken for the page in front of the human.
|
|
105
|
+
- After switching chats, commands belonging to the old chat are refused with an
|
|
106
|
+
explanation instead of running against a page nobody can see.
|
|
107
|
+
- When the browser has to be opened, it opens in the **requesting** chat via
|
|
108
|
+
`openTabIn(sessionId, …)`, not in whichever chat happens to be on screen.
|
|
109
|
+
|
|
110
|
+
That is what stops the old failure mode: ask for a page in chat A, switch to chat
|
|
111
|
+
B before it opens, and find the browser in B.
|
|
112
|
+
|
|
113
|
+
> Why the renderer does the opening: the sidebar lives entirely in the client —
|
|
114
|
+
> its host module contributes nothing — so there is no host service to call. The
|
|
115
|
+
> client half publishes `sidebarRight` through `ctx.reflect.provide`, which is
|
|
116
|
+
> what this plugin uses.
|
|
117
|
+
|
|
118
|
+
## Why this shape
|
|
119
|
+
|
|
120
|
+
| | This plugin | Patching the shipped bundle |
|
|
121
|
+
|---|---|---|
|
|
122
|
+
| Click / type / read / scroll | yes | yes |
|
|
123
|
+
| Export a page's charts as PNG | yes, via the page's own canvas/SVG | yes |
|
|
124
|
+
| Native window screenshots | no | yes |
|
|
125
|
+
| Survives a DSH upgrade | yes | no — re-patch every release |
|
|
126
|
+
| Risk to existing features | none | can break sidebar link handling |
|
|
127
|
+
|
|
128
|
+
## How it works
|
|
129
|
+
|
|
130
|
+
```
|
|
131
|
+
agent tool call
|
|
132
|
+
│ bridge.call(method, params) lib/bridge.js
|
|
133
|
+
▼
|
|
134
|
+
command queue ──────► panel long-polls client.js
|
|
135
|
+
│
|
|
136
|
+
├─ no guest yet → open the sidebar browser (openTabIn)
|
|
137
|
+
├─ page commands → injected script in the guest
|
|
138
|
+
├─ browser commands → guest element (loadURL/goBack…)
|
|
139
|
+
└─ tab commands → sidebar service
|
|
140
|
+
│
|
|
141
|
+
settle(result)
|
|
142
|
+
▼
|
|
143
|
+
tool result
|
|
144
|
+
|
|
145
|
+
right-click a tab ──► decorate the menu (client.js) ──► close others / all / to the right
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
- **`lib/bridge.js`** — the queue and the long poll. Cold calls stay queued long
|
|
149
|
+
enough for the client to open a tab; a call that never gets a panel fails with
|
|
150
|
+
an actionable message instead of hanging.
|
|
151
|
+
- **`lib/agent-script.js`** — the script injected into every guest document. Pure
|
|
152
|
+
DOM: snapshot with stable `ref`s, click, type, press, select, hover, scroll,
|
|
153
|
+
read text, evaluate, wait, attach files, capture canvas/SVG to PNG.
|
|
154
|
+
- **`lib/rpc.js`** — one prefix route on the DSH web server (`/dsh-live-browser`)
|
|
155
|
+
speaking the panel's small request/response envelope. Every request is checked
|
|
156
|
+
for a same-origin browser and loopback host, because the route can write files
|
|
157
|
+
(`saveImage`) and read them (`readUpload`).
|
|
158
|
+
- **`lib/config.js` / `lib/schema.js`** — the settings values and the form DSH
|
|
159
|
+
renders. Values are dependency-free so the tool layer and tests can read them
|
|
160
|
+
without a `node_modules` tree.
|
|
161
|
+
- **`client.js`** — finds the guest inside **the current conversation's** sidebar
|
|
162
|
+
(`webview[data-sidebar-browser-frame="webview"]`), injects the script, opens the
|
|
163
|
+
tab when needed, and runs the poll loop. Background chats keep their sidebars
|
|
164
|
+
mounted (`keepMounted`) but `hidden`, so they are never mistaken for the one on
|
|
165
|
+
screen; among several tabs it picks the one that is actually laid out, not the
|
|
166
|
+
last mounted. It also decorates the tab context menu: a `contextmenu` listener
|
|
167
|
+
waits for DSH's menu portal and appends the three entries, still without
|
|
168
|
+
touching DSH. IIFE-wrapped: DSH batches plugin bundles into one combo
|
|
169
|
+
`<script>`, so a top-level `const` would collide with a sibling.
|
|
170
|
+
|
|
171
|
+
### Why not `connection.rpc.handle`
|
|
172
|
+
|
|
173
|
+
On dsh 0.2.1-rc.2 that API builds its route with
|
|
174
|
+
`owner.effect(() => owner.webServer.register(route))`, where `owner` is the
|
|
175
|
+
connection plugin's own context — which never declared a `webServer` dependency.
|
|
176
|
+
The property read throws `cannot get property "webServer" without inject`, the
|
|
177
|
+
effect swallows it, and every panel request falls through to the static frontend
|
|
178
|
+
fallback as **HTTP 405**. Registering on `webServer` directly is the same call
|
|
179
|
+
without the broken indirection.
|
|
180
|
+
|
|
181
|
+
## Tools
|
|
182
|
+
|
|
183
|
+
| Tool | Purpose |
|
|
184
|
+
|---|---|
|
|
185
|
+
| `browser_status` | Is the panel open, what page is showing |
|
|
186
|
+
| `browser_open_panel` | Open the sidebar browser (also happens automatically) |
|
|
187
|
+
| `browser_navigate` / `back` / `forward` / `reload` | Navigation |
|
|
188
|
+
| `browser_snapshot` | Accessibility-style node list with stable `ref`s |
|
|
189
|
+
| `browser_read_text` | Visible body text, optionally scoped by selector |
|
|
190
|
+
| `browser_click` / `type` / `press` / `select` / `hover` | Page interaction |
|
|
191
|
+
| `browser_scroll` / `wait` | Scrolling and waiting |
|
|
192
|
+
| `browser_eval` | Run a JS expression in the page |
|
|
193
|
+
| `browser_screenshot` | Export a chart (canvas/SVG) or element as a PNG file |
|
|
194
|
+
| `browser_tabs` / `tab_new` | List tabs, open a new browser tab |
|
|
195
|
+
| `browser_upload` | Attach local files to a file input |
|
|
196
|
+
| `browser_downloads` | List files in the downloads folder |
|
|
197
|
+
| `browser_confirm` | Ask the human to confirm an irreversible action |
|
|
198
|
+
|
|
199
|
+
### Tabs
|
|
200
|
+
|
|
201
|
+
The sidebar keeps its tab ids internal. The plugin acts on the focused guest, or
|
|
202
|
+
on the tab that is actually laid out in the current chat's sidebar when nothing
|
|
203
|
+
is focused: every visited tab stays mounted, so document order is not what the
|
|
204
|
+
human sees. There is no `tab` argument to aim it elsewhere: the parameter used to
|
|
205
|
+
exist and was silently dropped, which let a caller believe it had targeted a
|
|
206
|
+
specific tab. `browser_tabs` still lists what is open.
|
|
207
|
+
|
|
208
|
+
The tab's right-click menu gains three entries: **close other tabs**, **close all
|
|
209
|
+
tabs**, and **close tabs to the right** (everything after this tab; the current
|
|
210
|
+
tab stays, and entries with nothing to close are disabled). Order comes from the
|
|
211
|
+
strip of the pane the menu was opened in, so split panes stay independent; the
|
|
212
|
+
menu itself still belongs to DSH. Floating panes have no strip, so they do not get
|
|
213
|
+
these entries.
|
|
214
|
+
|
|
215
|
+
### Honest limits
|
|
216
|
+
|
|
217
|
+
Three things DSH does not let a plugin do, reported as structured results rather
|
|
218
|
+
than silent failures:
|
|
219
|
+
|
|
220
|
+
- **`browser_tab_close` / `browser_tab_activate`** — the sidebar keeps tab ids
|
|
221
|
+
internal; the tool tells you to click the tab yourself.
|
|
222
|
+
- **`browser_reset_profile`** — the browser session is not exposed for clearing;
|
|
223
|
+
the tool returns step-by-step instructions.
|
|
224
|
+
- **Native window screenshots** — the guest renders natively, so there are no
|
|
225
|
+
pixels at the DOM level. `browser_screenshot` instead asks the page to
|
|
226
|
+
rasterise its own `<canvas>`/`<svg>`, which is exactly what charts are.
|
|
227
|
+
|
|
228
|
+
## Security
|
|
229
|
+
|
|
230
|
+
- The plugin never patches DSH. It reads the DOM the official sidebar browser
|
|
231
|
+
renders and injects a script into guests the official plugin mounted.
|
|
232
|
+
- `browser_eval` runs arbitrary JavaScript in whatever page is loaded. That is the
|
|
233
|
+
point of the tool and the reason `allowEval` exists.
|
|
234
|
+
- Actions marked `sensitive` ask the user first while `confirmSensitive` is on.
|
|
235
|
+
A missing question channel denies rather than allows.
|
|
236
|
+
- The `/dsh-live-browser` route refuses cross-site and non-loopback callers. A
|
|
237
|
+
local process could still spoof the headers, but it can already read and write
|
|
238
|
+
the same files as you.
|
|
239
|
+
|
|
240
|
+
## Tests
|
|
241
|
+
|
|
242
|
+
```bash
|
|
243
|
+
node test/test-all.mjs
|
|
244
|
+
```
|
|
245
|
+
|
|
246
|
+
Thirteen suites. Two groups:
|
|
247
|
+
|
|
248
|
+
- **No browser needed** — package integrity, tool contracts, the tool layer, the
|
|
249
|
+
bridge, the end-to-end path, the client driver, session binding, the tab menu,
|
|
250
|
+
settings, DOM behaviour. These run anywhere.
|
|
251
|
+
- **Need a live page** — scenarios and chart capture. They drive a real page
|
|
252
|
+
through the CDP harness in `test/fixtures/`, which exists **only** for testing;
|
|
253
|
+
the shipped plugin never touches Chrome directly. Start one with
|
|
254
|
+
`node scripts/ensure-browser.mjs`, or let them report `SKIP`.
|
|
255
|
+
|
|
256
|
+
`test/test-rpc-http.mjs` needs a running DSH:
|
|
257
|
+
|
|
258
|
+
```bash
|
|
259
|
+
node test/test-rpc-http.mjs "http://127.0.0.1:PORT/?token=..."
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
Paths for the test browser can be overridden with `DSH_TEST_CDP_PORT`,
|
|
263
|
+
`DSH_TEST_PROFILE_DIR` and `DSH_TEST_DOWNLOAD_DIR`.
|
|
264
|
+
|
|
265
|
+
> After editing the source locally, **DSH must be restarted**: host plugin tool
|
|
266
|
+
> registration happens only at process start, so file changes do not take effect.
|
|
267
|
+
> Until the restart, the `browser_*` tools and the `/dsh-live-browser` route do not
|
|
268
|
+
> exist (an HTTP 405 from that route is the signal that it is not loaded).
|
|
269
|
+
|
|
270
|
+
## Publish
|
|
271
|
+
|
|
272
|
+
The plugin is installable straight from GitHub or npm with no registry step. To
|
|
273
|
+
have it listed in the community plugin markets that index GitHub, the repository
|
|
274
|
+
carries the `dsh-plugin` topic; those indexes rescan on their own schedule.
|
|
275
|
+
|
|
276
|
+
## Changelog
|
|
277
|
+
|
|
278
|
+
### 0.4.0
|
|
279
|
+
|
|
280
|
+
- Fixed: the first sidebar in the document was treated as the current chat's
|
|
281
|
+
sidebar, so the agent could read another chat's / another tab's page. It now
|
|
282
|
+
follows the sidebar of the chat on screen and the tab that is actually shown.
|
|
283
|
+
- Fixed: after switching chats, commands belonging to the old chat are refused
|
|
284
|
+
with an explanation instead of running against a page nobody can see.
|
|
285
|
+
- Fixed: `browser_status` / `browser_open_panel` report the panel of the current
|
|
286
|
+
chat only.
|
|
287
|
+
- Added: tab context menu entries — close other tabs / close all tabs / close tabs
|
|
288
|
+
to the right.
|
|
289
|
+
- Fixed: `autoOpenPanel` was declared but never honoured; turning it off now stops
|
|
290
|
+
the automatic open.
|
|
291
|
+
|
|
292
|
+
### 0.3.0
|
|
293
|
+
|
|
294
|
+
- Cold-start auto-open, per-session binding, tab listing and chart capture.
|
|
295
|
+
|
|
296
|
+
## License
|
|
297
|
+
|
|
298
|
+
MIT
|
package/README.md
CHANGED
|
@@ -1,3 +1,312 @@
|
|
|
1
|
-
#
|
|
1
|
+
# dsh-live-browser
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
[](https://dsh-plugin.org/zh/plugins/c347087870/dsh-live-browser)
|
|
4
|
+
|
|
5
|
+
让 agent 操作 DSH 自带的那个浏览器 —— 就在 DSH 里,什么都不离开窗口。
|
|
6
|
+
|
|
7
|
+
中文 | [English](README.en.md)
|
|
8
|
+
|
|
9
|
+
DSH 自带的 `@deepseek-ai/dsh-client-ui-sidebar-browser` 会在右侧栏渲染一个真正的
|
|
10
|
+
Electron `<webview>`,带地址栏、历史记录和独立的浏览器会话。但它只开放了导航能力:
|
|
11
|
+
agent 能打开页面,却点不了、输不进、读不到、滚不动。
|
|
12
|
+
|
|
13
|
+
这个插件在**不修改 DSH 任何文件**的前提下补上这一段:找到官方插件挂载的那些 guest,
|
|
14
|
+
往每个文档里注入一套操作接口,再给模型 25 个 `browser_*` 工具,操作的正是用户眼前那张页面。
|
|
15
|
+
顺带把标签页右键菜单补全:**关闭其他标签页 / 关闭全部标签页 / 关闭右侧所有标签页**。
|
|
16
|
+
|
|
17
|
+
## 安装
|
|
18
|
+
|
|
19
|
+
需要 DSH 0.2.x 与 Node.js 20 以上。插件在 `package.json` 里声明了 DSH 组合包,安装时
|
|
20
|
+
DSH 会自动注册。
|
|
21
|
+
|
|
22
|
+
### 从 GitHub 安装
|
|
23
|
+
|
|
24
|
+
```bash
|
|
25
|
+
# Web / CLI profile
|
|
26
|
+
dsh plugin --profile web add github:c347087870/dsh-live-browser
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
桌面版的 profile 由 Electron 应用独占,请从界面安装:**插件 → 添加插件**,粘贴
|
|
30
|
+
`https://github.com/c347087870/dsh-live-browser`,然后重启 DSH。
|
|
31
|
+
|
|
32
|
+
### 从 npm 安装
|
|
33
|
+
|
|
34
|
+
npm 包尚未发布;发布后安装命令为 `dsh plugin --profile web add dsh-live-browser`。
|
|
35
|
+
|
|
36
|
+
### 确认装好了
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
node -e "const p=require('./node_modules/dsh-live-browser/package.json');console.log(p.version, p.dsh)"
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
`dsh.bundle.patch` 必须存在,`lib/index.js` 必须存在。重启后日志里会出现
|
|
43
|
+
`[dsh-live-browser] loaded: 25 tools, driving the DSH sidebar browser`。
|
|
44
|
+
|
|
45
|
+
> 本地开发**不要**用 junction(`mklink /J`、`ln -s`):Node 会解析真实路径,于是
|
|
46
|
+
> `schemastery` 去源码目录找而不是去 profile 找。要么复制整个包目录,要么用
|
|
47
|
+
> `dsh plugin` 安装。
|
|
48
|
+
|
|
49
|
+
## 配置项
|
|
50
|
+
|
|
51
|
+
DSH 会把这些渲染在插件的设置卡片里。每一项都有默认值,不动也能用。
|
|
52
|
+
|
|
53
|
+
| 配置 | 默认 | 含义 |
|
|
54
|
+
|---|---|---|
|
|
55
|
+
| `confirmSensitive` | `true` | 标记为 `sensitive` 的动作执行前先问用户 |
|
|
56
|
+
| `allowEval` | `true` | 是否允许 `browser_eval` 在页面里执行任意 JavaScript |
|
|
57
|
+
| `autoOpenPanel` | `true` | 命令到达但没有页面时,自动打开侧边栏浏览器 |
|
|
58
|
+
| `commandTimeoutMs` | `30000` | 单次命令等待面板执行的超时(毫秒) |
|
|
59
|
+
| `openPanelWaitMs` | `8000` | 打开面板后等待其就绪的时间 |
|
|
60
|
+
| `maxSnapshotNodes` | `400` | `browser_snapshot` 默认节点上限 |
|
|
61
|
+
| `maxTextChars` | `12000` | `browser_read_text` 默认字符上限 |
|
|
62
|
+
| `waitTimeoutMs` | `10000` | `browser_wait` 默认超时(毫秒) |
|
|
63
|
+
| `uploadMaxBytes` | `16777216` | `browser_upload` 单个文件的上限 |
|
|
64
|
+
| `imageDir` | 空 | `browser_screenshot` 的图片目录;留空用 `~/.dsh/storages/dsh-live-browser/images` |
|
|
65
|
+
| `downloadsDir` | 空 | `browser_downloads` 读取的目录;留空用系统「下载」文件夹 |
|
|
66
|
+
|
|
67
|
+
想让 agent 只读页面、不在页面里跑代码,就把 `allowEval` 关掉。`browser_eval` 随后
|
|
68
|
+
返回 `executed: false`,并提示改用 `browser_snapshot` / `browser_read_text`。
|
|
69
|
+
|
|
70
|
+
## 浏览器会自己打开
|
|
71
|
+
|
|
72
|
+
agent 不会先让你去开浏览器,也永远不会去开外部窗口:
|
|
73
|
+
|
|
74
|
+
- 工具把命令排进宿主队列,并标记它来自**哪个对话**。
|
|
75
|
+
- 插件的客户端 bundle 一直在轮询,即使当前一个浏览器标签页都没有。
|
|
76
|
+
- 命令到达时如果没有页面,客户端会自己调用侧边栏的
|
|
77
|
+
`openTabIn(sessionId, 'browser', …)` 打开浏览器标签页,等 guest 挂载好再执行。
|
|
78
|
+
|
|
79
|
+
所以冷启动时一句 `browser_navigate`,表现就是右侧栏自己弹出一个标签页并加载目标地址。
|
|
80
|
+
|
|
81
|
+
### 一个对话一个浏览器
|
|
82
|
+
|
|
83
|
+
DSH 每个 Session 一个侧边栏,插件遵守这个边界:
|
|
84
|
+
|
|
85
|
+
- 每条命令都带着发起它的 Session id(宿主侧取 `exec.agent.session.id`)。
|
|
86
|
+
- 面板只收到属于自己 Session 的命令,别的对话的面板同时在轮询也抢不走。
|
|
87
|
+
- 面板只认**屏幕上那个对话**的侧边栏:后台对话的侧边栏还挂在 DOM 里(`keepMounted`),
|
|
88
|
+
但它们被标了 `hidden`,不能被当成你眼前这一页。
|
|
89
|
+
- 切走对话后,属于旧对话的命令会被拒绝并说明原因,而不是在看不见的页面上继续执行。
|
|
90
|
+
- 需要打开浏览器时,用 `openTabIn(sessionId, …)` 在**发起请求的那个对话**里打开,
|
|
91
|
+
而不是在屏幕上恰好显示的那个对话里。
|
|
92
|
+
|
|
93
|
+
这消除了老的故障模式:在 A 对话要页面,没等它出来就切到 B,结果浏览器出现在 B。
|
|
94
|
+
|
|
95
|
+
> 为什么打开动作放在渲染层:侧边栏整个在客户端,它的宿主模块什么都不贡献,所以没有
|
|
96
|
+
> 宿主服务可调。客户端那一半通过 `ctx.reflect.provide` 发布 `sidebarRight`,插件用的就是这个。
|
|
97
|
+
|
|
98
|
+
## 为什么用这个形态
|
|
99
|
+
|
|
100
|
+
| | 本插件 | 直接改官方 bundle |
|
|
101
|
+
|---|---|---|
|
|
102
|
+
| 点击 / 输入 / 读取 / 滚动 | 可以 | 可以 |
|
|
103
|
+
| 把页面图表导成 PNG | 可以,用页面自己的 canvas/SVG | 可以 |
|
|
104
|
+
| 截原生窗口 | 不行 | 可以 |
|
|
105
|
+
| 经得起 DSH 升级 | 可以 | 不行,每个版本都要重打补丁 |
|
|
106
|
+
| 对现有功能的风险 | 无 | 可能破坏侧边栏的链接处理 |
|
|
107
|
+
|
|
108
|
+
## 工作原理
|
|
109
|
+
|
|
110
|
+
```
|
|
111
|
+
工具调用
|
|
112
|
+
│ bridge.call(method, params) lib/bridge.js
|
|
113
|
+
▼
|
|
114
|
+
命令队列 ──────► 面板长轮询 client.js
|
|
115
|
+
│
|
|
116
|
+
├─ 还没有 guest → 打开侧边栏浏览器(openTabIn 本对话)
|
|
117
|
+
├─ 页面命令 → 注入到 guest 里的脚本
|
|
118
|
+
├─ 浏览器命令 → guest 元素本身(loadURL/goBack…)
|
|
119
|
+
└─ 标签页命令 → 侧边栏服务
|
|
120
|
+
│
|
|
121
|
+
settle(result)
|
|
122
|
+
▼
|
|
123
|
+
工具结果
|
|
124
|
+
|
|
125
|
+
右键标签页 ──► 菜单 DOM 装饰(client.js)──► 关闭其他 / 全部 / 右侧
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
- **`lib/bridge.js`** —— 队列与长轮询。冷调用会在队列里留够时间让客户端把标签页开出来;
|
|
129
|
+
始终等不到面板的调用会带着可操作的提示失败,而不是一直挂着。
|
|
130
|
+
- **`lib/agent-script.js`** —— 注入每个 guest 文档的脚本。纯 DOM:带稳定 `ref` 的快照、
|
|
131
|
+
点击、输入、按键、选择、悬停、滚动、读正文、执行表达式、等待、附加文件、canvas/SVG 转 PNG。
|
|
132
|
+
- **`lib/rpc.js`** —— 挂在 DSH web server 上的一个前缀路由(`/dsh-live-browser`),说面板那套
|
|
133
|
+
简单的请求/响应信封。每个请求都校验同源与回环主机,因为这个路由既能写文件(`saveImage`)
|
|
134
|
+
也能读文件(`readUpload`)。
|
|
135
|
+
- **`lib/config.js` / `lib/schema.js`** —— 配置值与 DSH 渲染的表单。取值部分不依赖第三方包,
|
|
136
|
+
所以工具层和测试不需要 `node_modules` 就能跑。
|
|
137
|
+
- **`client.js`** —— 找到**当前对话**那个侧边栏里的
|
|
138
|
+
`webview[data-sidebar-browser-frame="webview"]`,注入脚本,需要时打开标签页,跑轮询循环。
|
|
139
|
+
后台对话的侧边栏还挂在 DOM 里(`keepMounted`),但被标了 `hidden`,插件不会误认;
|
|
140
|
+
同一栏里有多个标签时,挑真正有布局盒子的那个,而不是最后挂载的那个。标签页右键菜单
|
|
141
|
+
也在这里补三项:监听 `contextmenu`,等 DSH 的菜单 portal 出现后追加按钮,仍然不改 DSH。
|
|
142
|
+
整体包在 IIFE 里:DSH 会把多个插件 bundle 塞进同一个 combo `<script>`,顶层 `const`
|
|
143
|
+
会跟邻居撞名。
|
|
144
|
+
|
|
145
|
+
### 为什么不用 `connection.rpc.handle`
|
|
146
|
+
|
|
147
|
+
在 dsh 0.2.1-rc.2 上,这个 API 用
|
|
148
|
+
`owner.effect(() => owner.webServer.register(route))` 建路由,而 `owner` 是 connection
|
|
149
|
+
插件自己的 context —— 它从没声明过 `webServer` 依赖。于是属性读取抛
|
|
150
|
+
`cannot get property "webServer" without inject`,effect 把错误吞掉,面板请求全部落到
|
|
151
|
+
静态前端兜底,表现为 **HTTP 405**。直接在 `webServer` 上注册是同一个调用,只是绕开了
|
|
152
|
+
那层坏掉的间接。
|
|
153
|
+
|
|
154
|
+
## 工具
|
|
155
|
+
|
|
156
|
+
| 工具 | 作用 |
|
|
157
|
+
|---|---|
|
|
158
|
+
| `browser_status` | 面板是否打开、当前是什么页面 |
|
|
159
|
+
| `browser_open_panel` | 打开侧边栏浏览器(也会自动发生) |
|
|
160
|
+
| `browser_navigate` / `back` / `forward` / `reload` | 导航 |
|
|
161
|
+
| `browser_snapshot` | 无障碍风格的节点列表,带稳定 `ref` |
|
|
162
|
+
| `browser_read_text` | 可见正文,可用选择器限定范围 |
|
|
163
|
+
| `browser_click` / `type` / `press` / `select` / `hover` | 页面交互 |
|
|
164
|
+
| `browser_scroll` / `wait` | 滚动与等待 |
|
|
165
|
+
| `browser_eval` | 在页面里执行一段 JS 表达式 |
|
|
166
|
+
| `browser_screenshot` | 把图表(canvas/SVG)或元素导出成 PNG |
|
|
167
|
+
| `browser_tabs` / `tab_new` | 列出标签页、新建标签页 |
|
|
168
|
+
| `browser_upload` | 给文件输入框附加本地文件 |
|
|
169
|
+
| `browser_downloads` | 列出下载目录里的文件 |
|
|
170
|
+
| `browser_confirm` | 请用户确认一个不可撤销的动作 |
|
|
171
|
+
|
|
172
|
+
### 关于标签页
|
|
173
|
+
|
|
174
|
+
侧边栏把标签页 id 留在内部。插件操作的是获得焦点的那个 guest;没有焦点时,用当前对话的
|
|
175
|
+
右侧栏里真正显示着的那个标签页 —— 访问过的标签页都会保活挂在 DOM 里,按挂载顺序取最后一个
|
|
176
|
+
会点到看不见的页面。也没有 `tab` 参数去指别处 —— 这个参数以前存在却被静默丢弃,会让调用方
|
|
177
|
+
误以为自己指定了某个标签页。`browser_tabs` 仍然可以列出当前有哪些。
|
|
178
|
+
|
|
179
|
+
右键标签页的菜单里,插件补了三个动作:**关闭其他标签页**、**关闭全部标签页**、
|
|
180
|
+
**关闭右侧所有标签页**(当前标签右侧的全部,当前标签保留;没得关时置灰)。顺序取自当前
|
|
181
|
+
分栏的标签条,分栏两侧互不影响;菜单本体仍是 DSH 的,插件不接管它。浮动分栏没有标签条,
|
|
182
|
+
因此不提供这组动作。
|
|
183
|
+
|
|
184
|
+
### 说实话的限制
|
|
185
|
+
|
|
186
|
+
有三件事 DSH 不开放给插件,插件以结构化结果如实报告,而不是假装成功:
|
|
187
|
+
|
|
188
|
+
- **`browser_tab_close` / `browser_tab_activate`** —— 侧边栏的标签 id 是内部的,工具会
|
|
189
|
+
请你自己去点标签。
|
|
190
|
+
- **`browser_reset_profile`** —— 浏览器会话没有开放清理接口,工具返回操作步骤。
|
|
191
|
+
- **截原生窗口** —— guest 渲染在原生层,DOM 层面没有像素。`browser_screenshot` 改为让
|
|
192
|
+
页面自己栅格化它的 `<canvas>`/`<svg>`,而这正是图表。
|
|
193
|
+
|
|
194
|
+
## 安全
|
|
195
|
+
|
|
196
|
+
- 插件不改 DSH。它读官方侧边栏浏览器渲染出来的 DOM,往官方插件挂载的 guest 里注入脚本。
|
|
197
|
+
- `browser_eval` 会在当前加载的任何页面里执行任意 JavaScript。这是这个工具存在的意义,
|
|
198
|
+
也是 `allowEval` 存在的原因。
|
|
199
|
+
- `confirmSensitive` 开着时,标记为 `sensitive` 的动作会先问用户;没有确认通道时是拒绝,
|
|
200
|
+
不是放行。
|
|
201
|
+
- `/dsh-live-browser` 路由拒绝跨站与非回环的调用方。本机进程仍能伪造这些头,但它本来就能
|
|
202
|
+
读写和你一样的文件。
|
|
203
|
+
|
|
204
|
+
## 测试
|
|
205
|
+
|
|
206
|
+
```bash
|
|
207
|
+
node test/test-all.mjs
|
|
208
|
+
```
|
|
209
|
+
|
|
210
|
+
十三个套件,分两类:
|
|
211
|
+
|
|
212
|
+
- **不需要浏览器** —— 包完整性、工具契约、工具层、桥接、端到端链路、客户端驱动、
|
|
213
|
+
会话绑定、标签菜单、设置项、DOM 行为。任何环境都能跑,`npm test` 即可。
|
|
214
|
+
- **需要真实页面** —— 场景、注入脚本、图表导出。它们通过 `test/fixtures/` 里的 CDP
|
|
215
|
+
夹具驱动真实页面,这个夹具**只为测试而存在**;实际发的插件从不直接碰 Chrome。
|
|
216
|
+
用 `node scripts/ensure-browser.mjs` 起一个,或者让它们报 `SKIP`。
|
|
217
|
+
|
|
218
|
+
「DOM 行为」和「端到端链路」原本也要真浏览器,现在改成在 jsdom 里跑同一份注入脚本,
|
|
219
|
+
所以不需要 Chrome。图表导出没跟着改,因为 jsdom 的 canvas 要额外装原生包 `canvas`,
|
|
220
|
+
为了跑测试把编译依赖架到使用者机器上不划算。
|
|
221
|
+
|
|
222
|
+
`test/test-rpc-http.mjs` 需要一个正在运行的 DSH:
|
|
223
|
+
|
|
224
|
+
```bash
|
|
225
|
+
node test/test-rpc-http.mjs "http://127.0.0.1:PORT/?token=..."
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
测试浏览器的路径可以用 `DSH_TEST_CDP_PORT`、`DSH_TEST_PROFILE_DIR`、
|
|
229
|
+
`DSH_TEST_DOWNLOAD_DIR` 覆盖。
|
|
230
|
+
|
|
231
|
+
> 本地改源码后,**必须重启 DSH**:宿主插件的工具注册只在进程启动时发生,文件变化
|
|
232
|
+
> 不会热生效。重启前 `browser_*` 工具和 `/dsh-live-browser` 路由都不存在(路由返回
|
|
233
|
+
> HTTP 405 就是没加载的信号)。
|
|
234
|
+
|
|
235
|
+
### 文件不能带 BOM
|
|
236
|
+
|
|
237
|
+
`package.json` 这类文件如果开头混进 UTF-8 BOM,DSH 会报
|
|
238
|
+
|
|
239
|
+
```
|
|
240
|
+
installation rejected: Cannot validate installed package dsh-live-browser:
|
|
241
|
+
SyntaxError: Unexpected token ... is not valid JSON
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
然后把 `package.json`、`pnpm-lock.yaml` 和 `node_modules` **整体回滚**——插件装不上,
|
|
245
|
+
但代码本身毫无问题,现象是「工具和路由都不存在」,极难排查。
|
|
246
|
+
|
|
247
|
+
Windows 上用 PowerShell 的 `Set-Content`、`Out-File` 写文件很容易带上 BOM。仓库里带了
|
|
248
|
+
巡检脚本:
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
npm run strip-bom # 扫掉全仓库文本文件的 BOM
|
|
252
|
+
```
|
|
253
|
+
|
|
254
|
+
`test/test-package-integrity.mjs` 也会检查这一点,并核对 `files` 里声明的每个文件是否
|
|
255
|
+
真实存在、`exports` 指向的入口是否存在、运行时代码有没有 `console.log` 残留。
|
|
256
|
+
|
|
257
|
+
### 在内置浏览器里实测
|
|
258
|
+
|
|
259
|
+
凡是依赖真实渲染的行为,都在 DSH 侧边栏的内置浏览器里跑过一遍。做法是先
|
|
260
|
+
`node scripts/verify-in-browser.mjs --serve` 起本地页面,再用 `browser_*` 工具驱动。
|
|
261
|
+
|
|
262
|
+
结论(实测通过):
|
|
263
|
+
|
|
264
|
+
- **pointer 事件** —— 一次点击产生的序列是
|
|
265
|
+
`pointerover, pointerenter, pointerdown, mousedown, pointerup, mouseup, click`,
|
|
266
|
+
且每个事件恰好一次。只监听 `pointerdown` 的库(拖拽类、React 合成事件)也能收到。
|
|
267
|
+
- **编辑键** —— Backspace 删一个字符、Shift+A 输入大写 A(不是 `+A`)、
|
|
268
|
+
Control+a 清空、Enter 在 textarea 里插入换行。
|
|
269
|
+
- **contenteditable** —— `browser_type` 能写入,编辑键也能作用在可编辑 div 上。
|
|
270
|
+
- **ref 生命周期** —— 连续多次 `browser_snapshot`,ref 数量稳定,不随快照次数增长。
|
|
271
|
+
- **链接跳转** —— 点击 `<a>` 会真实导航,落点页面的标题与正文都变了。
|
|
272
|
+
|
|
273
|
+
这几个行为里有三个曾经是真 bug:pointer 事件根本没派发、编辑键只发 keydown
|
|
274
|
+
不改值、修饰键被模板字符串的反斜杠吞掉导致 `Control+a` 解析成 `+a`。
|
|
275
|
+
|
|
276
|
+
### 开发脚本
|
|
277
|
+
|
|
278
|
+
`scripts/` 下这几个只在开发和维护时用,发布产物里不含它们:
|
|
279
|
+
|
|
280
|
+
| 脚本 | 用途 |
|
|
281
|
+
| --- | --- |
|
|
282
|
+
| `ensure-browser.mjs` | 起一个测试用的 Chrome,供需要真实页面的套件使用 |
|
|
283
|
+
| `strip-bom.mjs` | 扫掉全仓库文本文件的 BOM(见上节,DSH 会因此拒绝安装) |
|
|
284
|
+
| `verify-in-browser.mjs` | 生成内置浏览器实机验证用的页面,`--serve` 起本地服务,`--check` 自检 |
|
|
285
|
+
| `audit-tools.mjs` | 核对「工具 → 方法 → 客户端处理器」这条链有没有断的 |
|
|
286
|
+
| `audit-plugins.mjs` | 扫一遍本地 profile 里装了哪些插件、版本与兼容性 |
|
|
287
|
+
| `wrap-client-iife.mjs` | 把客户端 bundle 包成 IIFE,避免与同页其他插件顶层声明撞名 |
|
|
288
|
+
| `verify-isolation.mjs` | 验证包成 IIFE 后,与别的插件声明混在同一个 script 里仍能解析 |
|
|
289
|
+
|
|
290
|
+
## 发布
|
|
291
|
+
|
|
292
|
+
插件可以直接从 GitHub 或 npm 安装,不需要任何注册步骤。如果要被那些索引 GitHub 的社区
|
|
293
|
+
插件市场收录,仓库带上 `dsh-plugin` topic 即可,它们会按自己的节奏重新扫描。
|
|
294
|
+
|
|
295
|
+
## 更新记录
|
|
296
|
+
|
|
297
|
+
### 0.4.0
|
|
298
|
+
|
|
299
|
+
- 修复:把「文档里第一个右侧栏」当成当前对话的侧边栏,导致 agent 读到别的对话/别的标签页;
|
|
300
|
+
现在只认屏幕上那个对话的侧边栏,并按真正显示的标签取 guest。
|
|
301
|
+
- 修复:切走对话后,属于旧对话的命令会被拒绝并说明原因,不再落到看不见的页面上。
|
|
302
|
+
- 修复:`browser_status` / `browser_open_panel` 按当前对话判断面板,别的对话的面板不再算数。
|
|
303
|
+
- 新增:标签页右键菜单补齐「关闭其他标签页 / 关闭全部标签页 / 关闭右侧所有标签页」。
|
|
304
|
+
- 修复:`autoOpenPanel` 此前只声明未生效,现在关闭后命令不会自动打开面板。
|
|
305
|
+
|
|
306
|
+
### 0.3.0
|
|
307
|
+
|
|
308
|
+
- 冷启动自动打开侧边栏浏览器、会话绑定、标签页列表与截图导出。
|
|
309
|
+
|
|
310
|
+
## 许可
|
|
311
|
+
|
|
312
|
+
MIT
|