dsh-builtin-browser 0.1.23 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +91 -3
- package/README.md +98 -4
- package/client.js +334 -0
- package/cordis.patch.yml +24 -5
- package/desktop-bridge/install.mjs +98 -0
- package/desktop-bridge/plugin-browser-bridge.js +385 -0
- package/docs/README.md +1 -1
- package/docs/architecture.md +1 -1
- package/docs/tool-reference.md +2 -1
- package/lib/browser/runtime.d.ts +18 -1
- package/lib/browser/runtime.js +19 -0
- package/lib/browser/types.d.ts +33 -0
- package/lib/browser-electron/bridge-connection.d.ts +58 -0
- package/lib/browser-electron/bridge-connection.js +133 -0
- package/lib/browser-electron/desktop-bridge-host.d.ts +124 -0
- package/lib/browser-electron/desktop-bridge-host.js +297 -0
- package/lib/browser-electron/entry.js +195 -11
- package/lib/browser-electron/history-store.d.ts +75 -0
- package/lib/browser-electron/history-store.js +162 -0
- package/lib/browser-electron/host-main.js +28 -1
- package/lib/browser-electron/provider.d.ts +122 -1
- package/lib/browser-electron/provider.js +233 -21
- package/lib/browser-electron/remote-host.d.ts +17 -1
- package/lib/browser-electron/remote-host.js +81 -9
- package/lib/browser-electron/settings-store.d.ts +92 -0
- package/lib/browser-electron/settings-store.js +183 -0
- package/lib/browser-electron/system-browser.d.ts +100 -0
- package/lib/browser-electron/system-browser.js +322 -0
- package/lib/browser-electron/virtual-cursor.d.ts +67 -0
- package/lib/browser-electron/virtual-cursor.js +141 -0
- package/lib/index.d.ts +16 -2
- package/lib/index.js +18 -2
- package/lib/tool-browser/index.js +122 -16
- package/package.json +14 -1
- package/src/browser/runtime.ts +22 -0
- package/src/browser/types.ts +31 -0
- package/src/browser-electron/bridge-connection.ts +145 -0
- package/src/browser-electron/desktop-bridge-host.ts +305 -0
- package/src/browser-electron/entry.ts +217 -11
- package/src/browser-electron/history-store.ts +171 -0
- package/src/browser-electron/host-main.ts +26 -1
- package/src/browser-electron/provider.ts +284 -21
- package/src/browser-electron/remote-host.ts +84 -7
- package/src/browser-electron/settings-store.ts +227 -0
- package/src/browser-electron/system-browser.ts +318 -0
- package/src/browser-electron/virtual-cursor.ts +152 -0
- package/src/index.ts +20 -2
- package/src/tool-browser/index.ts +121 -15
package/README.en.md
CHANGED
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
| --- | --- |
|
|
19
19
|
| Why a shared real browser, and how it differs from headless approaches | [Why a shared real browser](docs/why-browser.md) |
|
|
20
20
|
| Installation, configuration, day-to-day use | [User guide](docs/user-guide.md) |
|
|
21
|
-
| All
|
|
21
|
+
| All 34 tools: parameters, output, examples | [Tool reference](docs/tool-reference.md) |
|
|
22
22
|
| How the seam / provider / tools layers and self-hosting work | [Architecture](docs/architecture.md) |
|
|
23
23
|
| Documentation index and README split | [Docs index](docs/README.md) |
|
|
24
24
|
|
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
|
|
29
29
|
- **A real view, not a relay**: the browser is a native `WebContentsView`; the human sees every step the agent takes and can grab control at any time;
|
|
30
30
|
- **Install-and-use**: with a desktop shell the shell's embedded view is used; on plain `dsh web` the plugin **self-hosts** — it spawns its own Electron window with zero extra configuration;
|
|
31
|
-
- **One plugin, one toolset**: after install the agent automatically gets
|
|
31
|
+
- **One plugin, one toolset**: after install the agent automatically gets 34 `browser_*` tools (open, a11y tree, wait, semantic/coordinate interaction, scroll, back/forward, batch and single-control form filling, keys, structured scraping, screenshot, download, auth management…).
|
|
32
32
|
|
|
33
33
|
In one sentence: **installing the plugin gives you a real browser that is shared with the user and drivable by the agent.**
|
|
34
34
|
|
|
@@ -117,6 +117,26 @@ See the full list in [Tool reference](#tool-reference).
|
|
|
117
117
|
<p><code>browser_screenshot</code> supports <code>savePath</code> to write the PNG to disk, ready for vision models (modlens etc.) to locate elements visually.</p>
|
|
118
118
|
</td>
|
|
119
119
|
</tr>
|
|
120
|
+
<tr>
|
|
121
|
+
<td width="50%" valign="top">
|
|
122
|
+
<h3>Searchable browsing history</h3>
|
|
123
|
+
<p><code>browser_visited</code> reads the <b>persistent</b> record of visited pages (stored beside the browser profile, so it survives closing the browser and restarting DSH), filterable by domain and reopenable. It is a different thing from <code>browser_history</code>, which is the session's operation log.</p>
|
|
124
|
+
</td>
|
|
125
|
+
<td width="50%" valign="top">
|
|
126
|
+
<h3>Synthetic cursor</h3>
|
|
127
|
+
<p>While the agent works, a virtual pointer and click ripple are drawn inside the page — <b>its appearance means the agent has taken over that tab</b>. DOM-level actions (set value, check, select) also show a landing point. Can be switched off in settings.</p>
|
|
128
|
+
</td>
|
|
129
|
+
</tr>
|
|
130
|
+
<tr>
|
|
131
|
+
<td width="50%" valign="top">
|
|
132
|
+
<h3>A "Browser" section in Settings</h3>
|
|
133
|
+
<p>Keep history / keep cookies / auto-expand the side panel / close the browser when a session ends / show the cursor / vision strategy / allow credential reads — switches take effect <b>immediately</b>, no restart.</p>
|
|
134
|
+
</td>
|
|
135
|
+
<td width="50%" valign="top">
|
|
136
|
+
<h3>Unambiguous teardown</h3>
|
|
137
|
+
<p><b>Closing the browser window ends that session</b> (the next open is a clean one) while browsing history and login state survive; <b>collapsing the interface is only a collapse</b> and the browser keeps running in the background.</p>
|
|
138
|
+
</td>
|
|
139
|
+
</tr>
|
|
120
140
|
</table>
|
|
121
141
|
|
|
122
142
|
## Why this plugin
|
|
@@ -135,6 +155,7 @@ See the full list in [Tool reference](#tool-reference).
|
|
|
135
155
|
| `browser_snapshot` | Numbered inventory of interactive elements (inputs/buttons/links; pierces same-origin iframes and Shadow DOM) | – |
|
|
136
156
|
| `browser_a11y` | Accessibility tree: semantic role/name/value/states + coordinates per interactive node (pierces same-origin iframes and Shadow DOM) | – |
|
|
137
157
|
| `browser_execute` | Run JS in the page; args arrive as `arguments[0..n]` | ✅ |
|
|
158
|
+
| `browser_visited` | Read the persistent browsing history (visited pages, filterable by domain / capped); reopen with `browser_open` | – |
|
|
138
159
|
| `browser_content` | Fetch the page as html / markdown / txt / json (selector, maxChars, timeoutMs) | – |
|
|
139
160
|
| `browser_click` | Click a semantic target (`target`: css/text/xpath, scrolls into view and clicks center) or viewport coordinates (vision-located) | ✅ |
|
|
140
161
|
| `browser_type` | Type text (optionally focusing a `target` element first; CDP `Input.insertText`) | ✅ |
|
|
@@ -221,7 +242,52 @@ agent (browser_* tools)
|
|
|
221
242
|
|
|
222
243
|
## Division of labor with the desktop shell
|
|
223
244
|
|
|
224
|
-
|
|
245
|
+
There are three carriers, and the plugin picks one automatically — **no configuration**:
|
|
246
|
+
|
|
247
|
+
**① Desktop: drive the official sidebar's page (one page for both parties)**
|
|
248
|
+
|
|
249
|
+
DSH Desktop is two layers: an Electron shell plus an `--expose-internals` **Node-mode host** (the plugin runs there, with **no Electron API**). 0.2 removed `electronViewHost`, and the host/shell event set carries nothing view-related either — so the plugin borrows a **small bridge**: a loopback + token service inside the shell's main process hands the plugin CDP access to the sidebar browser's guest, i.e. the very page you see on screen.
|
|
250
|
+
|
|
251
|
+
The result: **the page the agent works on is the page the human looks at**. The plugin no longer spawns its own Electron and no second window appears.
|
|
252
|
+
|
|
253
|
+
Install that bridge (it modifies an **installed** desktop app, so it is replayable):
|
|
254
|
+
|
|
255
|
+
```bash
|
|
256
|
+
node desktop-bridge/install.mjs # idempotent; backs up main.js.before-bridge on first run
|
|
257
|
+
node desktop-bridge/install.mjs --revert # roll back
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
> **Re-run `install.mjs` after every desktop upgrade** — the upgrade replaces `resources/app/` and takes the bridge with it. With no bridge the plugin falls back to self-hosting: nothing breaks, you just get a separate window again.
|
|
261
|
+
|
|
262
|
+
> **⚠️ What the sandbox boundary change means**
|
|
263
|
+
>
|
|
264
|
+
> The official sidebar browser is built on the premise that its pages are **not readable from outside** — it uses a separate partition and the shell refuses cross-site content access. Letting the agent drive that guest **deliberately breaks that premise**:
|
|
265
|
+
>
|
|
266
|
+
> - the agent can read the content of **any page you open in the sidebar** (that is precisely what "one page for both parties" means);
|
|
267
|
+
> - the agent can read the **cookies and login state** in that partition, and `browser_auth` can export them (controlled by a setting);
|
|
268
|
+
> - your actions in the sidebar and the agent's actions act on **the same page** and can affect each other (the agent will not overwrite what you are typing, but navigation changes what you both see).
|
|
269
|
+
>
|
|
270
|
+
> That is the inherent cost of one shared page. We think it is worth it — it turns "the agent is doing something in a window you cannot see" into "you can watch it work and take over" — but you are entitled to know it exists, so there are switches: with **credential access** off the agent stops reading cookies and login state, and with the **vision strategy** set to non-visual any coordinate click that depends on a screenshot is refused. If you would rather not accept the boundary change at all, removing the plugin from the desktop profile returns you to the old separate-window shape.
|
|
271
|
+
|
|
272
|
+
**② Use the browser you already have (Chrome / Edge)**
|
|
273
|
+
|
|
274
|
+
The settings panel can point the plugin at an **installed Chrome or Edge** (`browser.channel`: `bundled` / `auto` / `chrome` / `edge`). The approach is the same one Codex Browser Use takes: launch it with `--remote-debugging-port=0`, read the port it writes into `DevToolsActivePort`, and drive it entirely over CDP (through Node 22's built-in `WebSocket` — **no new dependency**).
|
|
275
|
+
|
|
276
|
+
**Your data is not touched**: the plugin launches it with a **separate profile** (`$DSH_HOME/dsh-builtin-browser-host/<chrome|edge>-profile`). Your everyday windows, bookmarks and logins are never opened, locked or modified, and closing the plugin never closes your browser.
|
|
277
|
+
|
|
278
|
+
**What happens to login state**:
|
|
279
|
+
|
|
280
|
+
- `cookies.persist` **on** (default) → that fixed profile is kept, so **you stay signed in across DSH restarts**, and `browser_auth` can still export/restore its cookies.
|
|
281
|
+
- `cookies.persist` **off** → a **throwaway profile** each time, deleted when the browser is released; no login trace is left behind.
|
|
282
|
+
- The trade-off, stated plainly: a separate profile **does not see** the sites you are signed into in your everyday browser. Sign in once in the window the plugin opens and the session stays in its own profile.
|
|
283
|
+
|
|
284
|
+
**③ A shell that provides `electronViewHost`** (older desktop shells): that view is used directly.
|
|
285
|
+
|
|
286
|
+
**④ No shell at all (plain `dsh web`)**: self-hosted — the plugin spawns the Electron it ships.
|
|
287
|
+
|
|
288
|
+
> The visible view and column layout always belong to the host shell; the plugin owns the seam, the provider and the tools. Across all carriers the **toolset, browsing history, settings panel, synthetic cursor and teardown rules are identical** — only the carrier of the page differs.
|
|
289
|
+
>
|
|
290
|
+
> **Precedence**: an explicitly chosen installed browser > the desktop sidebar > self-hosting. If the chosen browser is **missing or fails to start**, a warning is logged and the bundled browser is kept — it is **never silently swapped** for something else.
|
|
225
291
|
|
|
226
292
|
## Requirements
|
|
227
293
|
|
|
@@ -243,6 +309,25 @@ The browser's **visible view**, the **browser column layout**, and the **column-
|
|
|
243
309
|
|
|
244
310
|
> The plugin declares `electron >= 30`; it has **only been verified on Windows** (macOS/Linux untested, not yet promised).
|
|
245
311
|
|
|
312
|
+
## Updating (the two hosts differ)
|
|
313
|
+
|
|
314
|
+
The plugin has one installation per host, and the two are updated separately — **updating one does not update the other**.
|
|
315
|
+
|
|
316
|
+
**Desktop (DSH Desktop)**
|
|
317
|
+
- The plugin is a dependency of the desktop profile (usually `$DSH_HOME/profiles/desktop`). Updating means moving that dependency to the new version and then **restarting DSH Desktop**, which is when the settings panel and the tools pick up the new code.
|
|
318
|
+
- The browser engine is the desktop app's own Electron; the plugin never downloads a second copy.
|
|
319
|
+
- Upgrading the desktop app itself does not carry the plugin along; update it as described above.
|
|
320
|
+
|
|
321
|
+
**Web (`dsh web`)**
|
|
322
|
+
- The plugin is a dependency of the web profile (`$DSH_HOME/profiles/web`); **restart `dsh web`** after updating.
|
|
323
|
+
- There is no desktop shell, so the shared browser is self-hosted by the plugin: the first install may need an Electron binary. If the package manager's build allow-list blocked it (pnpm v10+ blocks `electron`'s postinstall), run `npx install-electron` once to fetch it.
|
|
324
|
+
- Update it the same way you installed it (npm package `dsh-builtin-browser`, the GitHub repo `wqty123/dsh-browser`, or a local directory).
|
|
325
|
+
|
|
326
|
+
**What is the same on both**
|
|
327
|
+
- The toolset (34 `browser_*` tools), the "Browser" section in Settings, and how browsing history and cookies persist are identical; only the carrier of the browser window differs (desktop shell vs. plugin-hosted).
|
|
328
|
+
- Upgrading loses no data: history and settings live in `$DSH_HOME/dsh-builtin-browser-host/` (`history.jsonl`, `settings.json`), and login state sits in the same profile directory.
|
|
329
|
+
- If history behaves unexpectedly after an upgrade, check **Settings → Browser**: history defaults to **on**, one-time auto-expand defaults to **on**, and closing the browser when a session ends defaults to **off**.
|
|
330
|
+
|
|
246
331
|
## Known limitations
|
|
247
332
|
|
|
248
333
|
- JPEG screenshots are available only on the self-hosted native path (`capturePage` `toJPEG`); the desktop shell's CDP fallback stays PNG (CDP JPEG hangs on Electron 43).
|
|
@@ -314,6 +399,9 @@ Code layout:
|
|
|
314
399
|
|
|
315
400
|
| 19 | 2026-10-01 | **DSH 0.2 compatibility**: DSH moved to the 0.2 line (`@deepseek-ai/dsh@0.2.0-rc.2`, with `dsh-llm`/`dsh-tools`/`dsh-system-prompt` following to `0.2.0-rc.2`), while our declaration `>=0.1.1-rc.1 <0.2.0` shut 0.2 out → verified the plugin's (narrow) runtime dependency surface against a **real 0.2.0-rc.2 host** (`cordis` Context/Service, `dsh-tools` defineTool, `dsh-llm` HarnessError, `schemastery`) and found no breaking change: session, navigation, snapshot and the screenshot trio (outside-path refused / legal write / overwrite refused) all pass → peer ranges for the three dsh packages widened to `>=0.1.1-rc.2 <0.3.0`, `dsh.compatibility.dsh` widened to `>=0.1.1-rc.1 <0.3.0`, and `dshReleases` gained `0.2.0-rc.1`/`0.2.0-rc.2` = compatible |
|
|
316
401
|
| **0.1.23** | 2026-10-01 | **Release**: round 18 (PR #15: the Windows synthesized-input trio + Electron probe self-healing + locate verdicts) and round 19 (DSH 0.2 compatibility) ship as **0.1.23** (build clean, **47/47 tests pass**, tag `v0.1.23`) |
|
|
402
|
+
| Round 20 | 2026-10-01 | **Browsing history / settings panel / synthetic cursor / teardown**: (1) **persistent browsing history** (`history-store`: append-only JSONL beside the browser profile; capped at 5000 entries or 90 days, whichever comes first; a damaged line loses only itself) plus the new **`browser_visited`** tool (tool count **33 → 34**), reopening via `browser_open`; (2) a "Browser" section in Settings (hand-written client bundle registered into `settings.section` with `order: 60`, below the host's own rows) served by `GET/PUT /dsh-builtin-browser/settings` (same-origin guard, 64 KiB cap) over a `settings-store` document (per-field validation, unknown keys dropped, malformed file falls back to defaults) — **switches take effect immediately** because the provider reads the document on every use; (3) an in-page **synthetic cursor** (inline styles + Web Animations, so a page's `style-src` CSP cannot drop it; `buildTargetScript` now attaches the element centre as `__point`, which gives click / type / setValue / check / select / clear **a landing point**) — **its appearance means the agent has taken over that tab**; (4) **closing a window ends its session**: on `closed` the host releases every view's `webContents` (a BrowserWindow does not destroy child views, so each window would otherwise leak a renderer) and reports `viewClosed`; the provider ends that session and the seam gains `exists()` so the tool layer re-checks a cached session — the next call gets a **clean session** while browsing history and login state survive. 21 new tests (**68/68 pass**) |
|
|
403
|
+
| Round 20 addendum | 2026-10-01 | **Root cause of the invisible settings panel, plus crash diagnostics**: (1) On a real host the panel did not appear — the cause is that all three `cordis.patch.yml` rows were registered under **subpath** names (`dsh-builtin-browser/browser`), while the host's client-module scan resolves a row's specifier to a **package root** (`exactPackageSpecifier` returns `undefined` as soon as it sees a `/`), leaving the package with no row to read its `dsh.client` from. Fixed by adding an inert root row under the bare package name and giving the root entry `export const name` plus an empty `apply()`. Measured: boot entries 67 → **68**, the panel appears below "规则设定", and switches persist immediately. (2) Three host-log defects fixed: ISO timestamps, both paths plus their existence recorded before spawning, and a 2 MiB rotation that leaves a dated marker instead of wiping the file (which is how weeks of history vanished); the `exit` line gained `pid=` and `entryExists=` so "present at spawn, absent at exit" names an installation replaced underneath a running host. (3) Test pollution of the real log fixed: three spawn-based tests restored `DSH_HOME` in a `finally` while `dispose()` kills children asynchronously, so their synthetic exit lines landed in the operator's real log — now isolated at module scope. **70/70 tests** |
|
|
404
|
+
| **0.2.0** | 2026-10-01 | **Release**: round 20 (persistent browsing history / the "Browser" settings section / synthetic cursor / teardown semantics) and its addendum ship as **0.2.0** — tool count **33 → 34** (new `browser_visited`), plus a new "Browser" section in Settings. Build clean, **70/70 tests pass**, tag `v0.2.0` |
|
|
317
405
|
|
|
318
406
|
> The npm badge at the top is the authority on the registry's latest version: `0.1.23` is committed and tagged; if the badge still reads `0.1.22`, that version is not published yet.
|
|
319
407
|
|
package/README.md
CHANGED
|
@@ -18,7 +18,7 @@
|
|
|
18
18
|
| --- | --- |
|
|
19
19
|
| 了解插件为什么存在、与无头方案的区别 | [为什么做共享真实浏览器](docs/why-browser.md) |
|
|
20
20
|
| 安装、配置与日常使用 | [用户指南](docs/user-guide.md) |
|
|
21
|
-
| 全部
|
|
21
|
+
| 全部 34 个工具的参数、输出与示例 | [工具参考](docs/tool-reference.md) |
|
|
22
22
|
| 了解 seam / provider / 工具三层与自托管实现 | [架构说明](docs/architecture.md) |
|
|
23
23
|
| 查看全部文档与 README 分工 | [文档索引](docs/README.md) |
|
|
24
24
|
|
|
@@ -28,7 +28,7 @@
|
|
|
28
28
|
|
|
29
29
|
- **真实视图,而非转播**:浏览器是原生 `WebContentsView`,用户直接看到 agent 在做什么,随时可以上手接管;
|
|
30
30
|
- **装好即用**:有桌面外壳时嵌入外壳视图;纯 `dsh web` 也能**自托管**——插件自己拉起一个 Electron 窗口,不需要任何额外配置;
|
|
31
|
-
- **一插件即一套工具**:安装后 agent 自动获得
|
|
31
|
+
- **一插件即一套工具**:安装后 agent 自动获得 34 个 `browser_*` 工具(打开、查看、无障碍树、等待、语义/坐标操作、滚动、回退、批量/单控件填表、按键、结构化提取、截图、下载、登录态管理……)。
|
|
32
32
|
|
|
33
33
|
一句话:**安装插件 = 获得一个与用户共享、可被 agent 驱动的真实浏览器。**
|
|
34
34
|
|
|
@@ -117,6 +117,26 @@ dsh plugin --profile web add <本仓库路径>
|
|
|
117
117
|
<p><code>browser_screenshot</code> 支持 <code>savePath</code> 直接落盘 PNG,交给视觉模型(modlens 等)做基于视觉的元素定位。</p>
|
|
118
118
|
</td>
|
|
119
119
|
</tr>
|
|
120
|
+
<tr>
|
|
121
|
+
<td width="50%" valign="top">
|
|
122
|
+
<h3>可查的浏览历史</h3>
|
|
123
|
+
<p><code>browser_visited</code> 读取<b>持久化</b>的访问记录(与 cookie 同址落盘,关闭浏览器、重启 DSH 后仍在),可按域名过滤并重新打开任意一条。它与 <code>browser_history</code>(会话内的操作日志)是两件事。</p>
|
|
124
|
+
</td>
|
|
125
|
+
<td width="50%" valign="top">
|
|
126
|
+
<h3>可视化鼠标</h3>
|
|
127
|
+
<p>agent 操作时在页面内绘制虚拟光标与点击涟漪 —— <b>光标出现即表示它已接管该标签页</b>;DOM 级操作(填值/勾选/选择)同样有落点。可在设置里关闭。</p>
|
|
128
|
+
</td>
|
|
129
|
+
</tr>
|
|
130
|
+
<tr>
|
|
131
|
+
<td width="50%" valign="top">
|
|
132
|
+
<h3>设置页里的「浏览器」栏</h3>
|
|
133
|
+
<p>历史与 cookie 是否保留、侧栏是否自动展开、会话结束时是否关闭浏览器、是否显示光标、视觉策略、是否允许读取凭据 —— 开关<b>即时生效</b>,无需重启。</p>
|
|
134
|
+
</td>
|
|
135
|
+
<td width="50%" valign="top">
|
|
136
|
+
<h3>收尾明确</h3>
|
|
137
|
+
<p><b>关掉浏览器窗口 = 结束该会话</b>(下次打开是干净的新会话),浏览历史与登录状态保留;<b>收起界面只是收起</b>,浏览器继续在后台运行。</p>
|
|
138
|
+
</td>
|
|
139
|
+
</tr>
|
|
120
140
|
</table>
|
|
121
141
|
|
|
122
142
|
## 为什么选它
|
|
@@ -135,6 +155,7 @@ dsh plugin --profile web add <本仓库路径>
|
|
|
135
155
|
| `browser_snapshot` | 交互元素(输入框/按钮/链接)带编号清单(穿透同源 iframe 与 Shadow DOM) | – |
|
|
136
156
|
| `browser_a11y` | 无障碍树:每个交互节点的语义角色/名称/值/状态 + 坐标(穿透同源 iframe 与 Shadow DOM) | – |
|
|
137
157
|
| `browser_execute` | 在页面执行 JS;参数以 `arguments[0..n]` 传入 | ✅ |
|
|
158
|
+
| `browser_visited` | 读取持久化浏览历史(访问过的页面,可按域名过滤/限量);重开用 `browser_open` | – |
|
|
138
159
|
| `browser_content` | 以 html / markdown / txt / json 抓取页面(selector、maxChars、timeoutMs) | – |
|
|
139
160
|
| `browser_click` | 点击:语义目标(`target`: css/text/xpath,滚动到元素并点中心)或视口坐标(配合截图视觉定位) | ✅ |
|
|
140
161
|
| `browser_type` | 输入文本(可先按 `target` 聚焦元素;CDP `Input.insertText`) | ✅ |
|
|
@@ -211,7 +232,7 @@ agent (browser_* 工具)
|
|
|
211
232
|
|
|
212
233
|
- **seam 层**(`browser` 行)提供 `ctx.browser` 服务:provider 注册、会话生命周期、错误码,与具体实现解耦;
|
|
213
234
|
- **provider 层**(`browser-electron` 行)通过 `ElectronBrowserViewHost` 接缝操作视图(创建/销毁/显示/`sendCommand`),由真实外壳用 Electron 对象实现;
|
|
214
|
-
- **工具层**(`tool-browser` 行)提供模型侧的
|
|
235
|
+
- **工具层**(`tool-browser` 行)提供模型侧的 34 个 `browser_*` 工具,按调用方任务(DSH 会话)维护独立的浏览器会话。
|
|
215
236
|
|
|
216
237
|
**自托管模式**:没有桌面外壳时,插件自己拉起一个 Electron 子进程(`host-main.js`),通过本机 TCP JSON-RPC 驱动。RPC 带随机 token 认证,token 经 **stdin + 环境变量双通道**传递——Windows 上 Electron 是 GUI 子系统进程、收不到 piped stdin,环境变量兜底保证握手稳定。子进程崩溃会自动重启;优先使用随插件安装的 electron 包(**打包应用如 DSH Desktop.exe 不会被误当作可复用二进制**,避免 spawn 秒退);截图优先走 Electron 原生 `capturePage`(CDP 截图在多视图下会挂起);Electron 的定位顺序见下(33.x 有合成器缺陷,建议 ≥ 40;44+ 的 electron 包不再随安装自动下载二进制,首次使用若缺失会按报错提示先 `npx install-electron`,需联网)。
|
|
217
238
|
|
|
@@ -221,7 +242,52 @@ agent (browser_* 工具)
|
|
|
221
242
|
|
|
222
243
|
## 与桌面外壳的分工
|
|
223
244
|
|
|
224
|
-
|
|
245
|
+
有三种形态,插件会自动选择,**无需配置**:
|
|
246
|
+
|
|
247
|
+
**① 桌面端:驱动官方侧栏的页面(人机同页)**
|
|
248
|
+
|
|
249
|
+
DSH Desktop 是两层结构:Electron 外壳 + 一个 `--expose-internals` 的 **Node 模式宿主**(插件就跑在宿主里,**没有 Electron API**)。0.2 移除了 `electronViewHost`,宿主与外壳之间也没有任何承载视图的通道,所以插件**借道一条小桥**:外壳主进程里跑一个 loopback + token 的 bridge,把侧栏浏览器的 guest(就是你在界面上看到的那个页面)的 CDP 交给插件。
|
|
250
|
+
|
|
251
|
+
结果是:**agent 操作的页面就是人看的页面**,插件不再 spawn 自己的 Electron,也不再出现第二个窗口。
|
|
252
|
+
|
|
253
|
+
安装这条桥(改的是**已安装的桌面端**,所以做成可重放):
|
|
254
|
+
|
|
255
|
+
```bash
|
|
256
|
+
node desktop-bridge/install.mjs # 幂等;首次会备份 main.js.before-bridge
|
|
257
|
+
node desktop-bridge/install.mjs --revert # 回滚
|
|
258
|
+
```
|
|
259
|
+
|
|
260
|
+
> **桌面端升级后要重跑一次 `install.mjs`** —— 升级会替换 `resources/app/`,bridge 随之消失。bridge 不在时插件自动退回自托管,功能不会中断,只是会多出一个独立窗口。
|
|
261
|
+
|
|
262
|
+
> **⚠️ 沙箱边界的变化(请明示知悉)**
|
|
263
|
+
>
|
|
264
|
+
> 官方侧栏浏览器的前提是:侧栏里的页面**不被外部读取** —— 它有独立的 partition,且宿主拒绝跨站内容访问。让 Agent 驱动该 guest,等于**有意打破这个前提**:
|
|
265
|
+
>
|
|
266
|
+
> - Agent 能读到你在侧栏访问的**任何页面的内容**(这正是"人机同页"的意义);
|
|
267
|
+
> - Agent 能读到该 partition 中的 **Cookie / 登录态**,`browser_auth` 可将其导出(由设置项控制);
|
|
268
|
+
> - 反过来,你在侧栏里的操作与 Agent 的操作**作用于同一个页面**,可能互相影响(Agent 不会主动覆盖你的输入,但导航会改变双方看到的内容)。
|
|
269
|
+
>
|
|
270
|
+
> 这是"人机同页"的必然代价。我们认为值得(它把"Agent 在一个你看不见的窗口里操作"变成"你能看着它操作并随时接手"),但你有权知道它存在 —— 因此也提供了开关:**「凭据访问」关闭后 Agent 不再读取 Cookie/登录态**;**「视觉策略」设为纯非视觉后,任何依赖截图定位的坐标点击都会被拒绝**。不想接受这个边界变化时,把 desktop profile 里的插件移除即可回到"独立窗口"的旧形态。
|
|
271
|
+
|
|
272
|
+
**② 用你自己的浏览器(Chrome / Edge)**
|
|
273
|
+
|
|
274
|
+
设置里可以把载体改成**本机已安装的 Chrome 或 Edge**(`browser.channel`:`bundled` / `auto` / `chrome` / `edge`)。做法与 Codex Browser Use 一致:以 `--remote-debugging-port=0` 启动,读浏览器自己写下的 `DevToolsActivePort` 得到端口,再全程走 CDP(用 Node 22 内置的 `WebSocket`,**不新增依赖**)。
|
|
275
|
+
|
|
276
|
+
**你自己的数据不会被碰**:插件用的是**独立 profile**(`$DSH_HOME/dsh-builtin-browser-host/<chrome|edge>-profile`),不会打开、占用或修改你日常的窗口、书签与登录状态;插件退出也不会关掉你的浏览器。
|
|
277
|
+
|
|
278
|
+
**登录态怎么办**:
|
|
279
|
+
|
|
280
|
+
- `cookies.persist` **开**(默认)→ 上面那个固定 profile 会保留,**重启 DSH 后仍是登录状态**;`browser_auth` 也照常可导出/恢复该 profile 的 Cookie。
|
|
281
|
+
- `cookies.persist` **关** → 每次用**临时 profile**,释放浏览器时整个目录被删除,不留登录痕迹。
|
|
282
|
+
- 代价要说清:独立 profile **看不到**你日常浏览器里已登录的站点 —— 在插件打开的窗口里登录一次即可,之后登录态就存在它自己的 profile 里。
|
|
283
|
+
|
|
284
|
+
**③ 有 `electronViewHost` 的宿主(旧版桌面外壳)**:直接使用外壳提供的视图。
|
|
285
|
+
|
|
286
|
+
**④ 没有外壳(纯 `dsh web`)**:自托管 —— spawn 插件自带的 Electron 窗口,功能照常。
|
|
287
|
+
|
|
288
|
+
> 可见视图与列布局始终属于宿主外壳;插件只负责 seam、provider 与工具。各形态下**工具集、浏览历史、设置栏、可视化鼠标、收尾语义完全一致**,差别只在页面由谁承载。
|
|
289
|
+
>
|
|
290
|
+
> **优先级**:设置里显式选择的本机浏览器 > 桌面端侧栏 > 自托管。选定的浏览器**没装或起不来**时会记一条警告并继续用内置浏览器 —— **不会静默换成别的**。
|
|
225
291
|
|
|
226
292
|
## 环境要求
|
|
227
293
|
|
|
@@ -243,6 +309,31 @@ agent (browser_* 工具)
|
|
|
243
309
|
|
|
244
310
|
> 插件声明 `electron >= 30`;**当前仅在 Windows 环境实测**(macOS/Linux 未验证,暂不承诺)。
|
|
245
311
|
|
|
312
|
+
## 更新方式(两端不同)
|
|
313
|
+
|
|
314
|
+
插件在两种宿主里各有一份安装,更新路径也不同 —— **更新其中一端不会连带更新另一端**。
|
|
315
|
+
|
|
316
|
+
**桌面端(DSH Desktop)**
|
|
317
|
+
- 插件是桌面端 profile 里的依赖(profile 目录通常是 `$DSH_HOME/profiles/desktop`)。更新它 = 把该 profile 里的依赖更新到新版本,然后**重启 DSH Desktop**,客户端设置栏与工具才会换成新代码。
|
|
318
|
+
- **桌面端还多一步,而 Web 端没有**:让插件驱动侧栏的那条 bridge 装在**桌面端自己的安装目录**里(`resources/app/`),插件更新**不会**带上它。桌面端升级会替换该目录、bridge 随之消失,所以请重跑一次:
|
|
319
|
+
```bash
|
|
320
|
+
node desktop-bridge/install.mjs # 幂等;已装则只刷新模块
|
|
321
|
+
node desktop-bridge/install.mjs --revert # 回滚
|
|
322
|
+
```
|
|
323
|
+
bridge 不在时插件自动退回自托管(多出一个独立窗口),功能不中断。
|
|
324
|
+
- 浏览器内核默认由桌面端自带的 Electron 提供,插件不会再下载一份 Electron;也可以在设置里改用它自己装的 Chrome / Edge。
|
|
325
|
+
|
|
326
|
+
**Web 端(`dsh web`)**
|
|
327
|
+
- 插件是 web profile 里的依赖(`$DSH_HOME/profiles/web`),更新后**重启 `dsh web`** 生效。
|
|
328
|
+
- Web 端**没有侧栏、也没有 bridge**:共享浏览器由插件自托管拉起。首次安装可能需要 Electron 二进制;若包管理器的构建白名单拦下了它(pnpm v10+ 会拦 `electron` 的 postinstall),执行一次 `npx install-electron` 补上即可。
|
|
329
|
+
- 同样可以在设置里改用本机 Chrome / Edge —— 这是两端**行为一致**的选项。
|
|
330
|
+
- 更新方式与你首次安装它时一致(按 npm 包名 `dsh-builtin-browser`、按 GitHub 仓库 `wqty123/dsh-browser`,或本地目录)。
|
|
331
|
+
|
|
332
|
+
**两端一致的体验**
|
|
333
|
+
- 工具集(34 个 `browser_*`)、设置页里的「浏览器」栏、浏览历史与 cookie 的持久化行为完全相同;差别只在页面由谁承载(桌面端=官方侧栏,Web 端=插件自托管窗口,或你在设置里指定的本机浏览器)。
|
|
334
|
+
- 升级不会丢数据:浏览历史与设置都在 `$DSH_HOME/dsh-builtin-browser-host/`(`history.jsonl`、`settings.json`),登录状态在同一 profile 目录里 —— 包括使用本机浏览器时的 `<chrome|edge>-profile`。
|
|
335
|
+
- 升级后如果历史记录不符合预期,先去「设置 → 浏览器」确认这些开关:历史默认**开**、侧栏自动展开默认**开**、会话结束时自动关闭浏览器默认**关**、载体默认**内置**。
|
|
336
|
+
|
|
246
337
|
## 已知限制
|
|
247
338
|
|
|
248
339
|
- JPEG 截图仅自托管原生路径可用(`capturePage` 的 `toJPEG`);桌面壳的 CDP 回退路径仍为 PNG(CDP JPEG 在 Electron 43 上挂起)。
|
|
@@ -314,6 +405,9 @@ npm run build
|
|
|
314
405
|
|
|
315
406
|
| 第十九轮 | 2026-10-01 | **DSH 0.2 兼容性**:DSH 进入 0.2 线(`@deepseek-ai/dsh@0.2.0-rc.2`,peer 包 `dsh-llm`/`dsh-tools`/`dsh-system-prompt` 同步到 `0.2.0-rc.2`),而原声明 `>=0.1.1-rc.1 <0.2.0` 把 0.2 挡在门外 → 在**真实 0.2.0-rc.2 宿主**上实测插件的运行时依赖面(`cordis` 的 Context/Service、`dsh-tools` 的 defineTool、`dsh-llm` 的 HarnessError、`schemastery`),确认无破坏性变更:会话、导航、快照、截图三态(越界拒绝/合法写入/覆盖拒绝)全部通过 → `peerDependencies` 三个 dsh 包范围改为 `>=0.1.1-rc.2 <0.3.0`,`dsh.compatibility.dsh` 放宽为 `>=0.1.1-rc.1 <0.3.0`,`dshReleases` 增加 `0.2.0-rc.1`/`0.2.0-rc.2` = compatible |
|
|
316
407
|
| **0.1.23** | 2026-10-01 | **发布**:第十八轮(PR #15:Windows 合成输入三连 + Electron 探测自愈 + 定位判词)与第十九轮(DSH 0.2 兼容)随 **0.1.23** 发布(构建零错误、**47 项测试全绿**,tag `v0.1.23`) |
|
|
408
|
+
| 第二十轮 | 2026-10-01 | **浏览历史 / 设置栏 / 可视化鼠标 / 收尾语义**:①新增**持久化浏览历史**(`history-store`:追加式 JSONL,落在 cookie 同一 profile 目录;5000 条或 90 天先到者为准;损坏行只丢该行)+ 新工具 **`browser_visited`**(工具数 **33 → 34**),重新打开沿用 `browser_open`;②新增设置页「浏览器」栏(手写客户端 bundle 注册到 `settings.section`,`order: 60` 排在宿主自带栏目下方)+ `GET/PUT /dsh-builtin-browser/settings`(同源防护、64 KiB 上限)+ 设置文档(`settings-store`:字段逐个校验、未知键丢弃、损坏文件按默认值),**开关即时生效**(provider 每次读取,无需重启);③新增页面内**虚拟光标**(内联样式 + Web Animations 以规避页面 `style-src` CSP;`buildTargetScript` 统一附加元素中心 `__point`,于是 click / type / setValue / check / select / clear **都有落点**),**光标出现即代表 agent 已接管该标签页**;④**关掉窗口 = 结束该会话**:窗口 `closed` 时释放其全部视图的 `webContents`(BrowserWindow 不连带销毁子视图,否则每个窗口泄漏一个渲染进程)并上报 `viewClosed`,provider 结束对应会话,seam 新增 `exists()` 供工具层核验会话缓存 —— 下一次调用得到**干净的新会话**,浏览历史与登录状态保留;新增 21 条测试(**68/68 全绿**) |
|
|
409
|
+
| 第二十轮补记 | 2026-10-01 | **客户端设置栏"隐形"的根因 + 崩溃诊断**:①真机验证发现设置栏不出现 —— 根因是 `cordis.patch.yml` 三行全用**子路径名**(`dsh-builtin-browser/browser`)注册,而宿主的客户端模块扫描只接受**精确包名**(`exactPackageSpecifier` 遇 `/` 即返回 `undefined`),于是这个包在客户端侧没有任何行可供读取 `dsh.client`;修复 = 增加一行以**包名**注册的惰性根行 + 给根入口补 `export const name` 与空 `apply()`;实测启动图条目 67 → **68**、设置栏出现并排在「规则设定」下方、开关即时落盘。②宿主日志三处缺陷修复:加 **ISO 时间戳**、spawn 前记录 **两条路径与存在性**、2 MiB 轮转改为**留时间戳标记**(原先整体清空,几周历史就是这样消失的);`exit` 行加 `pid=`/`entryExists=`,使"启动时存在、退出时不存在"自动命名"安装被就地替换"。③修复**测试污染真实日志**:三个 spawn 类测试因 `dispose()` 异步杀子进程,exit 行写在 `finally` 恢复 `DSH_HOME` 之后 → 改为模块级隔离。测试 **70/70** |
|
|
410
|
+
| **0.2.0** | 2026-10-01 | **发布**:第二十轮(浏览历史持久化 / 设置页「浏览器」栏 / 可视化鼠标 / 收尾语义)及其补记随 **0.2.0** 发布 —— 工具数 **33 → 34**(新增 `browser_visited`),设置页新增「浏览器」栏。构建零错误、**70/70 测试全绿**,tag `v0.2.0` |
|
|
317
411
|
|
|
318
412
|
> registry 上的最新版本以顶部 npm 徽章为准:本次 `0.1.23` 已入库并打 tag;若徽章仍显示 `0.1.22`,说明该版本尚未 publish。
|
|
319
413
|
|