dsh-builtin-browser 0.4.2 → 0.4.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (49) hide show
  1. package/CHANGELOG.md +1753 -0
  2. package/README.en.md +63 -20
  3. package/README.md +60 -20
  4. package/client.js +433 -24
  5. package/desktop-bridge/install.mjs +66 -6
  6. package/desktop-bridge/plugin-browser-bridge.js +425 -163
  7. package/docs/architecture.md +4 -4
  8. package/docs/tool-reference.md +3 -3
  9. package/docs/user-guide.md +50 -2
  10. package/lib/browser/types.d.ts +9 -2
  11. package/lib/browser-electron/bridge-connection.d.ts +18 -21
  12. package/lib/browser-electron/bridge-connection.js +42 -1
  13. package/lib/browser-electron/desktop-bridge-host.d.ts +69 -36
  14. package/lib/browser-electron/desktop-bridge-host.js +151 -86
  15. package/lib/browser-electron/entry.d.ts +20 -0
  16. package/lib/browser-electron/entry.js +110 -7
  17. package/lib/browser-electron/history-store.d.ts +11 -2
  18. package/lib/browser-electron/history-store.js +27 -5
  19. package/lib/browser-electron/host-main.js +11 -1
  20. package/lib/browser-electron/provider.d.ts +62 -2
  21. package/lib/browser-electron/provider.js +28 -12
  22. package/lib/browser-electron/remote-host.js +19 -6
  23. package/lib/browser-electron/rpc-ops.d.ts +18 -0
  24. package/lib/browser-electron/rpc-ops.js +36 -0
  25. package/lib/browser-electron/settings-route.d.ts +6 -6
  26. package/lib/browser-electron/settings-route.js +48 -8
  27. package/lib/browser-electron/settings-store.js +57 -16
  28. package/lib/browser-electron/system-browser.d.ts +31 -9
  29. package/lib/browser-electron/system-browser.js +116 -27
  30. package/lib/index.js +0 -1
  31. package/lib/tool-browser/element-fields.d.ts +145 -0
  32. package/lib/tool-browser/element-fields.js +152 -0
  33. package/lib/tool-browser/index.js +26 -60
  34. package/package.json +6 -2
  35. package/src/browser/types.ts +9 -2
  36. package/src/browser-electron/bridge-connection.ts +37 -1
  37. package/src/browser-electron/desktop-bridge-host.ts +146 -84
  38. package/src/browser-electron/entry.ts +113 -8
  39. package/src/browser-electron/history-store.ts +32 -7
  40. package/src/browser-electron/host-main.ts +10 -1
  41. package/src/browser-electron/provider.ts +91 -18
  42. package/src/browser-electron/remote-host.ts +19 -6
  43. package/src/browser-electron/rpc-ops.ts +41 -0
  44. package/src/browser-electron/settings-route.ts +54 -13
  45. package/src/browser-electron/settings-store.ts +56 -17
  46. package/src/browser-electron/system-browser.ts +121 -28
  47. package/src/index.ts +3 -2
  48. package/src/tool-browser/element-fields.ts +167 -0
  49. package/src/tool-browser/index.ts +31 -60
@@ -44,13 +44,13 @@ agent (browser_* 工具)
44
44
  - 会话生命周期绑定 agent 作用域 ctx:agent(DSH 会话)销毁时自动关闭对应浏览器会话,任务结束后不再泄漏窗口/视图;
45
45
  - 会话/开启动态/白名单都是**每插件实例(每 context)作用域**的,多 context 互不共享、互不污染;
46
46
  - `browser_restrict` 维护本任务白名单,守卫所有非只读工具;
47
- - 输出 schema 与返回值严格一致(DSH 运行时会校验,`additionalProperties: false` 下多一个字段都会报错)。
47
+ - 输出 schema 与返回值严格一致(DSH 运行时会校验,`additionalProperties: false` 下多一个字段都会报错):元素与节点的字段集在 `element-fields.ts` 里**只声明一次**,item schema 与投影都由它导出 —— 手写白名单漏一个字段会**静默丢数据**、多一个会让**整个结果被拒**,同源之后这两个方向都不可能单独出错(`browser_a11y` 的 `selector` 与 `browser_screenshot` 的 `width`/`height` 都是这么丢的)。
48
48
 
49
49
  ### 输出尺寸(工具层)
50
50
 
51
51
  工具输出花的是调用方的上下文,所以工具层对几处返回设了明确上限,并在截断时**说明截断**:
52
52
 
53
- - `browser_a11y` 默认 150 个节点(上限 5000)、一层一个空格缩进、`states` 只列非默认项;
53
+ - `browser_a11y` 默认 150 个节点(上限 5000)、一层一个空格缩进、`states` 只省略字面量 `enabled`(`unchecked`、`collapsed` 等仍会打印);
54
54
  - `browser_snapshot` 与 `browser_a11y` 在元素有 `id`/`name` 时给出 `{#id}` / `{[name=x]}` 引用选择器(页面脚本一直算,工具层此前丢掉;输出 schema 必须声明该字段,否则 `additionalProperties: false` 会拒绝整个结果);
55
55
  - `browser_content` 的上限按格式:html/json 50 000,txt/markdown 20 000,`maxChars` 可覆盖;
56
56
  - `browser_execute` 的返回值上限 50 000 字符;
@@ -68,10 +68,10 @@ RemoteElectronViewHost ──TCP JSON-RPC──▶ host-main.js
68
68
  DeferredRemoteView(物化缓存) webContents.debugger(CDP)
69
69
  ```
70
70
 
71
- - **协议**:本机 loopback TCP,每行一个 JSON(`{ id, op, ... }` ↔ `{ id, ok, result|err }`);
71
+ - **协议**:本机 loopback TCP,每行一个 JSON(`{ id, op, ... }` ↔ `{ id, ok, result|err }`);op 名清单只写在 `rpc-ops.ts` 一处,并由一个读**子端 switch 与父端调用两侧源码**的测试断言一致 —— 单侧改名因此会当场失败,而不是编译通过、运行时报 "unknown op";
72
72
  - **认证**:每次 spawn 生成随机 token,经 **stdin 首行 + `DSH_BROWSER_RPC_TOKEN` 环境变量双通道**传给子进程(绝不进 argv,避免进程列表泄露;Windows GUI 进程收不到 piped stdin,环境变量兜底),子进程首条消息必须回传该 token(`hello`);服务端只接受**一个**连接,其余连接直接断开——本机其他进程无法伪冒子进程或注入回复;
73
73
  - **Electron 定位**:① `ELECTRON_PATH`(显式覆盖,优先于一切自动发现)→ ② 随插件安装的 electron 包(纯文件系统探测,含 pnpm store 布局,不触发 44+ 懒下载)→ ③ 锚点与 pnpm store 中**版本最新**者(33.x 有合成器缺陷,建议 ≥ 40)→ ④ 当前进程是**裸** Electron(dev 模式)时复用宿主二进制(`process.execPath`)→ ⑤ 进程祖先树中的裸 Electron 二进制(Windows 用 PowerShell CIM 查询,仅在其它路径全部落空时才执行)。**打包应用不参与复用**:旁有 `resources/app.asar` 的可执行文件(如 DSH Desktop.exe)不能按脚本参数拉起——spawn 会启动应用本体并秒退(单实例锁,即 "browser host exited (code=0)"),一律跳过;
74
- - **稳健性**:子进程/套接字都有 `error` 监听(否则未捕获事件会炸掉整个 DSH 进程);子进程退出自动重启,已物化的视图句柄在下次调用时**自动重建并重试一次**(会话跨崩溃存活,仅页面状态丢失,不再出现 "browser host is not running" 僵尸态);物化失败可重试;下载仅限 HTTP(S)、`savePath` 必须绝对路径(`downloadDir` 已配置时用它;未配置时按序探测 —— 存在的 `XDG_DOWNLOAD_DIR` → home 下第一个存在的 `Downloads`/`下载`/`下載` → 回退 `~/Downloads`,首次使用时创建)、流式限流 + Content-Length 提前拒绝、256MB 上限与 60s 超时,文件由**子进程直接落盘**(临时文件 + rename,不再经 RPC 传 base64);cookie 导出/恢复有 30s 超时,且两者都受设置里 `credentials.allowRead` 门控 —— 关闭时抛 `BROWSER_AUTH_DISABLED`(导出**与恢复**都拒绝),有原生方法时优先用它,否则走 CDP 的 `Network.getCookies` / `Storage.setCookies`(三种载体都可用);**导出的范围收敛到本会话当前所在页面的 URL**,页面 URL 取不到时**拒绝而不是放宽**(CDP 载体报 `BROWSER_AUTH_SCOPE_UNKNOWN`,自托管侧栏按 view 自己的 URL 过滤);`cookies.get`/`cookies.set`(后者每条 cookie 一次)与下载的 `Runtime.evaluate` 都套同一套 `COMMAND_TIMEOUT_MS` —— 这三条曾经是无上限的 await,会让串行操作队列永久停在那一条上;
74
+ - **稳健性**:子进程/套接字都有 `error` 监听(否则未捕获事件会炸掉整个 DSH 进程);子进程退出自动重启,已物化的视图句柄在下次调用时**自动重建并重试一次**(会话跨崩溃存活,仅页面状态丢失,不再出现 "browser host is not running" 僵尸态);物化失败可重试;下载仅限 HTTP(S)、`savePath` 必须绝对路径(`downloadDir` 已配置时用它;未配置时按序探测 —— 存在的 `XDG_DOWNLOAD_DIR` → home 下第一个存在的 `Downloads`/`下载`/`下載` → 回退 `~/Downloads`,首次使用时创建)、流式限流 + Content-Length 提前拒绝、256MB 上限与 60s 超时,文件由**子进程直接落盘**(临时文件 + rename,不再经 RPC 传 base64);cookie 导出/恢复有 30s 超时,两者受**不同**的门控:导出是**读**动作,读 `credentials.allowRead`,关闭时抛 `BROWSER_AUTH_DISABLED`;恢复是**写**动作,读 `actions.allowCredentialWrite`,关闭时抛 `BROWSER_AUTH_WRITE_DISABLED`(允许读取的操作者因此仍能拒绝写入)。有原生方法时优先用它,否则走 CDP 的 `Network.getCookies` / `Storage.setCookies`(三种载体都可用);**导出的范围收敛到本会话当前所在页面的 URL**,页面 URL 取不到时**拒绝而不是放宽**(CDP 载体报 `BROWSER_AUTH_SCOPE_UNKNOWN`,自托管侧栏按 view 自己的 URL 过滤);`cookies.get`/`cookies.set`(后者每条 cookie 一次)与下载的 `Runtime.evaluate` 都套同一套 `COMMAND_TIMEOUT_MS` —— 这三条曾经是无上限的 await,会让串行操作队列永久停在那一条上;
75
75
  - **视图可见性**:多标签/多会话时,`showView` 隐藏其他视图并把目标视图置顶(remove+re-add),确保用户看到的是活动标签;目标是当前可见视图时**跳过** remove/re-add(否则每次操作都闪烁);窗口标题实时显示当前可见会话的任务标识 + 页面标题/URL(`showView` 携带会话 label,子进程读 `getTitle`/`getURL`);窗口 resize 时视图 bounds 自动跟随;
76
76
  - **孤儿防护**:父进程断开时子进程自动退出,不留僵尸窗口;
77
77
  - **cookie 落盘**:子进程使用独立 userData 目录(`<DSH_HOME>/dsh-builtin-browser-host`),登录态跨重启保留(另有 `browser_auth` 手动导出/恢复)。
@@ -49,7 +49,7 @@
49
49
  | 工具 | 参数 | 输出 | 守卫 | 说明 |
50
50
  | --- | --- | --- | --- | --- |
51
51
  | `browser_history` | `verbose?` | `{ entries[] }` | – | 操作日志(最新在后),含 seq/action/ok/params/result/error。**默认只渲染最近 20 条,且不显示每条的参数**(参数就是你刚发出去的),首行先给总条数,免得把尾部当成全部;要看参数时传 `verbose: true`。返回的完整 `entries[]` 不受条数限制 |
52
- | `browser_visited` | `limit`(默认 30,上限 200), `domain`(主机名包含匹配), `query`(URL 或标题包含匹配), `session`(按来源会话标签过滤) | `{ count, entries[] }` | – | **持久化浏览历史**(访问过的页面,最新在前):与 cookie 同址落盘(`$DSH_HOME/dsh-builtin-browser-host/history.jsonl`),关闭浏览器与重启 DSH 后仍在;上限 5000 条或 90 天。重开某条用 `browser_open`;与 `browser_history`(会话内操作日志,随会话消失)是两件事;可在设置里关闭记录。渲染时**时间戳精确到分钟**,会话标记**只在切换时打印一次并取 8 位前缀** |
52
+ | `browser_visited` | `limit`(默认 30,上限 200), `domain`(主机名包含匹配), `query`(URL 或标题包含匹配), `session`(按来源会话标签过滤) | `{ count, entries[] }` | – | **持久化浏览历史**(访问过的页面,最新在前):与 cookie 同址落盘(`$DSH_HOME/dsh-builtin-browser-host/history.jsonl`),关闭浏览器与重启 DSH 后仍在;上限默认 5000 条或 90 天(设置文档里的 `history.maxEntries` / `history.maxAgeDays` 可改)。重开某条用 `browser_open`;与 `browser_history`(会话内操作日志,随会话消失)是两件事;可在设置里关闭记录。渲染时**时间戳精确到分钟**,会话标记**只在切换时打印一次并取 8 位前缀** |
53
53
  | `browser_replay` | `seq`(必填) | `{ replayed }` | ✅ | 按序号回放某一步(navigate/execute/click/type/scroll/key) |
54
54
  | `browser_download` | `url`(必填), `savePath`(必填) | `{ path }` | ✅ | 带会话 cookie 下载到本地(仅 http(s);`savePath` 必须为绝对路径且位于 `downloadDir` 内——默认系统下载目录,自动识别 `Downloads`/`下载`/`下載` 与 `XDG_DOWNLOAD_DIR`;不覆盖已有文件;上限 256MB,受 CORS 约束;由子进程直接落盘)。受设置「允许下载文件到磁盘」门控(`BROWSER_DOWNLOAD_DISABLED`) |
55
55
 
@@ -57,14 +57,14 @@
57
57
 
58
58
  | 工具 | 参数 | 输出 | 守卫 | 说明 |
59
59
  | --- | --- | --- | --- | --- |
60
- | `browser_auth` | `action`(flush/restore,必填), `cookies?` | `{ cookies[]? / restored? }` | ✅ | 导出/恢复 cookie —— **三种载体都可用**:自托管走原生会话,桌面侧栏与本机 Chrome/Edge 走 CDP 的 `Network.getCookies` / `Storage.setCookies`;flush 返回 cookie 列表,restore 带列表写回。**flush 只导出当前页所属站点的 cookie**(不是整台机器的共享 cookie 罐),范围按本会话所在页面的 URL 收敛;**范围未知时直接拒绝而不是放宽**(自托管侧栏载体报"cookie 范围未知",CDP 载体报 `BROWSER_AUTH_SCOPE_UNKNOWN`)。`restored` 反映真的提交了多少(无法构成有效 URL 的 cookie 会被丢弃并报出条数)。**受白名单约束**:`restore` 会向任意域写 cookie,是动作而非观察,因此该工具**不在只读豁免集合**里。设置里「允许读取 cookies / 导出登录状态」关闭时,flush 与 restore **都**抛 `BROWSER_AUTH_DISABLED`。导出受设置「凭据」门控(`BROWSER_AUTH_DISABLED`),写入受「允许写入登录状态」门控(`BROWSER_AUTH_WRITE_DISABLED`)—— 两者独立 |
60
+ | `browser_auth` | `action`(flush/restore,必填), `cookies?` | `{ cookies[]? / restored? }` | ✅ | 导出/恢复 cookie —— **三种载体都可用**:自托管走原生会话,桌面侧栏与本机 Chrome/Edge 走 CDP 的 `Network.getCookies` / `Storage.setCookies`;flush 返回 cookie 列表,restore 带列表写回。**flush 只导出当前页所属站点的 cookie**(不是整台机器的共享 cookie 罐),范围按本会话所在页面的 URL 收敛;**范围未知时直接拒绝而不是放宽**(自托管侧栏载体报"cookie 范围未知",CDP 载体报 `BROWSER_AUTH_SCOPE_UNKNOWN`)。`restored` 反映真的提交了多少(无法构成有效 URL 的 cookie 会被丢弃并报出条数)。**受白名单约束**:`restore` 会向任意域写 cookie,是动作而非观察,因此该工具**不在只读豁免集合**里。**两个门控是分开的**:`flush`(读)受设置「凭据 / 允许读取 cookies」管,关闭时抛 `BROWSER_AUTH_DISABLED`;`restore`(写)受「允许写入登录状态」管,关闭时抛 `BROWSER_AUTH_WRITE_DISABLED` —— 关掉读不会连带拒绝写,反之亦然 |
61
61
  | `browser_restrict` | `allowed?` | `{ restrictedTo[] }` | – | 设置动作白名单;空列表解除;**只校验名字是否以 `browser_` 开头**(写错的前缀会报错,`browser_typo` 这种拼错但前缀合法的名字会被接受、等同于拦掉该名字,不额外报错);守卫按名字匹配,因此白名单里列什么名字就只放行什么名字。**软护栏**——模型可自行解除,非安全边界 |
62
62
 
63
63
  ## 截图
64
64
 
65
65
  | 工具 | 参数 | 输出 | 守卫 | 说明 |
66
66
  | --- | --- | --- | --- | --- |
67
- | `browser_screenshot` | `fullPage?`, `savePath?`, `format?`(png/jpeg), `quality?`, `maxWidth?`, `maxHeight?` | `{ dataUrl, path? }` | – | 截图;PNG 默认,**JPEG 在自托管与本机 Chrome/Edge 上都可用**(自托管在原生 `capturePage` 路径编码,本机浏览器经 CDP 声明 `supportsCdpJpeg`;**桌面侧栏**的 Electron CDP JPEG 编码器会挂起,该载体下 JPEG 请求返回 PNG);`maxWidth`/`maxHeight` 等比缩放**三种载体都支持**;`fullPage` 会跳过原生 `capturePage`、三种载体统一走 CDP `captureBeyondViewport`;`savePath` 落盘供视觉模型读取(与 `browser_download` 同一准入门:必须位于 `downloadDir` 内且不覆盖已有文件) |
67
+ | `browser_screenshot` | `fullPage?`, `savePath?`, `format?`(png/jpeg), `quality?`, `maxWidth?`, `maxHeight?` | `{ dataUrl, path?, width?, height? }` | – | 截图;PNG 默认,**JPEG 在自托管与本机 Chrome/Edge 上都可用**(自托管在原生 `capturePage` 路径编码,本机浏览器经 CDP 声明 `supportsCdpJpeg`;**桌面侧栏**的 Electron CDP JPEG 编码器会挂起,该载体下 JPEG 请求返回 PNG);`maxWidth`/`maxHeight` 等比缩放**三种载体都支持**;`fullPage` 会跳过原生 `capturePage`、三种载体统一走 CDP `captureBeyondViewport`;`savePath` 落盘供视觉模型读取(与 `browser_download` 同一准入门:必须位于 `downloadDir` 内且不覆盖已有文件) |
68
68
 
69
69
  ## 常用组合
70
70
 
@@ -35,10 +35,11 @@ dsh plugin --profile web add <本仓库路径>
35
35
  | `browser-electron` | `viewHost` | 对象 | 可选 | 宿主提供的 `ElectronBrowserViewHost`(通常 `!!js ctx.get('electronViewHost')`)。**不传时插件自己选载体**:桌面端驱动官方侧栏、否则自托管;设置里显式选择的 Chrome/Edge 优先于两者 |
36
36
  | `browser-electron` | `httpOnly` | 布尔 | `true` | 仅允许 HTTP(S) 导航;`file:`/`data:` 等拒绝 |
37
37
  | `browser-electron` | `snapshotMaxElements` | 数字 | `60` | 快照最多收录的交互元素数 |
38
- | `browser-electron` | `contentMaxChars` | 数字 | **不再被读取(已废弃)** | 曾经的"内容抓取默认字符上限"。**现在没有任何代码读它,改它不改变任何行为** —— 上限按格式固定:html 50 000、json 50 000、txt 20 000、markdown 20 000,单次调用用 `maxChars` 覆盖。保留该键只为不让既有配置因未知键而报错 |
38
+ | `browser-electron` | `contentMaxChars` | 数字 | 未设置(按格式分档) | 内容抓取的字符上限。**不设**时按格式取分档默认(html 50 000、json 50 000、txt 20 000、markdown 20 000);**设了**就是操作者的值,覆盖分档默认。单次调用的 `maxChars` 优先级最高 |
39
39
  | `browser-electron` | `downloadDir` | 字符串 | 系统下载目录(自动识别 `Downloads`/`下载`/`下載`,或 `XDG_DOWNLOAD_DIR`) | 限定 `browser_download` 与 `browser_screenshot` 保存路径必须位于该目录内且不覆盖已有文件;默认收敛到系统下载目录,可改沙箱目录 |
40
40
  | `tool-browser` | `timeoutMs` | 数字 | `60000` | 工具协作超时(ms) |
41
41
  | `tool-browser` | `tabTools` | 布尔 | `true` | 是否注册标签管理工具 |
42
+ | `tool-browser` | `allowedActions` | 字符串数组 | 不限制 | **部署级**动作白名单,对该部署里的每个任务生效(单个任务仍可用 `browser_restrict` 再收窄);`READ_ONLY_TOOLS` 与 `browser_restrict` 无论列表怎么写都不拦。`cordis.patch.yml` 默认给的是空 `config: {}`(等于不限制) |
42
43
 
43
44
  ## 快速上手(给 agent 的提示词示例)
44
45
 
@@ -54,12 +55,59 @@ dsh plugin --profile web add <本仓库路径>
54
55
 
55
56
  ## 操作纪律
56
57
 
58
+ **先确认视觉策略。** 设置里的 `vision.strategy` 决定坐标点击能不能用,**而且只决定这一件事**:
59
+
60
+ - **`auto`(默认)**:坐标与语义都允许,适合能读图的模型。
61
+ - **`nonVisual`**:**坐标点击被明确拒绝**,报 `BROWSER_NON_VISUAL_COORDINATES`,错误信息直接给出改法 —— 传语义 target,如 `target { by: "text", value: "Sign in" }`。**这是配置,不是故障,不要重试同一次坐标点击。**
62
+
63
+ **两种策略下 DOM 查询、无障碍树、文本提取与结构化抓取完全一致** —— 所以下面这些纪律对**所有模型**都成立,不是因为谁看不了图。
64
+
65
+ **定位:先用语义,再考虑坐标**
66
+
67
+ - **`browser_a11y` 是理解页面的首选**,它给出每个交互节点的**角色**(button/textbox/…)与**可访问名称**,比读 DOM 更接近"这页面上有什么";`browser_snapshot` 给出可点击元素的**引用与坐标**。
68
+ - **两者都带 `selector`**:无 id 且无 name 时是空串。**有 `selector` 就优先用它**,它比坐标稳定 —— 页面滚动、布局变化都不会让它失效,而坐标会。
57
69
  - **优先用 DOM 语义而非坐标**:表单提交优先 `form.requestSubmit()`;点击优先 `element.click()`;坐标点击是最后手段。
70
+ - **`browser_click` 的 `target`(css/text)本质上就是语义定位**,能用它就别自己算坐标 —— 它会把元素滚进视野并在中心点击,坐标抖动、像素比、iframe 偏移都由它处理。
58
71
  - **选中正确的元素**:页面常有隐藏副本(如移动端按钮),用 `browser_execute` 过滤可见元素(`getBoundingClientRect()` 宽高 > 0、`getComputedStyle` 非 `display:none`),再取坐标。
59
72
  - **取坐标后立即点击**:中间不要插入其他操作(填表、滚动会移动元素,旧坐标立即失效)。
60
73
  - **点击前验证命中**:`document.elementFromPoint(x, y)` 确认该坐标确实是目标元素,再执行真实点击。
61
74
  - **DPR 注意**:CDP 输入使用 CSS 像素;高 DPI 屏上若点击落空,用 `elementFromPoint` 校准,不要盲试坐标。
62
75
 
76
+ **截图与内容:先问"我要的是判断还是像素"**
77
+
78
+ - **能用文本就用文本。** `browser_snapshot` / `browser_a11y` / `browser_content` 给出同一页面的文本表示;**没有图像输入能力的模型看截图什么也得不到**,只会花钱。截图留给布局、图表、设计核对这类**必须看像素**的事。
79
+ - **`browser_screenshot` 带 `maxWidth`/`maxHeight` 时,结果里会返回 `width`/`height`**(实际像素尺寸)。需要控制成本时用它确认降采样生效了,而不是自己解码 data URL。
80
+ - **`browser_content` 的 `maxChars` 是按格式分档的**(html/json 默认 5 万,其余 2 万),单次传 `maxChars` 优先于设置里的 `contentMaxChars`。
81
+
82
+ **动作开关:三个门是操作者(人)的,不是你的**
83
+
84
+ 设置页里的**执行页面脚本**、**下载文件到磁盘**、**写入登录态** 由人控制,**任何 `browser_*` 工具都改不了它们**。关掉时对应调用会明确报错,一眼能认出:
85
+
86
+ | 谁被关 | 报的错误码 | 哪些工具会遇到 |
87
+ | --- | --- | --- |
88
+ | 执行页面脚本 | `BROWSER_EXECUTE_DISABLED` | `browser_execute`、以及需要页内脚本的工具 |
89
+ | 下载文件到磁盘 | `BROWSER_DOWNLOAD_DISABLED` | `browser_download` |
90
+ | 写入登录态 | `BROWSER_AUTH_WRITE_DISABLED` | `browser_auth` 的 `restore` |
91
+ | 读取 cookie | `BROWSER_AUTH_DISABLED` | `browser_auth` 的 `flush` |
92
+
93
+ **看到这些码不要重试、不要绕路** —— **它是人的决定,不是可以克服的故障**,直接告诉人"需要他在设置里打开"。
94
+
95
+ 另外几个容易认错来源的码:`BROWSER_DOWNLOAD_BLOCKED` / `BROWSER_SCREENSHOT_BLOCKED` 是**保存路径被沙箱拒绝**(不是开关问题,换路径而不是找开关);`BROWSER_DOWNLOAD_UNSUPPORTED` 是**该载体没有这个能力**(比如非自托管浏览器)。
96
+
97
+ `browser_restrict` **不一样** —— 那是**你自己**设的临时白名单,你有权设也有权解除;它不该被用来绕过上面三个门。
98
+
99
+ **失败怎么读**
100
+
101
+ - **`CDP error: ...` 是浏览器答了但拒绝**(比如元素已经不在)—— 多半是页面变了,重新快照再试,不要盲目重复同一动作。
102
+ - **超时 / 连接类错误**才是浏览器可能有问题;这类错误会触发一次自动探测,通常下一条命令就能恢复。
103
+ - **`BROWSER_SESSION_UNKNOWN`** 说明会话没了(浏览器被替换过);`browser_reset_session` 重建本任务的会话。
104
+ - **同一个动作连续失败两次就换策略**:换 `selector`、换语义定位、或者用 `browser_snapshot` 重新看页面 —— **不要第三次用同样的坐标**。
105
+
106
+ **`browser_history` 与动作集**
107
+
108
+ - `browser_history` **默认只给 20 条**,要更多自己传 `limit`。
109
+ - **可重放的动作有六种**:`navigate` / `click` / `type` / `scroll` / `key` / `execute`。不是每种历史记录都能 `browser_replay`,遇到不可重放的会明确报错。
110
+
63
111
  ## 多任务并行
64
112
 
65
113
  每个 DSH 会话(任务)拥有独立的浏览器会话(独立标签页与历史),并发任务互不干扰:
@@ -86,7 +134,7 @@ DSH Desktop 上②命中即可用(0.1.18+ 插件自带 electron 包;44+ 二进
86
134
  窗口标题为 `dsh-browser`(自托管)。若子进程崩溃(或宿主 DSH 重启)会自动重启;崩溃前已打开的会话在**下一次调用时自动重建**——仅页面状态丢失,无需手动 `browser_reset_session`。`browser_reset_session` 仍可用于主动重置。
87
135
 
88
136
  **Q:设置里的 `contentMaxChars` 改了没反应?**
89
- 因为它**已经没有任何代码在读** —— 那是内容上限改为按格式固定之后的遗留键(html/json 50 000,txt/markdown 20 000)。上限请用单次调用的 `maxChars` 覆盖。
137
+ 先确认改的是哪一层。优先级从高到低是:**单次调用的 `maxChars`** > **配置里的 `contentMaxChars`** > **按格式的分档默认**(html/json 50 000,txt/markdown 20 000)。`contentMaxChars` 的作用就是给这个部署换一个默认值:不设时才用分档,设了就盖过分档。若改了没反应,通常是那一次调用自己传了 `maxChars`。
90
138
 
91
139
  **Q:本机 Chrome / Edge 被关掉之后还能继续用吗?**
92
140
  能。该载体观察进程退出并清掉**已失效的会话映射**,下一次调用会**重新拉起**浏览器并继续工作;重新拉起前会先 kill 旧进程并**最多等 3 秒**它真正退出,且**只认本次启动写下的 `DevToolsActivePort`**(被杀掉的浏览器没机会清理那个文件,照读会连到别的 Chromium 或 `node --inspect` 上)。反过来,页面只是**慢**不会让插件杀掉你的窗口:命令超时只把连接标记为存疑,下一次调用先用 `Browser.getVersion` 探一次再决定是否丢弃,只有 socket 已关闭才重启。
@@ -366,8 +366,15 @@ export interface BrowserA11yNode {
366
366
  readonly role: string;
367
367
  /** Accessible name (aria-label, label text, placeholder, own text, …). */
368
368
  readonly name: string;
369
- /** Current value of input-like nodes (null when not applicable). */
370
- readonly value: string | null;
369
+ /**
370
+ * Current value of input-like nodes; absent when the node has none.
371
+ *
372
+ * The provider produces `null` for "not applicable" and the tool layer drops it, so absence
373
+ * and null mean the same thing here — which is what the output schema declares (`string`,
374
+ * not nullable). Saying `string | null` described the provider's internal shape rather than
375
+ * the contract, and the two disagreed at every call.
376
+ */
377
+ readonly value?: string;
371
378
  /** ARIA/DOM states: enabled/disabled/checked/unchecked/expanded/… */
372
379
  readonly states: readonly string[];
373
380
  /** DOM depth (within its document), for reconstructing the tree. */
@@ -1,22 +1,3 @@
1
- /**
2
- * Transport for the desktop shell's browser bridge: one long-lived connection to a
3
- * loopback TCP service.
4
- *
5
- * WHY THIS IS ITS OWN MODULE
6
- * The bridge used to be reached with a fresh connection per request, which measured
7
- * at ~25ms per call (TCP handshake + token exchange) against ~0.2ms when the socket
8
- * is reused. A single browser action issues several CDP commands, so that cost was
9
- * multiplied on every tool call — the difference between a snappy agent and a
10
- * visibly sluggish one. Reuse is therefore a property of the transport, and keeping
11
- * it here means the view host above does not have to think about sockets at all.
12
- *
13
- * PROTOCOL
14
- * The first line on a connection carries the token; the service marks the socket
15
- * authenticated and ignores that line as a command. Requests and answers are then
16
- * newline-delimited JSON, one answer per request, in order.
17
- *
18
- * @module dsh-browser/browser-electron/bridge-connection
19
- */
20
1
  /** Where the shell publishes its bridge endpoint. */
21
2
  export interface BridgeEndpoint {
22
3
  readonly port: number;
@@ -40,7 +21,8 @@ export interface BridgeEndpoint {
40
21
  * become a dead plugin.
41
22
  */
42
23
  export declare class BridgeConnection {
43
- private readonly endpoint;
24
+ private endpoint;
25
+ private readonly endpointPath?;
44
26
  private socket;
45
27
  private buffer;
46
28
  /** Outstanding requests by id. */
@@ -54,8 +36,23 @@ export declare class BridgeConnection {
54
36
  private serialisable;
55
37
  /**
56
38
  * @param endpoint - the shell's published bridge endpoint.
39
+ * @param endpointPath - where that endpoint is republished, so a moved port can be followed.
40
+ */
41
+ constructor(endpoint: BridgeEndpoint, endpointPath?: string | undefined);
42
+ /**
43
+ * Re-read the published endpoint, and adopt it if the shell has moved.
44
+ *
45
+ * The bridge binds an ephemeral port and picks a NEW one whenever its server is recreated. The
46
+ * endpoint file is rewritten every fifteen seconds, but a host that adopted the old one keeps
47
+ * dialling a dead port forever: measured here as `connect ECONNREFUSED 127.0.0.1:52276` while
48
+ * the live bridge sat on 52411. Discarding the socket is not enough — the comment below says a
49
+ * dead connection must never become a dead plugin, and a stale PORT makes exactly that happen.
50
+ *
51
+ * Cheap and safe: one small file read per call, no socket work.
52
+ *
53
+ * @returns true when a different endpoint was adopted.
57
54
  */
58
- constructor(endpoint: BridgeEndpoint);
55
+ private refreshEndpoint;
59
56
  /**
60
57
  * Send one request, reusing the socket if it is still healthy.
61
58
  *
@@ -1,3 +1,4 @@
1
+ import { readFileSync } from 'node:fs';
1
2
  /**
2
3
  * Transport for the desktop shell's browser bridge: one long-lived connection to a
3
4
  * loopback TCP service.
@@ -49,6 +50,7 @@ const REQUEST_ID_FIELD = 'bridgeRequestId';
49
50
  */
50
51
  export class BridgeConnection {
51
52
  endpoint;
53
+ endpointPath;
52
54
  socket;
53
55
  buffer = '';
54
56
  /** Outstanding requests by id. */
@@ -62,9 +64,46 @@ export class BridgeConnection {
62
64
  serialisable = false;
63
65
  /**
64
66
  * @param endpoint - the shell's published bridge endpoint.
67
+ * @param endpointPath - where that endpoint is republished, so a moved port can be followed.
65
68
  */
66
- constructor(endpoint) {
69
+ constructor(endpoint, endpointPath) {
67
70
  this.endpoint = endpoint;
71
+ this.endpointPath = endpointPath;
72
+ }
73
+ /**
74
+ * Re-read the published endpoint, and adopt it if the shell has moved.
75
+ *
76
+ * The bridge binds an ephemeral port and picks a NEW one whenever its server is recreated. The
77
+ * endpoint file is rewritten every fifteen seconds, but a host that adopted the old one keeps
78
+ * dialling a dead port forever: measured here as `connect ECONNREFUSED 127.0.0.1:52276` while
79
+ * the live bridge sat on 52411. Discarding the socket is not enough — the comment below says a
80
+ * dead connection must never become a dead plugin, and a stale PORT makes exactly that happen.
81
+ *
82
+ * Cheap and safe: one small file read per call, no socket work.
83
+ *
84
+ * @returns true when a different endpoint was adopted.
85
+ */
86
+ refreshEndpoint() {
87
+ if (this.endpointPath === undefined)
88
+ return false;
89
+ try {
90
+ const raw = JSON.parse(readFileSync(this.endpointPath, 'utf8'));
91
+ if (typeof raw.port !== 'number' || typeof raw.token !== 'string')
92
+ return false;
93
+ if (raw.port === this.endpoint.port && raw.token === this.endpoint.token)
94
+ return false;
95
+ this.endpoint = {
96
+ port: raw.port,
97
+ token: raw.token,
98
+ pid: typeof raw.pid === 'number' ? raw.pid : this.endpoint.pid,
99
+ };
100
+ // The old socket belongs to the old port.
101
+ this.reset(new Error('dsh-builtin-browser: the bridge moved; reconnecting to the new port'));
102
+ return true;
103
+ }
104
+ catch {
105
+ return false;
106
+ }
68
107
  }
69
108
  /**
70
109
  * Send one request, reusing the socket if it is still healthy.
@@ -85,6 +124,8 @@ export class BridgeConnection {
85
124
  // straight-line re-dial rather than a retry loop: an earlier revision re-entered call()
86
125
  // from inside its own promise executor, which left an orphaned pending entry behind and,
87
126
  // when the condition could not clear, spun until the process ran out of heap.
127
+ // Follow the shell if its bridge moved to a new port (see refreshEndpoint).
128
+ this.refreshEndpoint();
88
129
  let socket = this.ensureSocket();
89
130
  if (this.serialisable && this.queue.length > 0) {
90
131
  this.reset(new Error('dsh-builtin-browser: bridge connection reopened so replies stay attributable'));
@@ -36,14 +36,51 @@ export declare function bridgeEndpointPath(): string;
36
36
  */
37
37
  export declare class DesktopBridgeViewHost implements ElectronBrowserViewHost {
38
38
  private readonly connection;
39
+ /**
40
+ * One entry per view this host handed out: the conversation it serves, and the guest backing it.
41
+ *
42
+ * The conversation belongs to the ENTRY, not to this host. A single plugin process serves every
43
+ * conversation — DSH runs them all through one host instance — so an owner held on the host
44
+ * would be one conversation's id applied to all of them, and a page could still land in
45
+ * another's sidebar however carefully the bridge checked it.
46
+ */
39
47
  private readonly views;
40
48
  /**
41
- * Guests whose view is gone but whose page may still be open in the sidebar.
49
+ * This process's identity with the bridge, for tab ownership.
50
+ *
51
+ * Every DSH session is its own plugin process, and the sidebar is one surface they all share.
52
+ * A session can only see its own `views` map, so before the bridge kept a ledger it adopted
53
+ * whichever tab it found — including one another session had opened, which is how a URL from
54
+ * one session appeared in another's sidebar. The id is per process, so two sessions never
55
+ * collide and one session's tabs are never handed to another; it is deliberately not the DSH
56
+ * session id, which this layer has no access to and does not need.
57
+ *
58
+ * The whole ledger rests on that premise, so it was measured rather than assumed: with two
59
+ * sessions running, the desktop had TWO host processes (plus one main, two renderers and a
60
+ * gpu/utility pair). If sessions ever shared a process, every owner check in the bridge would
61
+ * compare equal and the bleed would return in a new shape — so a change to how DSH spawns
62
+ * sessions invalidates this file, not just this comment.
63
+ *
64
+ * It is now the CONVERSATION id rather than an invented uuid, read from the plugin's ctx by the
65
+ * same path DSH's own desktop host uses. That matters because the shell keeps one sidebar per
66
+ * conversation and writes its session id onto each sidebar container (on the React fiber,
67
+ * measured on the running app) — so the bridge can only answer "is this sidebar mine" if it has
68
+ * a comparable id. With an invented uuid it never could, which is how a command came to be typed
69
+ * into another conversation's address bar.
70
+ *
71
+ * Falls back to a random uuid when the host cannot supply one — an older DSH, or a non-desktop
72
+ * composition. That is no worse than before: the bridge then knows only "this process", and it
73
+ * refuses to operate on a sidebar it cannot prove is its own.
74
+ */
75
+ private readonly owner;
76
+ /**
77
+ * Guests whose view is gone but whose page may still be open in the sidebar, with the
78
+ * conversation each belongs to.
42
79
  *
43
- * The provider destroys its view handles before it asks for a release, so the
44
- * mapping is dropped by then — keeping the guest ids here is what lets us close
45
- * only our own tabs when several sessions are running (requirements §4: each
46
- * session gets its own page).
80
+ * The provider destroys its view handles before it asks for a release, so the mapping is dropped
81
+ * by then — keeping the guest ids here is what lets us close only our own tabs when several
82
+ * conversations are running (requirements §4: each session gets its own page). The owner rides
83
+ * along for the same reason it does in {@link views}: a release names one conversation.
47
84
  */
48
85
  private readonly orphaned;
49
86
  /**
@@ -69,49 +106,43 @@ export declare class DesktopBridgeViewHost implements ElectronBrowserViewHost {
69
106
  * caller falls back to self-hosting, which always works.
70
107
  * @returns the host, or undefined when no bridge is available.
71
108
  */
72
- static discover(): Promise<DesktopBridgeViewHost | undefined>;
109
+ static discover(sessionId?: string): Promise<DesktopBridgeViewHost | undefined>;
73
110
  /** Whether this host can back views: the bridge already answered `list`. */
74
111
  available(): boolean;
75
112
  /**
76
- * The guest id backing a view, materialized on first use.
113
+ * The guest backing a view, materialized on first use.
77
114
  *
78
- * The sidebar browser is itself a multi-tab surface, so each view gets its own
79
- * tab's guest: the provider's tab bookkeeping then maps onto real tabs the human
80
- * can see and switch between. Two rules keep that honest:
81
- * - a cached guest is used as-is: probing it first cost a round-trip on every
82
- * command, so liveness is established by the command failing instead;
83
- * - a fresh view takes an unclaimed guest, growing the tab strip only when
84
- * every existing guest is already spoken for.
115
+ * The conversation that called owns exactly one sidebar page on this carrier — a human and the
116
+ * agent look at the same page — so every view resolves to that conversation's own guest, and the
117
+ * page does not multiply behind the provider's own tab bookkeeping.
118
+ *
119
+ * A cached guest is used as-is: probing it first cost a round-trip on every command, so liveness
120
+ * is established by the command failing instead.
85
121
  * @param viewId - the view whose guest is wanted.
86
- * @param url - address to use when a sidebar browser has to be opened first.
122
+ * @param url - address to use when this conversation has no page yet.
87
123
  */
88
124
  private guestFor;
89
125
  /**
90
- * Ask for at least `count` sidebar tabs and return their guest ids.
91
- * @param count - minimum number of tabs.
126
+ * @param owner - the conversation this view serves. Recorded WITH the view: one host instance
127
+ * serves every conversation, so the owner cannot live on the host.
92
128
  */
93
- private guestIds;
94
- createView(): ElectronViewHandle;
129
+ createView(owner?: string): ElectronViewHandle;
95
130
  destroyView(handle: ElectronViewHandle): void;
96
131
  /**
97
- * Nothing to show: the sidebar is already on screen, and showing it again is
98
- * the shell's business (it owns the tab strip and the visibility toggle).
99
- */
100
- showView(): void;
101
- /** Same as {@link showView}: the page is already presented by the shell. */
102
- presentView(): Promise<void>;
103
- /**
104
- * The shell owns window grouping, and one sidebar serves every view, so there
105
- * is nothing to record here. (It must not write a placeholder guest id either:
106
- * a view whose guest is not yet materialized has to stay absent from the map,
107
- * or the first real command would be sent to the placeholder.)
132
+ * Bring this view's tab to the front in the sidebar.
133
+ *
134
+ * This used to be empty, on the reasoning that the sidebar is already on screen so there is
135
+ * nothing to show. That confused "the sidebar is visible" with "this page is the one being
136
+ * looked at": opening a page left it behind whatever the human had in front, which is issue
137
+ * #23. The other two carriers both do this — the self-hosted window re-adds the view and the
138
+ * system browser calls `Page.bringToFront` — so this carrier was the odd one out.
139
+ *
140
+ * Best-effort by design: the tab strip belongs to the shell's renderer, and failing to raise
141
+ * it must not fail the navigation that has already happened.
142
+ * @param handle - the view to bring forward.
108
143
  */
109
- groupView(): void;
110
- /** Focus is the shell's to manage, and there is no separate window to raise. */
111
- focus(): Promise<void>;
144
+ showView(handle: ElectronViewHandle): void;
112
145
  onUserAction(handler: (action: BrowserUserAction) => void): void;
113
- /** The shell reports its own window lifecycle; nothing to subscribe to here. */
114
- onViewClosed(): void;
115
146
  /**
116
147
  * Release the sidebar's browser pages (requirements §3, `ui.closeWithSession`).
117
148
  *
@@ -119,9 +150,11 @@ export declare class DesktopBridgeViewHost implements ElectronBrowserViewHost {
119
150
  * belongs to the shell and would happily keep the page (and its renderer) alive.
120
151
  * Closing the tabs is what actually ends the page — and only the page: cookies
121
152
  * live in the partition, history on disk, so both survive.
153
+ * @param owner - the conversation whose pages to close. Omit to close every conversation's,
154
+ * which is only correct from a teardown that is itself global.
122
155
  * @returns a promise that settles once the shell has been asked.
123
156
  */
124
- releasePage(): Promise<void>;
157
+ releasePage(owner?: string): Promise<void>;
125
158
  /**
126
159
  * Fold the sidebar away without ending the page (`ui.autoExpandOnce` is off, or
127
160
  * the caller wants the screen back while work continues).