dsh-sidebar-browser-control 0.1.0 → 0.2.0

package/README.md

@@ -38,35 +38,68 @@ automation, downloads or screenshots.
 
 | Tool | Actions |
 |---|---|
-| `browser_tabs` | list open built-in-browser tabs: id, live title, URL, owning session, panel state, whether page control is available |
+| `browser_tabs` | list open built-in-browser tabs: id, live title, URL, owning session, panel state, whether page control is available, plus capture diagnostics |
 | `browser_control` | `open` · `navigate` · `back` · `forward` · `reload` · `close` · `focus` · `expand` · `collapse` · `fullscreen` · `split` · `float` · `dock` · `external` |
-| `browser_page` | `text` · `links` · `elements` · `eval` · `click` · `type` · `scroll` · `wait` *(Desktop only)* |
-| `/browser` | `open <url>` · `tabs` · `back` · `forward` · `reload` · `close` · `focus` · `expand` · `collapse` · `fullscreen` · `split` · `float` · `dock` · `external` · `read` |
+| `browser_page` | `text` · `links` · `elements` · `eval` · `click` · `type` · `fill` · `key` · `login` · `scroll` · `wait` · `images` · `image` · `screenshot` *(page actions: Desktop only)* |
+| `/browser` | `open <url>` · `tabs` · `back` · `forward` · `reload` · `close` · `focus` · `expand` · `collapse` · `fullscreen` · `split` · `float` · `dock` · `external` · `read` · `images` · `image <N>` · `screenshot` |
 
 The tools and the slash command share one implementation, so `/browser open …`
 costs no model round trip and behaves exactly like the tool.
 
+### Operating the page, not just reading it
+
+- `elements` reports what a person sees: `type`, `name`, `label`, `placeholder`,
+  `autocomplete`, `value`, `checked`, plus a selector for `click`.
+- `fill` takes several fields at once and matches them by **label or
+  placeholder**, not only by CSS selector — `[{ label: "用户名", value: "alice" },
+  { label: "密码", value: "…" }, { label: "记住我", value: true }]` — and can
+  submit. Passwords are masked in every result.
+- `login` is the whole flow in one call: find the account and password fields
+  (also inside a same-origin iframe), fill, submit, wait, then report which
+  fields were found, whether the password field disappeared, the address it
+  landed on and an excerpt of the page. It never echoes the password back.
+- `key` presses Enter/Tab/Escape; Enter submits the surrounding form.
+- `wait` can wait for a selector, a text, or `url_contains`.
+
+### Seeing the page
+
+Two ways to give the model actual pixels:
+
+- `images` lists the pictures in the viewport; `image { index }` attaches one of
+  them, so the model can look at a photo, chart, QR code or captcha. HTTP(S)
+  sources are fetched by the Host (the page's own `fetch` is bound by CORS);
+  `data:`, `blob:` and `<canvas>` sources are decoded inside the page.
+- `screenshot` captures the Browser pane itself and attaches it. It deliberately
+  avoids every native capture API — `webview.capturePage()` crashed the renderer
+  (see below) — and instead reads the pane's geometry from the page, then
+  captures the DSH window with `PrintWindow` through PowerShell and crops to that
+  rectangle. Because it renders the window's own pixels, it also works while
+  other applications cover the window. A **minimized** window has no rendered
+  surface, so the tool refuses with that reason — pass `restore: true` to let it
+  bring the window back first. This path is Windows-only today; elsewhere the
+  tool says so instead of failing quietly.
+
 ## Install
 
 The plugin is a normal DSH bundle, so any spec the plugin manager accepts works.
 
-**From GitHub** (works before the first npm release). Ask the agent, no restart
-needed:
+**From npm** (published as `dsh-sidebar-browser-control`). Ask the agent, no
+restart needed:
 
 ```
-plugin_manager { action: install_bundle, target: "github:XUFENGCHINA/dsh-sidebar-browser-control" }
+plugin_manager { action: install_bundle, target: "dsh-sidebar-browser-control" }
 ```
 
 or with the CLI:
 
 ```sh
-dsh plugin --profile desktop add github:XUFENGCHINA/dsh-sidebar-browser-control
+dsh plugin --profile desktop add dsh-sidebar-browser-control
 ```
 
-**From npm**, once published:
+**From GitHub**, if you prefer tracking `main`:
 
 ```
-plugin_manager { action: install_bundle, target: "dsh-sidebar-browser-control" }
+plugin_manager { action: install_bundle, target: "github:XUFENGCHINA/dsh-sidebar-browser-control" }
 ```
 
 **From a checkout**, for development:
@@ -89,6 +122,17 @@ Just ask, in any session whose Sidebar is on screen:
 - "search Bing in the built-in browser for DeepSeek Harness and give me the first three results"
 - `/browser open https://example.com` — straight from the composer, no model call
 
+Two behaviours worth knowing while driving it:
+
+- **A link that opens a new window becomes a new sidebar tab** — DSH turns the
+  popup request into a tab of its own. `browser_page click` reports it as
+  `opened_tab`/`opened_url`; continue with that id, because the original tab
+  stays where it was.
+- **A page that fails to load says so** — DNS, network, certificate and bot-check
+  failures leave Chromium's error page in the tab, and page actions report that
+  explicitly instead of returning empty content that reads as "the page is
+  empty".
+
 ## How it works
 
 The browser lives in the Web renderer, tools live in the Host, and DSH exposes no
@@ -126,9 +170,11 @@ while tab and layout control keep working; the table of those dependencies is in
 
 ## Known limitations
 
-- **No screenshots.** `webview.capturePage()` natively crashed the Electron
-  renderer on the verified deployment, so the capability is removed and the call
-  refuses with the reason. Evidence and survivable alternatives:
+- **No screenshots through Electron.** `webview.capturePage()` natively crashed
+  the Electron renderer on the verified deployment, so it is not used anywhere.
+  `browser_page screenshot` goes through PrintWindow instead: the window must not
+  be minimized, and only Windows is implemented (macOS/Linux report that).
+  Evidence and survivable alternatives:
   [docs/architecture.md](https://github.com/XUFENGCHINA/dsh-sidebar-browser-control/blob/main/docs/architecture.md#why-there-is-no-screenshot).
 - Page-level actions are **Desktop only**.
 - Tab ids are unique per session; an ambiguous id is reported and accepted as

package/README.zh.md

@@ -45,22 +45,22 @@ npm 上已有一个走 CDP 的 DSH 浏览器插件（`dsh-browser-control`）。
 
 它就是一个普通 DSH bundle，plugin manager 接受的任何 spec 都能用。
 
-**从 GitHub 安装**（还没发 npm 时用这个）。让 Agent 装，无需重启：
+**从 npm 安装**（已发布为 `dsh-sidebar-browser-control`）。让 Agent 装，无需重启：
 
 ```
-plugin_manager { action: install_bundle, target: "github:XUFENGCHINA/dsh-sidebar-browser-control" }
+plugin_manager { action: install_bundle, target: "dsh-sidebar-browser-control" }
 ```
 
 或用 CLI：
 
 ```sh
-dsh plugin --profile desktop add github:XUFENGCHINA/dsh-sidebar-browser-control
+dsh plugin --profile desktop add dsh-sidebar-browser-control
 ```
 
-**从 npm 安装**（发布之后）：
+**从 GitHub 安装**（想跟 `main` 分支时用）：
 
 ```
-plugin_manager { action: install_bundle, target: "dsh-sidebar-browser-control" }
+plugin_manager { action: install_bundle, target: "github:XUFENGCHINA/dsh-sidebar-browser-control" }
 ```
 
 **从本地 checkout 安装**（开发用）：
@@ -83,6 +83,14 @@ dsh plugin --profile desktop add link:/绝对路径/到/本仓库
 - 「在内置浏览器用 Bing 搜 DeepSeek Harness，给我前三条结果」
 - `/browser open https://example.com` —— 输入框直接敲，不经过模型
 
+驱动它时值得知道的两条实际行为：
+
+- **新窗口打开的链接会变成一个**新的侧栏标签**：DSH 把 popup 请求转成了独立的 tab。
+  `browser_page click` 会把它报成 `opened_tab`/`opened_url`，请接着用那个 id——
+  原标签仍停在原来的页面上。
+- **加载失败的页面会明确报出来**：DNS、网络、证书、以及反爬挑战失败时，标签里是
+  Chromium 错误页；页面级动作会明确说明这一点，而不是返回空内容让你误以为"页面是空的"。
+
 ## 工作原理
 
 浏览器活在 Web 渲染进程里，工具活在 Host 进程里，而 DSH 没有面向第三方插件的