npm - dsh-builtin-browser - Versions diffs - 0.1.11 → 0.1.12 - Mend

dsh-builtin-browser 0.1.11 → 0.1.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (3) hide show

package/README.en.md CHANGED Viewed

@@ -1,106 +1,169 @@
-# dsh-builtin-browser (built-in shared real browser plugin for DeepSeek Harness)
+<p align="center">
+  <img src="https://img.shields.io/github/stars/wqty123/dsh-browser?style=flat&amp;label=%E2%98%85&amp;color=08C" alt="GitHub stars">
+  <img src="https://img.shields.io/npm/v/dsh-builtin-browser?style=flat&amp;label=npm&amp;color=CB3837" alt="npm version">
+  <img src="https://img.shields.io/badge/license-MIT-2EA44F?style=flat" alt="MIT License">
+  <img src="https://img.shields.io/badge/DSH-Plugin-47848F?style=flat" alt="DeepSeek Harness plugin">
+  <img src="https://img.shields.io/badge/Windows%20%7C%20macOS%20%7C%20Linux-4493F8?style=flat-square" alt="Platforms">
+</p>
-中文 | [English](README.en.md)
+<p align="center"><a href="README.md">中文</a> · English</p>
-A shared **real browser** capability plugin for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness): install and it just works — the agent drives a real, visible browser over CDP that the human can watch and take over at any time. Human and agent operate the **very same page**.
+<h3 align="center">A <b>shared real browser</b> plugin for the DeepSeek Harness ecosystem (install-and-use, human and agent on the same page)</h3>
-**Why this plugin**
+<h4 align="center">The agent drives a real, visible browser the human can watch and take over at any time — both operate the <b>same page</b>.</h4>
-- **Install-and-use.** With a desktop shell the shell's embedded view is used; on plain `dsh web` the plugin **self-hosts** — it spawns its own Electron window with zero extra configuration.
-- **Real and visible, human-takeover friendly.** Not a headless screenshot or a relay: the user sees the page and can grab control directly.
-- **DOM-level driving, framework-friendly.** React/Vue controlled inputs fill reliably.
-- **Concurrency-safe.** Task-level browser session isolation — parallel tasks never fight over the page or pollute each other.
-- **Built for the real world.** CAPTCHA detection, login persistence, batch form filling, authenticated downloads, operation replay, and action restriction — all included.
+## What is this
-## Features
+`dsh-builtin-browser` adds browser capability to [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness):
-**Browser capability**
+- **A real view, not a relay**: the browser is a native `WebContentsView`; the human sees every step the agent takes and can grab control at any time;
+- **Install-and-use**: with a desktop shell the shell's embedded view is used; on plain `dsh web` the plugin **self-hosts** — it spawns its own Electron window with zero extra configuration;
+- **One plugin, one toolset**: after install the agent automatically gets 20 `browser_*` tools (open, inspect, interact, fill forms, screenshot, download, auth management…).
-- **Real view, not a relay.** The browser is a native view (`WebContentsView`) the human can see and operate directly; the agent drives the very same page. The view is provided by the host shell; the plugin drives it.
-- **DOM-referenced, not coordinate-guessing.** `browser_snapshot` returns numbered interactive elements; `browser_execute` runs JS in the page (native setters for framework inputs), so interaction works on React/Vue pages.
-- **Multi-tab sessions.** Open URLs in parallel tabs, list/switch/close/reset, all keeping state.
-- **Multi-format content.** Fetch pages as html / markdown / txt / json, scoped by selector, capped by length and timeout.
+In one sentence: **installing the plugin gives you a real browser that is shared with the user and drivable by the agent.**
-**Engineering capability**
-- **Per-task isolation.** Each DSH task (session) gets its own browser session (own tabs and history); concurrent tasks never interfere, and calls within one task reuse the same session (`browser_session` / `browser_reset_session`).
-- **Login persistence.** `browser_auth` exports/restores cookies, so logins survive host restarts.
-- **CAPTCHA / bot-detection awareness.** Cloudflare, reCAPTCHA, hCaptcha, Turnstile and generic challenges are detected automatically (`browser_challenge`, and flagged in snapshots); instead of blindly retrying, the agent asks the human to complete it in the shared window.
-- **Batch form filling.** `browser_fill` sets many fields in one call — matched by selector/name/label, handling controlled inputs, selects, checkboxes and radio groups, with optional submit.
-- **Operation history & replay.** `browser_history` records operations; `browser_replay` re-runs one step.
-- **Authenticated downloads.** `browser_download` fetches a URL with the session's cookies straight to a local file.
-- **Safety restriction.** `browser_restrict` limits which browser actions are allowed, preventing stray clicks/navigation.
-## Requirements
-- DeepSeek Harness (dsh) with the `web` profile
-- **Electron runtime** (optional peer dependency): the desktop shell carries it; on plain `dsh web` the plugin must be able to locate an Electron binary (see below)
-- With a desktop shell (`ctx.electronViewHost`) the shell's embedded view is used; without one the plugin **self-hosts**: it spawns its own Electron window and the `browser_*` tools still work
-**Electron lookup order**: ① `require('electron')` (peer installed) → ② DSH install anchors → ③ `node_modules/.pnpm` virtual store → ④ `ELECTRON_PATH` env var. A clear error tells you when none is found.
-### Verified versions
-| Component | Version |
-|---|---|
-| DeepSeek Harness (dsh) | `0.1.0-rc.5` |
-| Electron | `43.4.0` |
-| Node.js | `22.20.0` |
-| dsh-builtin-browser | `0.1.7` |
-| OS | Windows 10 (10.0.26200) |
-> The plugin declares `electron >= 30`; other platforms (macOS/Linux) run the same protocol but were only verified on the Windows environment above.
-## Install
+## Quick start
 ```sh
-dsh plugin --profile web add dsh-builtin-browser  # from npm once published
-# or from a checkout (one plugin, one repository):
+# Option 1: install from npm (published)
+dsh plugin --profile web add dsh-builtin-browser
+# Option 2: install from a checkout (one plugin, one repository)
 dsh plugin --profile web add <path-to-this-repo>
 ```
-This links the plugin, adds `dsh-builtin-browser` to the profile's bundle layer, and mounts:
-| Row | Subpath | Role |
-|---|---|---|
-| `browser` | `dsh-builtin-browser/browser` | `ctx.browser` capability seam (always mounted) |
-| `browser-electron` | `dsh-builtin-browser/browser-electron` | Electron CDP provider (needs `electronViewHost`) |
-| `tool-browser` | `dsh-builtin-browser/tool-browser` | `browser_*` model-facing tools |
-The provider and tools are gated on `ctx.get('electronViewHost')` being present, so a composition without a desktop shell simply keeps the seam and nothing else.
+After install the agent can use the browser tools, e.g.:
+| What you want | Tool | Notes |
+| --- | --- | --- |
+| Open a page | `browser_open` | Opens a URL and returns a numbered snapshot |
+| Understand a page | `browser_snapshot` | Numbered inventory of inputs/buttons/links to target |
+| Operate a page | `browser_execute` | Runs JS in the page (native setters, framework-friendly) |
+| Fill a form | `browser_fill` | Fills many fields in one call, optional submit |
+| See the page | `browser_screenshot` | PNG capture, optionally saved for a vision model |
+See the full list in [Tool reference](#tool-reference).
+## Main features
+<table>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>Shared real browser</h3>
+      <p>A native view, not a headless screenshot. Human and agent operate the same page: the user sees every step and can take over; the agent drives the very window in front of the user.</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>DOM-level driving, framework-friendly</h3>
+      <p><code>browser_snapshot</code> returns numbered interactive elements; <code>browser_execute</code> runs JS in the page (native setters + input/change events for controlled inputs), so React/Vue pages work reliably.</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>Multi-tab sessions</h3>
+      <p>Open URLs in parallel tabs; list/switch/close/reset tabs while each session keeps its own state.</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>Multi-format content</h3>
+      <p>Fetch pages as html / markdown / txt / json, scoped by CSS selector, capped by character limit and timeout.</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>Per-task session isolation</h3>
+      <p>Each DSH task (session) gets its own browser session (own tabs and history); concurrent tasks never fight over the page or pollute each other. Calls within one task reuse the same session.</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>Login persistence</h3>
+      <p><code>browser_auth</code> exports/restores cookies so logins survive host restarts; self-hosted cookies are also persisted to disk.</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>CAPTCHA / bot-detection awareness</h3>
+      <p>Detects Cloudflare, reCAPTCHA, hCaptcha, Turnstile and generic challenges (<code>browser_challenge</code>, also flagged in snapshots); instead of retrying blindly, the agent asks the human to complete it in the shared window.</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>Batch form filling</h3>
+      <p><code>browser_fill</code> fills many fields at once — matched by selector/name/label, handling controlled inputs, selects, checkboxes and radio groups, with optional submit; one failing field never aborts the rest.</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>Operation history &amp; replay</h3>
+      <p><code>browser_history</code> logs operations (open/execute/click/type/fill/download/auth); <code>browser_replay</code> re-runs one step by sequence number.</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>Authenticated downloads</h3>
+      <p><code>browser_download</code> fetches a file in the page context with the session's cookies and writes it to disk — content behind login works too.</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>Safety restriction</h3>
+      <p><code>browser_restrict</code> limits which browser actions are allowed (allow-list) to prevent stray clicks/navigation; read-only tools (snapshot/content/screenshot) are never blocked.</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>Screenshot, save and read</h3>
+      <p><code>browser_screenshot</code> supports <code>savePath</code> to write the PNG to disk, ready for vision models (modlens etc.) to locate elements visually.</p>
+    </td>
+  </tr>
+</table>
+## Why this plugin
+- **Install-and-use, zero config**: no desktop shell or extra startup step required; on plain `dsh web` it self-hosts an Electron window and the `browser_*` tools just work.
+- **Human-in-the-loop, non-interfering**: the user sees and can take over every agent action; per-task isolation gives each parallel task its own tabs and history.
+- **Built for the real world**: CAPTCHA detection, login persistence, batch form filling, authenticated downloads, operation replay, action restriction — real-browser automation that is actually reliable.
+- **Testable, replaceable architecture**: the provider talks to Electron through the `ElectronBrowserViewHost` seam, so a future headless relay provider can serve remote deployments without touching the tool layer.
+## Tool reference
+| Tool | Purpose | Guard |
+| --- | --- | --- |
+| `browser_open` | Open a URL (optionally in a new tab); returns a page snapshot | ✅ |
+| `browser_snapshot` | Numbered inventory of interactive elements (inputs/buttons/links) | – |
+| `browser_execute` | Run JS in the page; args arrive as `arguments[0..n]` | ✅ |
+| `browser_content` | Fetch the page as html / markdown / txt / json (selector, maxChars, timeoutMs) | – |
+| `browser_click` | Click at viewport coordinates (for vision-located targets) | ✅ |
+| `browser_type` | Type text into the focused element (CDP `Input.insertText`) | ✅ |
+| `browser_fill` | Batch form fill (selector/name/label matching, controlled inputs, selects, checkbox/radio, optional submit) | ✅ |
+| `browser_screenshot` | PNG capture, optional `fullPage` and `savePath` | – |
+| `browser_list_tabs` | List the session's tabs | – |
+| `browser_switch_tab` | Switch to a tab by id (also switches the visible view when self-hosted) | ✅ |
+| `browser_close_tab` | Close a tab by id; closing the active tab activates the next | – |
+| `browser_reset` | Close all tabs of this task, back to one blank tab | ✅ |
+| `browser_session` | Show this task's browser session and tabs | – |
+| `browser_reset_session` | Close and rebuild this task's browser session | ✅ |
+| `browser_history` | Operation log (newest last), with per-step success/error and result summary | – |
+| `browser_replay` | Replay one step by sequence number (navigate/execute/click/type) | ✅ |
+| `browser_download` | Download a URL with session cookies to a local file (256 MB cap) | ✅ |
+| `browser_auth` | Export/restore cookies (login persistence, self-hosted) | ✅ |
+| `browser_challenge` | Detect a human-verification challenge (CAPTCHA / Cloudflare / reCAPTCHA / hCaptcha / Turnstile) | – |
+| `browser_restrict` | Restrict allowed browser actions (allow-list; empty list lifts it) | – |
+> "Guard" column: ✅ actions are governed by the `browser_restrict` allow-list; read-only tools (snapshot/content/screenshot/list_tabs/session/challenge/history) are never blocked.
+### Operating discipline (click/fill)
+- **Prefer DOM semantics over coordinates**: submit forms with `form.requestSubmit()`; click with `element.click()`; coordinate clicks are the last resort.
+- **Target the right element**: pages often have hidden duplicates (e.g. mobile buttons); filter visible elements with `browser_execute` (`getBoundingClientRect()` w/h > 0, `getComputedStyle` not `display:none`), then take coordinates.
+- **Click right after taking coordinates**: do not insert other operations in between (filling/scrolling moves elements and invalidates old coordinates).
+- **Verify before clicking**: use `document.elementFromPoint(x, y)` to confirm the coordinate hits the intended element (button/link), then perform the real click.
+- **DPR awareness**: CDP input uses CSS pixels; on high-DPI screens calibrate with `elementFromPoint` instead of guessing coordinates.
 ## Configuration
-The plugin mounts through `cordis.patch.yml`; per-row config:
+The plugin mounts through `cordis.patch.yml` (three rows); per-row config:
 | Row | Key | Type | Default | Description |
-|---|---|---|---|---|
+| --- | --- | --- | --- | --- |
 | `browser-electron` | `viewHost` | object | required | `ElectronBrowserViewHost` supplied by the host shell (typically `!!js ctx.get('electronViewHost')`) |
-| `browser-electron` | `httpOnly` | boolean | `true` | Allow HTTP(S) navigation only; other protocols (e.g. `file:`/`data:`) are rejected (`BROWSER_NAVIGATION_BLOCKED`) |
+| `browser-electron` | `httpOnly` | boolean | `true` | Allow HTTP(S) navigation only; other protocols (e.g. `file:`/`data:`) rejected (`BROWSER_NAVIGATION_BLOCKED`) |
 | `browser-electron` | `snapshotMaxElements` | number | `60` | Max snapshot elements before truncation |
 | `browser-electron` | `contentMaxChars` | number | `100000` | Default content character cap |
 | `tool-browser` | `timeoutMs` | number | `60000` | Cooperative tool timeout (ms) |
 | `tool-browser` | `tabTools` | boolean | `true` | Register tab-management tools (`browser_list_tabs` etc.) |
-## Tools
-| Tool | Purpose |
-|---|---|
-| `browser_open` | Open a URL (optionally a new tab); returns a snapshot |
-| `browser_snapshot` | Numbered inventory of interactive elements (inputs/buttons/links) |
-| `browser_execute` | Run JS in the page; args arrive as `arguments[0..n]` |
-| `browser_content` | Fetch the page as html / markdown / txt / json (selector, maxChars, timeoutMs) |
-| `browser_screenshot` | PNG capture, optional `fullPage` |
-| `browser_list_tabs` / `browser_switch_tab` / `browser_close_tab` / `browser_reset` | Multi-tab session management |
-### Interaction discipline (clicking / filling forms)
-- **Prefer DOM semantics over coordinates**: submit forms with `form.requestSubmit()`, click with `element.click()`; coordinate clicks are the last resort.
-- **Target the right element**: pages often carry hidden copies (e.g. mobile buttons). Use `browser_execute` to filter visible elements (`getBoundingClientRect()` with width/height > 0, computed style not `display:none`) before reading coordinates.
-- **Click immediately after reading coordinates**: do not interleave other actions (filling, scrolling moves elements; stale coordinates fail instantly).
-- **Verify the hit before clicking**: `document.elementFromPoint(x, y)` confirms the coordinate actually lands on the target button/link before issuing a real click.
-- **Mind DPR**: CDP input uses CSS pixels; on high-DPI screens, calibrate with `elementFromPoint` instead of blind coordinate tries.
 ## How it works
 ```
@@ -111,26 +174,73 @@ agent (browser_* tools)
   → WebContentsView + webContents.debugger (CDP)
 ```
-The provider is Electron-agnostic by construction: it operates through the `ElectronBrowserViewHost` seam (create/destroy/show views, `sendCommand`), which a real shell implements with Electron objects. The same seam lets a future relay provider (headless Chromium screenshot stream) serve remote deployments without touching the tools.
+- **Seam** (`browser` row): provides the `ctx.browser` service — provider registration, session lifecycle, error codes — decoupled from any implementation.
+- **Provider** (`browser-electron` row): operates views through the `ElectronBrowserViewHost` seam (create/destroy/show, `sendCommand`), implemented with real Electron objects by the shell.
+- **Tools** (`tool-browser` row): the 20 model-facing `browser_*` tools, maintaining one browser session per calling task (DSH session).
+**Self-hosted mode**: without a desktop shell, the plugin spawns its own Electron child process (`host-main.js`) and drives it over loopback TCP JSON-RPC (window title `dsh-browser`). The child auto-restarts after a crash; screenshots prefer Electron's native `capturePage` (CDP capture can hang with multiple views in the window); the plugin automatically picks the **newest** Electron in the environment (33.x has a compositor defect; ≥ 40 recommended).
+**Electron lookup order**: ① `require('electron')` (peer dependency) → ② `ELECTRON_PATH` (explicit override) → ③ the newest among DSH install anchors and pnpm virtual stores. A clear error tells you when none is found.
 ## Division of labor with the desktop shell
-The browser's **visible view**, the **browser column layout**, and the **column-to-view alignment** belong to the host shell (e.g. dsh's `apps/desktop`), not this plugin. This plugin only consumes the shell-provided `electronViewHost` and owns the seam, provider, and tools. Installing the plugin without a matching shell leaves the capability disabled.
+The browser's **visible view**, the **browser column layout**, and the **column-to-view alignment** belong to the host shell (e.g. dsh's `apps/desktop`), not this plugin. This plugin consumes the shell-provided `electronViewHost` and owns the seam, provider, and tools. Without a shell the plugin **self-hosts** and everything still works.
+## Requirements
+- DeepSeek Harness (dsh) with the `web` profile
+- **Electron runtime** (optional peer dependency): the desktop shell carries it; on plain `dsh web` the plugin locates an Electron binary automatically (see above; ≥ 40 recommended)
+### Verified versions
+| Component | Version |
+| --- | --- |
+| DeepSeek Harness (dsh) | `0.1.0-rc.5` |
+| Electron | `43.4.0` (≥ 40 recommended; 33.x has a compositor defect) |
+| Node.js | `22.20.0` |
+| dsh-builtin-browser | `0.1.11` |
+| OS | Windows 10 (10.0.26200) |
+> The plugin declares `electron >= 30`; other platforms (macOS/Linux) run the same protocol but were only verified on the Windows environment above.
 ## Known limitations
 - Screenshots are PNG-only (CDP JPEG hangs on Electron 43); JPEG awaits a non-CDP conversion path.
 - Self-hosted captures prefer Electron's native `capturePage` (CDP `captureScreenshot` can hang with multiple views in the window); the target tab is raised before capturing.
-- **Electron >= 40 is recommended**: 33.x has a compositor defect that intermittently breaks capture ("display surface not available"). The plugin automatically picks the newest Electron in the environment (peer dependency > `ELECTRON_PATH` > newest among anchors/pnpm store).
 - `fullPage` capture is flaky under software compositing on some hosts.
-- Sessions are isolated per task (each DSH session gets its own browser session with own tabs/history; concurrent tasks never interfere; calls within one task reuse the same session). Logins (cookies) are shared and can be exported/restored with `browser_auth`.
-- CAPTCHA cannot be solved automatically: snapshots flag detected challenges (`browser_challenge` checks explicitly); ask the human to complete it in the shared window instead of retrying.
+- CAPTCHA cannot be solved automatically: snapshots flag detected challenges; ask the human to complete it in the shared window instead of retrying.
 - Private mode (`privateMode`) is not implemented: it needs Electron session partitioning, which is host-layer territory; this plugin does not promise it.
 - `browser_download` fetches in the page context (keeps logins) and is subject to same-origin/CORS constraints; single files are capped at 256 MB.
 - The `browser_auth` cookie round-trip does not preserve `hostOnly`/`sameSite` (host-only cookies come back as domain cookies); it is available on the self-hosted browser only.
 - After a self-hosted child crash the browser host restarts automatically, but sessions opened before the crash are gone — call `browser_reset_session` to rebuild.
 - This plugin contains no browser-column UI — that is host-shell territory; do not treat "browser column" as a plugin feature.
+## Development
+```sh
+# Type-check + build (lib/)
+pnpm run build
+# Functional tests: start a local page server + Electron probe (see repo test scripts)
+```
+Code layout:
+| Directory | Responsibility |
+| --- | --- |
+| `src/browser/` | The `ctx.browser` seam and all request/result types |
+| `src/browser-electron/` | Electron CDP provider, self-hosted child (`host-main.ts`), RPC layer |
+| `src/tool-browser/` | Model-facing `browser_*` tools |
+| `src/types/` | Electron ambient types (shim; no hard electron type dependency) |
+## Acknowledgements
+Special thanks to the [DeepSeek Harness repository](https://github.com/deepseek-ai/deepseek-harness) and the DeepSeek AI team: the seam, the tool runtime, and the plugin system this plugin builds on all come from that project.
+Thanks as well to [Cordis](https://github.com/cordiverse/cordis) for the plugin foundation, and to everyone in the community who discussed, tested, gave feedback, and built plugins.
 ## License
-MIT
+MIT License, see [LICENSE](LICENSE).
+> This project is a community plugin for DeepSeek Harness, not an official DeepSeek product.

package/README.md CHANGED Viewed

@@ -1,80 +1,162 @@
-# dsh-builtin-browser（DeepSeek Harness 内置共享真实浏览器插件）
+<p align="center">
+  <img src="https://img.shields.io/github/stars/wqty123/dsh-browser?style=flat&amp;label=%E2%98%85&amp;color=08C" alt="GitHub stars">
+  <img src="https://img.shields.io/npm/v/dsh-builtin-browser?style=flat&amp;label=npm&amp;color=CB3837" alt="npm version">
+  <img src="https://img.shields.io/badge/license-MIT-2EA44F?style=flat" alt="MIT License">
+  <img src="https://img.shields.io/badge/DSH-Plugin-47848F?style=flat" alt="DeepSeek Harness plugin">
+  <img src="https://img.shields.io/badge/Windows%20%7C%20macOS%20%7C%20Linux-4493F8?style=flat-square" alt="Platforms">
+</p>
-[English](README.en.md) | 中文
+<p align="center"><sub>中文 · <a href="README.en.md">English</a></sub></p>
-面向 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 的**共享真实浏览器**能力插件:装好即可用——agent 通过 CDP 驱动一个真实、可见、可随时人工接管的浏览器,人与 agent 操作的是**同一个页面**。
+<h3 align="center">为 DeepSeek Harness 生态打造的<b>共享真实浏览器</b>插件（装好即用，人机同页）</h3>
-**我们的优势**
+<h4 align="center">agent 驱动一个真实、可见、可随时人工接管的浏览器——人与 agent 操作的是<b>同一个页面</b>。</h4>
-- **装好即用**:有桌面外壳时嵌入外壳视图;纯 `dsh web` 也能**自托管**拉起自己的 Electron 窗口,不需要任何额外配置。
-- **真实可见、人工可接管**:不是无头截屏或转播,用户随时能看到页面并直接接手操作。
-- **DOM 级驱动,框架友好**:React/Vue 受控组件也能可靠填写。
-- **并发安全**:任务级浏览器会话隔离,多个任务并行互不抢页面、不互相污染。
-- **面向真实世界**:人机验证识别、登录态持久化、批量填表、带登录态下载、操作回放、动作限制,一应俱全。
+## 这是什么
-## 功能
+`dsh-builtin-browser` 给 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 提供浏览器能力:
-**浏览器能力**
+- **真实视图,而非转播**:浏览器是原生 `WebContentsView`,用户直接看到 agent 在做什么,随时可以上手接管;
+- **装好即用**:有桌面外壳时嵌入外壳视图;纯 `dsh web` 也能**自托管**——插件自己拉起一个 Electron 窗口,不需要任何额外配置;
+- **一插件即一套工具**:安装后 agent 自动获得 20 个 `browser_*` 工具(打开、查看、操作、填表、截图、下载、登录态管理……)。
-- **真实视图,而非转播。** 浏览器是原生视图(`WebContentsView`),用户可直接看到并操作;agent 驱动的是同一个页面。视图由宿主外壳提供,插件负责驱动。
-- **DOM 引用,而非猜坐标。** `browser_snapshot` 返回带编号的交互元素;`browser_execute` 在页面里执行 JS(框架输入用原生 setter),在 React/Vue 页面上也能可靠交互。
-- **多标签会话。** 并行打开 URL、查看/切换/关闭/重置标签,状态保持。
-- **多格式内容。** 以 html / markdown / txt / json 抓取页面,支持 selector 限定、长度与超时上限。
+一句话:**安装插件 = 获得一个与用户共享、可被 agent 驱动的真实浏览器。**
-**工程化能力**
-- **按任务隔离。** 每个 DSH 任务(会话)拥有独立的浏览器会话(独立标签页与历史),并发任务互不干扰;同一任务内多次调用复用同一会话(`browser_session` / `browser_reset_session`)。
-- **登录态持久化。** `browser_auth` 导出/恢复 cookie,重启后登录态不丢。
-- **人机验证识别。** 自动检测 Cloudflare / reCAPTCHA / hCaptcha / Turnstile 等挑战(`browser_challenge`,快照也会标注),提示人工在共享窗口完成,不再盲目重试。
-- **批量表单填充。** `browser_fill` 一次填写多个字段:按选择器/名称/标签匹配,支持受控输入、下拉、单选/复选,可选提交。
-- **操作历史与回放。** `browser_history` 记录操作日志,`browser_replay` 可回放某一步。
-- **带登录态下载。** `browser_download` 用会话 cookie 把文件取到本地,登录后内容可直接落盘。
-- **安全限制。** `browser_restrict` 限制允许的浏览器动作,防误点/误导航。
-## 环境要求
-- DeepSeek Harness(dsh)且安装了 `web` profile
-- **Electron 运行时**(可选 peer 依赖):桌面外壳自带;纯 `dsh web` 下需要能定位到 Electron 二进制(见下)
-- 有桌面外壳(`ctx.electronViewHost`)时用外壳嵌入视图;没有时插件**自托管**:自己拉起一个 Electron 窗口,`browser_*` 工具照常可用
-**Electron 定位顺序**:① `require('electron')`(peer 依赖已装)→ ② DSH 安装锚点 → ③ `node_modules/.pnpm` 虚拟仓库 → ④ `ELECTRON_PATH` 环境变量。找不到时工具会报清晰的错误提示。
-### 验证过的版本
-| 组件 | 版本 |
-|---|---|
-| DeepSeek Harness(dsh) | `0.1.0-rc.5` |
-| Electron | `43.4.0` |
-| Node.js | `22.20.0` |
-| dsh-builtin-browser | `0.1.7` |
-| 操作系统 | Windows 10 (10.0.26200) |
-> 插件声明 `electron >= 30`;其他平台(如 macOS/Linux)按同一协议运行,但仅在上表 Windows 环境实测。
-## 安装
+## 快速开始
 ```sh
-dsh plugin --profile web add dsh-builtin-browser   # 发布到 npm 后
-# 或从源码目录(独立仓库,一插件一仓库):
+# 方式一:从 npm 安装(已发布)
+dsh plugin --profile web add dsh-builtin-browser
+# 方式二:从源码目录安装(独立仓库,一插件一仓库)
 dsh plugin --profile web add <本仓库路径>
 ```
-这会链接插件、把 `dsh-builtin-browser` 加入 profile 的 bundle 层,并挂载:
+安装后,agent 即可使用浏览器工具,例如:
+| 想做什么 | 用哪个工具 | 说明 |
+| --- | --- | --- |
+| 打开页面 | `browser_open` | 打开 URL,返回带编号元素的快照 |
+| 了解页面 | `browser_snapshot` | 输入框/按钮/链接的编号清单,可据此定位 |
+| 操作页面 | `browser_execute` | 在页面里执行 JS(原生 setter,框架友好) |
+| 填写表单 | `browser_fill` | 一次填写多个字段,可选提交 |
+| 看到页面 | `browser_screenshot` | PNG 截图,可存文件交给视觉模型 |
+完整清单见[工具参考](#工具参考)。
+## 主要功能
+<table>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>共享真实浏览器</h3>
+      <p>原生视图而非无头截屏。用户与 agent 操作同一个页面:用户能看到每一步,随时接管;agent 驱动的就是用户眼前那个窗口。</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>DOM 级驱动,框架友好</h3>
+      <p><code>browser_snapshot</code> 返回带编号的交互元素;<code>browser_execute</code> 在页面内执行 JS(受控输入用原生 setter + input/change 事件),React/Vue 页面也能可靠交互。</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>多标签会话</h3>
+      <p>并行打开 URL,查看/切换/关闭/重置标签,每个会话的状态独立保持。</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>多格式内容</h3>
+      <p>以 html / markdown / txt / json 抓取页面,支持 CSS selector 限定、字符上限与超时控制。</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>任务级会话隔离</h3>
+      <p>每个 DSH 任务(会话)拥有独立的浏览器会话(独立标签页与历史),并发任务互不抢页面、互不污染;同一任务内多次调用复用同一会话。</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>登录态持久化</h3>
+      <p><code>browser_auth</code> 导出/恢复 cookie,重启后登录态不丢;自托管实例的 cookie 本身也落盘持久。</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>人机验证识别</h3>
+      <p>自动检测 Cloudflare / reCAPTCHA / hCaptcha / Turnstile 等挑战(<code>browser_challenge</code>,快照也会标注),提示人工在共享窗口完成,不再盲目重试。</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>批量表单填充</h3>
+      <p><code>browser_fill</code> 一次填写多个字段:按选择器/名称/标签匹配,支持受控输入、下拉、单选/复选,可选提交;单个字段失败不影响其余字段。</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>操作历史与回放</h3>
+      <p><code>browser_history</code> 记录操作日志(打开/执行/点击/输入/填表/下载/登录),<code>browser_replay</code> 可按序号回放某一步。</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>带登录态下载</h3>
+      <p><code>browser_download</code> 在页面上下文内携带会话 cookie 拉取文件并落盘,登录后才能访问的内容也能直接下载。</p>
+    </td>
+  </tr>
+  <tr>
+    <td width="50%" valign="top">
+      <h3>安全限制</h3>
+      <p><code>browser_restrict</code> 限制允许的浏览器动作(白名单),防止 agent 误点、误导航;只读工具(snapshot/content/screenshot)不受限。</p>
+    </td>
+    <td width="50%" valign="top">
+      <h3>截图即存即读</h3>
+      <p><code>browser_screenshot</code> 支持 <code>savePath</code> 直接落盘 PNG,交给视觉模型(modlens 等)做基于视觉的元素定位。</p>
+    </td>
+  </tr>
+</table>
+## 为什么选它
+- **装好即用,零配置**:不需要桌面外壳、不需要额外启动步骤;纯 `dsh web` 环境自托管拉起 Electron 窗口,`browser_*` 工具照常可用。
+- **人机协同,互不干扰**:用户能看到并接管 agent 的每一个动作;任务级会话隔离让多个并行任务各自拥有独立的标签页与历史。
+- **面向真实世界的自动化**:人机验证识别、登录态持久化、批量填表、带登录态下载、操作回放、动作限制——把"真实浏览器"变成可靠的 agent 能力。
+- **可测试、可替换的架构**:provider 与 Electron 通过 `ElectronBrowserViewHost` 接缝解耦,同一套工具层未来可对接无头转播 provider,无需改动模型侧。
+## 工具参考
+| 工具 | 用途 | 守卫 |
+| --- | --- | --- |
+| `browser_open` | 打开 URL(可选新标签),返回页面快照 | ✅ |
+| `browser_snapshot` | 交互元素(输入框/按钮/链接)带编号清单 | – |
+| `browser_execute` | 在页面执行 JS;参数以 `arguments[0..n]` 传入 | ✅ |
+| `browser_content` | 以 html / markdown / txt / json 抓取页面(selector、maxChars、timeoutMs) | – |
+| `browser_click` | 按视口坐标点击(配合截图做视觉定位) | ✅ |
+| `browser_type` | 向聚焦元素输入文本(CDP `Input.insertText`) | ✅ |
+| `browser_fill` | 批量填充表单(选择器/名称/标签匹配,受控输入、下拉、单选/复选,可选提交) | ✅ |
+| `browser_screenshot` | PNG 截图,可选 `fullPage` 与 `savePath` | – |
+| `browser_list_tabs` | 当前会话的标签列表 | – |
+| `browser_switch_tab` | 按 id 切换标签(自托管下同步切换可见视图) | ✅ |
+| `browser_close_tab` | 按 id 关闭标签;关闭活动标签后激活下一个 | – |
+| `browser_reset` | 关闭本任务所有标签,回到一个空白标签 | ✅ |
+| `browser_session` | 查看本任务的浏览器会话与标签 | – |
+| `browser_reset_session` | 关闭并重建本任务的浏览器会话 | ✅ |
+| `browser_history` | 操作日志(最新在后),含成功/失败与结果摘要 | – |
+| `browser_replay` | 按序号回放某一步(navigate/execute/click/type) | ✅ |
+| `browser_download` | 带会话 cookie 下载 URL 到本地文件(上限 256MB) | ✅ |
+| `browser_auth` | 导出/恢复 cookie(登录态持久化,自托管可用) | ✅ |
+| `browser_challenge` | 检测人机验证(CAPTCHA / Cloudflare / reCAPTCHA / hCaptcha / Turnstile) | – |
+| `browser_restrict` | 限制允许的浏览器动作(白名单;空列表解除) | – |
+> 「守卫」列:打 ✅ 的动作受 `browser_restrict` 白名单约束;只读工具(snapshot/content/screenshot/list_tabs/session/challenge/history)永不拦截。
-| 行 | 子路径 | 角色 |
-|---|---|---|
-| `browser` | `dsh-builtin-browser/browser` | `ctx.browser` 能力 seam(始终挂载) |
-| `browser-electron` | `dsh-builtin-browser/browser-electron` | Electron CDP provider(需要 `electronViewHost`) |
-| `tool-browser` | `dsh-builtin-browser/tool-browser` | `browser_*` 模型侧工具 |
+### 操作纪律(点击/填表)
-provider 与工具以 `ctx.get('electronViewHost')` 是否存在为门控,因此没有桌面外壳的组合只会保留 seam,其余不启用。
+- **优先用 DOM 语义而非坐标**:表单提交优先 `form.requestSubmit()`;点击优先 `element.click()`;坐标点击是最后手段。
+- **选中正确的元素**:页面常有隐藏副本(如移动端按钮),用 `browser_execute` 过滤可见元素(`getBoundingClientRect()` 宽高 > 0、`getComputedStyle` 非 `display:none`),再取坐标。
+- **取坐标后立即点击**:中间不要插入其他操作(填表、滚动会移动元素,旧坐标立即失效)。
+- **点击前验证命中**:`document.elementFromPoint(x, y)` 确认该坐标确实是目标元素(按钮/链接),再执行真实点击。
+- **DPR 注意**:CDP 输入使用 CSS 像素;高 DPI 屏上若点击落空,用 `elementFromPoint` 校准,不要盲试坐标。
 ## 配置
-插件通过 `cordis.patch.yml` 挂载,各行的配置:
+插件通过 `cordis.patch.yml` 挂载三行,各行配置:
 | 行 | 配置项 | 类型 | 默认 | 说明 |
-|---|---|---|---|---|
+| --- | --- | --- | --- | --- |
 | `browser-electron` | `viewHost` | 对象 | 必填 | 宿主提供的 `ElectronBrowserViewHost` 实例(通常 `!!js ctx.get('electronViewHost')`) |
 | `browser-electron` | `httpOnly` | 布尔 | `true` | 仅允许 HTTP(S) 导航;其余协议(如 `file:`/`data:`)拒绝(`BROWSER_NAVIGATION_BLOCKED`) |
 | `browser-electron` | `snapshotMaxElements` | 数字 | `60` | 快照最多收录的交互元素数,超出截断 |
@@ -82,32 +164,6 @@ provider 与工具以 `ctx.get('electronViewHost')` 是否存在为门控,因此
 | `tool-browser` | `timeoutMs` | 数字 | `60000` | 工具协作超时(ms) |
 | `tool-browser` | `tabTools` | 布尔 | `true` | 是否注册标签管理工具(`browser_list_tabs` 等) |
-## 工具
-| 工具 | 用途 |
-|---|---|
-| `browser_open` | 打开 URL(可选新标签);返回快照 |
-| `browser_snapshot` | 交互元素(输入框/按钮/链接)带编号清单 |
-| `browser_execute` | 在页面执行 JS;参数以 `arguments[0..n]` 传入 |
-| `browser_content` | 以 html / markdown / txt / json 抓取页面(selector、maxChars、timeoutMs) |
-| `browser_screenshot` | PNG 截图,可选 `fullPage` |
-| `browser_list_tabs` / `browser_switch_tab` / `browser_close_tab` / `browser_reset` | 多标签会话管理 |
-| `browser_session` / `browser_reset_session` | 查看/重置当前任务的浏览器会话(按任务隔离) |
-| `browser_history` / `browser_replay` | 查看操作记录并回放某一步 |
-| `browser_download` | 带会话 cookie 下载 URL 到本地文件 |
-| `browser_restrict` | 限制允许的浏览器动作,防误点/误导航 |
-| `browser_auth` | 导出/恢复 cookie,持久化登录态 |
-| `browser_challenge` | 检测人机验证(CAPTCHA / Cloudflare / reCAPTCHA / hCaptcha / Turnstile)是否拦截当前页 |
-| `browser_fill` | 批量填充表单(选择器/名称/标签匹配,支持受控输入、下拉、单选/复选,可选提交) |
-### 操作纪律(点击/填表)
-- **优先用 DOM 语义而非坐标**:表单提交优先 `form.requestSubmit()`;点击优先 `element.click()`;坐标点击是最后手段。
-- **选中正确的元素**:页面常有隐藏副本(如移动端按钮),用 `browser_execute` 过滤可见元素(`getBoundingClientRect()` 宽高 > 0、`getComputedStyle` 非 `display:none`),再取坐标。
-- **取坐标后立即点击**:中间不要插入其他操作(填表、滚动会移动元素,旧坐标立即失效)。
-- **点击前验证命中**:`document.elementFromPoint(x, y)` 确认该坐标确实是目标元素(按钮/链接),再执行真实点击。
-- **DPR 注意**:CDP 输入使用 CSS 像素;高 DPI 屏上若点击落空,用 `elementFromPoint` 校准,不要盲试坐标。
 ## 工作原理
 ```
@@ -118,26 +174,73 @@ agent (browser_* 工具)
   → WebContentsView + webContents.debugger (CDP)
 ```
-provider 按构造与 Electron 解耦:它通过 `ElectronBrowserViewHost` 接缝操作(创建/销毁/显示视图、`sendCommand`),由真实外壳用 Electron 对象实现。同一接缝也让未来的转播 provider(无头 Chromium 截图流)服务远程部署,而无需改动工具。
+- **seam 层**(`browser` 行)提供 `ctx.browser` 服务:provider 注册、会话生命周期、错误码,与具体实现解耦;
+- **provider 层**(`browser-electron` 行)通过 `ElectronBrowserViewHost` 接缝操作视图(创建/销毁/显示/`sendCommand`),由真实外壳用 Electron 对象实现;
+- **工具层**(`tool-browser` 行)提供模型侧的 20 个 `browser_*` 工具,按调用方任务(DSH 会话)维护独立的浏览器会话。
+**自托管模式**:没有桌面外壳时,插件自己拉起一个 Electron 子进程(`host-main.js`),通过本机 TCP JSON-RPC 驱动,窗口标题 `dsh-browser`。子进程崩溃会自动重启;截图优先走 Electron 原生 `capturePage`(CDP 截图在多视图下会挂起),并自动选择环境中**最新版本**的 Electron(33.x 有合成器缺陷,建议 ≥ 40)。
+**Electron 定位顺序**:① `require('electron')`(peer 依赖)→ ② `ELECTRON_PATH`(显式覆盖)→ ③ DSH 安装锚点与 pnpm 虚拟仓库中**版本最新**者。找不到时工具会报清晰的错误提示。
 ## 与桌面外壳的分工
-浏览器**可见视图**、**浏览器列布局**、**列与视图的对齐**都属于宿主外壳(如 dsh 的 `apps/desktop`),不在本插件内。本插件只消费外壳提供的 `electronViewHost`,负责 seam、provider 与工具。若你只装插件而没有配套外壳,插件功能保持禁用。
+浏览器**可见视图**、**浏览器列布局**、**列与视图的对齐**都属于宿主外壳(如 dsh 的 `apps/desktop`),不在本插件内。本插件只消费外壳提供的 `electronViewHost`,负责 seam、provider 与工具。没有配套外壳时插件**自托管**,功能照常可用。
+## 环境要求
+- DeepSeek Harness(dsh)且安装了 `web` profile
+- **Electron 运行时**(可选 peer 依赖):桌面外壳自带;纯 `dsh web` 下插件自动定位 Electron 二进制(见上,建议 ≥ 40)
+### 验证过的版本
+| 组件 | 版本 |
+| --- | --- |
+| DeepSeek Harness(dsh) | `0.1.0-rc.5` |
+| Electron | `43.4.0`(推荐 ≥ 40;33.x 存在合成器缺陷) |
+| Node.js | `22.20.0` |
+| dsh-builtin-browser | `0.1.11` |
+| 操作系统 | Windows 10 (10.0.26200) |
+> 插件声明 `electron >= 30`;其他平台(macOS/Linux)按同一协议运行,但仅在上表 Windows 环境实测。
 ## 已知限制
 - 截图仅 PNG(CDP JPEG 在 Electron 43 上挂起);JPEG 等待非 CDP 转换路径。
 - 自托管截图优先走 Electron 原生 `capturePage`(CDP `captureScreenshot` 在多视图下会挂起);截图前自动把目标标签置顶。
-- **Electron 版本建议 ≥ 40**:33.x 存在合成器缺陷,会间歇性导致截图失败("display surface not available")。插件会**自动选择环境中最新版本的 Electron**(peer 依赖 > `ELECTRON_PATH` > 锚点/pnpm store 中最新版)。
 - 部分主机在软件合成下 `fullPage` 截图不稳定。
-- 会话按任务隔离:每个调用方任务(DSH 会话)拥有独立的浏览器会话(独立标签页与历史),并发任务互不干扰;同一任务的多次调用复用同一会话。登录态(cookie)为共享,可用 `browser_auth` 导出/恢复。
-- 人机验证(CAPTCHA)无法自动解决:快照会标注检测到的挑战(`browser_challenge` 可显式检查),此时应请用户在共享窗口中人工完成,而不是反复重试。
+- 人机验证(CAPTCHA)无法自动解决:快照会标注检测到的挑战,此时应请用户在共享窗口中人工完成,而不是反复重试。
 - 无痕模式(`privateMode`)未实现:它需要 Electron 的 session 分区能力,属于宿主层,本插件不承诺。
 - `browser_download` 在页面上下文内 `fetch`(带登录态),受同源/CORS 约束;单文件上限 256MB。
 - `browser_auth` 的 cookie 往返不保留 `hostOnly`/`sameSite` 字段(host-only cookie 恢复后变成 domain cookie);仅自托管浏览器可用。
 - 自托管浏览器子进程崩溃后会自动重启,但崩溃前已打开的会话视图已失效,调用 `browser_reset_session` 重建即可。
 - 本插件不含浏览器列 UI——那是宿主外壳的配套,别把"浏览器列"当成插件能力。
-## 许可证
+## 开发
+```sh
+# 类型检查 + 构建(lib/)
+pnpm run build
+# 功能测试:启动本地页面服务器 + Electron probe(见仓库测试脚本)
+```
+代码结构:
+| 目录 | 职责 |
+| --- | --- |
+| `src/browser/` | `ctx.browser` seam 与全部请求/结果类型 |
+| `src/browser-electron/` | Electron CDP provider、自托管子进程(`host-main.ts`)与 RPC 层 |
+| `src/tool-browser/` | 模型侧 `browser_*` 工具 |
+| `src/types/` | electron 环境类型(shim,避免强制依赖 electron 类型) |
+## 特别感谢
+特别感谢 [DeepSeek Harness 原始仓库](https://github.com/deepseek-ai/deepseek-harness) 与 DeepSeek AI 团队:本插件的 seam、工具运行时与插件体系都构建在这个项目之上。
+同时感谢 [Cordis](https://github.com/cordiverse/cordis) 提供的插件化基础,以及所有参与讨论、测试、反馈和插件开发的社区成员。
+## License
+本项目遵循 [MIT License](LICENSE)。
-MIT
+> 本项目是 DeepSeek Harness 的社区插件,并非 DeepSeek 官方产品。

package/package.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "dsh-builtin-browser",
-  "version": "0.1.11",
+  "version": "0.1.12",
   "description": "Shared real browser plugin for DeepSeek Harness: install-and-use — a visible native browser the human can take over, driven by the agent over CDP. DOM-level interaction (React/Vue safe), per-task session isolation, cookie persistence (browser_auth), CAPTCHA detection, batch form fill (browser_fill), authenticated download, history/replay, action restriction",
   "type": "module",
   "main": "lib/index.js",