@restregular/dsh-visual-browser 0.0.0-stage → 0.1.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 RestRegular
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,124 @@
-# Temporary Holding Version
+# dsh-visual-browser
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+A DeepSeek Harness plugin that gives the Agent a **real Chrome** and gives you a **window into it**.
+
+- The Agent never reads the DOM. It reads a numbered list of *visible interactive
+  elements* plus a short text excerpt — a few hundred tokens per page instead of a
+  full snapshot.
+- Every action the Agent takes (click, type, key, scroll, navigation) is mirrored
+  into a **面板 (panel)** with the page frame, an operation timeline you can replay,
+  and a picture you can click to take over the browser yourself.
+- Cookies persist across restarts (one dedicated Chrome profile), so a login the
+  Agent performs once keeps working.
+
+## Install
+
+```bash
+# The plugin is a normal DSH bundle; install it from the registry
+dsh plugin add @restregular/dsh-visual-browser
+```
+
+Or from the Harness UI: **Plugins → Add → `@restregular/dsh-visual-browser`**.
+
+Requirements: Google Chrome, Microsoft Edge, or Chromium on the machine
+(auto-detected), and Node.js 22 or newer (the plugin ships no runtime
+dependencies; it talks CDP over the built-in WebSocket).
+
+## Use
+
+1. Click the **浏览器 / Browser** icon in the sidebar to open the panel.
+2. Ask the Agent something that needs a website, for example
+   *"打开 example.com 看看首页写了什么"*. The Agent calls `browser_open`.
+3. Watch the frame update. Click the picture to click the page; wheel over it to
+   scroll; use the toolbar for back / forward / reload / live view / stop.
+
+## Tools the Agent gets
+
+| Tool | Purpose |
+| --- | --- |
+| `browser_open` | Open a URL (launches Chrome on first use) and return the numbered view. |
+| `browser_see` | Fresh numbered view of the current page. |
+| `browser_act` | `click` / `type` / `key` / `hover` / `scroll` / `select` / `back` / `forward` / `reload` / `wait` / `close`, then return the new view. |
+| `browser_text` | Read text of the page or of one element (a link also returns its `href`). |
+| `browser_screenshot` | Save a PNG/JPEG to disk and return the path (feed it to `read_image`). |
+
+### What the Agent sees
+
+```
+URL: https://example.com/login
+TITLE: 登录 - Example
+VIEW: 1280x800 scroll 0/2400 · 4/6 interactive
+[1] textbox "手机号" value=13800000000
+[2] textbox "密码" value=••••••
+[3] button "登录"
+[4] link "忘记密码" → /forgot
+HEADINGS: 用户登录 | 没有账号?
+TEXT: 用户登录 手机号 密码 登录 忘记密码
+ACTIVE: [1] textbox "手机号"
+```
+
+Numbers are reassigned on every view, so an action always uses the newest
+numbers; a stale one fails with a clear "take a fresh snapshot" message instead
+of clicking the wrong element.
+
+## Configuration
+
+Row config (in the profile's `cordis.patch.yml`, override by row id
+`dsh-visual-browser`):
+
+```yaml
+- id: dsh-visual-browser
+  config:
+    headless: true            # false = visible Chrome window
+    executablePath: ''        # explicit chrome.exe path
+    userDataDir: ''           # default: ~/.dsh/visual-browser/chrome-profile
+    viewport:
+      width: 1280
+      height: 800
+    maxElements: 100          # cap on the numbered list
+    maxTextChars: 320         # cap on the text excerpt
+    frameQuality: 62          # JPEG quality of panel frames
+    timelineFrames: 24        # replayable frames kept in memory
+    allowUserControl: true    # allow clicks from the panel
+    actionTimeoutMs: 12000
+    navigationTimeoutMs: 30000
+```
+
+## How it works
+
+- **Host** (`index.js`, `lib/`): launches Chrome with
+  `--remote-debugging-port=0 --user-data-dir=<profile>`, parses the DevTools
+  endpoint from stderr, attaches one flattened CDP session, and exposes:
+  - five tools through the `tools` service (registration is effect-scoped),
+  - a same-origin prefix route `/visual-browser/api` (`/state`, `/frame`,
+    `/live`, `/start`, `/stop`, `/capture`, `/click`, `/wheel`, `/key`, `/nav`)
+    that the panel polls and posts to.
+- **Snapshot** (`lib/page-scripts.js`): one serialized function runs in the page,
+  filters interactive elements to the viewport, assigns `data-dsh-vb-ref`
+  numbers, and returns structured JSON. Nothing else about the DOM leaves the
+  page.
+- **Frames** stay on the host: the panel displays the newest JPEG by sequence
+  number, so a live picture costs no session tokens. The timeline holds the last
+  N frames; clicking a row replays that moment.
+- **One code path**: panel clicks, wheel events and keys run the same
+  `BrowserManager` methods the tools use.
+
+Chrome is killed with its process tree when the plugin unloads.
+
+## Notes and limits
+
+- Cross-origin iframes are counted but not inspected; open the direct URL when a
+  flow lives inside a frame.
+- Blocking JS dialogs are auto-accepted and logged in the timeline.
+- The panel route is served by the Harness web server on its existing origin;
+  it performs no extra authentication, matching the other local panel routes.
+- Headless mode is the default. Some sites behave differently headless; set
+  `headless: false` when a site insists on a window.
+
+## Development
+
+```bash
+npm test    # syntax checks + snapshot/format/route/package smoke tests
+```
+
+MIT © RestRegular