dsh-sidebar-control 0.1.2 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +80 -0
- package/LICENSE +1 -3
- package/README.md +295 -63
- package/client.js +487 -214
- package/index.js +97 -14
- package/package.json +2 -1
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.2.1 — 2026-11-08
|
|
4
|
+
|
|
5
|
+
Live browser-control campaign (90+ actions against the real sidebar Browser; fixture pages, Bing, DeepSeek,
|
|
6
|
+
example.com) then fixes. Full evidence: [`test/campaign-0.1.3.md`](test/campaign-0.1.3.md).
|
|
7
|
+
|
|
8
|
+
### Fixed — correctness
|
|
9
|
+
|
|
10
|
+
- `select` with an unknown value silently blanked the `<select>` and fired `change`; it now fails with the
|
|
11
|
+
available `value=label` pairs, and matches labels trimmed and case-insensitively.
|
|
12
|
+
- `scroll` ignored `selector` (it scrolled by the default amount), `left`/`right` also moved vertically, and a
|
|
13
|
+
no-op was reported as success. It now resolves `ref`/`selector`, scrolls only on the requested axis, and says
|
|
14
|
+
when nothing moved.
|
|
15
|
+
- `wait` for a selector matched hidden elements (a closed `<details>` counted as "appeared"); it now requires a
|
|
16
|
+
visible box. The wait is client-driven, so a navigation while waiting no longer kills it.
|
|
17
|
+
- `eval` accepted only a single expression and returned an opaque `Script failed to execute` for anything else.
|
|
18
|
+
It now evaluates expressions and statement blocks (`eval` completion value), serialises DOM nodes, functions,
|
|
19
|
+
BigInt and circular objects, and reports parse/runtime errors as readable messages.
|
|
20
|
+
- `type` on a non-input threw `Illegal invocation`; number/date/maxlength fields that rejected or truncated the
|
|
21
|
+
text were reported as success. It now rejects unsupported targets, verifies the stored value, and presses
|
|
22
|
+
Enter when the field is not inside a form.
|
|
23
|
+
- `click` on an input/textarea did not focus it, so `click` → `press` typed into `<body>`. Clicking a focusable
|
|
24
|
+
field now focuses it, and the click reply names inputs by their label/placeholder instead of `""`.
|
|
25
|
+
- `open` could adopt a Browser frame owned by another Session and navigate it; frames are now scoped to the
|
|
26
|
+
current Session, and `open` reports the tab id it actually uses.
|
|
27
|
+
- Snapshot ref attributes were never cleared, so a stale ref could hit a different element after a new
|
|
28
|
+
snapshot. They are cleared before every snapshot.
|
|
29
|
+
- `focus` / `close_tab` accepted unknown ids and silently did nothing. Both validate against the current
|
|
30
|
+
Session's Browser tabs and explain foreign/unknown ids.
|
|
31
|
+
- `tabs` listed other Sessions' tabs (duplicate ids) without context; it now lists this Session's tabs and
|
|
32
|
+
counts hidden ones.
|
|
33
|
+
- Checkbox/radio labels showed the value (`"on"`) instead of the associated `<label>` text.
|
|
34
|
+
- `batch` steps bypassed host validation (relative URLs failed mid-batch); the host now validates every step
|
|
35
|
+
before running any, the reply says `ran N of M`, and `read`/`diff`/`snapshot` payloads are kept.
|
|
36
|
+
|
|
37
|
+
### Fixed — robustness
|
|
38
|
+
|
|
39
|
+
- Long actions looked like disconnections: the client capped every command at 45 s while the host allowed 90 s
|
|
40
|
+
(wait up to 300 s), and a busy page was reported as "no page connected". The client deadline now follows the
|
|
41
|
+
requested timeout, and the host distinguishes busy from disconnected.
|
|
42
|
+
- `press` had no modifiers and a partial key table. It now supports `modifiers` (ctrl/alt/shift/meta), maps
|
|
43
|
+
DOM key names to Electron accelerators (`ArrowDown` → `Down`, `Escape` → `Esc`), and no longer runs a second
|
|
44
|
+
synthetic Enter pass that could submit twice.
|
|
45
|
+
- Screenshots larger than the bridge body limit are downscaled before sending instead of being dropped.
|
|
46
|
+
- `snapshot` hrefs are trimmed to 240 characters (Bing tracking URLs were dominating output).
|
|
47
|
+
- `snapshot` with `include_text: false` no longer prints an empty `text:` section; iframes on the page are
|
|
48
|
+
reported as not traversed.
|
|
49
|
+
|
|
50
|
+
### Added
|
|
51
|
+
|
|
52
|
+
- `open` accepts `new_tab: true`, `about:blank`, Windows paths, and bare `localhost:port` URLs (treated as
|
|
53
|
+
`http`). `file://` is accepted at the validation layer, but the current desktop Browser refuses to load local
|
|
54
|
+
files; that failure now explains how to serve the file over http instead.
|
|
55
|
+
- `press` accepts `modifiers`.
|
|
56
|
+
- `screenshot` warns that `full_page` is not supported yet.
|
|
57
|
+
- `status` prints the plugin version, so a stale client after an update is obvious.
|
|
58
|
+
- `snapshot` / `read` / `diff` now report `N chars ≈ M tokens`, so the cost of a page read is visible in the
|
|
59
|
+
result instead of being guessed.
|
|
60
|
+
|
|
61
|
+
### Docs
|
|
62
|
+
|
|
63
|
+
- README: 10-second demo GIF (`docs/demo.gif`), a measured token table, a "Built-in Browser vs this plugin"
|
|
64
|
+
section, a three-column compatibility matrix, and two English plus two Chinese tutorials.
|
|
65
|
+
- `test/long-article.html` backs the token table; `test/record-demo.ps1` and `test/build-demo-gif.py` produced
|
|
66
|
+
the demo GIF from a real window capture.
|
|
67
|
+
|
|
68
|
+
### Compatibility
|
|
69
|
+
|
|
70
|
+
Tested against DeepSeek Harness Desktop **0.2.2-rc.1** (the current release); `engines.dsh` allows 0.2.x.
|
|
71
|
+
|
|
72
|
+
## 0.1.3 — 2026-11-07
|
|
73
|
+
|
|
74
|
+
- First release on npm and GitHub: sidebar Browser control with snapshot refs, click/type/press/select/scroll,
|
|
75
|
+
query-scoped `read` (BM25 over Latin words + CJK bigrams), `diff`, `batch`, multi-tab management, screenshots.
|
|
76
|
+
|
|
77
|
+
## 0.1.0 – 0.1.2 — 2026-11-07
|
|
78
|
+
|
|
79
|
+
- First iterations: packaging, MIT license, npm-first install docs and marketplace metadata.
|
|
80
|
+
|
package/LICENSE
CHANGED
|
@@ -1,8 +1,6 @@
|
|
|
1
1
|
MIT License
|
|
2
2
|
|
|
3
|
-
Copyright (c) 2026 Hongrui Zhao
|
|
4
|
-
Copyright (c) 2026 The dsh-sidebar-browser authors
|
|
5
|
-
(portions derived from dsh-sidebar-browser, https://www.npmjs.com/package/dsh-sidebar-browser, MIT)
|
|
3
|
+
Copyright (c) 2026 Hongrui Zhao
|
|
6
4
|
|
|
7
5
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
8
6
|
of this software and associated documentation files (the "Software"), to deal
|
package/README.md
CHANGED
|
@@ -9,6 +9,11 @@ Agent control for the **Browser panel DeepSeek Harness already ships** in the ri
|
|
|
9
9
|
browser, no bundled Chromium, no Playwright. The agent reads the real page, clicks real controls, types,
|
|
10
10
|
presses keys, scrolls, waits and takes screenshots inside the panel the person is already looking at.
|
|
11
11
|
|
|
12
|
+

|
|
13
|
+
|
|
14
|
+
*10-second screen capture (no cuts): `type` + `submit`, `read`, a refs-only `snapshot`, and a `screenshot` —
|
|
15
|
+
all inside the right-Sidebar Browser the user is already watching.*
|
|
16
|
+
|
|
12
17
|
> Built on the MIT-licensed [`dsh-sidebar-browser`](https://www.npmjs.com/package/dsh-sidebar-browser):
|
|
13
18
|
> its action surface and long-poll bridge are reused. This package adds **query-scoped reads**, **change
|
|
14
19
|
> diffs** and **multi-tab management**, and keeps the whole thing dependency-free.
|
|
@@ -37,24 +42,81 @@ after a fixed sleep.
|
|
|
37
42
|
|
|
38
43
|
Measured on a Windows desktop against the live Browser panel: `batch` of **type + submit + wait for results +
|
|
39
44
|
query read** on the Baidu homepage finished in **1043 ms**, and the same three steps on Bing in **832 ms**.
|
|
40
|
-
|
|
45
|
+
|
|
46
|
+
## Token measurements
|
|
47
|
+
|
|
48
|
+
Exact character counts from the plugin's own `… chars ≈ … tokens` note, on a **15,521-character** article
|
|
49
|
+
fixture ([`test/long-article.html`](https://github.com/clusterstar1229/dsh-sidebar-control/blob/main/test/long-article.html)):
|
|
50
|
+
|
|
51
|
+
| Task | Returned | Approx. tokens |
|
|
52
|
+
|---|---|---|
|
|
53
|
+
| `snapshot` (default 12k cap) | 12,000 chars, truncated | **≈ 3,000** |
|
|
54
|
+
| `read { query: "wall-clock" }` | 199 chars · 1 of 1 passage | **≈ 50** |
|
|
55
|
+
| `read { query: "batch latency" }` | 3,906 chars · 11 of 15 passages | **≈ 977** |
|
|
56
|
+
| `diff` after one paragraph changed | 288 chars | **≈ 72** |
|
|
57
|
+
|
|
58
|
+
That is a **98%** reduction for a targeted question versus a full snapshot, and a **~40×** difference on
|
|
59
|
+
`diff`. The estimator counts Latin text at ~4 characters/token and CJK at ~1.5 characters/token, so treat the
|
|
60
|
+
token column as an order of magnitude, not a tokenizer: the character counts are exact. Re-run the same
|
|
61
|
+
experiment with `python -m http.server 8799 --bind 127.0.0.1 --directory test` and the fixture above.
|
|
62
|
+
|
|
63
|
+
## Built-in Browser vs this plugin
|
|
64
|
+
|
|
65
|
+
DSH 0.2.2-rc.1 ships the right-Sidebar Browser as an **app surface**: a tab host plus an Electron
|
|
66
|
+
`<webview>`, with its own tabs, address bar and session. In this build we found no model-facing tool
|
|
67
|
+
registered for that surface — the panel is UI-only until something exposes it to the agent.
|
|
68
|
+
|
|
69
|
+
This package is that something. It registers exactly one model-facing tool, and it is deliberately called
|
|
70
|
+
`builtin_browser`: it drives the built-in panel. Consequences:
|
|
71
|
+
|
|
72
|
+
- **Same tab, same cookies.** `open` / `navigate` / `click` act on the page the person is already looking at,
|
|
73
|
+
including signed-in sessions. There is no second browser profile and no automation browser to log in again.
|
|
74
|
+
- **No extra binaries.** Nothing is downloaded and no Chromium/Playwright process starts; the Electron
|
|
75
|
+
`<webview>` DSH already renders is the only engine.
|
|
76
|
+
- **Name overlap, not a fork.** If a future Harness build registers its own tool with the same name, do not
|
|
77
|
+
enable both at once — two registrations with one name compete. Keep one; the action surface here stays
|
|
78
|
+
compatible in spirit (`status`, `open`, `snapshot`, `click`, `type`, …).
|
|
79
|
+
- **The panel chrome is unchanged.** This plugin adds read/diff/batch and validation on the tool side; it does
|
|
80
|
+
not replace the Browser UI, inject CSS or zoom pages.
|
|
81
|
+
|
|
82
|
+
## Versions and updates
|
|
83
|
+
|
|
84
|
+
| Version | Date | What this version solved |
|
|
85
|
+
|---|---|---|
|
|
86
|
+
| **0.2.1** | 2026-11-08 | **Live-campaign hardening.** A 110-action run against the real panel found 20 defects; this release fixes them. *Correctness:* `select` fails on an unknown option instead of blanking the control, `scroll` honours `selector` and axis, `wait` requires a visible element, `eval` accepts statements and serialises real data, `click` focuses text fields, `type` verifies the stored value. *Safety:* `open` / `tabs` / `focus` / `close_tab` are Session-scoped, and stale snapshot refs are cleared. *Reliability:* client deadlines follow the requested timeout (long waits no longer break), a busy page is no longer reported as disconnected, `batch` validates every step and reports `ran N of M`, oversized screenshots are downscaled, and results now state their size (`chars ≈ tokens`). |
|
|
87
|
+
| 0.1.3 | 2026-11-07 | First stable public release on npm and GitHub: sidebar Browser control with snapshot refs, `click` / `type` / `press` / `select` / `scroll`, query-scoped `read` (BM25 over Latin words + CJK bigrams), `diff`, `batch`, multi-tab management and screenshots. Superseded by 0.2.1 for daily use. |
|
|
88
|
+
| 0.1.0 – 0.1.2 | 2026-11-07 | First iterations: packaging, the MIT license, npm-first install docs and marketplace metadata. |
|
|
89
|
+
|
|
90
|
+
**Current release: 0.2.1.** **Upgrading from 0.1.x:** install 0.2.1 with the same command (or the Plugin
|
|
91
|
+
Manager **Update** button); there is no configuration or data migration. If you installed from a checkout,
|
|
92
|
+
reinstall from npm so the Plugin Manager can offer future updates. Per-fix detail lives in
|
|
93
|
+
[CHANGELOG.md](CHANGELOG.md); the live evidence is in
|
|
94
|
+
[`test/campaign-0.2.1.md`](https://github.com/clusterstar1229/dsh-sidebar-control/blob/main/test/campaign-0.2.1.md)
|
|
95
|
+
and the original defect matrix in
|
|
96
|
+
[`test/campaign-0.1.3.md`](https://github.com/clusterstar1229/dsh-sidebar-control/blob/main/test/campaign-0.1.3.md).
|
|
97
|
+
|
|
98
|
+
## Compatibility
|
|
99
|
+
|
|
100
|
+
| DSH / platform | Status | Notes |
|
|
101
|
+
|---|---|---|
|
|
102
|
+
| Desktop **0.2.2-rc.1** (current) | ✅ Tested | Full action set, screenshots, multi-tab, session scoping; 110-action campaign on 2026-11-08 |
|
|
103
|
+
| Desktop 0.2.x (later) | ✅ Expected | `engines.dsh` allows `>=0.2.2-rc.1 <0.3.0`; requires the right-Sidebar Browser `<webview>` carrier |
|
|
104
|
+
| Desktop < 0.2.2-rc.1 | ❌ Unsupported | No right-Sidebar Browser service for the Client half to drive |
|
|
105
|
+
| Web build (`dsh web`) | ⚠️ Tabs only | The panel is a cross-origin iframe: no page JS, refs or capture; `tabs` / `focus` still work |
|
|
106
|
+
| macOS / Linux desktop | ⚠️ Expected | Same Electron webview path, not exercised in this Windows campaign |
|
|
41
107
|
|
|
42
108
|
## Install
|
|
43
109
|
|
|
44
110
|
```bash
|
|
45
|
-
dsh plugin --profile desktop add dsh-sidebar-control@0.1
|
|
111
|
+
dsh plugin --profile desktop add dsh-sidebar-control@0.2.1
|
|
46
112
|
```
|
|
47
113
|
|
|
48
|
-
or in the app: **Settings → Plugins → Install bundle** → `dsh-sidebar-control@0.1
|
|
114
|
+
or in the app: **Settings → Plugins → Install bundle** → `dsh-sidebar-control@0.2.1`.
|
|
49
115
|
|
|
50
116
|
An npm install gives the Plugin Manager an **Update** button for later releases. Prefer a checkout?
|
|
51
|
-
Use `github:clusterstar1229/dsh-sidebar-control#v0.1
|
|
52
|
-
|
|
53
|
-
Requirements: **DeepSeek Harness Desktop** 0.2.2-rc.1 (the Browser panel must be the Electron
|
|
54
|
-
`<webview>` carrier), Node 20+, and the built-in sidebar Browser enabled.
|
|
117
|
+
Use `github:clusterstar1229/dsh-sidebar-control#v0.2.1` instead.
|
|
55
118
|
|
|
56
|
-
**
|
|
57
|
-
`engines.dsh` allows the whole 0.2.x line.
|
|
119
|
+
Requirements: **DeepSeek Harness Desktop** 0.2.2-rc.1, Node 20+, and the built-in sidebar Browser enabled.
|
|
58
120
|
|
|
59
121
|
## Use
|
|
60
122
|
|
|
@@ -65,32 +127,78 @@ Requirements: **DeepSeek Harness Desktop** 0.2.2-rc.1 (the Browser panel must be
|
|
|
65
127
|
4. After an action: `diff` (cheap) or a new `snapshot` (full).
|
|
66
128
|
5. Asking a long page a question: `read` with `query` instead of `snapshot`.
|
|
67
129
|
|
|
68
|
-
### Actions
|
|
69
|
-
|
|
70
|
-
| action | what it does |
|
|
71
|
-
|
|
72
|
-
| `status` | platform, Browser frames, active tab, whether the page is on screen |
|
|
73
|
-
| `open` | open or reuse the Browser tab at a URL |
|
|
74
|
-
| `navigate` / `back` / `forward` / `reload` | navigation |
|
|
75
|
-
| `snapshot` | page text + numbered refs (`max_text_chars`, `max_refs`, `include_text: false` for refs only) |
|
|
76
|
-
| `read` | text, or only passages matching `query` (`max_chars`) |
|
|
77
|
-
| `diff` | paragraphs added/removed since the previous `diff` |
|
|
78
|
-
| `batch` | run up to 12 steps in one call (navigate/click/type/press/select/scroll/wait/eval/read/diff/snapshot) |
|
|
79
|
-
| `tabs` | list Browser tabs with ids |
|
|
80
|
-
| `focus` / `close_tab` | switch to / close a tab by `tab_id` |
|
|
81
|
-
| `click` | by `ref
|
|
82
|
-
| `type` | fill a field, `submit: true`
|
|
83
|
-
| `press` | Enter, Tab, Escape, arrows, PageUp/Down, Home/End, Space |
|
|
84
|
-
| `select` | choose an option
|
|
85
|
-
| `scroll` | by direction/amount, or a ref into view |
|
|
86
|
-
| `wait` | for a selector or text |
|
|
87
|
-
| `eval` |
|
|
88
|
-
| `screenshot` | PNG of the visible page, returned as an image |
|
|
89
|
-
| `close` | close the controlled Browser tab |
|
|
130
|
+
### Actions (as of 0.2.1)
|
|
131
|
+
|
|
132
|
+
| action | what it does | since |
|
|
133
|
+
|---|---|---|
|
|
134
|
+
| `status` | platform, Browser frames, active tab, whether the page is on screen, plugin version | 0.1.x |
|
|
135
|
+
| `open` | open or reuse the Browser tab at a URL; `new_tab: true` opens a separate tab and returns once it has a real document | 0.1.x / `new_tab` 0.2.1 |
|
|
136
|
+
| `navigate` / `back` / `forward` / `reload` | navigation | 0.1.x |
|
|
137
|
+
| `snapshot` | page text + numbered refs (`max_text_chars`, `max_refs`, `include_text: false` for refs only) | 0.1.x / size note 0.2.1 |
|
|
138
|
+
| `read` | text, or only passages matching `query` (`max_chars`); reports `chars ≈ tokens` | 0.1.x / size note 0.2.1 |
|
|
139
|
+
| `diff` | paragraphs added/removed since the previous `diff` | 0.1.x / size note 0.2.1 |
|
|
140
|
+
| `batch` | run up to 12 steps in one call (navigate/click/type/press/select/scroll/wait/eval/read/diff/snapshot) | 0.1.x / validation+payload 0.2.1 |
|
|
141
|
+
| `tabs` | list this Session's Browser tabs with ids | 0.1.x / Session-scoped 0.2.1 |
|
|
142
|
+
| `focus` / `close_tab` | switch to / close a tab by `tab_id` (validated) | 0.1.x / validated 0.2.1 |
|
|
143
|
+
| `click` | by `ref` or `selector`; focuses text fields | 0.1.x / focus 0.2.1 |
|
|
144
|
+
| `type` | fill a field; the value is verified, `submit: true` submits the form or presses Enter | 0.1.x / verification 0.2.1 |
|
|
145
|
+
| `press` | Enter, Tab, Escape, arrows, PageUp/Down, Home/End, Space, plus `modifiers` (ctrl/alt/shift/meta) | 0.1.x / modifiers 0.2.1 |
|
|
146
|
+
| `select` | choose by option value or label; an unknown option is an error listing the options | 0.1.x / error report 0.2.1 |
|
|
147
|
+
| `scroll` | by direction/amount, or a `ref`/`selector` into view | 0.1.x / selector 0.2.1 |
|
|
148
|
+
| `wait` | for a visible selector and/or text | 0.1.x / visibility 0.2.1 |
|
|
149
|
+
| `eval` | a JavaScript expression or statement block in the page | 0.1.x / statements 0.2.1 |
|
|
150
|
+
| `screenshot` | PNG of the visible page, returned as an image | 0.1.x / downscale 0.2.1 |
|
|
151
|
+
| `close` | close the controlled Browser tab | 0.1.x |
|
|
90
152
|
|
|
91
153
|
Refs come from the latest snapshot and go stale after navigation — take a new `snapshot` after a page
|
|
92
154
|
change, or just use `diff` when you only need to know what changed.
|
|
93
155
|
|
|
156
|
+
### Tutorial 1 — search once, read only the passages you need
|
|
157
|
+
|
|
158
|
+
One `batch` call does the whole search flow: fill the box, submit, wait for results, and read only the
|
|
159
|
+
passages that answer the question.
|
|
160
|
+
|
|
161
|
+
```js
|
|
162
|
+
builtin_browser({
|
|
163
|
+
action: "batch",
|
|
164
|
+
steps: [
|
|
165
|
+
{ action: "type", selector: "#sb_form_q", text: "DeepSeek Harness", submit: true },
|
|
166
|
+
{ action: "wait", text: "DeepSeek Harness" },
|
|
167
|
+
{ action: "read", query: "deepseek harness plugin", max_chars: 700 },
|
|
168
|
+
],
|
|
169
|
+
})
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
A real reply from the Bing results page:
|
|
173
|
+
|
|
174
|
+
```text
|
|
175
|
+
Batch: 3 of 3 step(s) ran in 276 ms.
|
|
176
|
+
1. type: Typed 16 characters into <input>. Submitted the form. · url: https://www.bing.com/search?q=DeepSeek+Harness …
|
|
177
|
+
2. wait: Appeared: {"found":true,"text":"DeepSeek Harness"}
|
|
178
|
+
3. read: Read 14 of 42 matching passage(s) for "DeepSeek Harness plugin". · 700 chars ≈ 206 tokens
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
The model never asked for the full page: the result block is 700 characters instead of a 12k snapshot.
|
|
182
|
+
|
|
183
|
+
### Tutorial 2 — read a local HTML report
|
|
184
|
+
|
|
185
|
+
`file://` is blocked by the current desktop build, so serve the folder over loopback and let the Browser
|
|
186
|
+
open it like any site:
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
python -m http.server 8000 --bind 127.0.0.1 --directory ./out
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
```js
|
|
193
|
+
builtin_browser({ action: "open", url: "http://127.0.0.1:8000/report.html" })
|
|
194
|
+
builtin_browser({ action: "snapshot", max_refs: 20 }) // controls + a size note
|
|
195
|
+
builtin_browser({ action: "read", query: "Q3 revenue", max_chars: 800 })
|
|
196
|
+
builtin_browser({ action: "screenshot" }) // PNG of the visible report
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
Use `snapshot` when you need to click a chart control, `read` when you only need the numbers, and
|
|
200
|
+
`screenshot` when the model should see the layout.
|
|
201
|
+
|
|
94
202
|
## What to be careful about
|
|
95
203
|
|
|
96
204
|
- **Desktop only.** The Web build renders the sidebar browser as a cross-origin iframe; the tool reports
|
|
@@ -104,14 +212,28 @@ change, or just use `diff` when you only need to know what changed.
|
|
|
104
212
|
- **Fixed-width sites.** Pages like `news.baidu.com` keep a desktop layout and may scroll sideways in a
|
|
105
213
|
narrow column. This plugin deliberately does not inject zoom or rewrite the page.
|
|
106
214
|
- **Refs are per-snapshot.** A ref used after navigation fails with a clear message; snapshot again.
|
|
215
|
+
- **Local `file://` pages may be blocked.** The current desktop build refuses them; serve local HTML over
|
|
216
|
+
http. If your build allows them, only open local HTML you trust.
|
|
107
217
|
|
|
108
218
|
## Related work
|
|
109
219
|
|
|
110
220
|
- [`dsh-sidebar-browser`](https://www.npmjs.com/package/dsh-sidebar-browser) — the MIT implementation this
|
|
111
|
-
package builds on
|
|
221
|
+
package builds on (Copyright (c) 2026 The dsh-sidebar-browser authors); the `read`/`diff`/`tabs` additions
|
|
222
|
+
are this package's own work.
|
|
112
223
|
- [`dsh-sidebar-browser-agent`](https://github.com/Hesitate-P/dsh-sidebar-browser-agent) — a broader
|
|
113
224
|
native-input implementation with tabs, tests and config switches.
|
|
114
225
|
|
|
226
|
+
## Development
|
|
227
|
+
|
|
228
|
+
```bash
|
|
229
|
+
npm run check # node --check on both halves
|
|
230
|
+
python -m http.server 8799 --bind 127.0.0.1 --directory test # fixture pages
|
|
231
|
+
```
|
|
232
|
+
|
|
233
|
+
`test/fixture.html` exercises every action; `test/long-article.html` backs the token table; the two campaign
|
|
234
|
+
files record the live matrix and the fixes. `test/record-demo.ps1` + `test/build-demo-gif.py` produced
|
|
235
|
+
`docs/demo.gif` (10-second capture, 700×405, 3.0 MB).
|
|
236
|
+
|
|
115
237
|
## Feedback
|
|
116
238
|
|
|
117
239
|
Bugs and feature requests: **hongruizhao0808@gmail.com** — or open an issue at
|
|
@@ -128,6 +250,10 @@ MIT. Derived from `dsh-sidebar-browser` (MIT); see [LICENSE](LICENSE).
|
|
|
128
250
|
**DeepSeek Harness 右栏内置浏览器的 Agent 控制器**:不另起浏览器、不下载 Chromium、不用 Playwright。
|
|
129
251
|
Agent 直接操作你正在看的那个 Browser 面板——读真实页面、点真实控件、输入、按键、滚动、等待、截图。
|
|
130
252
|
|
|
253
|
+

|
|
254
|
+
|
|
255
|
+
*10 秒真实录屏(未剪辑):`type` + `submit`、`read`、只回 refs 的 `snapshot`、最后 `screenshot`,全部发生在你正在看的右栏 Browser 里。*
|
|
256
|
+
|
|
131
257
|
> 本项目基于 MIT 许可的 [`dsh-sidebar-browser`](https://www.npmjs.com/package/dsh-sidebar-browser),复用了它的
|
|
132
258
|
> 动作设计与长轮询桥;在此基础上增加了**按问题检索的阅读**、**变化对比**和**多标签管理**,全程零依赖。
|
|
133
259
|
|
|
@@ -151,23 +277,76 @@ Agent 直接操作你正在看的那个 Browser 面板——读真实页面、
|
|
|
151
277
|
通常只回一两行;五步表单用 `batch` 只花一次模型调用,而不是五次。点击也不再固定睡眠,页面安静就返回。
|
|
152
278
|
|
|
153
279
|
在 Windows 桌面版、真实 Browser 面板上的实测:`batch` 执行 **输入 + 提交 + 等结果 + 按 query 阅读**,
|
|
154
|
-
百度首页 **1043 ms**、Bing **832 ms
|
|
280
|
+
百度首页 **1043 ms**、Bing **832 ms**。
|
|
281
|
+
|
|
282
|
+
### Token 实测
|
|
283
|
+
|
|
284
|
+
以下字符数来自插件自己输出的 `… chars ≈ … tokens`,测试页是 **15,521 字符**的
|
|
285
|
+
[`test/long-article.html`](https://github.com/clusterstar1229/dsh-sidebar-control/blob/main/test/long-article.html):
|
|
286
|
+
|
|
287
|
+
| 任务 | 返回 | 约合 token |
|
|
288
|
+
|---|---|---|
|
|
289
|
+
| `snapshot`(默认 12k 上限) | 12,000 字符,已截断 | **≈ 3,000** |
|
|
290
|
+
| `read { query: "wall-clock" }` | 199 字符 · 1/1 段命中 | **≈ 50** |
|
|
291
|
+
| `read { query: "batch latency" }` | 3,906 字符 · 11/15 段命中 | **≈ 977** |
|
|
292
|
+
| `diff`(改一段之后) | 288 字符 | **≈ 72** |
|
|
293
|
+
|
|
294
|
+
同一页面上,针对性问题比整页快照少 **98%**,`diff` 少约 **40 倍**。估算规则是英文约 4 字符/token、
|
|
295
|
+
中文约 1.5 字符/token,因此 token 列看数量级即可,字符数是精确值。用
|
|
296
|
+
`python -m http.server 8799 --bind 127.0.0.1 --directory test` 打开同一个 fixture 可以复现。
|
|
297
|
+
|
|
298
|
+
### 与宿主内置 Browser 的关系
|
|
299
|
+
|
|
300
|
+
DSH 0.2.2-rc.1 自带右栏 Browser 这个**界面**:标签宿主 + Electron `<webview>`,有自己的标签、地址栏和会话。
|
|
301
|
+
在我们检查的这个构建里,它**没有**向模型注册同名工具——面板先是一个纯 UI,需要插件把它暴露出来。
|
|
302
|
+
|
|
303
|
+
本插件就是那个"暴露层":只注册一个面向模型的工具,名字就叫 `builtin_browser`,因为它驱动的是内置面板。
|
|
304
|
+
由此带来几个事实:
|
|
305
|
+
|
|
306
|
+
- **同一个标签、同一份 cookie**:`open` / `navigate` / `click` 操作的是用户正在看的页面(含已登录会话),
|
|
307
|
+
不存在第二个浏览器配置,也不需要重新登录。
|
|
308
|
+
- **不下载额外二进制**:不启动 Chromium/Playwright 进程,DSH 已经渲染的 Electron `<webview>` 就是唯一引擎。
|
|
309
|
+
- **同名不是分叉**:如果未来 Harness 自己注册了 `builtin_browser`,不要同时启用两个——同名注册会互相竞争。
|
|
310
|
+
留一个即可,动作面(`status`、`open`、`snapshot`、`click`、`type`……)保持兼容。
|
|
311
|
+
- **面板 UI 不变**:本插件只在工具侧增加 read/diff/batch 与校验,不替换 Browser 界面,也不注入 CSS 或缩放页面。
|
|
312
|
+
|
|
313
|
+
### 版本与更新
|
|
314
|
+
|
|
315
|
+
| 版本 | 日期 | 这个版本解决了什么 |
|
|
316
|
+
|---|---|---|
|
|
317
|
+
| **0.2.1** | 2026-11-08 | **现场实战加固。** 在真实面板上跑了 110 步自动化,发现 20 个缺陷并全部修复。*正确性:*`select` 选不存在的值会明确报错而不是清空控件,`scroll` 支持 `selector` 与单一滚动轴,`wait` 只认可见元素,`eval` 支持语句块并返回可读数据,`click` 会聚焦输入框,`type` 会校验实际写入值。*安全性:*`open` / `tabs` / `focus` / `close_tab` 按会话隔离,快照旧 ref 会被清除。*可靠性:*客户端超时跟随请求的 timeout(长等待不再断),忙状态不再被误报为页面未连接,`batch` 会先校验步骤并报告 `ran N of M`,超大截图自动降采样,结果会标注 `chars ≈ tokens`。 |
|
|
318
|
+
| 0.1.3 | 2026-11-07 | 首个 npm / GitHub 正式公开版本:侧栏 Browser 控制、快照 refs、`click` / `type` / `press` / `select` / `scroll`、按问题检索的 `read`(英文词 + 中文二元组 BM25)、`diff`、`batch`、多标签与截图。日常使用请升级到 0.2.1。 |
|
|
319
|
+
| 0.1.0 – 0.1.2 | 2026-11-07 | 最初几次迭代:打包、MIT 许可、npm 优先的安装文档与市场元数据。 |
|
|
320
|
+
|
|
321
|
+
**当前发布:0.2.1。** **从 0.1.x 升级:** 用同样的命令安装 0.2.1(或在插件管理器点 **Update**);没有配置或
|
|
322
|
+
数据迁移。若当初是从源码 checkout 安装的,请改用 npm 安装,这样以后才能收到更新。逐条修复见
|
|
323
|
+
[CHANGELOG.md](CHANGELOG.md),现场证据见
|
|
324
|
+
[`test/campaign-0.2.1.md`](https://github.com/clusterstar1229/dsh-sidebar-control/blob/main/test/campaign-0.2.1.md),
|
|
325
|
+
最初的缺陷清单见
|
|
326
|
+
[`test/campaign-0.1.3.md`](https://github.com/clusterstar1229/dsh-sidebar-control/blob/main/test/campaign-0.1.3.md)。
|
|
327
|
+
|
|
328
|
+
### 兼容矩阵
|
|
329
|
+
|
|
330
|
+
| DSH / 平台 | 状态 | 说明 |
|
|
331
|
+
|---|---|---|
|
|
332
|
+
| 桌面版 **0.2.2-rc.1**(当前) | ✅ 已实测 | 完整动作集、截图、多标签、会话隔离;2026-11-08 跑了 110 步现场测试 |
|
|
333
|
+
| 桌面版 0.2.x(后续) | ✅ 预期可用 | `engines.dsh` 允许 `>=0.2.2-rc.1 <0.3.0`;需要右栏 Browser 的 `<webview>` 载体 |
|
|
334
|
+
| 桌面版 < 0.2.2-rc.1 | ❌ 不支持 | 没有可供客户端半驱动的右栏 Browser 服务 |
|
|
335
|
+
| Web 构建(`dsh web`) | ⚠️ 仅标签管理 | 面板是跨域 iframe:没有页面 JS、refs 或截图;`tabs` / `focus` 仍可用 |
|
|
336
|
+
| macOS / Linux 桌面版 | ⚠️ 预期可用 | 同一 Electron webview 路径,本次 Windows 测试未覆盖 |
|
|
155
337
|
|
|
156
338
|
### 安装
|
|
157
339
|
|
|
158
340
|
```bash
|
|
159
|
-
dsh plugin --profile desktop add dsh-sidebar-control@0.1
|
|
341
|
+
dsh plugin --profile desktop add dsh-sidebar-control@0.2.1
|
|
160
342
|
```
|
|
161
343
|
|
|
162
|
-
或在应用里:**设置 → 插件 → 安装 bundle** → `dsh-sidebar-control@0.1
|
|
344
|
+
或在应用里:**设置 → 插件 → 安装 bundle** → `dsh-sidebar-control@0.2.1`。
|
|
163
345
|
|
|
164
346
|
从 npm 安装后,插件管理器会为后续版本提供 **Update** 按钮。想用仓库源码,也可以改用
|
|
165
|
-
`github:clusterstar1229/dsh-sidebar-control#v0.1
|
|
347
|
+
`github:clusterstar1229/dsh-sidebar-control#v0.2.1`。
|
|
166
348
|
|
|
167
|
-
要求:**DeepSeek Harness 桌面版** 0.2.2-rc.1
|
|
168
|
-
并已启用内置侧栏浏览器。
|
|
169
|
-
|
|
170
|
-
**兼容性:** 0.1.1 已在当前桌面版 DSH **0.2.2-rc.1** 上实测通过;`engines.dsh` 允许整个 0.2.x 线。
|
|
349
|
+
要求:**DeepSeek Harness 桌面版** 0.2.2-rc.1、Node 20+,并已启用内置侧栏浏览器。
|
|
171
350
|
|
|
172
351
|
### 用法
|
|
173
352
|
|
|
@@ -177,31 +356,72 @@ dsh plugin --profile desktop add dsh-sidebar-control@0.1.2
|
|
|
177
356
|
4. 动作之后:`diff`(便宜)或重新 `snapshot`(完整)。
|
|
178
357
|
5. 只想问长页面一个问题:用 `read` + `query`,不要整页快照。
|
|
179
358
|
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
| action | 作用 |
|
|
183
|
-
|
|
184
|
-
| `status` | 平台、Browser
|
|
185
|
-
| `open` | 打开或复用 Browser
|
|
186
|
-
| `navigate` / `back` / `forward` / `reload` | 导航 |
|
|
187
|
-
| `snapshot` | 页面文字 + 编号 refs(`max_text_chars`、`max_refs`;`include_text: false` 只回控件) |
|
|
188
|
-
| `read` | 纯文本,或只回与 `query` 匹配的段落(`max_chars
|
|
189
|
-
| `diff` | 相对上一次 `diff` 新增/消失的段落 |
|
|
190
|
-
| `batch` | 一次调用执行最多 12 步(navigate/click/type/press/select/scroll/wait/eval/read/diff/snapshot) |
|
|
191
|
-
| `tabs` |
|
|
192
|
-
| `focus` / `close_tab` | 按 `tab_id` 切换 /
|
|
193
|
-
| `click` | 按 `ref` 或 `selector`
|
|
194
|
-
| `type` |
|
|
195
|
-
| `press` | Enter、Tab、Escape、方向键、PageUp/Down、Home/End、Space |
|
|
196
|
-
| `select` |
|
|
197
|
-
| `scroll` |
|
|
198
|
-
| `wait` |
|
|
199
|
-
| `eval` |
|
|
200
|
-
| `screenshot` | 可见页面的 PNG,作为图像返回 |
|
|
201
|
-
| `close` | 关闭当前受控标签 |
|
|
359
|
+
#### 动作(0.2.1)
|
|
360
|
+
|
|
361
|
+
| action | 作用 | 起始版本 |
|
|
362
|
+
|---|---|---|
|
|
363
|
+
| `status` | 平台、Browser 帧、当前标签、页面是否在屏幕上、插件版本 | 0.1.x |
|
|
364
|
+
| `open` | 打开或复用 Browser 标签;`new_tab: true` 另开一个,并在新标签页真正有文档后返回 | 0.1.x / `new_tab` 0.2.1 |
|
|
365
|
+
| `navigate` / `back` / `forward` / `reload` | 导航 | 0.1.x |
|
|
366
|
+
| `snapshot` | 页面文字 + 编号 refs(`max_text_chars`、`max_refs`;`include_text: false` 只回控件) | 0.1.x / 大小标注 0.2.1 |
|
|
367
|
+
| `read` | 纯文本,或只回与 `query` 匹配的段落(`max_chars`),并标注 `chars ≈ tokens` | 0.1.x / 大小标注 0.2.1 |
|
|
368
|
+
| `diff` | 相对上一次 `diff` 新增/消失的段落 | 0.1.x / 大小标注 0.2.1 |
|
|
369
|
+
| `batch` | 一次调用执行最多 12 步(navigate/click/type/press/select/scroll/wait/eval/read/diff/snapshot) | 0.1.x / 校验与内容保留 0.2.1 |
|
|
370
|
+
| `tabs` | 列出当前会话的 Browser 标签及其 id | 0.1.x / 会话隔离 0.2.1 |
|
|
371
|
+
| `focus` / `close_tab` | 按 `tab_id` 切换 / 关闭标签(会校验 id) | 0.1.x / 校验 0.2.1 |
|
|
372
|
+
| `click` | 按 `ref` 或 `selector` 点击;会聚焦输入框 | 0.1.x / 聚焦 0.2.1 |
|
|
373
|
+
| `type` | 填字段并校验写入结果,`submit: true` 提交表单,无表单则按 Enter | 0.1.x / 校验 0.2.1 |
|
|
374
|
+
| `press` | Enter、Tab、Escape、方向键、PageUp/Down、Home/End、Space,支持 `modifiers`(ctrl/alt/shift/meta) | 0.1.x / 修饰键 0.2.1 |
|
|
375
|
+
| `select` | 按值或标签选择;不存在的选项会报错并列出可选项 | 0.1.x / 报错 0.2.1 |
|
|
376
|
+
| `scroll` | 按方向/像素滚动,或用 `ref`/`selector` 滚入视口 | 0.1.x / selector 0.2.1 |
|
|
377
|
+
| `wait` | 等可见的 selector 和/或文字 | 0.1.x / 可见性 0.2.1 |
|
|
378
|
+
| `eval` | 在页面里执行 JS 表达式或语句块 | 0.1.x / 语句块 0.2.1 |
|
|
379
|
+
| `screenshot` | 可见页面的 PNG,作为图像返回 | 0.1.x / 降采样 0.2.1 |
|
|
380
|
+
| `close` | 关闭当前受控标签 | 0.1.x |
|
|
202
381
|
|
|
203
382
|
refs 来自最近一次快照,导航后会失效——页面变了就重新 `snapshot`;只想知道"变了什么"时用 `diff` 就够。
|
|
204
383
|
|
|
384
|
+
#### 教程 1 —— 一次 batch 完成搜索,只读命中的段落
|
|
385
|
+
|
|
386
|
+
```js
|
|
387
|
+
builtin_browser({
|
|
388
|
+
action: "batch",
|
|
389
|
+
steps: [
|
|
390
|
+
{ action: "type", selector: "#sb_form_q", text: "DeepSeek Harness", submit: true },
|
|
391
|
+
{ action: "wait", text: "DeepSeek Harness" },
|
|
392
|
+
{ action: "read", query: "deepseek harness plugin", max_chars: 700 },
|
|
393
|
+
],
|
|
394
|
+
})
|
|
395
|
+
```
|
|
396
|
+
|
|
397
|
+
Bing 结果页上的真实返回:
|
|
398
|
+
|
|
399
|
+
```text
|
|
400
|
+
Batch: 3 of 3 step(s) ran in 276 ms.
|
|
401
|
+
1. type: Typed 16 characters into <input>. Submitted the form. · url: https://www.bing.com/search?q=DeepSeek+Harness …
|
|
402
|
+
2. wait: Appeared: {"found":true,"text":"DeepSeek Harness"}
|
|
403
|
+
3. read: Read 14 of 42 matching passage(s) for "DeepSeek Harness plugin". · 700 chars ≈ 206 tokens
|
|
404
|
+
```
|
|
405
|
+
|
|
406
|
+
模型不需要整页:结果块是 700 字符,而不是 12k 的整页快照。
|
|
407
|
+
|
|
408
|
+
#### 教程 2 —— 读取本地 HTML 报告
|
|
409
|
+
|
|
410
|
+
当前桌面版会拦截 `file://`,所以用 loopback http 把目录提供出来:
|
|
411
|
+
|
|
412
|
+
```bash
|
|
413
|
+
python -m http.server 8000 --bind 127.0.0.1 --directory ./out
|
|
414
|
+
```
|
|
415
|
+
|
|
416
|
+
```js
|
|
417
|
+
builtin_browser({ action: "open", url: "http://127.0.0.1:8000/report.html" })
|
|
418
|
+
builtin_browser({ action: "snapshot", max_refs: 20 }) // 控件 + 大小标注
|
|
419
|
+
builtin_browser({ action: "read", query: "Q3 revenue", max_chars: 800 })
|
|
420
|
+
builtin_browser({ action: "screenshot" }) // 可见报告的 PNG
|
|
421
|
+
```
|
|
422
|
+
|
|
423
|
+
需要点图表控件时用 `snapshot`,只需要数字时用 `read`,需要看版式时用 `screenshot`。
|
|
424
|
+
|
|
205
425
|
### 需要注意
|
|
206
426
|
|
|
207
427
|
- **仅桌面版可用**:Web 构建下侧栏浏览器是跨域 iframe,工具会明确报错而不是假装成功;标签管理仍可用。
|
|
@@ -211,12 +431,24 @@ refs 来自最近一次快照,导航后会失效——页面变了就重新 `s
|
|
|
211
431
|
- **沙箱仍然是沙箱**:guest 没有 Node、没有 preload,也访问不到 Harness 源;下载、权限弹窗、顶层跳转由内置浏览器限制。
|
|
212
432
|
- **固定宽度站点**:如 `news.baidu.com` 这类页面在窄栏里仍会横向滚动;本插件刻意不注入缩放、不改写页面。
|
|
213
433
|
- **refs 只对一次快照有效**:导航后使用旧 ref 会得到明确报错,重新快照即可。
|
|
434
|
+
- **`file://` 可能被构建禁止**:当前桌面版会拒绝本地文件;请用 http 提供本地 HTML。
|
|
214
435
|
|
|
215
436
|
### 相关项目
|
|
216
437
|
|
|
217
|
-
- [`dsh-sidebar-browser`](https://www.npmjs.com/package/dsh-sidebar-browser) — 本项目的 MIT
|
|
438
|
+
- [`dsh-sidebar-browser`](https://www.npmjs.com/package/dsh-sidebar-browser) — 本项目的 MIT 基础实现
|
|
439
|
+
(Copyright (c) 2026 The dsh-sidebar-browser authors);`read`/`diff`/`tabs` 等增量为本项目原创。
|
|
218
440
|
- [`dsh-sidebar-browser-agent`](https://github.com/Hesitate-P/dsh-sidebar-browser-agent) — 更全面的原生输入实现(标签页、测试、配置开关)。
|
|
219
441
|
|
|
442
|
+
### 开发
|
|
443
|
+
|
|
444
|
+
```bash
|
|
445
|
+
npm run check # 对 host/client 两半做 node --check
|
|
446
|
+
python -m http.server 8799 --bind 127.0.0.1 --directory test # 测试 fixture
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
`test/fixture.html` 覆盖所有动作;`test/long-article.html` 是 token 表格的测试页;两个 campaign 文件记录现场
|
|
450
|
+
测试与修复。`test/record-demo.ps1` + `test/build-demo-gif.py` 生成 `docs/demo.gif`(10 秒、700×405、3.0 MB)。
|
|
451
|
+
|
|
220
452
|
### 反馈
|
|
221
453
|
|
|
222
454
|
Bug 与功能建议:**hongruizhao0808@gmail.com**,或到
|