dsh-builtin-browser 0.1.7 → 0.1.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +21 -3
- package/README.md +21 -3
- package/lib/browser/runtime.d.ts +2 -1
- package/lib/browser/runtime.js +15 -2
- package/lib/browser/types.d.ts +2 -1
- package/lib/browser-electron/entry.js +6 -1
- package/lib/browser-electron/host-main.js +26 -2
- package/lib/browser-electron/provider.js +50 -20
- package/lib/browser-electron/remote-host.d.ts +7 -4
- package/lib/browser-electron/remote-host.js +65 -15
- package/lib/index.d.ts +8 -7
- package/lib/index.js +7 -6
- package/lib/tool-browser/index.d.ts +1 -1
- package/lib/tool-browser/index.js +44 -20
- package/package.json +2 -2
- package/src/browser/runtime.ts +14 -2
- package/src/browser/types.ts +2 -1
- package/src/browser-electron/entry.ts +6 -1
- package/src/browser-electron/host-main.ts +22 -2
- package/src/browser-electron/provider.ts +67 -21
- package/src/browser-electron/remote-host.ts +62 -12
- package/src/index.ts +12 -6
- package/src/tool-browser/index.ts +44 -21
package/README.en.md
CHANGED
|
@@ -2,17 +2,35 @@
|
|
|
2
2
|
|
|
3
3
|
中文 | [English](README.en.md)
|
|
4
4
|
|
|
5
|
-
A shared real browser capability plugin for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness): a visible browser the human
|
|
5
|
+
A shared **real browser** capability plugin for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness): install and it just works — the agent drives a real, visible browser over CDP that the human can watch and take over at any time. Human and agent operate the **very same page**.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
**Why this plugin**
|
|
8
|
+
|
|
9
|
+
- **Install-and-use.** With a desktop shell the shell's embedded view is used; on plain `dsh web` the plugin **self-hosts** — it spawns its own Electron window with zero extra configuration.
|
|
10
|
+
- **Real and visible, human-takeover friendly.** Not a headless screenshot or a relay: the user sees the page and can grab control directly.
|
|
11
|
+
- **DOM-level driving, framework-friendly.** React/Vue controlled inputs fill reliably.
|
|
12
|
+
- **Concurrency-safe.** Task-level browser session isolation — parallel tasks never fight over the page or pollute each other.
|
|
13
|
+
- **Built for the real world.** CAPTCHA detection, login persistence, batch form filling, authenticated downloads, operation replay, and action restriction — all included.
|
|
8
14
|
|
|
9
15
|
## Features
|
|
10
16
|
|
|
17
|
+
**Browser capability**
|
|
18
|
+
|
|
11
19
|
- **Real view, not a relay.** The browser is a native view (`WebContentsView`) the human can see and operate directly; the agent drives the very same page. The view is provided by the host shell; the plugin drives it.
|
|
12
20
|
- **DOM-referenced, not coordinate-guessing.** `browser_snapshot` returns numbered interactive elements; `browser_execute` runs JS in the page (native setters for framework inputs), so interaction works on React/Vue pages.
|
|
13
21
|
- **Multi-tab sessions.** Open URLs in parallel tabs, list/switch/close/reset, all keeping state.
|
|
14
22
|
- **Multi-format content.** Fetch pages as html / markdown / txt / json, scoped by selector, capped by length and timeout.
|
|
15
23
|
|
|
24
|
+
**Engineering capability**
|
|
25
|
+
|
|
26
|
+
- **Per-task isolation.** Each DSH task (session) gets its own browser session (own tabs and history); concurrent tasks never interfere, and calls within one task reuse the same session (`browser_session` / `browser_reset_session`).
|
|
27
|
+
- **Login persistence.** `browser_auth` exports/restores cookies, so logins survive host restarts.
|
|
28
|
+
- **CAPTCHA / bot-detection awareness.** Cloudflare, reCAPTCHA, hCaptcha, Turnstile and generic challenges are detected automatically (`browser_challenge`, and flagged in snapshots); instead of blindly retrying, the agent asks the human to complete it in the shared window.
|
|
29
|
+
- **Batch form filling.** `browser_fill` sets many fields in one call — matched by selector/name/label, handling controlled inputs, selects, checkboxes and radio groups, with optional submit.
|
|
30
|
+
- **Operation history & replay.** `browser_history` records operations; `browser_replay` re-runs one step.
|
|
31
|
+
- **Authenticated downloads.** `browser_download` fetches a URL with the session's cookies straight to a local file.
|
|
32
|
+
- **Safety restriction.** `browser_restrict` limits which browser actions are allowed, preventing stray clicks/navigation.
|
|
33
|
+
|
|
16
34
|
## Requirements
|
|
17
35
|
|
|
18
36
|
- DeepSeek Harness (dsh) with the `web` profile
|
|
@@ -28,7 +46,7 @@ This plugin provides the **host-side capability** (browser seam, Electron CDP pr
|
|
|
28
46
|
| DeepSeek Harness (dsh) | `0.1.0-rc.5` |
|
|
29
47
|
| Electron | `43.4.0` |
|
|
30
48
|
| Node.js | `22.20.0` |
|
|
31
|
-
| dsh-builtin-browser | `0.1.
|
|
49
|
+
| dsh-builtin-browser | `0.1.7` |
|
|
32
50
|
| OS | Windows 10 (10.0.26200) |
|
|
33
51
|
|
|
34
52
|
> The plugin declares `electron >= 30`; other platforms (macOS/Linux) run the same protocol but were only verified on the Windows environment above.
|
package/README.md
CHANGED
|
@@ -2,17 +2,35 @@
|
|
|
2
2
|
|
|
3
3
|
[English](README.en.md) | 中文
|
|
4
4
|
|
|
5
|
-
面向 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)
|
|
5
|
+
面向 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 的**共享真实浏览器**能力插件:装好即可用——agent 通过 CDP 驱动一个真实、可见、可随时人工接管的浏览器,人与 agent 操作的是**同一个页面**。
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
**我们的优势**
|
|
8
|
+
|
|
9
|
+
- **装好即用**:有桌面外壳时嵌入外壳视图;纯 `dsh web` 也能**自托管**拉起自己的 Electron 窗口,不需要任何额外配置。
|
|
10
|
+
- **真实可见、人工可接管**:不是无头截屏或转播,用户随时能看到页面并直接接手操作。
|
|
11
|
+
- **DOM 级驱动,框架友好**:React/Vue 受控组件也能可靠填写。
|
|
12
|
+
- **并发安全**:任务级浏览器会话隔离,多个任务并行互不抢页面、不互相污染。
|
|
13
|
+
- **面向真实世界**:人机验证识别、登录态持久化、批量填表、带登录态下载、操作回放、动作限制,一应俱全。
|
|
8
14
|
|
|
9
15
|
## 功能
|
|
10
16
|
|
|
17
|
+
**浏览器能力**
|
|
18
|
+
|
|
11
19
|
- **真实视图,而非转播。** 浏览器是原生视图(`WebContentsView`),用户可直接看到并操作;agent 驱动的是同一个页面。视图由宿主外壳提供,插件负责驱动。
|
|
12
20
|
- **DOM 引用,而非猜坐标。** `browser_snapshot` 返回带编号的交互元素;`browser_execute` 在页面里执行 JS(框架输入用原生 setter),在 React/Vue 页面上也能可靠交互。
|
|
13
21
|
- **多标签会话。** 并行打开 URL、查看/切换/关闭/重置标签,状态保持。
|
|
14
22
|
- **多格式内容。** 以 html / markdown / txt / json 抓取页面,支持 selector 限定、长度与超时上限。
|
|
15
23
|
|
|
24
|
+
**工程化能力**
|
|
25
|
+
|
|
26
|
+
- **按任务隔离。** 每个 DSH 任务(会话)拥有独立的浏览器会话(独立标签页与历史),并发任务互不干扰;同一任务内多次调用复用同一会话(`browser_session` / `browser_reset_session`)。
|
|
27
|
+
- **登录态持久化。** `browser_auth` 导出/恢复 cookie,重启后登录态不丢。
|
|
28
|
+
- **人机验证识别。** 自动检测 Cloudflare / reCAPTCHA / hCaptcha / Turnstile 等挑战(`browser_challenge`,快照也会标注),提示人工在共享窗口完成,不再盲目重试。
|
|
29
|
+
- **批量表单填充。** `browser_fill` 一次填写多个字段:按选择器/名称/标签匹配,支持受控输入、下拉、单选/复选,可选提交。
|
|
30
|
+
- **操作历史与回放。** `browser_history` 记录操作日志,`browser_replay` 可回放某一步。
|
|
31
|
+
- **带登录态下载。** `browser_download` 用会话 cookie 把文件取到本地,登录后内容可直接落盘。
|
|
32
|
+
- **安全限制。** `browser_restrict` 限制允许的浏览器动作,防误点/误导航。
|
|
33
|
+
|
|
16
34
|
## 环境要求
|
|
17
35
|
|
|
18
36
|
- DeepSeek Harness(dsh)且安装了 `web` profile
|
|
@@ -28,7 +46,7 @@
|
|
|
28
46
|
| DeepSeek Harness(dsh) | `0.1.0-rc.5` |
|
|
29
47
|
| Electron | `43.4.0` |
|
|
30
48
|
| Node.js | `22.20.0` |
|
|
31
|
-
| dsh-builtin-browser | `0.1.
|
|
49
|
+
| dsh-builtin-browser | `0.1.7` |
|
|
32
50
|
| 操作系统 | Windows 10 (10.0.26200) |
|
|
33
51
|
|
|
34
52
|
> 插件声明 `electron >= 30`;其他平台(如 macOS/Linux)按同一协议运行,但仅在上表 Windows 环境实测。
|
package/lib/browser/runtime.d.ts
CHANGED
|
@@ -94,7 +94,8 @@ export declare class BrowserRuntime extends Service {
|
|
|
94
94
|
flushAuth(session: BrowserSessionId): Promise<readonly ExportedCookie[]>;
|
|
95
95
|
/** Import cookies into the session through the provider. */
|
|
96
96
|
restoreAuth(session: BrowserSessionId, cookies: readonly ExportedCookie[]): Promise<number>;
|
|
97
|
-
/** Close the session through the selected provider. Idempotent
|
|
97
|
+
/** Close the session through the selected provider. Idempotent; a missing
|
|
98
|
+
* provider is treated as already-closed so teardown paths stay no-ops. */
|
|
98
99
|
close(session: BrowserSessionId): Promise<void>;
|
|
99
100
|
}
|
|
100
101
|
export default BrowserRuntime;
|
package/lib/browser/runtime.js
CHANGED
|
@@ -158,9 +158,22 @@ export class BrowserRuntime extends Service {
|
|
|
158
158
|
async restoreAuth(session, cookies) {
|
|
159
159
|
return this.resolveProvider().restoreAuth(session, cookies);
|
|
160
160
|
}
|
|
161
|
-
/** Close the session through the selected provider. Idempotent
|
|
161
|
+
/** Close the session through the selected provider. Idempotent; a missing
|
|
162
|
+
* provider is treated as already-closed so teardown paths stay no-ops. */
|
|
162
163
|
async close(session) {
|
|
163
|
-
|
|
164
|
+
try {
|
|
165
|
+
await this.resolveProvider().close(session);
|
|
166
|
+
}
|
|
167
|
+
catch (error) {
|
|
168
|
+
const code = error instanceof BrowserError ? error.code : undefined;
|
|
169
|
+
if (code === 'BROWSER_PROVIDER_UNAVAILABLE'
|
|
170
|
+
|| code === 'BROWSER_PROVIDER_CONFIGURED_MISSING'
|
|
171
|
+
|| code === 'BROWSER_PROVIDER_CONFIGURED_UNAVAILABLE'
|
|
172
|
+
|| code === 'BROWSER_PROVIDER_AMBIGUOUS') {
|
|
173
|
+
return; // provider gone; nothing to close
|
|
174
|
+
}
|
|
175
|
+
throw error;
|
|
176
|
+
}
|
|
164
177
|
}
|
|
165
178
|
}
|
|
166
179
|
export default BrowserRuntime;
|
package/lib/browser/types.d.ts
CHANGED
|
@@ -294,7 +294,8 @@ export interface BrowserProvider {
|
|
|
294
294
|
export interface BrowserHistoryEntry {
|
|
295
295
|
/** Monotonic sequence number within the session. */
|
|
296
296
|
readonly seq: number;
|
|
297
|
-
/** The operation name: navigate | execute | click | type
|
|
297
|
+
/** The operation name. Replayable: navigate | execute | click | type.
|
|
298
|
+
* Also recorded (non-replayable): fill | download | flushAuth | restoreAuth. */
|
|
298
299
|
readonly action: string;
|
|
299
300
|
/** The operation's arguments. */
|
|
300
301
|
readonly params: Record<string, unknown>;
|
|
@@ -28,10 +28,15 @@ export function apply(ctx, config) {
|
|
|
28
28
|
// External host (desktop shell) wins; otherwise self-host. The self-hosted
|
|
29
29
|
// child is disposed with the fiber, mirroring the shell's lifetime.
|
|
30
30
|
const host = config.viewHost ?? new RemoteElectronViewHost(defaultHostMainPath());
|
|
31
|
+
// Own the disposer on THIS plugin's fiber: registerBrowserProvider's effect
|
|
32
|
+
// is bound to the seam's own fiber (the browser row), so a reload of this
|
|
33
|
+
// row would otherwise collide with the still-registered provider
|
|
34
|
+
// (BROWSER_DUPLICATE_PROVIDER) or leave a stale provider behind.
|
|
35
|
+
const unregister = ctx.browser.registerBrowserProvider(new ElectronBrowserProvider(host, { httpOnly: config.httpOnly }));
|
|
31
36
|
ctx.effect(() => () => {
|
|
37
|
+
unregister();
|
|
32
38
|
if (config.viewHost === undefined && host instanceof RemoteElectronViewHost) {
|
|
33
39
|
host.dispose();
|
|
34
40
|
}
|
|
35
41
|
});
|
|
36
|
-
ctx.browser.registerBrowserProvider(new ElectronBrowserProvider(host, { httpOnly: config.httpOnly }));
|
|
37
42
|
}
|
|
@@ -48,11 +48,14 @@ async function handle(op, msg) {
|
|
|
48
48
|
window.on('closed', () => { window = undefined; });
|
|
49
49
|
}
|
|
50
50
|
const view = new WebContentsView();
|
|
51
|
+
// New views start hidden: with several sessions/tabs only the shown
|
|
52
|
+
// one may be visible (they stack in contentView child order).
|
|
51
53
|
view.setVisible(false);
|
|
52
54
|
window.contentView.addChildView(view);
|
|
53
55
|
const [width, height] = window.getContentSize();
|
|
54
56
|
view.setBounds({ x: 0, y: 0, width: width ?? 0, height: height ?? 0 });
|
|
55
|
-
|
|
57
|
+
if (views.size === 0)
|
|
58
|
+
view.setVisible(true);
|
|
56
59
|
view.webContents.debugger.attach(CDP_VERSION);
|
|
57
60
|
views.set(viewId, { webContentsView: view });
|
|
58
61
|
reply(msg.id, { ok: true });
|
|
@@ -80,8 +83,23 @@ async function handle(op, msg) {
|
|
|
80
83
|
if (viewId === undefined)
|
|
81
84
|
throw new Error('showView missing viewId');
|
|
82
85
|
const entry = views.get(viewId);
|
|
83
|
-
if (entry !== undefined)
|
|
86
|
+
if (entry !== undefined) {
|
|
87
|
+
// Hide every other view, then show and RAISE the target so the
|
|
88
|
+
// human actually sees the active tab/session (topmost child wins).
|
|
89
|
+
for (const v of views.values()) {
|
|
90
|
+
if (v === entry)
|
|
91
|
+
continue;
|
|
92
|
+
try {
|
|
93
|
+
v.webContentsView.setVisible(false);
|
|
94
|
+
}
|
|
95
|
+
catch { /* destroyed */ }
|
|
96
|
+
}
|
|
84
97
|
entry.webContentsView.setVisible(true);
|
|
98
|
+
if (window !== undefined) {
|
|
99
|
+
window.contentView.removeChildView(entry.webContentsView);
|
|
100
|
+
window.contentView.addChildView(entry.webContentsView);
|
|
101
|
+
}
|
|
102
|
+
}
|
|
85
103
|
reply(msg.id, { ok: true });
|
|
86
104
|
return;
|
|
87
105
|
}
|
|
@@ -228,6 +246,12 @@ void app.whenReady().then(() => {
|
|
|
228
246
|
socket.on('error', error => {
|
|
229
247
|
process.stderr.write(`[dsh-browser host] socket error: ${String(error)}\n`);
|
|
230
248
|
});
|
|
249
|
+
// The parent owns our lifetime: when it closes the socket (dispose) or dies
|
|
250
|
+
// without cleanup, exit so no zombie Electron window is left behind.
|
|
251
|
+
socket.on('close', () => {
|
|
252
|
+
process.stderr.write('[dsh-browser host] parent connection closed, exiting\n');
|
|
253
|
+
app.exit(0);
|
|
254
|
+
});
|
|
231
255
|
// Keep the process alive until the parent closes the socket or kills us.
|
|
232
256
|
});
|
|
233
257
|
// Diagnostics go to stderr, which the parent never parses as protocol.
|
|
@@ -21,11 +21,11 @@ const CHALLENGE_DETECT_EXPRESSION = `(() => {
|
|
|
21
21
|
const lower = (title + '\\n' + bodyText).toLowerCase()
|
|
22
22
|
const frameSrcs = [...document.querySelectorAll('iframe')].map(f => f.src || '').join(' ')
|
|
23
23
|
const framesLower = frameSrcs.toLowerCase()
|
|
24
|
-
const hasCfInterstitial = /just a moment|checking your browser|attention required|cf_chl
|
|
24
|
+
const hasCfInterstitial = /just a moment|checking your browser|attention required|cf_chl/i.test(lower)
|
|
25
25
|
|| !!document.querySelector('#challenge-running, #challenge-stage, #cf-chl-container')
|
|
26
26
|
const hasHCaptcha = !!window.hcaptcha || !!document.querySelector('.h-captcha') || /hcaptcha\\.com/i.test(framesLower)
|
|
27
27
|
const hasRecaptcha = !!window.grecaptcha || !!document.querySelector('.g-recaptcha') || /recaptcha\\/api|google\\.com\\/recaptcha/i.test(framesLower)
|
|
28
|
-
const hasTurnstile = !!window.turnstile || /challenges\\.cloudflare\\.com/i.test(framesLower) || /turnstile/i.test(lower)
|
|
28
|
+
const hasTurnstile = !!window.turnstile || /challenges\\.cloudflare\\.com/i.test(framesLower) || /turnstile|challenge-platform/i.test(lower)
|
|
29
29
|
const verifyWording = /verify you are human|verify you are not a robot|\\u4eba\\u673a\\u9a8c\\u8bc1|\\u5b89\\u5168\\u9a8c\\u8bc1|enable javascript and cookies|\\u8bf7.*\\u9a8c\\u8bc1/i.test(lower)
|
|
30
30
|
if (hasCfInterstitial) return { blocked: true, kind: 'cloudflare', reason: 'Cloudflare "Just a moment" interstitial' }
|
|
31
31
|
if (hasHCaptcha) return { blocked: true, kind: 'hcaptcha', reason: 'hCaptcha verification' }
|
|
@@ -80,7 +80,7 @@ export class ElectronBrowserProvider {
|
|
|
80
80
|
open() {
|
|
81
81
|
const handle = this.host.createView();
|
|
82
82
|
const id = `browser:${randomUUID()}`;
|
|
83
|
-
this.sessions.set(id, { id, tabs: [{ id: `tab:${randomUUID()}`, handle }], activeIndex: 0, history: [] });
|
|
83
|
+
this.sessions.set(id, { id, tabs: [{ id: `tab:${randomUUID()}`, handle }], activeIndex: 0, history: [], nextSeq: 1 });
|
|
84
84
|
return Promise.resolve(id);
|
|
85
85
|
}
|
|
86
86
|
/** Open a URL in the active tab (default) or a new tab. */
|
|
@@ -175,15 +175,19 @@ export class ElectronBrowserProvider {
|
|
|
175
175
|
}
|
|
176
176
|
}
|
|
177
177
|
signal?.throwIfAborted();
|
|
178
|
-
|
|
178
|
+
// Page.navigate can hang on an unreachable/slow host; bound it like the
|
|
179
|
+
// evaluate paths so a wedged navigation surfaces as an error instead of
|
|
180
|
+
// blocking the tool call forever.
|
|
181
|
+
const timeoutMs = 30_000;
|
|
182
|
+
const result = await withTimeout(handle.sendCommand(CDP_PAGE_NAVIGATE, { url }), timeoutMs, signal, `browser: navigation timed out after ${timeoutMs}ms`);
|
|
179
183
|
// Page.navigate resolves even when the navigation fails; surface the
|
|
180
184
|
// failure instead of leaving a silent white screen.
|
|
181
185
|
const errorText = result.errorText;
|
|
182
186
|
if (typeof errorText === 'string' && errorText !== '') {
|
|
183
|
-
this.record(s, 'navigate', { url }, false, { error: errorText });
|
|
184
187
|
throw new BrowserError(`browser: navigation to "${url}" failed: ${errorText}`, 'BROWSER_NAVIGATION_FAILED');
|
|
185
188
|
}
|
|
186
189
|
this.record(s, 'navigate', { url }, true);
|
|
190
|
+
this.showActive(s);
|
|
187
191
|
}
|
|
188
192
|
catch (error) {
|
|
189
193
|
if (!(error instanceof BrowserError && error.code === 'BROWSER_NAVIGATION_BLOCKED')) {
|
|
@@ -230,10 +234,16 @@ export class ElectronBrowserProvider {
|
|
|
230
234
|
return { ok: false, exception };
|
|
231
235
|
}
|
|
232
236
|
const value = result.result?.value ?? null;
|
|
233
|
-
this.record(s, 'execute', {
|
|
237
|
+
this.record(s, 'execute', {
|
|
238
|
+
script: request.script,
|
|
239
|
+
...request.args !== undefined && request.args.length > 0 ? { args: request.args } : {},
|
|
240
|
+
}, true, { result: typeof value === 'string' ? value.slice(0, 500) : JSON.stringify(value).slice(0, 500) });
|
|
234
241
|
return { ok: true, value };
|
|
235
242
|
}
|
|
236
243
|
catch (error) {
|
|
244
|
+
if (error instanceof Error && error.name === 'TimeoutError') {
|
|
245
|
+
throw new BrowserError(`browser: execute timed out after ${request.timeoutMs ?? 30_000}ms`, 'BROWSER_EXECUTE_TIMEOUT', { cause: error });
|
|
246
|
+
}
|
|
237
247
|
throw new BrowserError(`browser: execute failed: ${String(error)}`, 'BROWSER_EXECUTE_FAILED', { cause: error });
|
|
238
248
|
}
|
|
239
249
|
}
|
|
@@ -333,13 +343,10 @@ export class ElectronBrowserProvider {
|
|
|
333
343
|
return { ok: true, content: content.slice(0, ${String(maxChars)}), truncated }
|
|
334
344
|
})()`;
|
|
335
345
|
// Honor a per-call timeout: content evaluation can hang on a heavy page,
|
|
336
|
-
// so a caller-supplied budget bounds it.
|
|
337
|
-
//
|
|
346
|
+
// so a caller-supplied budget bounds it. Unlike a bare signal entry check,
|
|
347
|
+
// withTimeout also interrupts a call already in flight.
|
|
338
348
|
const timeoutMs = request.timeoutMs ?? 30_000;
|
|
339
|
-
const
|
|
340
|
-
const result = signal !== undefined
|
|
341
|
-
? await handleSendEvaluate(tab.handle, script, AbortSignal.any([signal, timeoutSignal]))
|
|
342
|
-
: await handleSendEvaluate(tab.handle, script, timeoutSignal);
|
|
349
|
+
const result = await withTimeout(handleSendEvaluate(tab.handle, script), timeoutMs, signal, `browser: content timed out after ${timeoutMs}ms`);
|
|
343
350
|
if (!result.ok)
|
|
344
351
|
throw new BrowserError(`browser: content evaluation failed: ${result.exception}`, 'BROWSER_CONTENT_FAILED');
|
|
345
352
|
const value = result.value;
|
|
@@ -362,7 +369,9 @@ export class ElectronBrowserProvider {
|
|
|
362
369
|
const { handle } = this.activeTab(s);
|
|
363
370
|
signal?.throwIfAborted();
|
|
364
371
|
await handle.sendCommand('Input.insertText', { text: request.text });
|
|
365
|
-
|
|
372
|
+
// Store the full text so replay re-issues the same input; the history
|
|
373
|
+
// tool truncates long values when rendering.
|
|
374
|
+
this.record(s, 'type', { text: request.text }, true);
|
|
366
375
|
}
|
|
367
376
|
/**
|
|
368
377
|
* Fill a form's fields in one batch. Runs one page-context script that
|
|
@@ -421,7 +430,15 @@ export class ElectronBrowserProvider {
|
|
|
421
430
|
return vis.length > 0 ? vis : all
|
|
422
431
|
}
|
|
423
432
|
for (const spec of specs) {
|
|
424
|
-
|
|
433
|
+
let els
|
|
434
|
+
try {
|
|
435
|
+
els = candidates(spec)
|
|
436
|
+
} catch (e) {
|
|
437
|
+
// A malformed selector must not abort the whole batch; report the
|
|
438
|
+
// field as failed and continue with the rest.
|
|
439
|
+
out.push({ ok: false, error: String(e), target: describe(spec) })
|
|
440
|
+
continue
|
|
441
|
+
}
|
|
425
442
|
if (els.length === 0) { out.push({ ok: false, error: 'field not found', target: describe(spec) }); continue }
|
|
426
443
|
const el = els[0]
|
|
427
444
|
const tag = el.tagName
|
|
@@ -481,7 +498,9 @@ export class ElectronBrowserProvider {
|
|
|
481
498
|
let submitted = false
|
|
482
499
|
if (${submitFlag}) {
|
|
483
500
|
let last = null
|
|
484
|
-
for (const spec of specs) {
|
|
501
|
+
for (const spec of specs) {
|
|
502
|
+
try { const els = candidates(spec); if (els.length > 0) { last = els[0]; break } } catch { /* skip */ }
|
|
503
|
+
}
|
|
485
504
|
const form = last && (last.form || last.closest('form'))
|
|
486
505
|
if (form) { form.requestSubmit(); submitted = true }
|
|
487
506
|
}
|
|
@@ -553,7 +572,10 @@ export class ElectronBrowserProvider {
|
|
|
553
572
|
// a clip this yields the full-page image (CDP default is the viewport).
|
|
554
573
|
params.captureBeyondViewport = true;
|
|
555
574
|
}
|
|
556
|
-
|
|
575
|
+
// Bound the capture: a view that has not painted can wedge
|
|
576
|
+
// Page.captureScreenshot indefinitely (observed after view re-showing).
|
|
577
|
+
const timeoutMs = 30_000;
|
|
578
|
+
const result = await withTimeout(handle.sendCommand(CDP_PAGE_CAPTURE_SCREENSHOT, params), timeoutMs, signal, `browser: screenshot timed out after ${timeoutMs}ms`);
|
|
557
579
|
const data = result.data;
|
|
558
580
|
if (typeof data !== 'string') {
|
|
559
581
|
throw new BrowserError('browser: screenshot returned no image data', 'BROWSER_SCREENSHOT_FAILED');
|
|
@@ -576,7 +598,7 @@ export class ElectronBrowserProvider {
|
|
|
576
598
|
/** Append one operation to the session's history. */
|
|
577
599
|
record(s, action, params, ok, detail) {
|
|
578
600
|
const entry = {
|
|
579
|
-
seq: s.
|
|
601
|
+
seq: s.nextSeq++,
|
|
580
602
|
action,
|
|
581
603
|
params,
|
|
582
604
|
ok,
|
|
@@ -636,8 +658,10 @@ export class ElectronBrowserProvider {
|
|
|
636
658
|
const script = entry.params.script;
|
|
637
659
|
if (typeof script !== 'string')
|
|
638
660
|
throw new BrowserError(`browser: history seq ${seq} execute has no script`, 'BROWSER_HISTORY_INVALID');
|
|
639
|
-
const
|
|
640
|
-
|
|
661
|
+
const recordedArgs = entry.params.args;
|
|
662
|
+
const args = Array.isArray(recordedArgs) ? recordedArgs.filter((a) => typeof a === 'string') : undefined;
|
|
663
|
+
const result = await this.execute(session, { script, ...args !== undefined && args.length > 0 ? { args } : {} });
|
|
664
|
+
this.record(s, 'replay', { seq, of: entry.action, script, ...args !== undefined && args.length > 0 ? { args } : {} }, result.ok, result.ok ? { result: String(result.value) } : { error: result.exception });
|
|
641
665
|
return;
|
|
642
666
|
}
|
|
643
667
|
default:
|
|
@@ -703,7 +727,13 @@ function withTimeout(promise, ms, signal, message) {
|
|
|
703
727
|
if (done)
|
|
704
728
|
return;
|
|
705
729
|
done = true;
|
|
706
|
-
|
|
730
|
+
// A fired timeout must also release the abort listener; { once: true }
|
|
731
|
+
// only releases it on the next abort, which may never come.
|
|
732
|
+
if (signal !== undefined)
|
|
733
|
+
signal.removeEventListener('abort', onAbort);
|
|
734
|
+
const error = new Error(message);
|
|
735
|
+
error.name = 'TimeoutError';
|
|
736
|
+
reject(error);
|
|
707
737
|
}, ms);
|
|
708
738
|
const finish = (fn) => {
|
|
709
739
|
if (done)
|
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
* Self-hosted Electron browser host (parent side): an
|
|
3
3
|
* {@link ElectronBrowserViewHost} implementation that spawns the plugin's own
|
|
4
4
|
* Electron child process (host-main.js) and drives it over line-delimited
|
|
5
|
-
* JSON-RPC on
|
|
6
|
-
* desktop shell's electronViewHost (plain dsh web):
|
|
7
|
-
* enough — the browser window appears on first use.
|
|
5
|
+
* JSON-RPC on a loopback TCP socket. This is what makes the plugin work on
|
|
6
|
+
* surfaces without a desktop shell's electronViewHost (plain dsh web):
|
|
7
|
+
* installing the plugin is enough — the browser window appears on first use.
|
|
8
8
|
*
|
|
9
9
|
* Protocol (one JSON object per line, both directions):
|
|
10
10
|
* -> { id, op: 'createView' } | { id, op: 'destroyView', viewId } |
|
|
@@ -28,10 +28,13 @@ export declare class RemoteElectronViewHost implements ElectronBrowserViewHost {
|
|
|
28
28
|
private pendingSocket;
|
|
29
29
|
private readonly views;
|
|
30
30
|
private readyPromise;
|
|
31
|
+
private disposed;
|
|
31
32
|
constructor(hostMainPath: string);
|
|
32
|
-
/** Ensure the child is up and ready (lazy on first use). */
|
|
33
|
+
/** Ensure the child is up and ready (lazy on first use; restarts after a crash). */
|
|
33
34
|
private ready;
|
|
34
35
|
private start;
|
|
36
|
+
/** The child died: tear down so the next use starts a fresh child. */
|
|
37
|
+
private onChildExit;
|
|
35
38
|
createView(): ElectronViewHandle;
|
|
36
39
|
private ensureView;
|
|
37
40
|
showView(handle: ElectronViewHandle): void;
|
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
* Self-hosted Electron browser host (parent side): an
|
|
3
3
|
* {@link ElectronBrowserViewHost} implementation that spawns the plugin's own
|
|
4
4
|
* Electron child process (host-main.js) and drives it over line-delimited
|
|
5
|
-
* JSON-RPC on
|
|
6
|
-
* desktop shell's electronViewHost (plain dsh web):
|
|
7
|
-
* enough — the browser window appears on first use.
|
|
5
|
+
* JSON-RPC on a loopback TCP socket. This is what makes the plugin work on
|
|
6
|
+
* surfaces without a desktop shell's electronViewHost (plain dsh web):
|
|
7
|
+
* installing the plugin is enough — the browser window appears on first use.
|
|
8
8
|
*
|
|
9
9
|
* Protocol (one JSON object per line, both directions):
|
|
10
10
|
* -> { id, op: 'createView' } | { id, op: 'destroyView', viewId } |
|
|
@@ -147,6 +147,7 @@ function findElectronExe(store) {
|
|
|
147
147
|
class ElectronChildClient {
|
|
148
148
|
hostMainPath;
|
|
149
149
|
port;
|
|
150
|
+
onExit;
|
|
150
151
|
child;
|
|
151
152
|
pending = new Map();
|
|
152
153
|
nextId = 1;
|
|
@@ -154,9 +155,12 @@ class ElectronChildClient {
|
|
|
154
155
|
socket;
|
|
155
156
|
connected = false;
|
|
156
157
|
outbox = [];
|
|
157
|
-
|
|
158
|
+
/** Set once the child has exited; further calls fail fast instead of queueing. */
|
|
159
|
+
dead = false;
|
|
160
|
+
constructor(hostMainPath, port, onExit) {
|
|
158
161
|
this.hostMainPath = hostMainPath;
|
|
159
162
|
this.port = port;
|
|
163
|
+
this.onExit = onExit;
|
|
160
164
|
const electron = resolveElectronPath();
|
|
161
165
|
// ELECTRON_RUN_AS_NODE (even an empty string) makes Electron run as plain
|
|
162
166
|
// Node, breaking require('electron'); NODE_OPTIONS can inject flags that
|
|
@@ -174,25 +178,45 @@ class ElectronChildClient {
|
|
|
174
178
|
// Diagnostics only; never parse stderr as protocol.
|
|
175
179
|
process.stderr.write(`[dsh-browser host] ${String(chunk)}`);
|
|
176
180
|
});
|
|
181
|
+
// A failed spawn (bad/corrupt binary) emits 'error' — without a listener
|
|
182
|
+
// that would crash the whole DSH process.
|
|
183
|
+
this.child.on('error', error => {
|
|
184
|
+
process.stderr.write(`[dsh-browser host] spawn error: ${String(error)}\n`);
|
|
185
|
+
this.fail(new Error(`dsh-builtin-browser: browser host failed to start: ${String(error)}`));
|
|
186
|
+
});
|
|
177
187
|
this.child.on('exit', (code, signal) => {
|
|
178
|
-
|
|
179
|
-
for (const pending of this.pending.values())
|
|
180
|
-
pending.reject(err);
|
|
181
|
-
this.pending.clear();
|
|
188
|
+
this.fail(new Error(`dsh-builtin-browser: browser host exited (code=${String(code)} signal=${String(signal)})`));
|
|
182
189
|
});
|
|
183
190
|
}
|
|
191
|
+
/** Reject everything in flight, mark the client dead, and notify the host. */
|
|
192
|
+
fail(err) {
|
|
193
|
+
if (this.dead)
|
|
194
|
+
return;
|
|
195
|
+
this.dead = true;
|
|
196
|
+
this.connected = false;
|
|
197
|
+
for (const pending of this.pending.values())
|
|
198
|
+
pending.reject(err);
|
|
199
|
+
this.pending.clear();
|
|
200
|
+
this.outbox = [];
|
|
201
|
+
this.onExit?.();
|
|
202
|
+
}
|
|
184
203
|
/** Accept the child's connection (called by the server). */
|
|
185
204
|
attach(socket) {
|
|
186
205
|
this.socket = socket;
|
|
187
206
|
this.connected = true;
|
|
188
207
|
socket.setEncoding('utf8');
|
|
208
|
+
// Without an 'error' listener a remote reset (ECONNRESET/EPIPE) throws an
|
|
209
|
+
// uncaught 'error' event and crashes the whole DSH process; 'close' below
|
|
210
|
+
// does the cleanup.
|
|
211
|
+
socket.on('error', error => {
|
|
212
|
+
process.stderr.write(`[dsh-browser host] socket error: ${String(error)}\n`);
|
|
213
|
+
});
|
|
189
214
|
socket.on('data', chunk => this.onData(chunk));
|
|
190
215
|
socket.on('close', () => {
|
|
191
216
|
this.connected = false;
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
|
|
195
|
-
this.pending.clear();
|
|
217
|
+
if (!this.dead) {
|
|
218
|
+
this.fail(new Error('dsh-builtin-browser: browser host connection closed'));
|
|
219
|
+
}
|
|
196
220
|
});
|
|
197
221
|
// Flush anything queued while disconnected.
|
|
198
222
|
if (this.outbox.length > 0) {
|
|
@@ -231,6 +255,9 @@ class ElectronChildClient {
|
|
|
231
255
|
}
|
|
232
256
|
/** Send one command and await the reply. */
|
|
233
257
|
call(op, payload = {}) {
|
|
258
|
+
if (this.dead) {
|
|
259
|
+
return Promise.reject(new Error('dsh-builtin-browser: browser host is not running'));
|
|
260
|
+
}
|
|
234
261
|
const id = this.nextId++;
|
|
235
262
|
const line = JSON.stringify({ id, op, ...payload });
|
|
236
263
|
return new Promise((resolve, reject) => {
|
|
@@ -294,10 +321,11 @@ export class RemoteElectronViewHost {
|
|
|
294
321
|
pendingSocket;
|
|
295
322
|
views = new Map();
|
|
296
323
|
readyPromise;
|
|
324
|
+
disposed = false;
|
|
297
325
|
constructor(hostMainPath) {
|
|
298
326
|
this.hostMainPath = hostMainPath;
|
|
299
327
|
}
|
|
300
|
-
/** Ensure the child is up and ready (lazy on first use). */
|
|
328
|
+
/** Ensure the child is up and ready (lazy on first use; restarts after a crash). */
|
|
301
329
|
ready() {
|
|
302
330
|
if (this.readyPromise !== undefined)
|
|
303
331
|
return this.readyPromise;
|
|
@@ -316,10 +344,15 @@ export class RemoteElectronViewHost {
|
|
|
316
344
|
server.once('error', reject);
|
|
317
345
|
server.listen(0, '127.0.0.1', () => resolve());
|
|
318
346
|
});
|
|
347
|
+
// A later server error (rare on a loopback ephemeral port) must not crash
|
|
348
|
+
// the process; the client's fail path handles the actual recovery.
|
|
349
|
+
server.on('error', error => {
|
|
350
|
+
process.stderr.write(`[dsh-browser host] rpc server error: ${String(error)}\n`);
|
|
351
|
+
});
|
|
319
352
|
const address = server.address();
|
|
320
353
|
const port = typeof address === 'object' && address !== null ? address.port : 0;
|
|
321
354
|
this.server = server;
|
|
322
|
-
this.client = new ElectronChildClient(this.hostMainPath, port);
|
|
355
|
+
this.client = new ElectronChildClient(this.hostMainPath, port, () => this.onChildExit());
|
|
323
356
|
if (this.pendingSocket !== undefined) {
|
|
324
357
|
this.client.attach(this.pendingSocket);
|
|
325
358
|
this.pendingSocket = undefined;
|
|
@@ -327,6 +360,18 @@ export class RemoteElectronViewHost {
|
|
|
327
360
|
// Wait for the child's connection + readiness ping.
|
|
328
361
|
await withTimeout(this.client.call('ping'), READY_TIMEOUT_MS, 'browser host did not become ready');
|
|
329
362
|
}
|
|
363
|
+
/** The child died: tear down so the next use starts a fresh child. */
|
|
364
|
+
onChildExit() {
|
|
365
|
+
if (this.disposed)
|
|
366
|
+
return;
|
|
367
|
+
this.client = undefined;
|
|
368
|
+
this.server?.close();
|
|
369
|
+
this.server = undefined;
|
|
370
|
+
this.pendingSocket = undefined;
|
|
371
|
+
this.readyPromise = undefined;
|
|
372
|
+
// Keep the views map: handles still resolve to ids; a fresh child simply
|
|
373
|
+
// has no such views yet, and reset_session reopens clean sessions.
|
|
374
|
+
}
|
|
330
375
|
createView() {
|
|
331
376
|
// The seam is synchronous; the provider uses the handle immediately, so
|
|
332
377
|
// commands are deferred until the child is up and the view materialized.
|
|
@@ -346,7 +391,11 @@ export class RemoteElectronViewHost {
|
|
|
346
391
|
return view;
|
|
347
392
|
}
|
|
348
393
|
showView(handle) {
|
|
349
|
-
|
|
394
|
+
// Fire-and-forget by design (visibility is best-effort), but a rejected
|
|
395
|
+
// promise must not become an unhandled rejection (crash on Node >= 15).
|
|
396
|
+
void this.ready()
|
|
397
|
+
.then(() => this.client?.call('showView', { viewId: handle.id }))
|
|
398
|
+
.catch(() => { });
|
|
350
399
|
}
|
|
351
400
|
destroyView(handle) {
|
|
352
401
|
const view = this.views.get(handle.id);
|
|
@@ -359,6 +408,7 @@ export class RemoteElectronViewHost {
|
|
|
359
408
|
}
|
|
360
409
|
/** Shut the child and the RPC server down. */
|
|
361
410
|
dispose() {
|
|
411
|
+
this.disposed = true;
|
|
362
412
|
this.client?.kill();
|
|
363
413
|
this.client = undefined;
|
|
364
414
|
this.server?.close();
|
package/lib/index.d.ts
CHANGED
|
@@ -1,15 +1,16 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* dsh-browser plugin entry: aggregates the shared-browser capability
|
|
3
|
-
* The cordis.patch.yml rows reference subpath exports:
|
|
4
|
-
* - `dsh-browser/browser`
|
|
5
|
-
* - `dsh-browser/browser-electron`
|
|
6
|
-
* - `dsh-browser/tool-browser`
|
|
2
|
+
* dsh-builtin-browser plugin entry: aggregates the shared-browser capability
|
|
3
|
+
* pieces. The cordis.patch.yml rows reference subpath exports:
|
|
4
|
+
* - `dsh-builtin-browser/browser` -> the ctx.browser seam (Service)
|
|
5
|
+
* - `dsh-builtin-browser/browser-electron` -> the Electron CDP provider
|
|
6
|
+
* - `dsh-builtin-browser/tool-browser` -> the model-facing browser_* tools
|
|
7
7
|
* This root entry only re-exports for programmatic use; the loader rows are
|
|
8
8
|
* the composition surface.
|
|
9
|
-
* @module dsh-browser
|
|
9
|
+
* @module dsh-builtin-browser
|
|
10
10
|
*/
|
|
11
11
|
export { BrowserError } from './browser/types.ts';
|
|
12
|
-
export type { BrowserClickRequest, BrowserContentFormat, BrowserContentRequest, BrowserContentResult, BrowserExecuteRequest, BrowserExecuteResult, BrowserNavigateRequest, BrowserOpenRequest, BrowserProvider, BrowserScreenshotRequest, BrowserScreenshotResult, BrowserSessionId, BrowserSnapshotElement, BrowserSnapshotResult, BrowserTab, BrowserTypeRequest, } from './browser/types.ts';
|
|
12
|
+
export type { BrowserChallenge, BrowserClickRequest, BrowserContentFormat, BrowserContentRequest, BrowserContentResult, BrowserExecuteRequest, BrowserExecuteResult, BrowserFillField, BrowserFillRequest, BrowserFillResult, BrowserNavigateRequest, BrowserOpenRequest, BrowserProvider, BrowserScreenshotRequest, BrowserScreenshotResult, BrowserSessionId, BrowserSnapshotElement, BrowserSnapshotResult, BrowserTab, BrowserTypeRequest, ExportedCookie, } from './browser/types.ts';
|
|
13
13
|
export { BrowserRuntime } from './browser/runtime.ts';
|
|
14
14
|
export { ElectronBrowserProvider } from './browser-electron/provider.ts';
|
|
15
15
|
export type { ElectronBrowserViewHost, ElectronViewHandle } from './browser-electron/provider.ts';
|
|
16
|
+
export { RemoteElectronViewHost, defaultHostMainPath } from './browser-electron/remote-host.ts';
|
package/lib/index.js
CHANGED
|
@@ -1,13 +1,14 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* dsh-browser plugin entry: aggregates the shared-browser capability
|
|
3
|
-
* The cordis.patch.yml rows reference subpath exports:
|
|
4
|
-
* - `dsh-browser/browser`
|
|
5
|
-
* - `dsh-browser/browser-electron`
|
|
6
|
-
* - `dsh-browser/tool-browser`
|
|
2
|
+
* dsh-builtin-browser plugin entry: aggregates the shared-browser capability
|
|
3
|
+
* pieces. The cordis.patch.yml rows reference subpath exports:
|
|
4
|
+
* - `dsh-builtin-browser/browser` -> the ctx.browser seam (Service)
|
|
5
|
+
* - `dsh-builtin-browser/browser-electron` -> the Electron CDP provider
|
|
6
|
+
* - `dsh-builtin-browser/tool-browser` -> the model-facing browser_* tools
|
|
7
7
|
* This root entry only re-exports for programmatic use; the loader rows are
|
|
8
8
|
* the composition surface.
|
|
9
|
-
* @module dsh-browser
|
|
9
|
+
* @module dsh-builtin-browser
|
|
10
10
|
*/
|
|
11
11
|
export { BrowserError } from "./browser/types.js";
|
|
12
12
|
export { BrowserRuntime } from "./browser/runtime.js";
|
|
13
13
|
export { ElectronBrowserProvider } from "./browser-electron/provider.js";
|
|
14
|
+
export { RemoteElectronViewHost, defaultHostMainPath } from "./browser-electron/remote-host.js";
|
|
@@ -32,7 +32,7 @@ export interface Config {
|
|
|
32
32
|
export declare function apply(ctx: Context, config?: Config): void;
|
|
33
33
|
/** Test hook: inspect and reset the plugin-level session map (used by tests). */
|
|
34
34
|
export declare const internals: {
|
|
35
|
-
/**
|
|
35
|
+
/** A copy of the per-task session map (task key -> provider session id). */
|
|
36
36
|
readonly sessions: ReadonlyMap<string, BrowserSessionId>;
|
|
37
37
|
/** Drop one task's mapping without closing the provider session. */
|
|
38
38
|
clearSession(key?: string): void;
|