dsh-builtin-browser 0.1.6 → 0.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.en.md CHANGED
@@ -2,17 +2,35 @@
2
2
 
3
3
  中文 | [English](README.en.md)
4
4
 
5
- A shared real browser capability plugin for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness): a visible browser the human watches and can take over, driven by the agent over CDP.
5
+ A shared **real browser** capability plugin for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness): install and it just works — the agent drives a real, visible browser over CDP that the human can watch and take over at any time. Human and agent operate the **very same page**.
6
6
 
7
- This plugin provides the **host-side capability** (browser seam, Electron CDP provider, `browser_*` model tools). The browser's **native view itself** is supplied by the host (a desktop shell) through `ctx.electronViewHost`; the plugin contains no browser UI.
7
+ **Why this plugin**
8
+
9
+ - **Install-and-use.** With a desktop shell the shell's embedded view is used; on plain `dsh web` the plugin **self-hosts** — it spawns its own Electron window with zero extra configuration.
10
+ - **Real and visible, human-takeover friendly.** Not a headless screenshot or a relay: the user sees the page and can grab control directly.
11
+ - **DOM-level driving, framework-friendly.** React/Vue controlled inputs fill reliably.
12
+ - **Concurrency-safe.** Task-level browser session isolation — parallel tasks never fight over the page or pollute each other.
13
+ - **Built for the real world.** CAPTCHA detection, login persistence, batch form filling, authenticated downloads, operation replay, and action restriction — all included.
8
14
 
9
15
  ## Features
10
16
 
17
+ **Browser capability**
18
+
11
19
  - **Real view, not a relay.** The browser is a native view (`WebContentsView`) the human can see and operate directly; the agent drives the very same page. The view is provided by the host shell; the plugin drives it.
12
20
  - **DOM-referenced, not coordinate-guessing.** `browser_snapshot` returns numbered interactive elements; `browser_execute` runs JS in the page (native setters for framework inputs), so interaction works on React/Vue pages.
13
21
  - **Multi-tab sessions.** Open URLs in parallel tabs, list/switch/close/reset, all keeping state.
14
22
  - **Multi-format content.** Fetch pages as html / markdown / txt / json, scoped by selector, capped by length and timeout.
15
23
 
24
+ **Engineering capability**
25
+
26
+ - **Per-task isolation.** Each DSH task (session) gets its own browser session (own tabs and history); concurrent tasks never interfere, and calls within one task reuse the same session (`browser_session` / `browser_reset_session`).
27
+ - **Login persistence.** `browser_auth` exports/restores cookies, so logins survive host restarts.
28
+ - **CAPTCHA / bot-detection awareness.** Cloudflare, reCAPTCHA, hCaptcha, Turnstile and generic challenges are detected automatically (`browser_challenge`, and flagged in snapshots); instead of blindly retrying, the agent asks the human to complete it in the shared window.
29
+ - **Batch form filling.** `browser_fill` sets many fields in one call — matched by selector/name/label, handling controlled inputs, selects, checkboxes and radio groups, with optional submit.
30
+ - **Operation history & replay.** `browser_history` records operations; `browser_replay` re-runs one step.
31
+ - **Authenticated downloads.** `browser_download` fetches a URL with the session's cookies straight to a local file.
32
+ - **Safety restriction.** `browser_restrict` limits which browser actions are allowed, preventing stray clicks/navigation.
33
+
16
34
  ## Requirements
17
35
 
18
36
  - DeepSeek Harness (dsh) with the `web` profile
@@ -28,7 +46,7 @@ This plugin provides the **host-side capability** (browser seam, Electron CDP pr
28
46
  | DeepSeek Harness (dsh) | `0.1.0-rc.5` |
29
47
  | Electron | `43.4.0` |
30
48
  | Node.js | `22.20.0` |
31
- | dsh-builtin-browser | `0.1.3` |
49
+ | dsh-builtin-browser | `0.1.7` |
32
50
  | OS | Windows 10 (10.0.26200) |
33
51
 
34
52
  > The plugin declares `electron >= 30`; other platforms (macOS/Linux) run the same protocol but were only verified on the Windows environment above.
package/README.md CHANGED
@@ -2,17 +2,35 @@
2
2
 
3
3
  [English](README.en.md) | 中文
4
4
 
5
- 面向 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 的共享真实浏览器能力插件:一个用户可见、可随时接管的浏览器,由 agent 通过 CDP 驱动。
5
+ 面向 [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) 的**共享真实浏览器**能力插件:装好即可用——agent 通过 CDP 驱动一个真实、可见、可随时人工接管的浏览器,人与 agent 操作的是**同一个页面**。
6
6
 
7
- 本插件提供 **host 侧能力**(浏览器 seam、Electron CDP provider、`browser_*` 模型工具)。浏览器本身的**原生视图**由宿主(桌面外壳)通过 `ctx.electronViewHost` 提供;插件不包含浏览器界面 UI。
7
+ **我们的优势**
8
+
9
+ - **装好即用**:有桌面外壳时嵌入外壳视图;纯 `dsh web` 也能**自托管**拉起自己的 Electron 窗口,不需要任何额外配置。
10
+ - **真实可见、人工可接管**:不是无头截屏或转播,用户随时能看到页面并直接接手操作。
11
+ - **DOM 级驱动,框架友好**:React/Vue 受控组件也能可靠填写。
12
+ - **并发安全**:任务级浏览器会话隔离,多个任务并行互不抢页面、不互相污染。
13
+ - **面向真实世界**:人机验证识别、登录态持久化、批量填表、带登录态下载、操作回放、动作限制,一应俱全。
8
14
 
9
15
  ## 功能
10
16
 
17
+ **浏览器能力**
18
+
11
19
  - **真实视图,而非转播。** 浏览器是原生视图(`WebContentsView`),用户可直接看到并操作;agent 驱动的是同一个页面。视图由宿主外壳提供,插件负责驱动。
12
20
  - **DOM 引用,而非猜坐标。** `browser_snapshot` 返回带编号的交互元素;`browser_execute` 在页面里执行 JS(框架输入用原生 setter),在 React/Vue 页面上也能可靠交互。
13
21
  - **多标签会话。** 并行打开 URL、查看/切换/关闭/重置标签,状态保持。
14
22
  - **多格式内容。** 以 html / markdown / txt / json 抓取页面,支持 selector 限定、长度与超时上限。
15
23
 
24
+ **工程化能力**
25
+
26
+ - **按任务隔离。** 每个 DSH 任务(会话)拥有独立的浏览器会话(独立标签页与历史),并发任务互不干扰;同一任务内多次调用复用同一会话(`browser_session` / `browser_reset_session`)。
27
+ - **登录态持久化。** `browser_auth` 导出/恢复 cookie,重启后登录态不丢。
28
+ - **人机验证识别。** 自动检测 Cloudflare / reCAPTCHA / hCaptcha / Turnstile 等挑战(`browser_challenge`,快照也会标注),提示人工在共享窗口完成,不再盲目重试。
29
+ - **批量表单填充。** `browser_fill` 一次填写多个字段:按选择器/名称/标签匹配,支持受控输入、下拉、单选/复选,可选提交。
30
+ - **操作历史与回放。** `browser_history` 记录操作日志,`browser_replay` 可回放某一步。
31
+ - **带登录态下载。** `browser_download` 用会话 cookie 把文件取到本地,登录后内容可直接落盘。
32
+ - **安全限制。** `browser_restrict` 限制允许的浏览器动作,防误点/误导航。
33
+
16
34
  ## 环境要求
17
35
 
18
36
  - DeepSeek Harness(dsh)且安装了 `web` profile
@@ -28,7 +46,7 @@
28
46
  | DeepSeek Harness(dsh) | `0.1.0-rc.5` |
29
47
  | Electron | `43.4.0` |
30
48
  | Node.js | `22.20.0` |
31
- | dsh-builtin-browser | `0.1.3` |
49
+ | dsh-builtin-browser | `0.1.7` |
32
50
  | 操作系统 | Windows 10 (10.0.26200) |
33
51
 
34
52
  > 插件声明 `electron >= 30`;其他平台(如 macOS/Linux)按同一协议运行,但仅在上表 Windows 环境实测。
@@ -74,6 +92,13 @@ provider 与工具以 `ctx.get('electronViewHost')` 是否存在为门控,因此
74
92
  | `browser_content` | 以 html / markdown / txt / json 抓取页面(selector、maxChars、timeoutMs) |
75
93
  | `browser_screenshot` | PNG 截图,可选 `fullPage` |
76
94
  | `browser_list_tabs` / `browser_switch_tab` / `browser_close_tab` / `browser_reset` | 多标签会话管理 |
95
+ | `browser_session` / `browser_reset_session` | 查看/重置当前任务的浏览器会话(按任务隔离) |
96
+ | `browser_history` / `browser_replay` | 查看操作记录并回放某一步 |
97
+ | `browser_download` | 带会话 cookie 下载 URL 到本地文件 |
98
+ | `browser_restrict` | 限制允许的浏览器动作,防误点/误导航 |
99
+ | `browser_auth` | 导出/恢复 cookie,持久化登录态 |
100
+ | `browser_challenge` | 检测人机验证(CAPTCHA / Cloudflare / reCAPTCHA / hCaptcha / Turnstile)是否拦截当前页 |
101
+ | `browser_fill` | 批量填充表单(选择器/名称/标签匹配,支持受控输入、下拉、单选/复选,可选提交) |
77
102
 
78
103
  ### 操作纪律(点击/填表)
79
104
 
@@ -103,7 +128,8 @@ provider 按构造与 Electron 解耦:它通过 `ElectronBrowserViewHost` 接缝
103
128
 
104
129
  - 截图仅 PNG(CDP JPEG 在 Electron 43 上挂起);JPEG 等待非 CDP 转换路径。
105
130
  - 部分主机在软件合成下 `fullPage` 截图不稳定。
106
- - 会话生命周期为插件级(模型共享一个会话),尚未做到按 agent 隔离。
131
+ - 会话按任务隔离:每个调用方任务(DSH 会话)拥有独立的浏览器会话(独立标签页与历史),并发任务互不干扰;同一任务的多次调用复用同一会话。登录态(cookie)为共享,可用 `browser_auth` 导出/恢复。
132
+ - 人机验证(CAPTCHA)无法自动解决:快照会标注检测到的挑战(`browser_challenge` 可显式检查),此时应请用户在共享窗口中人工完成,而不是反复重试。
107
133
  - 无痕模式(`privateMode`)未实现:它需要 Electron 的 session 分区能力,属于宿主层,本插件不承诺。
108
134
  - 本插件不含浏览器列 UI——那是宿主外壳的配套,别把"浏览器列"当成插件能力。
109
135
 
@@ -8,9 +8,9 @@
8
8
  */
9
9
  import { Context, Service } from '@deepseek-ai/cordis';
10
10
  import z from '@deepseek-ai/schemastery';
11
- import type { BrowserClickRequest, BrowserContentRequest, BrowserContentResult, BrowserExecuteRequest, BrowserExecuteResult, BrowserNavigateRequest, BrowserOpenRequest, BrowserProvider, BrowserScreenshotRequest, BrowserScreenshotResult, BrowserSessionId, BrowserSnapshotResult, BrowserTab, BrowserTypeRequest } from './types.ts';
11
+ import type { BrowserClickRequest, BrowserContentRequest, BrowserContentResult, BrowserDownloadRequest, BrowserExecuteRequest, BrowserExecuteResult, BrowserFillRequest, BrowserFillResult, BrowserHistoryEntry, BrowserNavigateRequest, BrowserOpenRequest, BrowserProvider, BrowserScreenshotRequest, BrowserScreenshotResult, BrowserSessionId, BrowserSnapshotResult, BrowserTab, BrowserTypeRequest, BrowserChallenge, ExportedCookie } from './types.ts';
12
12
  export { BrowserError, } from './types.ts';
13
- export type { BrowserClickRequest, BrowserContentFormat, BrowserContentRequest, BrowserContentResult, BrowserExecuteRequest, BrowserExecuteResult, BrowserNavigateRequest, BrowserOpenRequest, BrowserProvider, BrowserScreenshotRequest, BrowserScreenshotResult, BrowserSessionId, BrowserSnapshotElement, BrowserSnapshotResult, BrowserTab, BrowserTypeRequest, } from './types.ts';
13
+ export type { BrowserChallenge, BrowserClickRequest, BrowserContentFormat, BrowserContentRequest, BrowserContentResult, BrowserDownloadRequest, BrowserExecuteRequest, BrowserExecuteResult, BrowserFillField, BrowserFillRequest, BrowserFillResult, BrowserHistoryEntry, BrowserNavigateRequest, BrowserOpenRequest, BrowserProvider, BrowserScreenshotRequest, BrowserScreenshotResult, BrowserSessionId, BrowserSnapshotElement, BrowserSnapshotResult, BrowserTab, BrowserTypeRequest, ExportedCookie, } from './types.ts';
14
14
  declare module '@deepseek-ai/cordis' {
15
15
  interface Context {
16
16
  browser: BrowserRuntime;
@@ -29,12 +29,12 @@ export interface BrowserRuntimeConfig {
29
29
  * context).
30
30
  *
31
31
  * Selection semantics (resolved at execution time, never order-dependent):
32
- * - A configured id that is registered and `available()` 鈫?that provider.
33
- * - A configured id not registered 鈫?`BROWSER_PROVIDER_CONFIGURED_MISSING`.
34
- * - A configured id registered but unavailable 鈫? * `BROWSER_PROVIDER_CONFIGURED_UNAVAILABLE`.
35
- * - No id configured, exactly one registered usable provider 鈫?that provider.
36
- * - No id configured, multiple usable providers 鈫?`BROWSER_PROVIDER_AMBIGUOUS`.
37
- * - No id configured, no usable provider 鈫?`BROWSER_PROVIDER_UNAVAILABLE`.
32
+ * - A configured id that is registered and `available()` → that provider.
33
+ * - A configured id not registered → `BROWSER_PROVIDER_CONFIGURED_MISSING`.
34
+ * - A configured id registered but unavailable → `BROWSER_PROVIDER_CONFIGURED_UNAVAILABLE`.
35
+ * - No id configured, exactly one registered usable provider → that provider.
36
+ * - No id configured, multiple usable providers → `BROWSER_PROVIDER_AMBIGUOUS`.
37
+ * - No id configured, no usable provider → `BROWSER_PROVIDER_UNAVAILABLE`.
38
38
  */
39
39
  export declare class BrowserRuntime extends Service {
40
40
  /** Provider selection config. */
@@ -76,8 +76,24 @@ export declare class BrowserRuntime extends Service {
76
76
  click(session: BrowserSessionId, request: BrowserClickRequest, signal?: AbortSignal): Promise<void>;
77
77
  /** Type into the focused element through the selected provider. */
78
78
  type(session: BrowserSessionId, request: BrowserTypeRequest, signal?: AbortSignal): Promise<void>;
79
+ /** Fill a form's fields in one batch through the selected provider. */
80
+ fillForm(session: BrowserSessionId, request: BrowserFillRequest, signal?: AbortSignal): Promise<BrowserFillResult>;
79
81
  /** Capture the current page through the selected provider. */
80
82
  screenshot(session: BrowserSessionId, request?: BrowserScreenshotRequest, signal?: AbortSignal): Promise<BrowserScreenshotResult>;
83
+ /** Check for a human-verification challenge on the active tab. */
84
+ detectChallenge(session: BrowserSessionId, signal?: AbortSignal): Promise<BrowserChallenge>;
85
+ /** Return the session's chronological operation log through the provider. */
86
+ history(session: BrowserSessionId): Promise<readonly BrowserHistoryEntry[]>;
87
+ /** Replay one recorded operation by sequence number through the provider. */
88
+ replay(session: BrowserSessionId, seq: number): Promise<void>;
89
+ /** Download a URL to a local file through the provider. */
90
+ download(session: BrowserSessionId, request: BrowserDownloadRequest, signal?: AbortSignal): Promise<{
91
+ readonly path: string;
92
+ }>;
93
+ /** Export the session's cookies through the provider. */
94
+ flushAuth(session: BrowserSessionId): Promise<readonly ExportedCookie[]>;
95
+ /** Import cookies into the session through the provider. */
96
+ restoreAuth(session: BrowserSessionId, cookies: readonly ExportedCookie[]): Promise<number>;
81
97
  /** Close the session through the selected provider. Idempotent. */
82
98
  close(session: BrowserSessionId): Promise<void>;
83
99
  }
@@ -15,12 +15,12 @@ export { BrowserError, } from "./types.js";
15
15
  * context).
16
16
  *
17
17
  * Selection semantics (resolved at execution time, never order-dependent):
18
- * - A configured id that is registered and `available()` 鈫?that provider.
19
- * - A configured id not registered 鈫?`BROWSER_PROVIDER_CONFIGURED_MISSING`.
20
- * - A configured id registered but unavailable 鈫? * `BROWSER_PROVIDER_CONFIGURED_UNAVAILABLE`.
21
- * - No id configured, exactly one registered usable provider 鈫?that provider.
22
- * - No id configured, multiple usable providers 鈫?`BROWSER_PROVIDER_AMBIGUOUS`.
23
- * - No id configured, no usable provider 鈫?`BROWSER_PROVIDER_UNAVAILABLE`.
18
+ * - A configured id that is registered and `available()` → that provider.
19
+ * - A configured id not registered → `BROWSER_PROVIDER_CONFIGURED_MISSING`.
20
+ * - A configured id registered but unavailable → `BROWSER_PROVIDER_CONFIGURED_UNAVAILABLE`.
21
+ * - No id configured, exactly one registered usable provider → that provider.
22
+ * - No id configured, multiple usable providers → `BROWSER_PROVIDER_AMBIGUOUS`.
23
+ * - No id configured, no usable provider → `BROWSER_PROVIDER_UNAVAILABLE`.
24
24
  */
25
25
  export class BrowserRuntime extends Service {
26
26
  /** Provider selection config. */
@@ -51,7 +51,7 @@ export class BrowserRuntime extends Service {
51
51
  yield () => this.providers.delete(provider.id);
52
52
  }.bind(this), 'browser.registerProvider()');
53
53
  // ctx.effect's disposer returns Promise<void>; our disposer API is
54
- // synchronous fire-and-forget 鈥?discard the (always-resolved) promise.
54
+ // synchronous fire-and-forget — discard the (always-resolved) promise.
55
55
  return () => void dispose();
56
56
  }
57
57
  /** Resolve the selected provider or throw the matching {@link BrowserError}. */
@@ -126,10 +126,38 @@ export class BrowserRuntime extends Service {
126
126
  async type(session, request, signal) {
127
127
  return this.resolveProvider().type(session, request, signal);
128
128
  }
129
+ /** Fill a form's fields in one batch through the selected provider. */
130
+ async fillForm(session, request, signal) {
131
+ return this.resolveProvider().fillForm(session, request, signal);
132
+ }
129
133
  /** Capture the current page through the selected provider. */
130
134
  async screenshot(session, request, signal) {
131
135
  return this.resolveProvider().screenshot(session, request, signal);
132
136
  }
137
+ /** Check for a human-verification challenge on the active tab. */
138
+ async detectChallenge(session, signal) {
139
+ return this.resolveProvider().detectChallenge(session, signal);
140
+ }
141
+ /** Return the session's chronological operation log through the provider. */
142
+ async history(session) {
143
+ return this.resolveProvider().history(session);
144
+ }
145
+ /** Replay one recorded operation by sequence number through the provider. */
146
+ async replay(session, seq) {
147
+ return this.resolveProvider().replay(session, seq);
148
+ }
149
+ /** Download a URL to a local file through the provider. */
150
+ async download(session, request, signal) {
151
+ return this.resolveProvider().download(session, request, signal);
152
+ }
153
+ /** Export the session's cookies through the provider. */
154
+ async flushAuth(session) {
155
+ return this.resolveProvider().flushAuth(session);
156
+ }
157
+ /** Import cookies into the session through the provider. */
158
+ async restoreAuth(session, cookies) {
159
+ return this.resolveProvider().restoreAuth(session, cookies);
160
+ }
133
161
  /** Close the session through the selected provider. Idempotent. */
134
162
  async close(session) {
135
163
  return this.resolveProvider().close(session);
@@ -3,14 +3,14 @@
3
3
  * provider registration, session lifecycle, cancellation, errors, and product
4
4
  * configuration; providers differ only in what backs a session (an Electron
5
5
  * `WebContentsView` in the desktop shell, a headless Chromium relay for
6
- * remote deployments, 鈥?.
6
+ * remote deployments, and so on).
7
7
  * @module dsh-browser/browser/types
8
8
  */
9
9
  import { HarnessError } from '@deepseek-ai/dsh-llm';
10
10
  /**
11
11
  * Stable identity of one browser session, resolved by the Host through a
12
12
  * Typert `LookupMap` to the live session object (the same mechanism
13
- * `Agent 鈫?agentId` uses). The id is opaque to the wire; only the provider
13
+ * `Agent → agentId` uses). The id is opaque to the wire; only the provider
14
14
  * and its host binding can interpret it.
15
15
  */
16
16
  export type BrowserSessionId = string;
@@ -25,7 +25,7 @@ export interface BrowserNavigateRequest {
25
25
  }
26
26
  /**
27
27
  * Click at viewport coordinates. Coordinates are viewport-relative pixels in
28
- * the session's own coordinate space 鈥?the same space the human interacts
28
+ * the session's own coordinate space — the same space the human interacts
29
29
  * with, so a real view and a relay agree without scaling.
30
30
  */
31
31
  export interface BrowserClickRequest {
@@ -39,6 +39,48 @@ export interface BrowserTypeRequest {
39
39
  /** The text to insert. */
40
40
  readonly text: string;
41
41
  }
42
+ /** One field of a batch form fill. Match by selector, or by name/label/placeholder. */
43
+ export interface BrowserFillField {
44
+ /** CSS selector; when present, candidates are scoped to it. */
45
+ readonly selector?: string;
46
+ /** Match by the field's `name` attribute. */
47
+ readonly name?: string;
48
+ /** Match by associated `<label>` text or `aria-label`. */
49
+ readonly label?: string;
50
+ /** Match by `placeholder` text. */
51
+ readonly placeholder?: string;
52
+ /** Field kind; defaults to text. */
53
+ readonly kind?: 'text' | 'textarea' | 'checkbox' | 'radio' | 'select';
54
+ /** Value to set: string, number, or boolean (checkbox/radio). For a
55
+ * multi-select or radio group, the option value (or visible text). */
56
+ readonly value: string | number | boolean;
57
+ }
58
+ /** Batch form fill: set several fields, optionally submitting the form. */
59
+ export interface BrowserFillRequest {
60
+ /** The fields to fill, in order. */
61
+ readonly fields: readonly BrowserFillField[];
62
+ /** Submit the containing form after filling. Default false. */
63
+ readonly submit?: boolean;
64
+ /** Per-field evaluation budget in ms. Default 30000. */
65
+ readonly timeoutMs?: number;
66
+ }
67
+ /** One field's outcome in a batch fill. */
68
+ export interface BrowserFillFieldResult {
69
+ readonly ok: boolean;
70
+ /** The field description matched (selector/name/label/placeholder). */
71
+ readonly target: string;
72
+ /** How the value was applied: input | textarea | select | checkbox | radio | contenteditable. */
73
+ readonly method?: string;
74
+ /** Error text when the field could not be filled. */
75
+ readonly error?: string;
76
+ }
77
+ /** Outcome of a batch form fill. */
78
+ export interface BrowserFillResult {
79
+ /** Per-field outcomes, in request order. */
80
+ readonly fields: readonly BrowserFillFieldResult[];
81
+ /** Whether the containing form was submitted. */
82
+ readonly submitted: boolean;
83
+ }
42
84
  /**
43
85
  * A screenshot of the current page state. `dataUrl` is a base64 data URL the
44
86
  * tool layer renders directly; the live view itself never crosses the wire
@@ -47,15 +89,38 @@ export interface BrowserTypeRequest {
47
89
  export interface BrowserScreenshotResult {
48
90
  /** Base64 data URL of the capture (PNG or JPEG per the request). */
49
91
  readonly dataUrl: string;
92
+ /** File path the capture was also saved to, when the request asked for it. */
93
+ readonly path?: string;
50
94
  }
51
95
  /** Screenshot capture options. PNG only (CDP JPEG hangs on Electron 43). */
52
96
  export interface BrowserScreenshotRequest {
53
97
  /** Capture the full scrollable page instead of the viewport. Default false. */
54
98
  readonly fullPage?: boolean;
99
+ /** Absolute file path to also save the PNG to (for vision-tool reads). */
100
+ readonly savePath?: string;
101
+ }
102
+ /** Download options: fetch a URL into a local file, keeping the session's cookies. */
103
+ export interface BrowserDownloadRequest {
104
+ /** The URL to download. */
105
+ readonly url: string;
106
+ /** Absolute path of the file to write. */
107
+ readonly savePath: string;
108
+ }
109
+ /** A serializable cookie, exported from or imported into the browser session. */
110
+ export interface ExportedCookie {
111
+ /** Cookie URL (scheme + domain + path), as accepted by cookies.set. */
112
+ readonly url: string;
113
+ readonly name: string;
114
+ readonly value: string;
115
+ readonly domain?: string;
116
+ readonly path?: string;
117
+ readonly secure?: boolean;
118
+ readonly httpOnly?: boolean;
119
+ readonly expirationDate?: number;
55
120
  }
56
121
  /**
57
122
  * Execute arbitrary JavaScript in the session's page context. This is the
58
- * primary interaction path 鈥?DOM-referenced, not screen-coordinate-based:
123
+ * primary interaction path — DOM-referenced, not screen-coordinate-based:
59
124
  * focus/input/click go through the page's own JS (native setters for
60
125
  * framework-controlled inputs), matching how a human-driven agent operates.
61
126
  */
@@ -97,7 +162,7 @@ export interface BrowserSnapshotElement {
97
162
  }
98
163
  /**
99
164
  * AI-friendly page snapshot: a compact, numbered inventory of interactive
100
- * elements plus page facts. Not raw HTML 鈥?the model dialogues with this.
165
+ * elements plus page facts. Not raw HTML — the model dialogues with this.
101
166
  */
102
167
  export interface BrowserSnapshotResult {
103
168
  /** Final page URL. */
@@ -108,6 +173,21 @@ export interface BrowserSnapshotResult {
108
173
  readonly elements: readonly BrowserSnapshotElement[];
109
174
  /** True when the snapshot was truncated (element cap reached). */
110
175
  readonly truncated: boolean;
176
+ /** Human-verification challenge blocking the page, when one is detected. */
177
+ readonly challenge?: BrowserChallenge;
178
+ }
179
+ /**
180
+ * A detected human-verification (bot-detection / CAPTCHA) challenge blocking
181
+ * the page. Detection is best-effort and marker-based; a clean result is not a
182
+ * guarantee that no challenge exists.
183
+ */
184
+ export interface BrowserChallenge {
185
+ /** Whether a challenge currently blocks the page. */
186
+ readonly blocked: boolean;
187
+ /** Machine-readable kind: cloudflare | hcaptcha | recaptcha | turnstile | generic. */
188
+ readonly kind?: string;
189
+ /** Short human-readable description, e.g. 'Cloudflare "Just a moment" interstitial'. */
190
+ readonly reason?: string;
111
191
  }
112
192
  /**
113
193
  * Content fetch formats. `html` is the raw DOM; `markdown` is a structured
@@ -163,7 +243,7 @@ export interface BrowserProvider {
163
243
  available(): boolean;
164
244
  /**
165
245
  * Open a new session. The provider mints the session id and prepares its
166
- * backing surface (a view, a headless page, 鈥?. Sessions are isolated from
246
+ * backing surface (a view, a headless page, and so on). Sessions are isolated from
167
247
  * each other; a session must not be visible to any other session's caller.
168
248
  */
169
249
  open(): Promise<BrowserSessionId>;
@@ -171,7 +251,7 @@ export interface BrowserProvider {
171
251
  openUrl(session: BrowserSessionId, request: BrowserOpenRequest, signal?: AbortSignal): Promise<void>;
172
252
  /** List the session's tabs. */
173
253
  listTabs(session: BrowserSessionId): Promise<readonly BrowserTab[]>;
174
- /** Switch to a tab by id. Unknown id 鈫?`BROWSER_TAB_UNKNOWN`. */
254
+ /** Switch to a tab by id. Unknown id -> `BROWSER_TAB_UNKNOWN`. */
175
255
  switchTab(session: BrowserSessionId, tabId: string): Promise<void>;
176
256
  /** Close one tab. Closing the active tab activates the next. */
177
257
  closeTab(session: BrowserSessionId, tabId: string): Promise<void>;
@@ -183,17 +263,50 @@ export interface BrowserProvider {
183
263
  execute(session: BrowserSessionId, request: BrowserExecuteRequest, signal?: AbortSignal): Promise<BrowserExecuteResult>;
184
264
  /** Produce an AI-friendly snapshot of the active tab. */
185
265
  snapshot(session: BrowserSessionId, signal?: AbortSignal): Promise<BrowserSnapshotResult>;
266
+ /** Check whether a human-verification challenge is blocking the active tab. */
267
+ detectChallenge(session: BrowserSessionId, signal?: AbortSignal): Promise<BrowserChallenge>;
186
268
  /** Fetch page content in a requested format. */
187
269
  content(session: BrowserSessionId, request: BrowserContentRequest, signal?: AbortSignal): Promise<BrowserContentResult>;
188
270
  /** Click at viewport coordinates (fallback path; execute is preferred). Honor `signal` for cancellation. */
189
271
  click(session: BrowserSessionId, request: BrowserClickRequest, signal?: AbortSignal): Promise<void>;
190
272
  /** Type into the focused element (fallback path). Honor `signal` for cancellation. */
191
273
  type(session: BrowserSessionId, request: BrowserTypeRequest, signal?: AbortSignal): Promise<void>;
274
+ /** Fill a form's fields in one batch. Honor `signal` for cancellation. */
275
+ fillForm(session: BrowserSessionId, request: BrowserFillRequest, signal?: AbortSignal): Promise<BrowserFillResult>;
192
276
  /** Capture the current page. Honor `signal` for cancellation. */
193
277
  screenshot(session: BrowserSessionId, request?: BrowserScreenshotRequest, signal?: AbortSignal): Promise<BrowserScreenshotResult>;
278
+ /** Download a URL to a local file. Honor `signal` for cancellation. */
279
+ download(session: BrowserSessionId, request: BrowserDownloadRequest, signal?: AbortSignal): Promise<{
280
+ readonly path: string;
281
+ }>;
282
+ /** Export the session's cookies (login state). */
283
+ flushAuth(session: BrowserSessionId): Promise<readonly ExportedCookie[]>;
284
+ /** Import cookies into the session (restore login state). */
285
+ restoreAuth(session: BrowserSessionId, cookies: readonly ExportedCookie[]): Promise<number>;
286
+ /** Return the session's chronological operation log. */
287
+ history(session: BrowserSessionId): Promise<readonly BrowserHistoryEntry[]>;
288
+ /** Replay one recorded operation by sequence number. */
289
+ replay(session: BrowserSessionId, seq: number): Promise<void>;
194
290
  /** Close the session and destroy its backing surface. Idempotent. */
195
291
  close(session: BrowserSessionId): Promise<void>;
196
292
  }
293
+ /** One recorded browser operation, in chronological order (seq 1, 2, 3…). */
294
+ export interface BrowserHistoryEntry {
295
+ /** Monotonic sequence number within the session. */
296
+ readonly seq: number;
297
+ /** The operation name: navigate | execute | click | type | replay. */
298
+ readonly action: string;
299
+ /** The operation's arguments. */
300
+ readonly params: Record<string, unknown>;
301
+ /** Whether the operation succeeded. */
302
+ readonly ok: boolean;
303
+ /** Truncated result text, when the operation returned one. */
304
+ readonly result?: string;
305
+ /** Error text, when the operation failed. */
306
+ readonly error?: string;
307
+ /** Epoch millis of the operation. */
308
+ readonly at: number;
309
+ }
197
310
  /**
198
311
  * Typed browser error with a machine-routable, open-string `code` and chained
199
312
  * `cause`. Consumers must tolerate provider-specific codes. Shared codes cover
@@ -3,7 +3,7 @@
3
3
  * provider registration, session lifecycle, cancellation, errors, and product
4
4
  * configuration; providers differ only in what backs a session (an Electron
5
5
  * `WebContentsView` in the desktop shell, a headless Chromium relay for
6
- * remote deployments, 鈥?.
6
+ * remote deployments, and so on).
7
7
  * @module dsh-browser/browser/types
8
8
  */
9
9
  import { HarnessError } from '@deepseek-ai/dsh-llm';
@@ -99,6 +99,91 @@ async function handle(op, msg) {
99
99
  reply(msg.id, { ok: true, result });
100
100
  return;
101
101
  }
102
+ case 'download': {
103
+ const viewId = msg.viewId;
104
+ const url = msg.url;
105
+ const savePath = msg.savePath;
106
+ if (viewId === undefined || typeof url !== 'string' || typeof savePath !== 'string') {
107
+ throw new Error('download missing viewId/url/savePath');
108
+ }
109
+ const entry = views.get(viewId);
110
+ if (entry === undefined)
111
+ throw new Error(`download: unknown view ${viewId}`);
112
+ // Fetch the URL inside the page context (keeps cookies/login), read
113
+ // the body as base64, and return it; the parent writes the file. This
114
+ // avoids Electron's download pipeline entirely (CDP debugger attach
115
+ // can interfere with will-download).
116
+ const result = await entry.webContentsView.webContents.debugger.sendCommand('Runtime.evaluate', {
117
+ expression: `(async () => {
118
+ const r = await fetch(${JSON.stringify(url)}, { credentials: 'include' })
119
+ if (!r.ok) throw new Error('HTTP ' + r.status)
120
+ const b = await r.arrayBuffer()
121
+ const bytes = new Uint8Array(b)
122
+ let bin = ''
123
+ for (let i = 0; i < bytes.length; i += 0x8000) bin += String.fromCharCode.apply(null, bytes.subarray(i, i + 0x8000))
124
+ return btoa(bin)
125
+ })()`,
126
+ awaitPromise: true,
127
+ returnByValue: true,
128
+ });
129
+ const value = result.result?.value;
130
+ if (typeof value !== 'string') {
131
+ const detail = result.exceptionDetails;
132
+ throw new Error(`download failed: ${detail?.exception?.description ?? 'no data'}`);
133
+ }
134
+ reply(msg.id, { ok: true, result: { base64: value, savePath } });
135
+ return;
136
+ }
137
+ case 'flushAuth': {
138
+ const viewId = msg.viewId;
139
+ if (viewId === undefined)
140
+ throw new Error('flushAuth missing viewId');
141
+ const entry = views.get(viewId);
142
+ if (entry === undefined)
143
+ throw new Error(`flushAuth: unknown view ${viewId}`);
144
+ // Export the session's cookies so login state can be saved/restored
145
+ // across browser hosts (or shared with another machine).
146
+ const cookies = await entry.webContentsView.webContents.session.cookies.get({});
147
+ const exported = cookies.map(c => ({
148
+ url: `http${c.secure ? 's' : ''}://${c.domain.startsWith('.') ? c.domain.slice(1) : c.domain}${c.path}`,
149
+ name: c.name,
150
+ value: c.value,
151
+ domain: c.domain,
152
+ path: c.path,
153
+ secure: c.secure,
154
+ httpOnly: c.httpOnly,
155
+ expirationDate: c.expirationDate,
156
+ }));
157
+ reply(msg.id, { ok: true, result: { cookies: exported } });
158
+ return;
159
+ }
160
+ case 'restoreAuth': {
161
+ const viewId = msg.viewId;
162
+ const cookies = msg.cookies;
163
+ if (viewId === undefined)
164
+ throw new Error('restoreAuth missing viewId');
165
+ const entry = views.get(viewId);
166
+ if (entry === undefined)
167
+ throw new Error(`restoreAuth: unknown view ${viewId}`);
168
+ if (!Array.isArray(cookies))
169
+ throw new Error('restoreAuth missing cookies array');
170
+ for (const c of cookies) {
171
+ if (typeof c.url !== 'string' || typeof c.name !== 'string' || typeof c.value !== 'string')
172
+ continue;
173
+ await entry.webContentsView.webContents.session.cookies.set({
174
+ url: c.url,
175
+ name: c.name,
176
+ value: c.value,
177
+ ...typeof c.domain === 'string' ? { domain: c.domain } : {},
178
+ ...typeof c.path === 'string' ? { path: c.path } : {},
179
+ ...typeof c.secure === 'boolean' ? { secure: c.secure } : {},
180
+ ...typeof c.httpOnly === 'boolean' ? { httpOnly: c.httpOnly } : {},
181
+ ...typeof c.expirationDate === 'number' ? { expirationDate: c.expirationDate } : {},
182
+ });
183
+ }
184
+ reply(msg.id, { ok: true, result: { restored: cookies.length } });
185
+ return;
186
+ }
102
187
  default:
103
188
  throw new Error(`unknown op ${op}`);
104
189
  }