dsh-web-fetch-playwright 0.2.0 → 0.2.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -12,8 +12,8 @@ A [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness) (DSH) plug
12
12
  - **Denoise pipeline** — Mozilla Readability extracts the article, DOMPurify removes layout/noise tags (nav, sidebar, footer, ads, forms), and Turndown with the GFM plugin converts to Markdown with the same style options as the shipped `tool-web` renderer.
13
13
  - **Two backends** — launch a local Playwright browser, or drive an already-running browser over its DevTools Protocol (CDP) endpoint.
14
14
  - **Browser resolution** — a configured path, a `playwright` CLI on `$PATH`, or the bundled `playwright-core`; CDP needs no local browser at all.
15
- - **Isolated sessions** — every fetch uses its own browser context. Local launches close their browser per fetch; the CDP backend keeps **one shared connection** to the remote browser and each fetch opens an isolated context (a tab) inside it, closed when done.
16
- - **Live configuration** — a settings card (设置 → 插件 → 插件配置) edits the backend, denoise toggle, and concurrency; changes apply to the next fetch without a restart.
15
+ - **Isolated or profile sessions (CDP)** — every fetch is scoped to exactly one tab. Local launches close their browser per fetch; the CDP backend keeps **one shared connection** to the remote browser and each fetch opens a tab inside it, closed when done. By default that tab lives in the remote browser's **real profile** (its cookies, localStorage, and persistent logins apply — like `playwright-cli open`); unchecking *Share the browser context* switches to a throwaway isolated context per fetch.
16
+ - **Live configuration** — a settings card (设置 → 插件 → 插件配置) edits the backend, context mode, denoise toggle, and concurrency; changes apply to the next fetch without a restart.
17
17
  - **Budget-aware** — per-fetch deadline (45s); concurrency is backend-priced (`maxConcurrency`, default 4 local browsers / **50 CDP tabs**; queued fetches fail fast with a retry hint after 20s instead of hanging); image/font/media subrequests aborted; body capped at 100k chars.
18
18
 
19
19
  ## How it works
@@ -61,11 +61,14 @@ After a bundle plugin is added to the profile layer stack, **restart `dsh web`**
61
61
 
62
62
  The settings card (设置 → 插件 → 插件配置 → *Playwright 网页爬取*) edits the `web-fetch-playwright` settings section live:
63
63
 
64
+ ![Playwright 网页爬取 plugin configuration card](./playwright-plugin-config.png)
65
+
64
66
  | Field | Default | Description |
65
67
  | --- | --- | --- |
66
68
  | `backend` | `local` | Radio: *Local Playwright* or *Remote CDP endpoint*, each with its own nested input. |
67
69
  | `playwrightPath` | (blank) | Local backend: path to a `playwright` executable or a Chromium-family browser binary. Blank = discover on `$PATH`, then fall back to the bundled `playwright-core`. |
68
70
  | `cdpEndpoint` | `127.0.0.1:9222` | Remote backend: `host:port`, `http(s)://…` or `ws(s)://…`. |
71
+ | `shareBrowserContext` | `true` | CDP backend only. **Checked (profile mode)**: each fetch is a tab in the remote browser's default context — its real profile — so cookies/localStorage are shared and its persistent logins apply; only the tab closes when the fetch ends. **Unchecked (isolated mode)**: a fresh incognito-like context per fetch, nothing shared. Local backend ignores this field. |
69
72
  | `denoise` | `true` | Run the denoise pipeline; off returns the full rendered HTML for the tool layer to convert. |
70
73
  | `maxConcurrency` | *(auto)* | How many fetches may render at once (1–200). Blank = backend default: **4** for local (each slot launches a browser) / **50** for CDP (each slot is just a tab in the already-running remote browser). Beyond the limit, fetches wait briefly; if no slot frees within 20s they fail with `WEB_FETCH_TIMEOUT` and a hint to retry or raise this setting, rather than hanging until the tool budget aborts. |
71
74
 
@@ -75,7 +78,25 @@ Local backend resolution order:
75
78
  2. A `playwright` executable on `$PATH` (its package knows that installation's browser registry).
76
79
  3. The bundled `playwright-core` — requires `PLAYWRIGHT_BROWSERS_PATH` or browsers in the default cache; otherwise the error suggests `playwright install chromium`.
77
80
 
78
- CDP mode needs no local browser: the provider holds **one shared connection** for its lifetime (reconnecting automatically if it drops, and reconnecting to the new endpoint when the setting changes), and every fetch leases a fresh isolated context — a tab in your browser — that closes when the fetch completes. Concurrency therefore counts tabs, which is why the CDP default is high (50). Unloading the plugin drops the shared connection.
81
+ CDP mode needs no local browser: the provider holds **one shared connection** for its lifetime (reconnecting automatically if it drops, and reconnecting to the new endpoint when the setting changes), and every fetch leases a tab in the remote browser that closes when the fetch completes. Concurrency therefore counts tabs, which is why the CDP default is high (50). Unloading the plugin drops the shared connection (the remote browser itself is never closed).
82
+
83
+ ### The CDP context modes (share the browser's profile or not)
84
+
85
+ With *Share the browser context* **checked** (default, profile mode), each fetch is a tab in the remote browser's default context — the real profile. Cookies and localStorage come from and are written back to that profile, so sites the browser is logged into are fetched logged-in, exactly like a tab you open by hand. The shared context is never closed; resource filtering and popup guards attach to the fetch's tab only, so your other tabs are untouched. With it **unchecked** (isolated mode), every fetch gets a fresh incognito-like context — anonymous reads, nothing persists.
86
+
87
+ **Profile-mode risk notes** — it upgrades `web_fetch` from "anonymous read" to "acts as the browser's logged-in user":
88
+
89
+ - A fetched page that talks the agent into a GET-style state-changing URL (logout, settings change, API call) will send it with your session cookies.
90
+ - Concurrent fetches to the same site share one cookie jar; one fetch's logout or `Set-Cookie` affects the others.
91
+ - Output starts depending on the browser's history (A/B buckets, language preferences). The remote profile also keeps accumulating site data; the plugin never cleans it.
92
+
93
+ Persistent logins require a persistent user-data-dir. Headful (recommended — log in by hand once):
94
+
95
+ ```sh
96
+ google-chrome --remote-debugging-port=9222 --user-data-dir="$HOME/.config/chrome-dsh-profile"
97
+ ```
98
+
99
+ Headless server (pre-seed logins in a headful environment first): `chromium --headless=new --remote-debugging-port=9222 --user-data-dir=/data/chrome-dsh-profile`. Do **not** add `--incognito` or a throwaway user-data-dir — either defeats profile mode. Design notes and verified playwright-core facts live in [`docs/context-mode-profile.md`](./docs/context-mode-profile.md).
79
100
 
80
101
  ## Development
81
102
 
@@ -104,7 +125,7 @@ See [CONTRIBUTING.md](./CONTRIBUTING.md) for the development and release workflo
104
125
 
105
126
  ## Security
106
127
 
107
- Same stance as the built-in HTTP provider: **no SSRF / private-network protection is implemented** — anything the browser can reach, this provider can fetch. The CDP endpoint is configured from the settings page with no loopback restriction, so only expose the settings page to trusted environments. Fetched pages are rendered locally; no data is sent anywhere beyond the target page itself.
128
+ Same stance as the built-in HTTP provider: **no SSRF / private-network protection is implemented** — anything the browser can reach, this provider can fetch. The CDP endpoint is configured from the settings page with no loopback restriction, so only expose the settings page to trusted environments. Fetched pages are rendered locally; no data is sent anywhere beyond the target page itself — but note that in profile mode the requests (and any state changes a malicious page tricks the agent into) carry the remote browser's logged-in sessions (see the risk notes above).
108
129
 
109
130
  ## License
110
131
 
package/README.zh-CN.md CHANGED
@@ -12,8 +12,8 @@
12
12
  - **降噪管线** —— Mozilla Readability 提取正文,DOMPurify 移除布局/噪音标签(导航、侧边栏、页脚、广告、表单),Turndown + GFM 插件按与内置 `tool-web` 渲染器一致的风格转成 Markdown。
13
13
  - **两种后端** —— 本地启动 Playwright 浏览器,或通过 DevTools 协议(CDP)驱动一个已在运行的浏览器。
14
14
  - **浏览器解析** —— 配置路径 → `$PATH` 上的 `playwright` CLI → 插件自带的 `playwright-core`;CDP 模式完全不需要本地浏览器。
15
- - **隔离会话** —— 每次抓取使用独立 browser context。本地后端每次抓取启动并关闭自己的浏览器;CDP 后端对远端浏览器保持**一条共享连接**,每次抓取只在其中开一个隔离 context(标签页),用完即关。
16
- - **热配置** —— 「设置 → 插件 → 插件配置」卡片可随时切换后端、降噪开关与并发数,改动对下一次抓取即时生效,无需重启。
15
+ - **共享或隔离会话(CDP)** —— 每次抓取严格限定为一个标签页。本地后端每次抓取启动并关闭自己的浏览器;CDP 后端对远端浏览器保持**一条共享连接**,每次抓取只在其里开一个标签页、用完即关。默认该标签页位于远端浏览器的**真实 profile**(沿用其 cookie、localStorage 与已登录会话,效果类似 `playwright-cli open`);取消勾选「共享浏览器上下文」则切换为每次抓取全新隔离 context。
16
+ - **热配置** —— 「设置 → 插件 → 插件配置」卡片可随时切换后端、上下文模式、降噪开关与并发数,改动对下一次抓取即时生效,无需重启。
17
17
  - **预算控制** —— 单次抓取 45s 超时;并发按后端定价(`maxConcurrency`,默认本地 4 个浏览器 / **CDP 50 个标签页**;排队的抓取等不到空位会在 20s 内尽快报错并提示重试,而不是一直挂到被工具层中止);拦截图片/字体/媒体子请求;返回体 10 万字符封顶。
18
18
 
19
19
  ## 工作原理
@@ -61,11 +61,14 @@ bundle 插件加入 profile 层栈后需**重启 `dsh web`** 生效;卸载用
61
61
 
62
62
  设置卡片(设置 → 插件 → 插件配置 → *Playwright 网页爬取*)实时编辑 `web-fetch-playwright` 设置段:
63
63
 
64
+ ![Playwright 网页爬取 plugin configuration card](./playwright-plugin-config.png)
65
+
64
66
  | 字段 | 默认 | 说明 |
65
67
  | --- | --- | --- |
66
68
  | `backend` | `local` | radio:本地 Playwright / 远端 CDP 地址,每个选项内嵌各自的填空。 |
67
69
  | `playwrightPath` | 空 | 本地后端:`playwright` 可执行文件或 Chromium 系浏览器二进制路径;留空按 `$PATH` 查找,再回退到内置 `playwright-core`。 |
68
70
  | `cdpEndpoint` | `127.0.0.1:9222` | 远端后端:`host:port`、`http(s)://…` 或 `ws(s)://…`。 |
71
+ | `shareBrowserContext` | `true` | 仅 CDP 后端。**勾选(profile 模式)**:每次抓取是远端浏览器默认 context(真实 profile)里的一个标签页,cookie/localStorage 与之互通、已登录会话直接生效,抓取结束只关标签页;**取消勾选(隔离模式)**:每次抓取使用全新隐身式 context,互不共享。本地后端忽略此字段。 |
69
72
  | `denoise` | `true` | 是否启用降噪;关闭时返回整页渲染 HTML,交由工具层转换。 |
70
73
  | `maxConcurrency` | *(自动)* | 同时渲染的页面上限(1–200)。留空按后端取默认:本地 **4**(每个槽位启动一个浏览器)/ CDP **50**(远端浏览器已就位,每个槽位只是一个标签页)。超出的请求短暂排队;20s 内等不到空位则以 `WEB_FETCH_TIMEOUT` 尽快失败并提示重试或调大该值,而不是一直挂起直到工具层预算中止。 |
71
74
 
@@ -75,7 +78,25 @@ bundle 插件加入 profile 层栈后需**重启 `dsh web`** 生效;卸载用
75
78
  2. `$PATH` 上的 `playwright`(其包自带该安装的浏览器注册表);
76
79
  3. 插件内置的 `playwright-core`——需要 `PLAYWRIGHT_BROWSERS_PATH` 或默认缓存里有浏览器,否则报错会提示 `playwright install chromium`。
77
80
 
78
- CDP 模式不需要本地浏览器:插件在生命周期内对远端浏览器保持**一条共享连接**(连接断开自动重连,地址改动后自动换连),每次抓取只租用一个全新的隔离 context——即远端浏览器里的一个标签页——抓取结束即关闭、不污染已有会话。因此并发数按"标签页"计,默认也更高(50)。插件卸载时断开共享连接。
81
+ CDP 模式不需要本地浏览器:插件在生命周期内对远端浏览器保持**一条共享连接**(连接断开自动重连,地址改动后自动换连),每次抓取只租用远端浏览器里的一个标签页,抓取结束即关闭。因此并发数按"标签页"计,默认也更高(50)。插件卸载时断开共享连接(绝不会关闭远端浏览器本身)。
82
+
83
+ ### CDP 上下文模式(是否共享浏览器 profile)
84
+
85
+ 「共享浏览器上下文」**勾选**(默认,profile 模式)时,每次抓取是远端浏览器默认 context——真实 profile——里的一个标签页:cookie 与 localStorage 双向互通,浏览器里已登录的站点会以登录态被抓取,和你手动开标签页一样。共享 context 永不关闭;资源过滤与弹窗守卫只挂在本次抓取自己的标签页上,不会干扰你人工打开的其他标签页。**取消勾选**(隔离模式)时,每次抓取使用全新隐身式 context——匿名读取,什么都不保留。
86
+
87
+ **profile 模式风险须知** —— 它把 `web_fetch` 从"匿名读网页"升级为"以浏览器登录身份行动":
88
+
89
+ - 被抓取的恶意页面若诱导 agent 请求 GET 型状态变更 URL(登出、改设置、API 操作),请求会自动携带会话 cookie。
90
+ - 同一站点的并发抓取共享一个 cookie jar,一方的登出 / `Set-Cookie` 会影响另一方。
91
+ - 输出开始依赖浏览器历史(A/B 分桶、语言偏好);远端 profile 的站点数据只增不减,插件不做清理。
92
+
93
+ 登录态持久的前提是 user-data-dir 持久化。有头(推荐,手动登录一次):
94
+
95
+ ```sh
96
+ google-chrome --remote-debugging-port=9222 --user-data-dir="$HOME/.config/chrome-dsh-profile"
97
+ ```
98
+
99
+ 无头服务器(先在有头环境预置登录态):`chromium --headless=new --remote-debugging-port=9222 --user-data-dir=/data/chrome-dsh-profile`。**不要**叠加 `--incognito` 或一次性 user-data-dir——都会让 profile 模式失效。设计依据与已核实的 playwright-core 源码事实见 [`docs/context-mode-profile.md`](./docs/context-mode-profile.md)。
79
100
 
80
101
  ## 开发
81
102
 
@@ -104,7 +125,7 @@ tests/ # 单元 + provider + 浏览器集成(可自跳过
104
125
 
105
126
  ## 安全边界
106
127
 
107
- 与内置 HTTP provider 同立场:**未实现 SSRF/私网防护**——浏览器能访问的目标,本 provider 就能抓。CDP 地址由设置页配置,不做回环限制,请在可信环境暴露设置页。抓取仅在本地渲染,除目标页面自身外不会向任何地方发送数据。
128
+ 与内置 HTTP provider 同立场:**未实现 SSRF/私网防护**——浏览器能访问的目标,本 provider 就能抓。CDP 地址由设置页配置,不做回环限制,请在可信环境暴露设置页。抓取仅在本地渲染,除目标页面自身外不会向任何地方发送数据——但 profile 模式下请求(以及恶意页面诱导 agent 触发的状态变更)会携带远端浏览器的登录会话,见上文风险须知。
108
129
 
109
130
  ## 许可证
110
131
 
package/dsh.plugin.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-web-fetch-playwright",
3
- "version": "0.2.0",
3
+ "version": "0.2.2",
4
4
  "description": "Playwright/CDP web-fetch provider for DeepSeek Harness: renders pages in a real browser, denoises them (Readability + DOMPurify), and returns markdown.",
5
5
  "entry": {
6
6
  "name": "dsh-web-fetch-playwright",
package/lib/client.js CHANGED
@@ -348,6 +348,7 @@ window.__ModuleLoader__.load({
348
348
  radioField("backend", ["local", "cdp"]),
349
349
  textField("playwrightPath"),
350
350
  textField("cdpEndpoint"),
351
+ checkboxField("shareBrowserContext"),
351
352
  checkboxField("denoise"),
352
353
  numberField("maxConcurrency", 1, 200)
353
354
  ]);
@@ -359,6 +360,7 @@ window.__ModuleLoader__.load({
359
360
  backend: this.form.field("backend"),
360
361
  playwrightPath: this.form.field("playwrightPath"),
361
362
  cdpEndpoint: this.form.field("cdpEndpoint"),
363
+ shareBrowserContext: this.form.field("shareBrowserContext"),
362
364
  denoise: this.form.field("denoise"),
363
365
  maxConcurrency: this.form.field("maxConcurrency")
364
366
  };
@@ -386,21 +388,21 @@ window.__ModuleLoader__.load({
386
388
  document.head.appendChild(tag);
387
389
  }
388
390
  var PluginCard_module_css_default = {
389
- "card": "UM8-Rq_card",
391
+ "chevron": "UM8-Rq_chevron",
390
392
  "chevronOpen": "UM8-Rq_chevronOpen",
391
- "readOnly": "UM8-Rq_readOnly",
392
- "footer": "UM8-Rq_footer",
393
- "pending": "UM8-Rq_pending",
394
- "header": "UM8-Rq_header",
395
393
  "save": "UM8-Rq_save",
396
- "chevron": "UM8-Rq_chevron",
394
+ "name": "UM8-Rq_name",
395
+ "discard": "UM8-Rq_discard",
397
396
  "failed": "UM8-Rq_failed",
398
- "headText": "UM8-Rq_headText",
399
- "body": "UM8-Rq_body",
397
+ "header": "UM8-Rq_header",
400
398
  "cardOpen": "UM8-Rq_cardOpen",
401
- "discard": "UM8-Rq_discard",
402
- "name": "UM8-Rq_name",
403
- "description": "UM8-Rq_description"
399
+ "description": "UM8-Rq_description",
400
+ "body": "UM8-Rq_body",
401
+ "card": "UM8-Rq_card",
402
+ "footer": "UM8-Rq_footer",
403
+ "readOnly": "UM8-Rq_readOnly",
404
+ "headText": "UM8-Rq_headText",
405
+ "pending": "UM8-Rq_pending"
404
406
  };
405
407
  //#endregion
406
408
  //#region src/client/PluginCard.tsx
@@ -507,7 +509,7 @@ window.__ModuleLoader__.load({
507
509
  }
508
510
  //#endregion
509
511
  //#region \0dsh-web-fetch-playwright-css:/opt/dsh/plugins/dsh-web-fetch-playwright/src/client/fields.module.css.mjs
510
- const css = ".CFSaVG_field{border:0;border-top:1px solid var(--dsw-alias-border-l2);flex-direction:column;gap:6px;margin:0;padding:12px 0;display:flex}.CFSaVG_field:first-child{border-top:0}.CFSaVG_fieldEmbedded{border:0;flex-direction:column;gap:6px;margin:0;padding:0;display:flex}.CFSaVG_head{align-items:center;gap:8px;display:flex}.CFSaVG_label{min-width:0;color:var(--dsw-alias-label-primary);flex:1;font-size:13px;font-weight:500;line-height:1.5}.CFSaVG_badges{align-items:center;gap:8px;display:inline-flex}.CFSaVG_badge{white-space:nowrap;background:var(--dsw-alias-bg-module-platform);color:var(--dsw-alias-label-secondary);border-radius:999px;padding:1px 8px;font-size:11px;font-weight:500;line-height:17px}.CFSaVG_reset{font:inherit;color:var(--dsw-alias-label-secondary);cursor:pointer;background:0 0;border:none;padding:0;font-size:12px;line-height:1.5}.CFSaVG_reset:hover:not(:disabled){color:var(--dsw-alias-label-primary)}.CFSaVG_reset:disabled{cursor:default}.CFSaVG_input{border:1px solid var(--dsw-alias-border-l2);background:var(--dsw-alias-bg-layer-3);height:34px;font:inherit;color:var(--dsw-alias-label-primary);border-radius:8px;padding:0 12px;font-size:13px;line-height:1.5}.CFSaVG_input:focus-visible{border-color:var(--dsw-alias-brand-primary);outline:none}.CFSaVG_input:disabled{color:var(--dsw-alias-label-tertiary);cursor:default}.CFSaVG_inputInvalid{border:1px solid var(--dsw-alias-label-error);background:var(--dsw-alias-bg-layer-3);height:34px;font:inherit;color:var(--dsw-alias-label-primary);border-radius:8px;padding:0 12px;font-size:13px;line-height:1.5}.CFSaVG_invalid{color:var(--dsw-alias-label-error);margin:0;font-size:12px;line-height:1.5}.CFSaVG_hint{color:var(--dsw-alias-label-tertiary);margin:0;font-size:12px;line-height:1.5}.CFSaVG_radioGroup{flex-direction:column;gap:8px;display:flex}.CFSaVG_radioOption{border:1px solid var(--dsw-alias-border-l2);background:var(--dsw-alias-bg-layer-3);border-radius:8px;flex-direction:column;gap:8px;padding:10px 12px;display:flex}.CFSaVG_radioOption:has(input[type=radio]:checked){border-color:var(--dsw-alias-brand-primary)}.CFSaVG_radioPick{cursor:pointer;align-items:flex-start;gap:10px;display:flex}.CFSaVG_radioPick input[type=radio]{width:14px;height:14px;accent-color:var(--dsw-alias-brand-primary);flex:none;margin-top:2px}.CFSaVG_radioPick input[type=radio]:focus-visible{outline:2px solid var(--dsw-alias-brand-primary);outline-offset:1px}.CFSaVG_radioText{flex-direction:column;gap:2px;min-width:0;display:flex}.CFSaVG_radioLabel{color:var(--dsw-alias-label-primary);font-size:13px;font-weight:500;line-height:1.5}.CFSaVG_radioHint{color:var(--dsw-alias-label-tertiary);font-size:12px;line-height:1.5}.CFSaVG_radioContent{margin-left:24px}.CFSaVG_radioContent input[type=text]:enabled{cursor:text}.CFSaVG_checkboxRow{align-items:center;gap:10px;display:flex}.CFSaVG_checkboxRow input[type=checkbox]{width:16px;height:16px;accent-color:var(--dsw-alias-brand-primary);flex:none;margin:0}.CFSaVG_checkboxRow input[type=checkbox]:focus-visible{outline:2px solid var(--dsw-alias-brand-primary);outline-offset:1px}";
512
+ const css = ".CFSaVG_field{border:0;border-top:1px solid var(--dsw-alias-border-l2);flex-direction:column;gap:6px;margin:0;padding:12px 0;display:flex}.CFSaVG_field:first-child{border-top:0}.CFSaVG_fieldEmbedded{border:0;flex-direction:column;gap:6px;margin:0;padding:0;display:flex}.CFSaVG_head{align-items:center;gap:8px;display:flex}.CFSaVG_label{min-width:0;color:var(--dsw-alias-label-primary);flex:1;font-size:13px;font-weight:500;line-height:1.5}.CFSaVG_badges{align-items:center;gap:8px;display:inline-flex}.CFSaVG_badge{white-space:nowrap;background:var(--dsw-alias-bg-module-platform);color:var(--dsw-alias-label-secondary);border-radius:999px;padding:1px 8px;font-size:11px;font-weight:500;line-height:17px}.CFSaVG_reset{font:inherit;color:var(--dsw-alias-label-secondary);cursor:pointer;background:0 0;border:none;padding:0;font-size:12px;line-height:1.5}.CFSaVG_reset:hover:not(:disabled){color:var(--dsw-alias-label-primary)}.CFSaVG_reset:disabled{cursor:default}.CFSaVG_input{border:1px solid var(--dsw-alias-border-l2);background:var(--dsw-alias-bg-layer-3);height:34px;font:inherit;color:var(--dsw-alias-label-primary);border-radius:8px;padding:0 12px;font-size:13px;line-height:1.5}.CFSaVG_input:focus-visible{border-color:var(--dsw-alias-brand-primary);outline:none}.CFSaVG_input:disabled{color:var(--dsw-alias-label-tertiary);cursor:default}.CFSaVG_inputInvalid{border:1px solid var(--dsw-alias-label-error);background:var(--dsw-alias-bg-layer-3);height:34px;font:inherit;color:var(--dsw-alias-label-primary);border-radius:8px;padding:0 12px;font-size:13px;line-height:1.5}.CFSaVG_invalid{color:var(--dsw-alias-label-error);margin:0;font-size:12px;line-height:1.5}.CFSaVG_hint{color:var(--dsw-alias-label-tertiary);margin:0;font-size:12px;line-height:1.5}.CFSaVG_hintIndented{margin-left:26px}.CFSaVG_radioGroup{flex-direction:column;gap:8px;display:flex}.CFSaVG_radioOption{border:1px solid var(--dsw-alias-border-l2);background:var(--dsw-alias-bg-layer-3);border-radius:8px;flex-direction:column;gap:8px;padding:10px 12px;display:flex}.CFSaVG_radioOption:has(input[type=radio]:checked){border-color:var(--dsw-alias-brand-primary)}.CFSaVG_radioPick{cursor:pointer;align-items:flex-start;gap:10px;display:flex}.CFSaVG_radioPick input[type=radio]{width:14px;height:14px;accent-color:var(--dsw-alias-brand-primary);flex:none;margin-top:2px}.CFSaVG_radioPick input[type=radio]:focus-visible{outline:2px solid var(--dsw-alias-brand-primary);outline-offset:1px}.CFSaVG_radioText{flex-direction:column;gap:2px;min-width:0;display:flex}.CFSaVG_radioLabel{color:var(--dsw-alias-label-primary);font-size:13px;font-weight:500;line-height:1.5}.CFSaVG_radioHint{color:var(--dsw-alias-label-tertiary);font-size:12px;line-height:1.5}.CFSaVG_radioContent{margin-left:24px}.CFSaVG_radioContent>*+*{margin-top:12px}.CFSaVG_radioContent input[type=text]:enabled{cursor:text}.CFSaVG_checkboxRow{align-items:center;gap:10px;display:flex}.CFSaVG_checkboxRow input[type=checkbox]{width:16px;height:16px;accent-color:var(--dsw-alias-brand-primary);flex:none;margin:0}.CFSaVG_checkboxRow input[type=checkbox]:focus-visible{outline:2px solid var(--dsw-alias-brand-primary);outline-offset:1px}";
511
513
  const tagId = "dsh-web-fetch-playwright/fields.module.css";
512
514
  if (typeof document !== "undefined" && document.querySelector("style[data-plugin-css=" + JSON.stringify(tagId) + "]") === null) {
513
515
  const tag = document.createElement("style");
@@ -517,25 +519,26 @@ window.__ModuleLoader__.load({
517
519
  document.head.appendChild(tag);
518
520
  }
519
521
  var fields_module_css_default = {
520
- "checkboxRow": "CFSaVG_checkboxRow",
521
- "radioOption": "CFSaVG_radioOption",
522
- "radioLabel": "CFSaVG_radioLabel",
523
- "head": "CFSaVG_head",
524
- "radioPick": "CFSaVG_radioPick",
525
522
  "badges": "CFSaVG_badges",
523
+ "invalid": "CFSaVG_invalid",
524
+ "radioContent": "CFSaVG_radioContent",
525
+ "hint": "CFSaVG_hint",
526
+ "radioLabel": "CFSaVG_radioLabel",
526
527
  "radioGroup": "CFSaVG_radioGroup",
527
- "fieldEmbedded": "CFSaVG_fieldEmbedded",
528
528
  "radioText": "CFSaVG_radioText",
529
- "radioHint": "CFSaVG_radioHint",
530
- "radioContent": "CFSaVG_radioContent",
531
- "field": "CFSaVG_field",
532
- "input": "CFSaVG_input",
533
529
  "badge": "CFSaVG_badge",
534
- "invalid": "CFSaVG_invalid",
535
- "hint": "CFSaVG_hint",
530
+ "fieldEmbedded": "CFSaVG_fieldEmbedded",
531
+ "checkboxRow": "CFSaVG_checkboxRow",
532
+ "hintIndented": "CFSaVG_hintIndented",
533
+ "input": "CFSaVG_input",
534
+ "radioHint": "CFSaVG_radioHint",
536
535
  "label": "CFSaVG_label",
537
536
  "reset": "CFSaVG_reset",
538
- "inputInvalid": "CFSaVG_inputInvalid"
537
+ "inputInvalid": "CFSaVG_inputInvalid",
538
+ "field": "CFSaVG_field",
539
+ "radioPick": "CFSaVG_radioPick",
540
+ "head": "CFSaVG_head",
541
+ "radioOption": "CFSaVG_radioOption"
539
542
  };
540
543
  //#endregion
541
544
  //#region src/client/fields.tsx
@@ -641,10 +644,14 @@ window.__ModuleLoader__.load({
641
644
  })]
642
645
  });
643
646
  }
644
- /** A staged checkbox: the denoise toggle. An absent stored value defaults to on. */
647
+ /**
648
+ * A staged checkbox: the denoise toggle and the CDP shared-context toggle.
649
+ * An absent stored value formats as '' — callers render the schema default
650
+ * (denoise on, context shared).
651
+ */
645
652
  function CheckboxField(props) {
646
653
  return /* @__PURE__ */ (0, react_jsx_runtime.jsxs)("div", {
647
- className: fields_module_css_default.field,
654
+ className: props.embedded ? fields_module_css_default.fieldEmbedded : fields_module_css_default.field,
648
655
  children: [/* @__PURE__ */ (0, react_jsx_runtime.jsxs)("div", {
649
656
  className: fields_module_css_default.checkboxRow,
650
657
  children: [
@@ -677,7 +684,7 @@ window.__ModuleLoader__.load({
677
684
  }) : null
678
685
  ]
679
686
  }), /* @__PURE__ */ (0, react_jsx_runtime.jsx)("p", {
680
- className: fields_module_css_default.hint,
687
+ className: props.embedded ? `${fields_module_css_default.hint} ${fields_module_css_default.hintIndented}` : fields_module_css_default.hint,
681
688
  children: props.hint
682
689
  })]
683
690
  });
@@ -739,7 +746,7 @@ window.__ModuleLoader__.load({
739
746
  value: "cdp",
740
747
  label: t("backendCdp"),
741
748
  hint: t("backendCdpHint"),
742
- content: /* @__PURE__ */ (0, react_jsx_runtime.jsx)(ValueField, {
749
+ content: /* @__PURE__ */ (0, react_jsx_runtime.jsxs)(react_jsx_runtime.Fragment, { children: [/* @__PURE__ */ (0, react_jsx_runtime.jsx)(ValueField, {
743
750
  embedded: true,
744
751
  id: "plugin-config-playwright-cdp",
745
752
  label: t("cdpEndpoint"),
@@ -756,7 +763,23 @@ window.__ModuleLoader__.load({
756
763
  onReset: () => {
757
764
  props.resetField("cdpEndpoint");
758
765
  }
759
- })
766
+ }), /* @__PURE__ */ (0, react_jsx_runtime.jsx)(CheckboxField, {
767
+ embedded: true,
768
+ id: "plugin-config-playwright-share-context",
769
+ label: t("shareBrowserContext"),
770
+ hint: t("shareBrowserContextHint"),
771
+ checked: state.shareBrowserContext.text !== "false",
772
+ overridden: state.shareBrowserContext.overridden,
773
+ overriddenLabel: t("overridden"),
774
+ resetLabel: t("reset"),
775
+ disabled: disabled || backend !== "cdp",
776
+ onEdit: (text) => {
777
+ props.edit("shareBrowserContext", text);
778
+ },
779
+ onReset: () => {
780
+ props.resetField("shareBrowserContext");
781
+ }
782
+ })] })
760
783
  }],
761
784
  text: state.backend.text,
762
785
  overridden: state.backend.overridden,
@@ -822,6 +845,8 @@ window.__ModuleLoader__.load({
822
845
  playwrightPathPlaceholder: "(auto: playwright from $PATH)",
823
846
  cdpEndpoint: "CDP endpoint",
824
847
  cdpEndpointHint: "host:port or http(s)/ws URL. Leave blank for 127.0.0.1:9222.",
848
+ shareBrowserContext: "Share the browser context (profile logins)",
849
+ shareBrowserContextHint: "Each fetch opens a tab in the remote browser’s real profile — cookies and localStorage are shared and its persistent logins apply. Unchecked: every fetch uses a fresh isolated context.",
825
850
  denoise: "Enable the denoise algorithm",
826
851
  denoiseHint: "Readability + DOMPurify strip nav bars, sidebars, footers, and ads before converting to markdown.",
827
852
  maxConcurrency: "Max concurrent fetches",
@@ -853,6 +878,8 @@ window.__ModuleLoader__.load({
853
878
  playwrightPathPlaceholder: "(自动:按 $PATH 查找 playwright)",
854
879
  cdpEndpoint: "CDP 地址",
855
880
  cdpEndpointHint: "host:port 或 http(s)/ws 地址;留空默认 127.0.0.1:9222。",
881
+ shareBrowserContext: "共享浏览器上下文(复用登录态)",
882
+ shareBrowserContextHint: "每次抓取在远端浏览器的真实 profile 里开一个标签页:共享 cookie 与 localStorage、复用已登录会话,页面会看到你的登录身份;取消勾选则每次抓取使用全新隔离上下文。",
856
883
  denoise: "启用降噪算法",
857
884
  denoiseHint: "使用 Readability + DOMPurify 清洗导航栏、侧边栏、页脚与贴片广告后再转为 Markdown。",
858
885
  maxConcurrency: "最大并发抓取数",
package/lib/index.js CHANGED
@@ -25,6 +25,7 @@ const Config = z.object({
25
25
  backend: z.union([z.const("local"), z.const("cdp")]).default("local"),
26
26
  playwrightPath: z.string().default(""),
27
27
  cdpEndpoint: z.string().default(""),
28
+ shareBrowserContext: z.boolean().default(true),
28
29
  denoise: z.boolean().default(true),
29
30
  maxConcurrency: z.number().step(1).min(1).max(200)
30
31
  });
@@ -41,6 +42,20 @@ function effectiveMaxConcurrency(config) {
41
42
  return config.backend === "cdp" ? 50 : 4;
42
43
  }
43
44
  /**
45
+ * The context mode a fetch actually runs with: profile (a tab in the remote
46
+ * browser's real profile) requires the CDP backend — a local launch has no
47
+ * meaningful shared profile — and the checkbox (which defaults on, so an
48
+ * absent value reads as `true`, mirroring the schema default); everything
49
+ * else collapses to isolated.
50
+ *
51
+ * @param config - the resolved settings section (or any partial of it).
52
+ * @returns the mode the pool acquires its lease with.
53
+ */
54
+ function effectiveContextMode(config) {
55
+ if (config.backend === "cdp" && config.shareBrowserContext !== false) return "profile";
56
+ return "isolated";
57
+ }
58
+ /**
44
59
  * Normalize a configured CDP endpoint for `chromium.connectOverCDP`:
45
60
  * blank becomes the default loopback endpoint; `wss?://` and `https?://`
46
61
  * pass through; a bare `host:port` gains the `http://` scheme Playwright's
@@ -59,7 +74,7 @@ function normalizeCdpEndpoint(input) {
59
74
  }
60
75
  //#endregion
61
76
  //#region src/cdp-pool.ts
62
- /** A reusable `connectOverCDP` session handing out per-fetch contexts. */
77
+ /** A reusable `connectOverCDP` session handing out per-fetch pages. */
63
78
  var CdpConnectionPool = class {
64
79
  connect;
65
80
  browser;
@@ -76,40 +91,70 @@ var CdpConnectionPool = class {
76
91
  this.connect = connect;
77
92
  }
78
93
  /**
79
- * Lease a fresh isolated context on the shared connection, connecting (or
94
+ * Lease a fetch's page on the shared connection, connecting (or
80
95
  * reconnecting) first if needed. Concurrent first fetches share one
81
96
  * connect attempt.
82
97
  *
83
98
  * @param endpoint - normalized CDP endpoint URL.
84
99
  * @param timeoutMs - connect timeout budget.
85
- * @returns the shared browser and a context the caller must release.
100
+ * @param mode - `isolated` (default): a fresh throwaway context plus a page
101
+ * in it; `profile`: a page in the remote browser's default context — the
102
+ * real profile — which is never closed, only the page is.
103
+ * @returns the shared browser, the page's context, and the page the caller
104
+ * must release.
86
105
  * @throws whatever the connect function throws when no connection can be
87
- * established (the provider wraps it in a structured WebError).
106
+ * established (the provider wraps it in a structured WebError), or a
107
+ * diagnostic error when `profile` mode finds no default context.
88
108
  */
89
- async acquire(endpoint, timeoutMs) {
109
+ async acquire(endpoint, timeoutMs, mode = "isolated") {
90
110
  const browser = await this.ensure(endpoint, timeoutMs);
91
111
  try {
92
- return {
93
- browser,
94
- context: await browser.newContext()
95
- };
112
+ return await this.openLease(browser, mode);
96
113
  } catch (error) {
97
114
  if (this.isLive(browser)) throw error;
98
115
  this.drop(browser);
99
116
  const fresh = await this.ensure(endpoint, timeoutMs);
117
+ return await this.openLease(fresh, mode);
118
+ }
119
+ }
120
+ /**
121
+ * Open one lease's page on a live connection. The profile-mode handle is
122
+ * taken fresh every call (never cached across connections): a reconnect
123
+ * yields a new Browser object whose `contexts()[0]` must be re-read.
124
+ */
125
+ async openLease(browser, mode) {
126
+ if (mode === "profile") {
127
+ const context = browser.contexts?.()[0];
128
+ if (context === void 0) throw new Error("the CDP endpoint exposed no default browser context (profile mode requires a real browser profile)");
100
129
  return {
101
- browser: fresh,
102
- context: await fresh.newContext()
130
+ browser,
131
+ context,
132
+ page: await context.newPage(),
133
+ persistent: true
103
134
  };
104
135
  }
136
+ const context = await browser.newContext();
137
+ try {
138
+ return {
139
+ browser,
140
+ context,
141
+ page: await context.newPage(),
142
+ persistent: false
143
+ };
144
+ } catch (error) {
145
+ await context.close().catch(() => {});
146
+ throw error;
147
+ }
105
148
  }
106
149
  /**
107
- * Close a fetch-owned context. The shared connection stays open for the
108
- * next fetch.
109
- * @param lease - the lease whose context goes away.
150
+ * Close a fetch-owned page, plus its context when the lease owns one. The
151
+ * remote default context (persistent leases) and the shared connection
152
+ * stay for the next fetch.
153
+ * @param lease - the lease whose page (and, when isolated, context) goes away.
110
154
  */
111
155
  async release(lease) {
112
- await lease.context.close().catch(() => {});
156
+ await lease.page.close().catch(() => {});
157
+ if (!lease.persistent) await lease.context.close().catch(() => {});
113
158
  }
114
159
  /**
115
160
  * Drop the shared connection (plugin teardown or endpoint change). Fetches
@@ -512,18 +557,21 @@ function existsAsFile(path) {
512
557
  *
513
558
  * Lifecycle: the local backend launches a browser per fetch and closes it —
514
559
  * nothing outlives the call. The CDP backend keeps ONE shared connection to
515
- * the remote browser for the provider's lifetime; each fetch only opens an
516
- * isolated context (a tab) inside it and closes that on completion, so the
517
- * concurrency cap counts tabs, not browsers. Either way, an aborted signal
518
- * closes the fetch's context. Concurrency is capped at the `maxConcurrency`
519
- * setting (explicit, or the backend default — {@link DEFAULT_MAX_CONCURRENCY_LOCAL}
520
- * browsers / {@link DEFAULT_MAX_CONCURRENCY_CDP} tabs); further fetches wait
521
- * briefly in a queue and fail fast (rather than hang until abort) when no
522
- * slot frees.
560
+ * the remote browser for the provider's lifetime; each fetch only opens a
561
+ * page (tab) inside it and closes that on completion — in a throwaway
562
+ * isolated context, or (the default, `shareBrowserContext`) in the remote
563
+ * browser's default context so its profile, cookies, and persistent logins
564
+ * apply; that default context is never closed. The concurrency cap counts
565
+ * tabs, not browsers. Either way, an aborted signal closes the fetch's page.
566
+ * Concurrency is capped at the `maxConcurrency` setting (explicit, or the
567
+ * backend default — {@link DEFAULT_MAX_CONCURRENCY_LOCAL} browsers /
568
+ * {@link DEFAULT_MAX_CONCURRENCY_CDP} tabs); further fetches wait briefly in
569
+ * a queue and fail fast (rather than hang until abort) when no slot frees.
523
570
  *
524
571
  * Private-network and SSRF protection is not implemented (same stance as the
525
572
  * shipped HTTP provider); a page this provider can reach is whatever the
526
- * browser can reach.
573
+ * browser can reach. Profile mode additionally acts WITH the remote
574
+ * browser's logged-in sessions (see the README's risk notes).
527
575
  *
528
576
  * @module dsh-web-fetch-playwright/provider
529
577
  */
@@ -752,7 +800,7 @@ var PlaywrightFetchProvider = class {
752
800
  }
753
801
  }
754
802
  /**
755
- * Open the configured backend and its per-fetch context. Split out so the
803
+ * Open the configured backend and its per-fetch page. Split out so the
756
804
  * test suite can substitute a fake browser.
757
805
  * @param config - the resolved settings section.
758
806
  * @param deadline - the fetch budget, applied to launch/connect timeouts.
@@ -764,37 +812,45 @@ var PlaywrightFetchProvider = class {
764
812
  const endpoint = normalizeCdpEndpoint(config.cdpEndpoint);
765
813
  const { source } = await resolveCdpBackend();
766
814
  try {
767
- const lease = await this.cdpPool.acquire(endpoint, timeout);
768
- await installResourceFilter(lease.context);
815
+ const lease = await this.cdpPool.acquire(endpoint, timeout, effectiveContextMode(config));
816
+ await installResourceFilter(lease.page);
817
+ guardPopups(lease.page);
769
818
  return {
770
819
  browser: lease.browser,
771
820
  context: lease.context,
772
- sharedBrowser: true
821
+ page: lease.page,
822
+ sharedBrowser: true,
823
+ persistent: lease.persistent
773
824
  };
774
825
  } catch (error) {
775
826
  throw new WebError(`cannot connect to the CDP endpoint ${endpoint} (${source}); is the browser started with --remote-debugging-port? ${String(error instanceof Error ? error.message : error)}`, "WEB_PROVIDER_ERROR", { cause: error });
776
827
  }
777
828
  }
778
829
  const { chromium, executablePath, source } = await resolvePlaywrightBackend(config.playwrightPath);
830
+ let browser;
779
831
  try {
780
- const browser = await chromium.launch({
832
+ browser = await chromium.launch({
781
833
  headless: true,
782
834
  ...executablePath !== void 0 ? { executablePath } : {},
783
835
  timeout
784
836
  });
785
837
  const context = await browser.newContext();
786
- await installResourceFilter(context);
838
+ const page = await context.newPage();
839
+ await installResourceFilter(page);
840
+ guardPopups(page);
787
841
  return {
788
842
  browser,
789
- context
843
+ context,
844
+ page
790
845
  };
791
846
  } catch (error) {
847
+ await browser?.close().catch(() => {});
792
848
  throw new WebError(`cannot launch the local browser (${source}); run \`playwright install chromium\` or point the settings path at a playwright/browser executable. ${String(error instanceof Error ? error.message : error)}`, "WEB_PROVIDER_ERROR", { cause: error });
793
849
  }
794
850
  }
795
- /** Navigate, settle, and decode one URL inside an open session. */
851
+ /** Navigate, settle, and decode one URL inside an open session's tab. */
796
852
  async retrieve(session, url, config, deadline) {
797
- const page = await session.context.newPage();
853
+ const page = session.page;
798
854
  const response = await page.goto(url.toString(), {
799
855
  waitUntil: "domcontentloaded",
800
856
  timeout: deadline.remainingMs()
@@ -841,14 +897,17 @@ function capResult(url, statusCode, body) {
841
897
  };
842
898
  }
843
899
  /**
844
- * Close a session's fetch-owned context, plus its browser when the session
845
- * launched one (local backend). CDP sessions ride the shared connection,
846
- * which stays open. Every close is grace-bounded so cleanup always completes
847
- * and the concurrency slot is released even when Playwright hangs.
900
+ * Close a session's fetch-owned tab, then its context unless the context is
901
+ * the remote default context (profile mode — closing it would tear down the
902
+ * whole shared connection), then its browser when the session launched one
903
+ * (local backend). CDP sessions ride the shared connection, which stays
904
+ * open. Every close is grace-bounded so cleanup always completes and the
905
+ * concurrency slot is released even when Playwright hangs.
848
906
  */
849
907
  async function closeSession(session) {
850
908
  if (session === void 0) return;
851
- await closeWithGrace(session.context);
909
+ await closeWithGrace(session.page);
910
+ if (session.persistent !== true) await closeWithGrace(session.context);
852
911
  if (session.sharedBrowser !== true) await closeWithGrace(session.browser);
853
912
  }
854
913
  /** One best-effort `close()` that resolves within the grace period. */
@@ -876,17 +935,32 @@ async function defaultCdpConnect(endpoint, timeoutMs) {
876
935
  /**
877
936
  * Abort image/font/media subrequests: the markdown output keeps their URLs
878
937
  * but never renders them, so downloading them only spends the budget.
879
- * Best-effort — a context that refuses interception still fetches.
938
+ * Installed on the PAGE (never the context): in profile mode a context-level
939
+ * route would intercept the remote browser's other tabs too. Best-effort —
940
+ * a page that refuses interception still fetches.
880
941
  */
881
- async function installResourceFilter(context) {
942
+ async function installResourceFilter(owner) {
882
943
  try {
883
- await context.route("**/*", async (route) => {
944
+ await owner.route("**/*", async (route) => {
884
945
  const type = route.request().resourceType();
885
946
  if (type === "image" || type === "font" || type === "media") await route.abort();
886
947
  else await route.continue();
887
948
  });
888
949
  } catch {}
889
950
  }
951
+ /**
952
+ * Close any popup a fetched page spawns so nothing outlives the fetch's tab
953
+ * — `page.close()` does not auto-close windows the page opened, and in
954
+ * profile mode a stray tab would stay in the user's remote browser.
955
+ * Best-effort: a page that refuses listeners just loses the guard.
956
+ */
957
+ function guardPopups(page) {
958
+ try {
959
+ page.on?.("popup", (popup) => {
960
+ popup.close().catch(() => {});
961
+ });
962
+ } catch {}
963
+ }
890
964
  //#endregion
891
965
  //#region src/index.ts
892
966
  /** Cordis plugin name used by loader diagnostics. */
@@ -911,4 +985,4 @@ function apply(ctx, config) {
911
985
  ctx.web.registerFetchProvider(provider);
912
986
  }
913
987
  //#endregion
914
- export { CdpConnectionPool, Config, DEFAULT_CDP_ENDPOINT, PLAYWRIGHT_FETCH_PROVIDER_ID, PlaywrightFetchProvider, WEB_FETCH_PLAYWRIGHT_SETTINGS_NAMESPACE, apply, htmlToMarkdown, inject, name, normalizeCdpEndpoint };
988
+ export { CdpConnectionPool, Config, DEFAULT_CDP_ENDPOINT, PLAYWRIGHT_FETCH_PROVIDER_ID, PlaywrightFetchProvider, WEB_FETCH_PLAYWRIGHT_SETTINGS_NAMESPACE, apply, effectiveContextMode, htmlToMarkdown, inject, name, normalizeCdpEndpoint };