dsh-read-url 0.4.7 → 0.4.9

package/CHANGELOG.md

@@ -1,5 +1,44 @@
 # Changelog
 
+## [0.4.9] - 2026-08-20
+
+### 稳定性：配置健壮性 + 严格宿主兼容 + 资源泄漏
+
+**修复（真 bug）**
+- `cordis.patch.yml` 数字加引号（`timeoutMs: "15000"`）时按字符串拼接进超时预算（`"1500020000"`）→ 加载时统一强转数字并钳制到合理范围（timeoutMs 500–120000 等），非法值回退默认
+- 严格 cordis 宿主上 `ctx.get('web')` 对未注入服务直接抛错（同 `settings` 踩过的坑）→ 移入 try，降级为全局 fetch
+- 超时预算公式低估最坏链路：SPA 渲染实际是 goto 30s 上限 + DOM 轮询 10s = 40s，原公式只留 20s → 修正为 `timeoutMs + 45s`，且随配置缩放（60s 配置得 105s 预算，不再钳制）
+- `read_url_site` 预算 120s 覆盖不了 50 页 × 长超时的爬取波次 → 改为随配置缩放（`timeoutMs × 25` 波）
+- `crawlSite` 在协作超时触发后不退出循环（对已中止的 fetch 空转清空队列）→ while 增加 signal aborted 检查
+- 无 body 响应（204/304）导致 `res.body` 迭代抛 TypeError → 空响应防护返回空 buffer
+- offset 续读到文末时误报"登录墙 / SPA 页" → 识别文末，提示"offset 已到文末"
+- playwright 启动失败被永久缓存为 rejected promise（装好后也不重试）→ 失败自愈重置；`closeBrowser` 对未完成的 launch 健壮化
+
+**测试**：39 → **42 断言**（+最坏链路预算随配置缩放、yml 字符串强转+钳制、严格宿主 seam 降级）+ test-spa 10 全绿
+
+**明确不做**：Web UI 设置卡片——宿主侧注册成功但官方 UI 只渲染自带浏览器半边的插件（"A served namespace no card claims renders nothing"），为一个 6 字段低频配置维护 React 构建链不划算；配置走 `cordis.patch.yml` 已足够
+
+## [0.4.8] - 2026-08-19
+
+### 41 站多类型实测 + 提取修复
+
+**新增 12 类站点测试**（29 → 41）：问答（知乎专栏/百度知道）、论坛（虎扑/贴吧）、百科（百度百科）、小说（起点）、政府（gov.cn）、教育（北邮）、视频（B 站视频页）、技术媒体（36 氪/中关村在线 GBK）、技术文档（Vue 文档）
+- 结果：**23 OK / 8 预期边界 / 10 网络·反爬边界 / 0 崩溃**；新增站 gov.cn/知乎专栏/虎扑/百度知道/B 站视频页/中关村在线(GBK) 全部干净提取
+
+**修复（真 bug）**
+- `pickMain` 只匹配 `role="main"` 属性形式，**裸 `<main>` 标签漏选**——VitePress 系文档站（vuejs.org）退化为整页 body，顶部导航（VPLocalNav "菜单/本页目录/赞助位"）混入正文 → 新增裸 `<main>` 匹配分支，文档站正文聚焦、噪音清除（实测 vuejs.org 修复）
+
+**修复（防御性）**
+- `readWindowsSystemProxy` 失败时**永久缓存空代理**——若 PowerShell 子进程被 EDR/策略临时拦截，整个会话都失去代理 → 失败缓存 60s 后允许重新探测
+
+**修复（测试脚本）**
+- `noiseCheck` 的 css 对象正则误报 JS 代码块（`{ data ()`）→ 收紧为 CSS 属性名白名单
+
+**验证**：38 → **39 断言**（+裸 main 优先于 body）+ test-spa 10 全绿；41 站 0 THREW
+
+### README 加真实案例
+- 中英「真实案例」章节：省 token（新浪 800/12398 截断）、竞速（BBC 633ms）、offset 续读（800+800 cached）、批量失败隔离——全部实测数据
+
 ## [0.4.7] - 2026-08-19
 
 ### 省 token：参数 schema 描述精简（固定开销 -9%）

package/README.md

@@ -4,9 +4,12 @@
 
 ![dsh-read-url](docs/banner.svg)
 
+[![npm](https://img.shields.io/npm/v/dsh-read-url)](https://www.npmjs.com/package/dsh-read-url)
+[![License](https://img.shields.io/github/license/2672243194/dsh-read-url)](https://github.com/2672243194/dsh-read-url/blob/main/LICENSE)
+
 URL reader plugin for DeepSeek Harness: fetch any webpage, **auto-detect encoding (GBK/GB2312/UTF-8/Big5)**, extract the clean main content, and return **token-efficient compact text or structured Markdown**.
 
-Zero dependencies (Node 20+ built-ins), no API key, no server side — install and use.
+Zero runtime dependencies (Node 20+ built-ins handle fetch/decode/extract), no API key, no server side — install and use.
 
 ## Why
 
@@ -59,6 +62,45 @@ Read https://example.com/article and summarize the key points
 Read https://docs.example.org/guide in markdown mode
 ```
 
+### Real examples (measured)
+
+**1. Token economy — return only what the model needs**
+
+`read_url` on a portal returns cleaned body capped at `maxChars` (default 6,000) — not the raw page with nav/ads/footers. Repeat reads hit the 5-min cache (`(cached)`), so the agent never re-fetches:
+
+```
+title: 新闻中心首页_新浪网
+charset utf-8
+(chars 800/12398 — 截断，offset 续读)
+```
+
+**2. Overseas sites — direct + proxy race**
+
+When a proxy is configured, the direct fetch and the proxy `curl` start together and the first to complete wins. A blocked overseas site that used to cost ~11s (direct-connect timeout + fallback) now reads in under a second:
+
+```
+BBC 中文: OK (633ms) — clean 4,000+ chars of headline news
+```
+
+**3. Long-article continuation (`offset`)**
+
+A 12,000-char article is read in slices; `offset` resumes from cache without repeating earlier text — the model keeps exactly what it needs in context:
+
+```
+chars 800+800/12398 · cached
+```
+
+**4. Batch research across pages**
+
+`read_url_batch` reads up to 10 pages in parallel (concurrency 4), each cleaned individually, failures isolated:
+
+```
+读取 2/4 页成功，2 页失败
+--- 阮一峰的网络日志 (491 字符) ---
+--- Example Domain (127 字符) ---
+[失败] https://zh.wikipedia.org/... — Fetch failed: HTTP 403 ...
+```
+
 ### Tools
 
 **`read_url(url, maxChars?, offset?, mode?, includeLinks?)`** — fetch and extract clean body
@@ -107,19 +149,23 @@ Read https://docs.example.org/guide in markdown mode
 
 ### Configuration (optional)
 
-Plugin-level config is overridable via the profile's `cordis.patch.yml` (defaults in the plugin's own `cordis.patch.yml`):
+Override via the profile's `cordis.patch.yml` (defaults in the plugin's own `cordis.patch.yml`):
 
 ```yaml
 - id: dsh-read-url
   config:
-    timeoutMs: 15000      # per-request timeout
+    timeoutMs: 15000      # per-request timeout (500-120000, clamped)
     maxBytes: 3145728     # response body cap (bytes)
     maxChars: 6000        # default body truncation
     maxLinks: 20          # read_url_links default count
     spaRender: true       # SPA rendering enhancement (needs playwright installed; degrades with a hint otherwise)
     userAgent: '...'      # request UA
+    cacheTtlMs: 300000    # success-cache TTL
+    cacheMax: 32          # cache entry cap
 ```
 
+Values are coerced and clamped to sane ranges at load — quoted numbers in YAML work, garbage falls back to defaults.
+
 ### Output (compact)
 
 ```json
@@ -172,9 +218,9 @@ const results = await Promise.all([
 - **Optional enhancement 2 (SPA page rendering)**: run `npm i playwright && npx playwright install chromium` in the DSH profile directory to auto-enable it. When the extracted body is empty and the page is script-heavy (likely Vue/React client-rendered), the plugin automatically renders it with headless Chromium before extracting (a `rendered` flag tells the model); rendering waits for the DOM to stabilize (content stops growing) instead of `networkidle` — heartbeat-polling sites never idle, so this avoids 30s timeouts; when not installed it degrades with a clear install hint, never errors — the core stays zero-dependency;
 - **Boundaries**: login-walled pages are not readable; SPA pages need the Playwright enhancement; **structured data (e.g. which like-count belongs to which comment) is out of text-extraction scope** — this plugin flattens HTML into readable text, so exact field↔value associations are lost; for precise fields, intercept the page's actual data API (see "Real-world validation" below).
 
-## Real-world validation (2026-08-19, v0.4.6)
+## Real-world validation (2026-08-19, v0.4.8)
 
-29-site sweep driven by `multi-site.mjs` (committed, re-runnable): **18 OK / 4 expected boundaries / 7 network boundaries / 0 crashes** — overseas sites (BBC/V2EX) are served through the user's proxy in ~1s via the direct+proxy race; wikipedia/httpbin remain clear attributed network boundaries.
+41-site sweep driven by `multi-site.mjs` (committed, re-runnable): **23 OK / 8 expected boundaries / 10 network·anti-bot boundaries / 0 crashes** — overseas sites (BBC/V2EX) served in ~1s via the direct+proxy race; doc sites (vuejs.org) fixed via bare-`<main>` picking; wikipedia/httpbin remain clear attributed network boundaries.
 
 | Category | Sites | Result |
 |---|---|---|
@@ -189,7 +235,7 @@ const results = await Promise.all([
 | Batch + failure isolation | 4-URL mix | ✅ 2/4 ok, failures isolated |
 | Site crawl | Ruan Yifeng blog | ✅ 5/5 pages tree map |
 
-- **38 zero-dep assertions** (incl. entity decoding, description-budget guard, link dedupe, table-separator escaping, proxy-fallback function, missing-args tolerance, race logic, empty-race guard, schema budget) + **10 SPA-test assertions** all green;
+- **42 zero-dep assertions** (incl. entity decoding, description-budget guard, link dedupe, table-separator escaping, proxy-fallback function, missing-args tolerance, race logic, empty-race guard, schema budget, bare-main pick, worst-case timeout budgets, yml string coercion + clamping, strict-host seam degradation) + **10 SPA-test assertions** all green;
 - Real case: on a Xiaoheihe post, comment like-counts (`up` field) could not be attributed from flattened text — **precise fields should come from the page's underlying data API** (e.g. `/bbs/app/link/tree` JSON). This is a shared boundary of text extractors, not a defect.
 
 ## Roadmap

package/README.zh.md

@@ -4,9 +4,12 @@
 
 ![dsh-read-url](docs/banner.svg)
 
+[![npm](https://img.shields.io/npm/v/dsh-read-url)](https://www.npmjs.com/package/dsh-read-url)
+[![License](https://img.shields.io/github/license/2672243194/dsh-read-url)](https://github.com/2672243194/dsh-read-url/blob/main/LICENSE)
+
 DeepSeek Harness 的 URL 阅读插件：抓取任意网页，**自动识别编码（GBK/GB2312/UTF-8/Big5）**，提取干净正文，输出**省 token 的紧凑文本或结构化 Markdown**。
 
-零依赖（Node 20+ 内置能力），免 API key，免服务端，装完即用。
+核心零运行时依赖（Node 20+ 内置能力完成抓取/转码/提取），免 API key，免服务端，装完即用。
 
 ## 为什么做它
 
@@ -24,7 +27,7 @@ DSH 的 Agent 能搜索（返回链接和片段），但缺"把 URL 读成干净
 | `ctx.effect` 卸载清理 | ✅ | ❌ | ❌ | ✅ |
 | 协作式超时（不暴露给模型） | ✅ | ⚠️ 自管 | ⚠️ 自管 | ✅ `timeoutMs` + `exec.signal` |
 | 模型视角输出 | 整页 Markdown | 紧凑文本 | 15 字段 JSON | **紧凑文本（无需解析 JSON）** |
-| 依赖 | 官方 | TS 需构建 | 零依赖 | 零依赖（JS ESM 即装即用） |
+| 依赖 | 官方 | TS 需构建 | 零依赖 | 核心零运行时依赖（JS ESM 即装即用） |
 | 反爬/降级响应（UA 与 TLS 指纹） | ⚠️ Node 默认 UA，实测 https 被中间设备按 TLS 指纹拦截、百度返回无热搜的降级版 | ❓ 未披露 | ❓ 未披露 | ✅ 完整浏览器 UA，实测获取完整版页面（百度热搜正常） |
 
 > 2026-08-16 实测（本机环境）：停用本插件后用官方 `web_fetch` 读 `https://www.baidu.com`——TLS 握手被中间设备按程序指纹拦截（退回 http 才成功），且百度对 Node UA 返回**服务端降级版**（热搜词条改由 JS 异步加载，静态 HTML 不含）；换回 `dsh-read-url` 后 https 正常、热搜完整可读。差异根因：请求的 UA 与 TLS 特征决定网站/中间设备是否按 bot 处理。
@@ -60,6 +63,45 @@ npx @deepseek-ai/dsh plugin --profile web add ./dsh-read-url
 同时读一下这几个网址，对比它们的观点：<url1> <url2> <url3>
 ```
 
+### 真实案例（实测数据）
+
+**1. 省 token——只返回模型真正需要的内容**
+
+门户页 `read_url` 返回按 `maxChars`（默认 6000）截断的干净正文——不是带导航/广告/页脚的原始页面。重复读取命中 5 分钟会话缓存（`(cached)`），模型不会重复抓取：
+
+```
+title: 新闻中心首页_新浪网
+charset utf-8
+(chars 800/12398 — 截断，offset 续读)
+```
+
+**2. 海外站——直连 + 代理并发竞速**
+
+检测到代理时，直连与代理 `curl` 同时发起、先完成者胜。原本被墙的海外站（直连超时 + 回退约 11s）现在 1 秒内读到：
+
+```
+BBC 中文: OK (633ms) — 4000+ 字符干净头条新闻
+```
+
+**3. 长文续读（`offset`）**
+
+1.2 万字符的文章分片读取，`offset` 从缓存续读且不重复前文——模型上下文只保留需要的部分：
+
+```
+chars 800+800/12398 · cached
+```
+
+**4. 跨页批量研究**
+
+`read_url_batch` 并行读最多 10 页（并发 4），逐页净化、失败隔离：
+
+```
+读取 2/4 页成功，2 页失败
+--- 阮一峰的网络日志 (491 字符) ---
+--- Example Domain (127 字符) ---
+[失败] https://zh.wikipedia.org/... — Fetch failed: HTTP 403 ...
+```
+
 ### 工具
 
 **`read_url(url, maxChars?, offset?, mode?, includeLinks?)`** — 抓取并提取干净正文
@@ -108,19 +150,23 @@ npx @deepseek-ai/dsh plugin --profile web add ./dsh-read-url
 
 ### 配置（可选）
 
-插件级配置通过 profile 的 `cordis.patch.yml` 覆盖（默认值见插件自带 `cordis.patch.yml`）：
+通过 profile 的 `cordis.patch.yml` 覆盖（默认值见插件自带 `cordis.patch.yml`）：
 
 ```yaml
 - id: dsh-read-url
   config:
-    timeoutMs: 15000      # 单请求超时
+    timeoutMs: 15000      # 单请求超时（500-120000，自动钳制）
     maxBytes: 3145728     # 响应体上限（字节）
     maxChars: 6000        # 默认正文截断
     maxLinks: 20          # read_url_links 默认条数
     spaRender: true       # SPA 渲染增强（需 playwright 已安装，未装自动降级提示）
     userAgent: '...'      # 请求 UA
+    cacheTtlMs: 300000    # 成功缓存 TTL
+    cacheMax: 32          # 缓存条目上限
 ```
 
+配置在加载时统一强转数字并钳制到合理范围——YAML 里加引号的数字也能用，非法值回退默认。
+
 ### 输出结构（紧凑）
 
 ```json
@@ -173,9 +219,9 @@ const results = await Promise.all([
 - **可选增强二（SPA 页面渲染）**：在 DSH profile 目录执行 `npm i playwright && npx playwright install chromium` 后自动启用。检测到正文为空且页面脚本密集（疑似 Vue/React 客户端渲染）时，自动用无头 Chromium 渲染后再提取（`rendered` 标记告知模型）；渲染采用 `domcontentloaded` + **DOM 稳定轮询**（内容停止增长即收，上限 10s）而非 `networkidle`——心跳轮询站永不空闲，避免 30s 超时；未安装时优雅提示安装方法、不报错——核心保持零依赖；
 - **边界**：登录墙页面无法读取；SPA 页面需安装 Playwright 增强后渲染读取（未安装时返回明确提示）；**结构化数据（如评论的点赞数归属、榜单数值）不在文本提取范围**——本插件把 HTML 扁平化为可读文本，字段与数值的精确对应关系会丢失；需要精确字段时，用 Playwright 拦截页面实际调用的数据 API 获取（见下方「真实世界验证」）。
 
-## 真实世界验证（2026-08-19，v0.4.6）
+## 真实世界验证（2026-08-19，v0.4.8）
 
-29 站全量实测（`multi-site.mjs` 已提交可复跑）：**18 OK / 4 预期边界 / 7 网络·反爬边界 / 0 崩溃**——海外站（BBC/V2EX）经竞速通道 ~1s 读回；wikipedia/httpbin 保持清晰的网络边界归因。
+41 站全量实测（`multi-site.mjs` 已提交可复跑）：**23 OK / 8 预期边界 / 10 网络·反爬边界 / 0 崩溃**——海外站（BBC/V2EX）经竞速通道 ~1s 读回；文档站（vuejs.org）经裸 `<main>` 修复后正文聚焦；wikipedia/httpbin 保持清晰的网络边界归因。
 
 | 类别 | 站点 | 结果 |
 |---|---|---|
@@ -190,7 +236,7 @@ const results = await Promise.all([
 | **批量 + 失败隔离** | 4 URL 混合 | ✅ 2/4 成功、失败隔离 |
 | **整站爬取** | 阮一峰博客 | ✅ 5/5 页树状站点地图 |
 
-- **38 个零依赖断言**（含实体解码、description 预算守卫、链接去重、表格分隔行转义、代理回退函数、空参容错、竞速逻辑、空竞速守卫、schema 预算）+ **10 个 SPA 测试断言**全绿；
+- **42 个单元断言**（含实体解码、description/schema 预算守卫、链接去重、表格分隔行转义、代理回退函数、空参容错、竞速逻辑、空竞速守卫、裸 main 提取、最坏链路超时预算、yml 字符串强转+钳制、严格宿主 seam 降级）+ **10 个 SPA 测试断言**全绿；
 - 一个真实案例：小黑盒帖子的评论点赞数（`up` 字段）无法从扁平文本确定归属——**精确字段应走页面背后的数据 API**（如 `/bbs/app/link/tree` JSON），这是同类文本提取器的共同边界，不是缺陷。
 
 ## Roadmap
@@ -203,7 +249,7 @@ const results = await Promise.all([
 ## 开发
 
 ```bash
-node test.mjs          # 零依赖自测（转码/提取/Markdown/截断/批量/站点爬取/缓存隔离）
+node test.mjs          # 单元自测（转码/提取/Markdown/截断/批量/站点爬取/缓存隔离/配置钳制）
 
 # SPA 渲染真实测试（需 playwright 已安装，未装自动 SKIP）
 node test-spa.mjs      # 10 断言：JS 正文/渲染后链接/工具不崩溃/缓存隔离

package/index.js

@@ -32,6 +32,8 @@ const DEFAULTS = {
   userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36',
 }
 
+// Session-level cache: url -> { time, full } on success, { time, error } on
+// failure (short TTL so a broken URL is not re-fetched in a loop).
 const cache = new Map()
 const decoders = new Map()
 
@@ -85,16 +87,21 @@ async function directFetch(url, signal, cfg) {
       signal,
       headers: { 'user-agent': cfg.userAgent, accept: 'text/html,application/xhtml+xml,*/*;q=0.8' },
     })
-    if (!res.ok) return { error: `HTTP ${res.status} ${res.statusText}` }
+    // Early-return paths must cancel the body stream, or the connection is
+    // held open undrained and cannot return to the keep-alive pool.
+    const discardBody = () => { if (res.body) res.body.cancel().catch(() => {}) }
+    if (!res.ok) { discardBody(); return { error: `HTTP ${res.status} ${res.statusText}` } }
     const contentType = res.headers.get('content-type') || ''
     if (contentType && !/text\/html|application\/xhtml|text\/plain/i.test(contentType)) {
+      discardBody()
       return { error: `Unsupported content-type: ${contentType.split(';')[0]}` }
     }
+    if (!res.body) return { buffer: Buffer.alloc(0), contentType, finalUrl: res.url }
     const chunks = []
     let size = 0
     for await (const chunk of res.body) {
       size += chunk.length
-      if (size > cfg.maxBytes) return { error: `Page exceeds ${cfg.maxBytes} bytes` }
+      if (size > cfg.maxBytes) { discardBody(); return { error: `Page exceeds ${cfg.maxBytes} bytes` } }
       chunks.push(chunk)
     }
     return { buffer: Buffer.concat(chunks), contentType, finalUrl: res.url }
@@ -178,9 +185,11 @@ async function fetchPage(url, externalSignal, cfg) {
 // Seam calls get the same cooperative timeout as fetchPage (a hanging
 // provider should not block the tool call forever).
 async function fetchViaWebSeam(ctx, url, externalSignal, cfg) {
-  const web = ctx && typeof ctx.get === 'function' ? ctx.get('web') : undefined
-  if (!web || typeof web.fetch !== 'function') return null
   try {
+    // ctx.get for a non-injected service throws on strict cordis hosts (seen
+    // with 'settings') — treat that as "seam absent" and fall through.
+    const web = ctx && typeof ctx.get === 'function' ? ctx.get('web') : undefined
+    if (!web || typeof web.fetch !== 'function') return null
     const timeoutSignal = AbortSignal.timeout(cfg.timeoutMs)
     const signal = externalSignal ? AbortSignal.any([externalSignal, timeoutSignal]) : timeoutSignal
     const res = await web.fetch(url, { signal })
@@ -241,7 +250,13 @@ export function pickMain(html) {
   // collect all of them instead of only the first.
   const articles = [...html.matchAll(/<article[\s>][\s\S]*?<\/article\s*>/gi)]
   if (articles.length) return articles.map((a) => a[0]).join('\n')
-  const mainTag = /<(main|div)[^>]*role=["']main["'][\s>][\s\S]*?<\/\1\s*>/i.exec(html)
+  const mainTag =
+    /<(main|div)[^>]*role=["']main["'][\s>][\s\S]*?<\/\1\s*>/i.exec(html) ||
+    // Bare <main> without role="main": doc generators (VitePress/VuePress,
+    // MDN, MDX sites) mark the body with a plain <main> tag. Falling back to
+    // the whole <body> would drag in the top/local nav (menu, outline, skip
+    // link) — the extraction quality loss we saw on vuejs.org.
+    /<main[\s>][\s\S]*?<\/main\s*>/i.exec(html)
   if (mainTag) return mainTag[0]
   const body = /<body[\s>]/i.exec(html)
   if (body) {
@@ -649,9 +664,13 @@ function renderResult(value) {
   if (r.cached) lines.push('(cached)')
   if (r.rendered) lines.push('(rendered — JS 执行后内容)')
   if (!r.text) {
-    lines.push(r.spaHint
-      ? `(无可读内容 — ${r.spaHint})`
-      : '(无可读内容 — 登录墙 / SPA 页 / 空页面；SPA 需安装 playwright)')
+    if (r.charsStart > 0 && !r.truncated) {
+      lines.push('(offset 已到文末，无更多内容)')
+    } else {
+      lines.push(r.spaHint
+        ? `(无可读内容 — ${r.spaHint})`
+        : '(无可读内容 — 登录墙 / SPA 页 / 空页面；SPA 需安装 playwright)')
+    }
   }
   lines.push('', r.text || '')
   if (Array.isArray(r.links) && r.links.length) {
@@ -686,7 +705,8 @@ function readLinksTool(ctx, cfg) {
       },
       required: ['url'],
     },
-    timeoutMs: 20000,
+    // Covers SPA-render fallback: full fetch timeout + playwright render cap.
+    timeoutMs: Math.max(45000, cfg.timeoutMs + 45000),
     output: {
       schema: {
         type: 'object',
@@ -785,7 +805,8 @@ function readUrlBatchTool(ctx, cfg) {
       },
       required: ['urls'],
     },
-    timeoutMs: Math.max(30000, cfg.timeoutMs + 15000),
+    // Worst case: ceil(10/4)=3 concurrency waves, each (fetch + SPA render).
+    timeoutMs: Math.max(60000, cfg.timeoutMs * 3 + 135000),
     output: {
       schema: {
         type: 'object',
@@ -871,7 +892,9 @@ async function crawlSite(entryUrl, cfg, opts, externalSignal) {
   const pages = []
   const failures = []
   const queue = [{ url: entryUrl, depth: 0 }]
-  while (queue.length && pages.length < maxPages) {
+  // Stop once the host's cooperative signal fires (tool budget hit): without
+  // this the loop keeps draining the queue on instantly-aborted fetches.
+  while (queue.length && pages.length < maxPages && !(externalSignal && externalSignal.aborted)) {
     // Never overshoot the page budget within a batch round.
     const batch = queue.splice(0, Math.min(2, maxPages - pages.length, queue.length))
     const results = await Promise.all(
@@ -951,7 +974,9 @@ function readUrlSiteTool(ctx, cfg) {
       },
       required: ['url'],
     },
-    timeoutMs: 120000,
+    // Worst case: ceil(maxPages/2)=25 waves of paired fetches (no SPA render
+    // here); the cooperative signal stops the loop early once the budget is hit.
+    timeoutMs: Math.max(120000, cfg.timeoutMs * 25),
     output: {
       schema: {
         type: 'object',
@@ -996,7 +1021,22 @@ function readUrlSiteTool(ctx, cfg) {
 
 export function apply(ctx, config) {
   // Plugin-level config from cordis.patch.yml (config row), merged over defaults.
+  // YAML is free-form: numbers can arrive quoted as strings, which would poison
+  // arithmetic and AbortSignal timeouts downstream — coerce to numbers once
+  // here, clamp to sane ranges, and fall back to defaults on garbage.
   const cfg = { ...DEFAULTS, ...(config || {}) }
+  const clamp = (v, d, min, max) => {
+    const n = Number(v)
+    return Number.isFinite(n) ? Math.min(max, Math.max(min, n)) : d
+  }
+  cfg.timeoutMs = clamp(cfg.timeoutMs, DEFAULTS.timeoutMs, 500, 120000)
+  cfg.maxBytes = clamp(cfg.maxBytes, DEFAULTS.maxBytes, 65536, 16 * 1024 * 1024)
+  cfg.maxChars = clamp(cfg.maxChars, DEFAULTS.maxChars, 500, 20000)
+  cfg.maxLinks = clamp(cfg.maxLinks, DEFAULTS.maxLinks, 1, 50)
+  cfg.cacheTtlMs = clamp(cfg.cacheTtlMs, DEFAULTS.cacheTtlMs, 0, 3600000)
+  cfg.cacheMax = clamp(cfg.cacheMax, DEFAULTS.cacheMax, 1, 256)
+  if (typeof cfg.userAgent !== 'string' || !cfg.userAgent.trim()) cfg.userAgent = DEFAULTS.userAgent
+  if (typeof cfg.spaRender !== 'boolean') cfg.spaRender = DEFAULTS.spaRender
 
   // Temporal composability: unload must fully revert side effects.
   ctx.effect(() => () => {
@@ -1025,7 +1065,10 @@ export function apply(ctx, config) {
       },
       required: ['url'],
     },
-    timeoutMs: Math.max(5000, cfg.timeoutMs + 5000),
+    // Pipeline budget must cover the real worst case: full fetch timeout
+    // (cfg.timeoutMs) + SPA render (playwright goto 30s cap + DOM-stability
+    // poll 10s) + margin. Derived from load-time cfg — scale with it, never clamp it.
+    timeoutMs: Math.max(45000, cfg.timeoutMs + 45000),
     output: {
       schema: {
         type: 'object',

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "dsh-read-url",
-  "version": "0.4.7",
-  "description": "DeepSeek Harness URL reader: fetch any webpage, auto-detect encoding (GBK/GB2312/UTF-8/Big5), extract clean main content, output compact text/Markdown to save tokens. Zero runtime dependencies.",
+  "version": "0.4.9",
+  "description": "DeepSeek Harness URL reader: fetch any webpage, auto-detect encoding (GBK/GB2312/UTF-8/Big5), extract clean main content, output compact text/Markdown to save tokens. Zero runtime dependencies, no API key, drop-in.",
   "type": "module",
   "main": "index.js",
   "files": [

package/proxy-fallback.js

@@ -16,14 +16,21 @@
 import { execFile, execFileSync } from 'node:child_process'
 
 let sysProxyCache = null // '' = checked & absent; string = proxy URL
+let sysProxyCheckedAt = 0 // cache timestamp; a failed read is re-tried after 60s
+const SYS_PROXY_RETRY_MS = 60 * 1000
 
 // Windows system proxy from the registry (HKCU Internet Settings). PowerShell
 // is used because `reg.exe` is frequently blocked by security policy; the
 // read is synchronous and cached after the first call (it only runs after a
 // direct-connect failure, so the ~1s cost is invisible on the happy path).
+// A FAILED read is cached as '' but re-tried after 60s — an environment that
+// transiently blocks the PowerShell child (EDR, policy) should not starve
+// the whole session of the proxy.
 function readWindowsSystemProxy() {
-  if (sysProxyCache !== null) return sysProxyCache
+  const now = Date.now()
+  if (sysProxyCache !== null && now - sysProxyCheckedAt < SYS_PROXY_RETRY_MS) return sysProxyCache
   sysProxyCache = ''
+  sysProxyCheckedAt = now
   try {
     const out = execFileSync(
       'powershell',
@@ -38,10 +45,11 @@ function readWindowsSystemProxy() {
     // ProxyServer may be "host:port" or "http=...;https=host:port"
     const httpsPart = server.split(';').map((s) => s.trim()).find((s) => /^https=/i.test(s))
     sysProxyCache = (httpsPart ? httpsPart.replace(/^https=/i, '') : server) || ''
+    return sysProxyCache
   } catch {
     sysProxyCache = ''
+    return sysProxyCache
   }
-  return sysProxyCache
 }
 
 // The user's proxy, in priority order: env vars, then Windows system proxy.

package/spa.js

@@ -20,8 +20,14 @@ export function looksLikeSpa(html) {
 
 async function getBrowser() {
   if (!browserPromise) {
-    const { chromium } = await import('playwright')
-    browserPromise = chromium.launch({ headless: true })
+    // A failed launch (playwright later installed / chromium download finished)
+    // must not stay cached as a rejected promise — reset so the next call retries.
+    browserPromise = import('playwright')
+      .then(({ chromium }) => chromium.launch({ headless: true }))
+      .catch((e) => {
+        browserPromise = null
+        throw e
+      })
   }
   return browserPromise
 }
@@ -73,9 +79,13 @@ export async function renderPage(url, externalSignal) {
 
 // Called on plugin unload (temporal composability): release the browser.
 export async function closeBrowser() {
-  if (browserPromise) {
-    const b = await browserPromise
+  if (!browserPromise) return
+  const p = browserPromise
+  browserPromise = null
+  try {
+    const b = await p
     await b.close().catch(() => {})
-    browserPromise = null
+  } catch {
+    // launch never completed — nothing to close
   }
 }