dsh-read-url 1.8.1 → 1.9.1

package/CHANGELOG.md

@@ -1,5 +1,38 @@
 # Changelog
 
+## [1.9.1] - 2026-09-28
+
+### 修复
+
+- 链接、分页、base 与 meta-refresh 按真实属性读取，支持等号空格和合法无引号属性；忽略 data-href/data-rel 等伪属性，识别 rel 中的 next 标记。
+- meta-refresh 正确解码实体并保留 URL 中的分号及查询参数，避免跳转地址被截断。
+- 有界标签扫描识别引号内的 `>`，空 href 不吞掉后续锚点；锚点正文单向扫描，避免损坏 HTML 触发反复搜索。
+- 链接扫描与站点爬取在取消后返回 cancelled，避免返回旧跳转壳或部分结果作为完整成功；Retry-After 等待结束或取消时清理监听与计时器。
+- 站点正文摘要按 maxCharsPerPage 显示，取消渲染层额外的 80 字符截断；默认仍只输出站点结构。
+- 对抗探针等待异步检查完成，异常设置非零退出码，避免未完成的 Promise 被打印为通过。
+
+### 验证
+
+- 352 个零依赖单元断言、15 个真实 Chromium SPA 断言、59 项对抗探针通过；新增 30 条单元检查与 6 条未闭合锚点探针。原始维护缺陷在 v1.8.1/v1.9.0 同输入均可复现，属于既有问题。
+- 152 站最终实网回归：103 OK / 20 THIN / 5 EMPTY / 24 ERR / 0 THREW，OK 站零噪声。Vue、React、酷狗及 Hatena 同 HTML 的全文续读与 800 字输出对照 v1.9.0 一致；微博动态波动经同 HTML/DOM 对照一致，错误站点集合未变。
+- DSH 0.1.7-rc.2 官方包字节差异及社区升级卡（截至 rc.1）交叉核查完成；v1.9.0 和当前维护代码各通过 17 项真实官方服务集成检查，发布版 patch 另通过 4 项真实加载器检查。已核查接口无需适配修改；受控 provider 检查不等同于完整宿主 UI/模型端到端验证。
+- 未闭合且含 href 的锚点 2.5 万/5 万/10 万输入本机约 27/57/118 ms；保留有界属性策略与零运行时依赖。四工具 description/schema 仍为 1,159/1,876 字符。
+
+## [1.9.0] - 2026-09-19
+
+### 新增
+
+- 新增仅 profile 可配置的 `directFetchOrigins`，默认空列表。按协议、域名和端口精确匹配，命中后直接读取，显式跳过宿主 provider 及其 SSRF 检查；未命中的初始 URL 维持原有行为，不因任意 provider 错误自动直连。
+- 例外链的 HTTP 重定向、meta-refresh、分页及爬取子页面均受完整列表约束；HTTP 跳转逐跳校验，每次尝试最多 5 跳并检测循环。例外不进入 curl 代理竞速或 SPA 渲染，失败边界/静态读取提示保留到工具输出。
+- 成功和失败缓存按规范化后的完整配置隔离；非法配置明确报错，工具参数不允许扩大直连范围。双语 README 增补配置方法及信任边界。
+
+### 验证
+
+- 322 个零依赖单元断言、15 个真实 Chromium SPA 断言、53 项对抗探针通过；新增 22 条直连策略检查与 3 条 SPA 限制检查。
+- 受控 HTTP 服务与 provider 桩覆盖默认拒绝、显式直连、重定向、分页、四工具、缓存隔离、取消、超时及正文上限。
+- 152 站实网回归：105 OK / 23 THIN+EMPTY / 24 ERR / 0 THREW，OK 站零噪声。Vue、React、酷狗三个已知 NOISE 站同输入 v1.8.1 对照的 text/Markdown 全文与 800 字符窗口逐字一致；ERR 均为网络、反爬、无效地址或预期不支持的类型。
+- 四工具 description 共 1,159 字符，参数 schema 共 1,876 字符；零运行时依赖，新增正则有界。
+
 ## [1.8.1] - 2026-09-19
 
 ### 修复

package/README.md

@@ -47,7 +47,7 @@ DSH agents can search (getting links and snippets) but lack the step of "reading
 
 Implemented per official docs (`docs/capability-seams.md`, `docs/cordis-primer.md`, `docs/tool-execution-pipeline.md`):
 
-1. **Web access via the `ctx.web` capability seam** — use `ctx.web.fetch({ url }, signal)` and its decoded `body.kind/content` result for reads, redirects and crawl requests; fall back to direct retrieval when the service or provider is unavailable. Provider HTTP errors and policy decisions remain errors;
+1. **Web access via the `ctx.web` capability seam** — use `ctx.web.fetch({ url }, signal)` and its decoded `body.kind/content` result for reads, redirects and crawl requests; fall back to direct retrieval when the service or provider is unavailable. Provider HTTP errors and policy decisions remain errors. The optional `directFetchOrigins` profile setting selects explicit direct retrieval for trusted origins;
 2. **Reversible side effects** — the session cache is registered under `ctx.effect`, auto-cleared on plugin unload (temporal composability);
 3. **Cooperative tool-call timeout** — `ToolDefinition.timeoutMs` declares the budget, `execute(args, exec)` forwards `exec.signal` to fetch; the timeout policy is enforced by the pipeline, never exposed to the model;
 4. **Model-facing simplicity** — render emits compact text (`title:` header + body); the model consumes it directly with no JSON parsing. Defaults are the most token-efficient; structured output is opt-in;
@@ -181,6 +181,27 @@ Override via the profile's `cordis.patch.yml` (defaults in the plugin's own `cor
 
 Values are coerced and clamped to sane ranges at load — quoted numbers in YAML work, garbage falls back to defaults.
 
+#### Explicit direct origins (v1.9.0)
+
+`directFetchOrigins` defaults to `[]`. For a trusted local reverse proxy (for example, a GitHub hostname mapped by your hosts file), explicitly list the origins in the plugin's **profile configuration**, then reload the plugin/profile:
+
+```yaml
+- id: dsh-read-url
+  config:
+    directFetchOrigins:
+      - https://github.com
+      - https://api.github.com
+      - https://raw.githubusercontent.com
+```
+
+Listed origins use direct fetch immediately, bypassing the host web provider **and its SSRF checks**. Add only origins you trust; this also permits their DNS/hosts mappings to local or private addresses. The option is not a tool argument. Unlisted initial URLs keep the existing provider selection and fallback behavior; a provider refusal does not enable this exception automatically.
+
+Matching is exact after URL normalization: scheme, hostname and port must match; subdomains are separate. At most 32 entries of 2,048 characters each are accepted. Only an HTTP(S) origin with an optional trailing `/` is valid: credentials, paths, queries, fragments, wildcards and ambiguous syntax cause a configuration error. Unlike numeric options, invalid entries do not fall back to defaults.
+
+Once a request selects explicit direct retrieval, every subsequent HTTP redirect, meta-refresh, pagination request and descendant crawl stays within the list. Redirects handled internally by the provider remain on the provider path. Direct HTTP requests follow at most 5 redirects per attempt and stop loops early. An outside target is stopped before requesting it; refresh/pagination blocked by this policy retain the readable static result with a hint. Explicit direct reads use neither the plugin's curl proxy race nor SPA rendering, preventing those mechanisms from widening the exception. When `spaRender` is enabled and `read_url` or `read_url_links` would otherwise render a script-only page, they return a static-reading hint. Cancellation, response-size limits and one bounded Retry-After retry still apply. All four tools share the policy; success and failure caches are isolated by the complete normalized list, including after changing or clearing it.
+
+Controlled HTTP servers and provider fixtures cover default refusals, explicit routing, redirects, pagination, all four tools, cache isolation, cancellation, timeouts and response-size limits.
+
 ### Output (compact)
 
 ```json
@@ -287,6 +308,21 @@ Plus a **15-item DSH acceptance round** (a real agent driving every read_url too
 - SPA cancellation closes the active page; plugin disposal awaits browser shutdown. Valid success-cache entries take precedence over concurrent failures. Crawl filtering uses path boundaries. Charset and Markdown attributes use bounded parsing.
 - Fixed model input cost remains **1,159 description characters / 1,876 schema characters**, within the 1,250 / 2,000 budgets.
 
+### v1.9.0 validation (2026-09-19)
+
+- **322 zero-dependency unit assertions, 15 real Chromium SPA assertions and 53 adversarial probes** passed. The suite includes 22 direct-origin policy checks and 3 SPA restriction checks.
+- The 152-site sweep returned **105 OK / 23 THIN+EMPTY / 24 ERR / 0 THREW**, with no noise in OK results.
+- Vue, React and Kugou's known NOISE cases matched v1.8.1 exactly using identical HTML and optional dependencies, in both text/Markdown and full/800-character outputs. Errors were network, anti-bot, invalid-address or expected content-type boundaries.
+- All four tools enforce the explicit origin policy, including redirect/crawl boundaries, cache isolation and visible static-reading hints. Model input remains **1,159 description characters / 1,876 schema characters**, within the 1,250 / 2,000 budgets.
+
+### v1.9.1 maintenance validation (2026-09-28)
+
+- **352 zero-dependency unit assertions, 15 real Chromium SPA assertions and 59 adversarial probes** passed. Navigation tests cover real attributes, quoted `>` characters, empty anchors, refresh entities and semicolons; cancellation, retry cleanup and the rendered site-summary limit have regression coverage. Asynchronous probes now await completion and report failures through the exit code.
+- The final 152-site sweep returned **103 OK / 25 THIN+EMPTY / 24 ERR / 0 THREW**, with no noise in OK results. Vue, React, Kugou and Hatena matched v1.9.0 using identical HTML, including full continuation and 800-character outputs. Weibo's variable static/SPA results also matched on identical HTML/DOM snapshots; the error-site set was unchanged.
+- Navigation uses bounded, quote-aware attribute scanning and advances through each anchor body once. Runs with 25k/50k/100k unclosed anchors containing `href` took about **27/57/118 ms** on this machine. The existing 1,000-character attribute boundary remains in place.
+- [DSH 0.1.7-rc.2](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.7-rc.2) compatibility was checked against official npm package byte differences and community upgrade notes (available through rc.1). Both published v1.9.0 and this maintenance code passed **17 actual official-service integration checks each**; the published patch passed **4 loader checks** covering loading, execution, configuration remount and removal. Web-provider/runtime contracts remain compatible; no adaptation change is needed for the checked interfaces. These checks used controlled provider responses, not a complete host UI/model session.
+- Core runtime dependencies remain zero. Model input stays at **1,159 description characters / 1,876 schema characters**; site body summaries remain opt-in and respect `maxCharsPerPage` without an extra 80-character render cutoff.
+
 ## Roadmap
 
 - [x] Single-page continuation (`offset` parameter)

package/README.zh.md

@@ -47,7 +47,7 @@ DSH 的 Agent 能搜索（返回链接和片段），但缺"把 URL 读成干净
 
 按官方文档实现（`docs/capability-seams.md`、`docs/cordis-primer.md`、`docs/tool-execution-pipeline.md`）：
 
-1. **网络访问走 `ctx.web` 能力缝**——读取、跳转与爬取使用 `ctx.web.fetch({ url }, signal)` 及解码后的 `body.kind/content`；服务或提供方不可用时回退直连。提供方 HTTP 错误与策略拒绝保留为错误；
+1. **网络访问走 `ctx.web` 能力缝**——读取、跳转与爬取使用 `ctx.web.fetch({ url }, signal)` 及解码后的 `body.kind/content`；服务或提供方不可用时回退直连。提供方 HTTP 错误与策略拒绝保留为错误。可选的 profile 配置 `directFetchOrigins` 为可信来源选择显式直连；
 2. **可逆副作用**——会话缓存注册在 `ctx.effect` 下，插件卸载即自动清理（时间可组合性）；
 3. **协作式工具调用超时**——`ToolDefinition.timeoutMs` 声明预算，`execute(args, exec)` 把 `exec.signal` 转发给 fetch，超时策略由管线强制执行，不把超时暴露给模型；
 4. **模型视角精简**——render 输出紧凑文本（`title:` 头部 + 正文），模型直接消费，无需解析 JSON；默认参数最省 token，结构化能力按需开启；
@@ -181,6 +181,27 @@ chars 800+800/12398 · cached
 
 配置在加载时统一强转数字并钳制到合理范围——YAML 里加引号的数字也能用，非法值回退默认。
 
+#### 显式直连来源（v1.9.0）
+
+`directFetchOrigins` 默认为 `[]`。对于可信的本地反向代理（例如通过 hosts 文件映射 GitHub 域名），在插件的 **profile 配置**中显式列出来源，再重载插件/profile：
+
+```yaml
+- id: dsh-read-url
+  config:
+    directFetchOrigins:
+      - https://github.com
+      - https://api.github.com
+      - https://raw.githubusercontent.com
+```
+
+命中的来源立即直连，跳过宿主 web provider **及其 SSRF 检查**。仅添加你信任的来源；其 DNS/hosts 映射到本机或私有地址也会被允许。此选项不开放为工具参数。未列出的初始 URL 保留既有 provider 选择与回退行为；provider 拒绝不会自动启用此例外。
+
+按 URL 规范化后的协议、域名、端口精确匹配，子域名需单独配置。最多 32 项，每项不超过 2,048 字符。仅接受 HTTP(S) origin，可带一个结尾 `/`；凭据、路径、查询、片段、通配符和歧义写法都会触发配置错误。此选项与数字选项不同，非法项不会静默回退默认值。
+
+请求选用显式直连后，后续 HTTP 重定向、meta-refresh、分页及爬取子页面均限制在列表内；provider 内部处理的跳转仍属于 provider 路径。直连每次尝试最多跟随 5 次 HTTP 重定向，遇循环提前停止。越界目标在发请求前停止；被此策略阻止的刷新跳转或分页保留可读的静态结果并附提示。显式直连不使用插件的 curl 代理竞速或 SPA 渲染，防止它们扩大例外范围；启用 `spaRender` 且 `read_url` 或 `read_url_links` 原本需要渲染脚本空壳时，会提示仅能静态读取。取消、正文大小限制及一次有界 Retry-After 重试继续生效。四个工具统一遵守此策略；成功与失败缓存按完整规范化列表隔离，修改或清空列表后不会误用旧策略的结果。
+
+受控 HTTP 服务与 provider 桩覆盖默认拒绝、显式直连、重定向、分页、四工具、缓存隔离、取消、超时及正文上限。
+
 ### 输出结构（紧凑）
 
 ```json
@@ -287,6 +308,21 @@ v1.4.0 复验（2026-08-28，代理环境）：python-docs stdtypes.html 锚点
 - SPA 取消关闭当前页面，插件卸载等待浏览器关闭；有效成功缓存优先于并发失败；爬取过滤遵循路径边界；编码与 Markdown 属性使用有界解析。
 - 模型固定输入成本仍为 **description 1,159 字符 / schema 1,876 字符**，低于 1,250 / 2,000 预算。
 
+### v1.9.0 验证（2026-09-19）
+
+- **322 个零依赖单元断言、15 个真实 Chromium SPA 断言、53 项对抗探针**通过；包括 22 条直连来源策略检查与 3 条 SPA 限制检查。
+- 152 站实网结果为 **105 OK / 23 THIN+EMPTY / 24 ERR / 0 THREW**，OK 站零噪声。
+- Vue、React、酷狗三个已知 NOISE 站采用同份 HTML 与相同可选依赖对照 v1.8.1，text/Markdown 全文及 800 字符窗口逐字一致；ERR 均为网络、反爬、无效地址或预期不支持的类型。
+- 四个工具统一遵守显式来源策略，覆盖跳转与爬取边界、缓存隔离及可见的静态读取提示。模型固定输入成本为 **description 1,159 字符 / schema 1,876 字符**，低于 1,250 / 2,000 预算。
+
+### v1.9.1 维护验证（2026-09-28）
+
+- **352 个零依赖单元断言、15 个真实 Chromium SPA 断言、59 项对抗探针**通过。导航测试覆盖真实属性、引号内的 `>`、空锚点、刷新实体和分号；取消、重试清理及站点摘要渲染上限均有回归覆盖。异步探针现在等待完成，并通过退出码报告失败。
+- 152 站最终结果为 **103 OK / 25 THIN+EMPTY / 24 ERR / 0 THREW**，OK 站零噪声。Vue、React、酷狗与 Hatena 使用同份 HTML 对照 v1.9.0，全文续读及 800 字输出一致；微博静态/SPA 波动也经同份 HTML/DOM 对照一致，错误站点集合未变。
+- 导航采用有界、识别引号的属性扫描，每段锚点正文只扫描一次。本机 2.5 万/5 万/10 万个含 `href` 的未闭合锚点耗时约 **27/57/118 ms**，保留原有 1,000 字符属性边界。
+- [DSH 0.1.7-rc.2](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.7-rc.2) 已完成官方 npm 接触包字节差异核查，并与社区升级卡交叉验证（卡片目前到 rc.1）。发布版 v1.9.0 与当前维护代码各通过 **17 项真实官方服务集成检查**；发布版 patch 另通过 **4 项加载器检查**，覆盖加载、调用、配置重挂和卸载。Web provider/runtime 契约兼容，已核查接口无需适配修改；测试使用受控 provider 响应，未覆盖完整宿主 UI/模型会话。
+- 核心仍为零运行时依赖，模型固定输入仍为 **description 1,159 字符 / schema 1,876 字符**；站点正文摘要仍默认关闭，启用后遵循 `maxCharsPerPage`，不再额外截成 80 字符。
+
 ## Roadmap
 
 - [x] 单页多段续读（`offset` 参数）

package/cordis.patch.yml

@@ -11,4 +11,5 @@
         spaRender: true
         paginate: true
         paginateMax: 3
+        directFetchOrigins: []
         userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36'

package/index.js

@@ -2,7 +2,8 @@
 // Zero external dependencies (Node 20+ built-ins only).
 // Aligned with DSH architecture:
 //   - all network access goes through the ctx.web capability seam (official
-//     docs/capability-seams.md), falling back to global fetch when absent
+//     docs/capability-seams.md), except explicit directFetchOrigins; falls
+//     back to global fetch when absent
 //   - cache is registered under ctx.effect, so unload fully reverts it
 //     (temporal composability, docs/cordis-primer.md)
 //   - cooperative tool-call timeout via ToolDefinition.timeoutMs + exec.signal
@@ -33,7 +34,8 @@ const DEFAULTS = {
   cacheMax: 32,
   spaRender: true,
   paginate: true,
-  paginateMax: 3,
+  paginateMax: 3,
+  directFetchOrigins: [],
   userAgent: 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0 Safari/537.36',
 }
 
@@ -51,7 +53,52 @@ function cacheStore(key, full, cacheMax) {
   cache.set(key, { time: Date.now(), full })
   failCache.delete(key)
 }
-const decoders = new Map()
+const decoders = new Map()
+
+// Owner configuration only: tool arguments cannot widen this trust boundary.
+// Preserve scheme/port, reject ambiguous origin syntax, and fail closed on
+// invalid configuration instead of silently accepting a partial allowlist.
+function directOrigins(values = []) {
+  const invalid = () => new Error('Invalid directFetchOrigins: expected at most 32 exact http(s) origins without credentials, paths, queries, fragments or wildcards')
+  if (!Array.isArray(values) || values.length > 32) throw invalid()
+  const origins = new Set()
+  for (const value of values) {
+    if (typeof value !== 'string' || value.length > 2048) throw invalid()
+    const raw = value.trim()
+    let u
+    try { u = new URL(raw) } catch { throw invalid() }
+    if (!raw.toLowerCase().startsWith(`${u.protocol}//`) ||
+        !['http:', 'https:'].includes(u.protocol) || u.username || u.password ||
+        u.pathname !== '/' || raw.includes('?') || raw.includes('#') ||
+        raw.includes('*') || raw.includes('\\') || raw.includes('@') ||
+        [...raw].some(c => c.charCodeAt(0) <= 32 || c.charCodeAt(0) === 127)) throw invalid()
+    // Do not accept URL parser repairs such as /a/.. or percent escapes in
+    // hostnames. Only an authority and one optional trailing slash are valid.
+    const authority = raw.slice(raw.indexOf('://') + 3)
+    if (authority.includes('%') || authority.replace(/\/$/, '').includes('/')) throw invalid()
+    origins.add(u.origin)
+  }
+  return [...origins].sort()
+}
+
+function createRoute(cfg) {
+  const origins = directOrigins(cfg.directFetchOrigins)
+  return { origins: new Set(origins), key: JSON.stringify(origins), restricted: false, hint: '' }
+}
+
+function matchesDirectOrigin(url, origins) {
+  try { return origins.has(new URL(url).origin) } catch { return false }
+}
+
+function allowedDirectUrl(url, origins) {
+  try {
+    const u = new URL(url)
+    return !u.username && !u.password && origins.has(u.origin) && ['http:', 'https:'].includes(u.protocol)
+  } catch { return false }
+}
+
+const DIRECT_BOUNDARY_ERROR = 'Direct fetch stopped: target is outside directFetchOrigins or contains credentials'
+const DIRECT_SPA_HINT = '已配置直连仅支持静态读取，未启用浏览器渲染'
 
 function getDecoder(enc) {
   let d = decoders.get(enc)
@@ -124,13 +171,31 @@ function retryAfterMs(res) {
 }
 
 // Single fetch attempt. Never throws; maps abort/timeout/network to { error }.
-async function directFetchOnce(url, signal, cfg) {
-  try {
-    const res = await fetch(url, {
-      redirect: 'follow',
-      signal,
-      headers: { 'user-agent': cfg.userAgent, accept: 'text/html,application/xhtml+xml,*/*;q=0.8' },
-    })
+async function directFetchOnce(url, signal, cfg, origins) {
+  try {
+    let currentUrl = url
+    let res
+    const seen = new Set()
+    for (let hops = 0; ; hops++) {
+      if (signal && signal.aborted) return { error: 'cancelled' }
+      if (origins && !allowedDirectUrl(currentUrl, origins)) return { error: DIRECT_BOUNDARY_ERROR }
+      if (origins) {
+        const normalized = normalizeUrl(currentUrl)
+        if (seen.has(normalized)) return { error: 'Direct fetch redirect loop' }
+        seen.add(normalized)
+      }
+      res = await fetch(currentUrl, {
+        redirect: origins ? 'manual' : 'follow',
+        signal,
+        headers: { 'user-agent': cfg.userAgent, accept: 'text/html,application/xhtml+xml,*/*;q=0.8' },
+      })
+      if (!origins || ![301, 302, 303, 307, 308].includes(res.status)) break
+      if (res.body) res.body.cancel().catch(() => {})
+      const location = res.headers.get('location')
+      if (!location) return { error: `HTTP ${res.status} without Location` }
+      if (hops >= 5) return { error: 'Direct fetch exceeded 5 redirects' }
+      currentUrl = new URL(location, currentUrl).href
+    }
     // Early-return paths must cancel the body stream, or the connection is
     // held open undrained and cannot return to the keep-alive pool.
     const discardBody = () => { if (res.body) res.body.cancel().catch(() => {}) }
@@ -150,7 +215,7 @@ async function directFetchOnce(url, signal, cfg) {
       discardBody()
       return { error: `Unsupported content-type: ${contentType.split(';')[0]}` }
     }
-    if (!res.body) return { buffer: Buffer.alloc(0), contentType, finalUrl: res.url }
+    if (!res.body) return { buffer: Buffer.alloc(0), contentType, finalUrl: res.url || currentUrl }
     const chunks = []
     let size = 0
     for await (const chunk of res.body) {
@@ -163,7 +228,7 @@ async function directFetchOnce(url, signal, cfg) {
     if (!contentType && looksBinary(buffer)) {
       return { error: 'Unsupported content-type (no header, binary body)' }
     }
-    return { buffer, contentType, finalUrl: res.url }
+    return { buffer, contentType, finalUrl: res.url || currentUrl }
   } catch (e) {
     if (e.name === 'AbortError' || e.name === 'TimeoutError') {
       return { error: `Timeout after ${cfg.timeoutMs}ms or cancelled` }
@@ -180,15 +245,24 @@ async function directFetchOnce(url, signal, cfg) {
 // Exported for tests: the double-throttle path must always produce an { error }
 // shape — a second `{ retryAfterMs }` would otherwise leak past the race
 // filter into decodeBuffer as a bufferless "success" (bare TypeError there).
-export async function directFetch(url, signal, cfg) {
-  const first = await directFetchOnce(url, signal, cfg)
-  if (first && first.retryAfterMs !== undefined && !(signal && signal.aborted)) {
+export async function directFetch(url, signal, cfg, origins) {
+  const first = await directFetchOnce(url, signal, cfg, origins)
+  if (first && first.retryAfterMs !== undefined) {
+    if (signal && signal.aborted) return { error: 'cancelled' }
     await new Promise((resolve) => {
-      const t = setTimeout(resolve, first.retryAfterMs)
-      if (signal) signal.addEventListener('abort', () => { clearTimeout(t); resolve() }, { once: true })
+      const done = () => {
+        clearTimeout(timer)
+        if (signal) signal.removeEventListener('abort', done)
+        resolve()
+      }
+      const timer = setTimeout(done, first.retryAfterMs)
+      if (signal) {
+        signal.addEventListener('abort', done, { once: true })
+        if (signal.aborted) done()
+      }
     })
     if (signal && signal.aborted) return { error: 'cancelled' }
-    const second = await directFetchOnce(url, signal, cfg)
+    const second = await directFetchOnce(url, signal, cfg, origins)
     // The retry budget is spent — report the rate limit instead of returning
     // a shapeless { retryAfterMs } (no buffer, no error) to the caller.
     if (second && second.retryAfterMs !== undefined) {
@@ -309,6 +383,27 @@ async function fetchViaWebSeam(ctx, url, externalSignal, cfg) {
     return { error: `Web provider failed: ${String((e && e.message) || e).slice(0, 160)}` }
   }
 }
+
+// Every fetch in a tool call passes here. Once a chain enters an explicit
+// direct origin it stays restricted, including meta refresh and pagination.
+// Never send an exception through curl -L or a browser: both may make further
+// requests outside this origin list. Unlisted initial URLs retain host policy.
+async function fetchResource(ctx, url, externalSignal, cfg, route) {
+  if (externalSignal && externalSignal.aborted) return { error: 'cancelled' }
+  if (route.restricted || matchesDirectOrigin(url, route.origins)) {
+    route.restricted = true
+    if (!allowedDirectUrl(url, route.origins)) {
+      route.hint = DIRECT_BOUNDARY_ERROR
+      return { error: DIRECT_BOUNDARY_ERROR }
+    }
+    const timeout = AbortSignal.timeout(cfg.timeoutMs)
+    const signal = externalSignal ? AbortSignal.any([externalSignal, timeout]) : timeout
+    const page = await directFetch(url, signal, cfg, route.origins)
+    if (page.error) route.hint = page.error
+    return page
+  }
+  return await fetchViaWebSeam(ctx, url, externalSignal, cfg) || await fetchPage(url, externalSignal, cfg)
+}
 
 // Decode common HTML entities in already-stripped text. Runs AFTER tag
 // stripping so escaped tags stay visible to the stripper and only remaining
@@ -1341,22 +1436,60 @@ function decodeUrlAttribute(value) {
 }
 
 // <base href> overrides the document base for relative URL resolution (old
-// forums / frame sites). Per HTML spec only the FIRST <base> in <head> counts;
+// forums / frame sites). Only the FIRST <base href> in <head> counts;
 // attribute order is arbitrary and the href may be protocol-relative. Empty
 // href (= document URL) is a no-op; non-http(s) targets are rejected. The
 // head scan is bounded — the tag cannot legally appear beyond it.
 function detectBaseHref(html, finalUrl) {
   const headEnd = html.search(/<\/head\s*>/i)
   const head = headEnd >= 0 ? html.slice(0, headEnd) : html.slice(0, 16384)
-  const base = /<base\b[^>]{0,1000}href=["']([^"']+)["'][^>]{0,1000}>/i.exec(head)
-  if (!base) return finalUrl
-  const href = decodeUrlAttribute(base[1]).trim()
-  if (!href) return finalUrl
-  try {
-    const u = new URL(href, finalUrl)
-    return u.protocol === 'http:' || u.protocol === 'https:' ? u.href : finalUrl
-  } catch {
-    return finalUrl
+  for (const { attrs } of navigationTags(head, /<base(?=[\s/>])/gi)) {
+    if (!Object.hasOwn(attrs, 'href')) continue
+    const href = decodeUrlAttribute(attrs.href).trim()
+    if (!href) return finalUrl
+    try {
+      const u = new URL(href, finalUrl)
+      return u.protocol === 'http:' || u.protocol === 'https:' ? u.href : finalUrl
+    } catch {
+      return finalUrl
+    }
+  }
+  return finalUrl
+}
+
+// Honor quoted '>' without nested regex alternatives. Keep the same total
+// tag-length bound as htmlAttributes; malformed/overlong tags stay inert.
+function* navigationTags(html, open) {
+  let m
+  while ((m = open.exec(html))) {
+    let quote = '', end = -1
+    const limit = Math.min(html.length, m.index + 1002)
+    for (let i = open.lastIndex; i < limit; i++) {
+      const ch = html[i]
+      if (quote) {
+        if (ch === quote) quote = ''
+      } else if (ch === '"' || ch === "'") quote = ch
+      else if (ch === '<') break
+      else if (ch === '>') { end = i + 1; break }
+    }
+    if (end < 0) continue
+    open.lastIndex = end
+    yield { attrs: htmlAttributes(html.slice(m.index, end)), end }
+  }
+}
+
+// Scan each anchor body once; tags without a real href cannot navigate.
+function* navigationAnchors(html) {
+  const open = /<a(?=[\s>])/gi
+  const close = /<\/a\s{0,1000}>/gi
+  for (const { attrs, end: start } of navigationTags(html, open)) {
+    if (!attrs.href?.trim()) continue
+    close.lastIndex = start
+    const end = close.exec(html)
+    if (!end) break
+    const body = html.slice(start, end.index)
+    open.lastIndex = close.lastIndex
+    yield { attrs, body }
   }
 }
 
@@ -1368,18 +1501,14 @@ function extractLinks(html, limit, baseUrl) {
   // A declared <base href> wins over the document URL (frame-site links would
   // otherwise land on the wrong host).
   const linkBase = detectBaseHref(html, baseUrl)
-  // Attribute runs bounded: an unclosed <a whose '>' is far away (or missing)
-  // must not make the regex scan the rest of the string at every anchor
-  // position. Real attributes never approach 1000 chars.
-  const re = /<a\b[^>]{0,1000}href=["']([^"']+)["'][^>]{0,1000}>([\s\S]*?)<\/a\s*>/gi
-  let m
   // Iteration cap: a page whose nav repeats the same links thousands of times
   // would otherwise scan the whole HTML to fill `limit` unique entries. Scan
   // at most limit*4 anchors; dedupe keeps the result bounded regardless.
   let scans = 0
   const maxScans = limit * 4
-  while ((m = re.exec(html)) && links.length < limit && scans++ < maxScans) {
-    const href = decodeUrlAttribute(m[1]).trim()
+  for (const { attrs, body } of navigationAnchors(html)) {
+    if (links.length >= limit || scans++ >= maxScans) break
+    const href = decodeUrlAttribute(attrs.href).trim()
     if (!href || /^(javascript|mailto|tel|data):/i.test(href)) continue
     let url
     try {
@@ -1395,7 +1524,7 @@ function extractLinks(html, limit, baseUrl) {
     // keeps the link list compact (tokens) without losing coverage
     if (seen.has(url)) continue
     seen.add(url)
-    const t = textOnly(m[2])
+    const t = textOnly(body)
     if (t) links.push({ title: t.slice(0, 80), url })
   }
   return links
@@ -1421,6 +1550,7 @@ export function findNextLink(html, baseUrl) {
   // document base, and a "next page" link resolves against it.
   const base = detectBaseHref(html || '', baseUrl)
   const resolve = (href) => {
+    if (!href?.trim()) return null
     try {
       const u = new URL(decodeUrlAttribute(href), base)
       return u.protocol === 'http:' || u.protocol === 'https:' ? u.href : null
@@ -1428,21 +1558,20 @@ export function findNextLink(html, baseUrl) {
       return null
     }
   }
-  const rel = /<(?:link|a)\b[^>]{0,1000}rel=["']next["'][^>]{0,1000}>/i.exec(html || '')
-  if (rel) {
-    const href = /href=["']([^"']+)["']/i.exec(rel[0])
-    const r = href && resolve(href[1])
+  for (const { attrs } of navigationTags(html || '', /<(?:link|a)(?=[\s/>])/gi)) {
+    const rel = decodeUrlAttribute(attrs.rel || '').toLowerCase().split(/[ \t\r\n\f]{1,1000}/)
+    if (!rel.includes('next')) continue
+    const r = resolve(attrs.href)
     if (r) return r
   }
-  const re = /<a\b[^>]{0,1000}href=["']([^"']+)["'][^>]{0,1000}>([\s\S]*?)<\/a\s*>/gi
-  let m
   let scans = 0
   // Bounded like extractLinks: a nav bar with thousands of anchors must not
   // scan the whole DOM hunting for a next-marker.
-  while ((m = re.exec(html || '')) && scans++ < 400) {
-    const t = textOnly(m[2])
+  for (const { attrs, body } of navigationAnchors(html || '')) {
+    if (scans++ >= 400) break
+    const t = textOnly(body)
     if (t.length <= 16 && NEXT_TEXT_RE.test(t)) {
-      const r = resolve(m[1])
+      const r = resolve(attrs.href)
       if (r) return r
     }
   }
@@ -1535,18 +1664,15 @@ export function metaRefreshTarget(html, baseUrl) {
   // Locate the refresh <meta> tag first (any attribute order / case), then
   // read content from inside that tag — a forward regex would demand
   // http-equiv before content and miss the reversed order.
-  const re = /<meta\b[^>]{0,1000}(?:http-equiv|name)=["']refresh["'][^>]{0,1000}>/gi
-  let m
-  while ((m = re.exec(html || ''))) {
-    const tag = m[0]
-    const contentM = /content=("([^"]*)"|'([^']*)')/i.exec(tag)
-    if (!contentM) continue // a refresh meta without content is inert
-    const content = (contentM[2] || contentM[3] || '').trim()
+  for (const { attrs } of navigationTags(html || '', /<meta(?=[\s/>])/gi)) {
+    if (![attrs['http-equiv'], attrs.name].some(value => value?.toLowerCase() === 'refresh')) continue
+    if (!Object.hasOwn(attrs, 'content')) continue
+    const content = decodeUrlAttribute(attrs.content).trim()
     // content="5" | content="0; url=http://…" | content=";url='…'" — the url
     // part may be single/double quoted or bare; "url=" must be a real token.
-    const urlM = /\burl\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s;]+))/i.exec(content)
+    const urlM = /\burl\s{0,1000}=\s{0,1000}(?:"([^"]{0,1000})"|'([^']{0,1000})'|([^\s]{1,1000}))/i.exec(content)
     if (!urlM) return null
-    const delayM = /^\s*(\d+(?:\.\d+)?)/.exec(content)
+    const delayM = /^\s{0,1000}(\d{1,1000}(?:\.\d{1,1000})?)/.exec(content)
     if (delayM && Number(delayM[1]) > 0) return null // timed refresh: keep the page
     const target = (urlM[1] || urlM[2] || urlM[3] || '').trim()
     if (!target) return null
@@ -1567,7 +1693,7 @@ export function metaRefreshTarget(html, baseUrl) {
 // cache key is the ORIGINAL url, and a shell re-request should re-follow
 // rather than serve a foreign page.
 const MAX_META_REFRESH_HOPS = 3
-async function followMetaRefresh(html, finalUrl, externalSignal, cfg, ctx) {
+async function followMetaRefresh(html, finalUrl, externalSignal, cfg, ctx, route) {
   let cur = html
   let curUrl = finalUrl
   let curCharset = '' // set only when a hop actually happened
@@ -1586,7 +1712,7 @@ async function followMetaRefresh(html, finalUrl, externalSignal, cfg, ctx) {
     const norm = normalizeUrl(target)
     if (!norm || seen.has(norm)) break
     seen.add(norm)
-    const page = await fetchViaWebSeam(ctx, target, hopSignal, cfg) || await fetchPage(target, hopSignal, cfg)
+    const page = await fetchResource(ctx, target, hopSignal, cfg, route)
     if (page.error) break
     const ct = page.kind === 'html' ? 'text/html' : (page.contentType || '').split(';')[0].toLowerCase().trim()
     if (!/html|xhtml/.test(ct)) break
@@ -1603,6 +1729,8 @@ export async function readUrl(args, ctx, externalSignal, cfg = DEFAULTS) {
   const url = String((args && args.url) || '').trim()
   if (!/^https?:\/\//i.test(url)) return { error: 'Only http/https URLs are supported' }
   if (externalSignal && externalSignal.aborted) return { error: 'cancelled' }
+  let route
+  try { route = createRoute(cfg) } catch (e) { return { error: e.message } }
   const maxChars = Math.max(500, Math.min(20000, Number(args.maxChars) || cfg.maxChars))
   const mode = args.mode === 'markdown' ? 'markdown' : 'text'
   const offset = Math.max(0, Number(args.offset) || 0)
@@ -1632,7 +1760,9 @@ export async function readUrl(args, ctx, externalSignal, cfg = DEFAULTS) {
   } catch {
     /* keep bare url */
   }
-  const cacheKey = `${cacheUrl}|${mode}|${args.includeLinks === true ? 'links' : 'no-links'}`
+  // Include the complete policy: an unlisted entry may reach a listed origin
+  // through refresh/pagination, and must not reuse content after revocation.
+  const cacheKey = `${route.key}|${cacheUrl}|${mode}|${args.includeLinks === true ? 'links' : 'no-links'}`
   const hit = cache.get(cacheKey)
   if (hit && Date.now() - hit.time < cfg.cacheTtlMs) {
     // Valid content takes precedence over a concurrent fetch failure.
@@ -1700,9 +1830,9 @@ export async function readUrl(args, ctx, externalSignal, cfg = DEFAULTS) {
     }
   }
   const dispatch = async () => {
-    const viaSeam = await fetchViaWebSeam(ctx, url, externalSignal, cfg)
-    const page = viaSeam || await fetchPage(url, externalSignal, cfg)
+    const page = await fetchResource(ctx, url, externalSignal, cfg, route)
     if (page.error) return { error: page.error }
+    const viaSeam = typeof page.html === 'string'
     const ct = viaSeam ? (page.kind === 'text' ? 'text/plain' : 'text/html') : (page.contentType || '').split(';')[0].toLowerCase().trim()
     const decoded = viaSeam ? { text: page.html, charset: 'provider-decoded' } : decodeBuffer(page.buffer, page.contentType)
     const meta = { finalUrl: page.finalUrl || url, sourceTruncated: page.sourceTruncated === true }
@@ -1782,7 +1912,7 @@ export async function readUrl(args, ctx, externalSignal, cfg = DEFAULTS) {
   // follow immediately, BEFORE extraction and SPA rendering — a shell that
   // redirects needs no headless round-trip, and extracting the stub is wasted
   // work. Fails open on any fetch problem; the follow is hop-bounded.
-  const followed = await followMetaRefresh(html, finalUrl, externalSignal, cfg, ctx)
+  const followed = await followMetaRefresh(html, finalUrl, externalSignal, cfg, ctx, route)
   if (externalSignal && externalSignal.aborted) return { error: 'cancelled' }
   html = followed.html
   finalUrl = followed.finalUrl
@@ -1826,7 +1956,10 @@ export async function readUrl(args, ctx, externalSignal, cfg = DEFAULTS) {
   let renderedHtml = null
   const needsRender = !extracted.text || extracted.text.length < 200
   const scriptShell = !extracted.text && /<script[\s>]/i.test(html)
-  if (cfg.spaRender !== false && needsRender && (looksLikeSpa(html) || scriptShell)) {
+  if (route.restricted && cfg.spaRender !== false && needsRender && (looksLikeSpa(html) || scriptShell)) {
+    spaHint = [spaHint, DIRECT_SPA_HINT].filter(Boolean).join('；')
+  }
+  if (!route.restricted && cfg.spaRender !== false && needsRender && (looksLikeSpa(html) || scriptShell)) {
     const rr = await renderPage(finalUrl || url, externalSignal)
     if (rr.html) {
       renderedHtml = rr.html
@@ -1882,7 +2015,7 @@ export async function readUrl(args, ctx, externalSignal, cfg = DEFAULTS) {
     const nu = normalizeUrl(nextUrl)
     if (!nu || seen.has(nu)) break
     seen.add(nu)
-    const page = await fetchViaWebSeam(ctx, nextUrl, externalSignal, cfg) || await fetchPage(nextUrl, externalSignal, cfg)
+    const page = await fetchResource(ctx, nextUrl, externalSignal, cfg, route)
     if (page.error) break
     const pageFinal = normalizeUrl(page.finalUrl || nextUrl)
     if (!pageFinal || !sameHost(pageFinal, startHost) || (pageFinal !== nu && seen.has(pageFinal))) break
@@ -1898,7 +2031,9 @@ export async function readUrl(args, ctx, externalSignal, cfg = DEFAULTS) {
     nextUrl = findNextLink(nextHtml, page.finalUrl || nextUrl)
   }
 
-  // A near-empty body (login wall, anti-bot shell) with no render hint yet
+  if (route.hint) spaHint = [spaHint, route.hint].filter(Boolean).join('；')
+
+  // A near-empty body (login wall, anti-bot shell) with no render hint yet
   // still looks like "the page content" to the model — one constant-cost line
   // tells it not to keep offset-reading a 40-char page.
   if (!spaHint && extracted.text && extracted.text.length < 60) {
@@ -1992,8 +2127,9 @@ function renderLinks(value) {
   if (typeof value === 'string') return value
   const r = value || {}
   if (r.error) return `Error: ${r.error}`
-  if (!Array.isArray(r.links) || r.links.length === 0) return `No links found on ${r.url}`
-  const lines = [`${r.count} link(s) on ${r.url}:`, UNTRUSTED_NOTICE]
+  if (!Array.isArray(r.links) || r.links.length === 0) return `No links found on ${r.url}${r.spaHint ? `\n提示: ${r.spaHint}` : ''}`
+  const lines = [`${r.count} link(s) on ${r.url}:`, UNTRUSTED_NOTICE]
+  if (r.spaHint) lines.push(`提示: ${r.spaHint}`)
   for (const l of r.links) lines.push(`- ${l.title || l.url} — ${l.url}`)
   return lines.join('\n')
 }
@@ -2033,14 +2169,19 @@ function readLinksTool(ctx, cfg) {
     async execute(args, exec) {
       const url = String((args && args.url) || '').trim()
       if (!/^https?:\/\//i.test(url)) return { error: 'Only http/https URLs are supported' }
-      const limit = Math.max(1, Math.min(50, Number(args.limit) || cfg.maxLinks))
-      const page = await fetchViaWebSeam(ctx, url, exec && exec.signal, cfg) || await fetchPage(url, exec && exec.signal, cfg)
+      const limit = Math.max(1, Math.min(50, Number(args.limit) || cfg.maxLinks))
+      const signal = exec && exec.signal
+      if (signal && signal.aborted) return { error: 'cancelled' }
+      const route = createRoute(cfg)
+      const page = await fetchResource(ctx, url, signal, cfg, route)
+      if (signal && signal.aborted) return { error: 'cancelled' }
       if (page.error) return { error: page.error }
       let html = typeof page.html === 'string' ? page.html : decodeBuffer(page.buffer, page.contentType).text
       let finalUrl = page.finalUrl || url
       // Same meta-refresh shell following as read_url: a link hop serves a
       // stub whose real links live at the target. Fails open, hop-bounded.
-      const followed = await followMetaRefresh(html, finalUrl, exec && exec.signal, cfg, ctx)
+      const followed = await followMetaRefresh(html, finalUrl, signal, cfg, ctx, route)
+      if (signal && signal.aborted) return { error: 'cancelled' }
       html = followed.html
       finalUrl = followed.finalUrl
       let links = extractLinks(html, limit, finalUrl)
@@ -2048,8 +2189,10 @@ function readLinksTool(ctx, cfg) {
       // render it and re-extract when the static result looks empty. Any
       // <script> counts as renderable when there are no links at all — the
       // static shell may be a JS-redirect (same rationale as read_url).
-      if (links.length < 3 && cfg.spaRender !== false && (looksLikeSpa(html) || (links.length === 0 && /<script[\s>]/i.test(html)))) {
-        const rr = await renderPage(finalUrl || url, exec && exec.signal)
+      const needsRender = links.length < 3 && cfg.spaRender !== false && (looksLikeSpa(html) || (links.length === 0 && /<script[\s>]/i.test(html)))
+      if (!route.restricted && needsRender) {
+        const rr = await renderPage(finalUrl || url, signal)
+        if (signal && signal.aborted) return { error: 'cancelled' }
         // A challenge interstitial's links are the challenge's own scripts —
         // never an improvement over whatever the static HTML offered.
         if (rr.html && !looksLikeChallenge(rr.html)) {
@@ -2060,7 +2203,8 @@ function readLinksTool(ctx, cfg) {
           }
         }
       }
-      return { url: finalUrl, count: links.length, links }
+      const spaHint = [route.hint, route.restricted && needsRender ? DIRECT_SPA_HINT : ''].filter(Boolean).join('；')
+      return { url: finalUrl, count: links.length, links, ...(spaHint ? { spaHint } : {}) }
     },
   }
 }
@@ -2091,7 +2235,8 @@ function renderBatch(value) {
       continue
     }
     const head = p.title || p.url
-    lines.push('', `--- ${head} (${p.chars} 字符${p.cached ? ' · cached' : ''}) ---`, p.text || '(无可读内容)')
+    lines.push('', `--- ${head} (${p.chars} 字符${p.cached ? ' · cached' : ''}) ---`, p.text || '(无可读内容)')
+    if (p.spaHint) lines.push(`提示: ${p.spaHint}`)
     if (Array.isArray(p.links) && p.links.length) {
       const ls = p.links.map((l) => l.title + ' — ' + l.url)
       lines.push(`links: ${ls.slice(0, 6).join(' | ')}${ls.length > 6 ? ` …共${ls.length}个` : ''}`)
@@ -2181,7 +2326,8 @@ function readUrlBatchTool(ctx, cfg) {
             cached: p.cached === true,
             text: p.text,
           }
-          if (Array.isArray(p.links) && p.links.length) out.links = p.links
+          if (Array.isArray(p.links) && p.links.length) out.links = p.links
+          if (p.spaHint) out.spaHint = p.spaHint
           return out
         }),
       }
@@ -2226,7 +2372,8 @@ function isNoiseUrl(url) {
 // failure is recorded and does not abort the crawl. No SPA rendering here —
 // that is read_url's job; crawling favors speed and breadth.
 async function crawlSite(entryUrl, cfg, opts, externalSignal, ctx) {
-  const { maxPages, maxDepth, includeContent, perMax } = opts
+  const { maxPages, maxDepth, includeContent, perMax } = opts
+  const policy = createRoute(cfg)
   let host = hostOf(entryUrl)
   const visited = new Set()
   const resolved = new Set()
@@ -2240,12 +2387,14 @@ async function crawlSite(entryUrl, cfg, opts, externalSignal, ctx) {
     // Never overshoot the page budget within a batch round.
     const batch = queue.splice(0, Math.min(2, maxPages - attempted, queue.length))
     const results = await Promise.all(
-      batch.map(async ({ url, depth }) => {
+      batch.map(async ({ url, depth, restricted = false }) => {
         const norm = normalizeUrl(url)
         if (!norm || visited.has(norm)) return null
         visited.add(norm)
         attempted++
-        const page = await fetchViaWebSeam(ctx, url, externalSignal, cfg) || await fetchPage(url, externalSignal, cfg)
+        const route = { ...policy, restricted }
+        const page = await fetchResource(ctx, url, externalSignal, cfg, route)
+        if (externalSignal && externalSignal.aborted) return null
         if (page.error) return { url, depth, error: page.error }
         let html = typeof page.html === 'string' ? page.html : decodeBuffer(page.buffer, page.contentType).text
         let pageUrl = page.finalUrl || url
@@ -2256,7 +2405,8 @@ async function crawlSite(entryUrl, cfg, opts, externalSignal, ctx) {
         // be crawled AS the page (title-less hop text pollutes results and the
         // real content is never reached). Fails open — fetch problems keep the
         // stub's own HTML.
-        const followed = await followMetaRefresh(html, pageUrl, externalSignal, cfg, ctx)
+        const followed = await followMetaRefresh(html, pageUrl, externalSignal, cfg, ctx, route)
+        if (externalSignal && externalSignal.aborted) return null
         html = followed.html
         pageUrl = followed.finalUrl
         if (depth === 0) host = hostOf(pageUrl)
@@ -2271,10 +2421,13 @@ async function crawlSite(entryUrl, cfg, opts, externalSignal, ctx) {
           title: ex.title || '',
           chars: ex.text.length,
           text: includeContent ? smartTruncate(ex.text, perMax).text : undefined,
-          links,
+          links,
+          route,
+          ...(route.hint ? { spaHint: route.hint } : {}),
         }
       }),
     )
+    if (externalSignal && externalSignal.aborted) break
     for (const r of results) {
       if (!r) continue
       if (r.error) {
@@ -2285,7 +2438,8 @@ async function crawlSite(entryUrl, cfg, opts, externalSignal, ctx) {
       if (!finalNorm || resolved.has(finalNorm)) continue
       resolved.add(finalNorm)
       visited.add(finalNorm)
-      pages.push(r)
+      const { route, ...publicPage } = r
+      pages.push(publicPage)
       if (r.depth + 1 <= maxDepth) {
         // Queue cap: a huge site (millions of links) would otherwise grow the
         // queue unboundedly while maxPages bounds only what we *process*.
@@ -2293,7 +2447,7 @@ async function crawlSite(entryUrl, cfg, opts, externalSignal, ctx) {
         for (const l of r.links) {
           if (!sameHost(l.url, host) || isNoiseUrl(l.url)) continue
           const n = normalizeUrl(l.url)
-          if (n && !visited.has(n)) queue.push({ url: l.url, depth: r.depth + 1 })
+          if (n && !visited.has(n)) queue.push({ url: l.url, depth: r.depth + 1, restricted: route.restricted })
         }
       }
     }
@@ -2309,8 +2463,9 @@ function renderSite(value) {
   for (const p of v.pages) {
     const indent = '  '.repeat(p.depth)
     const head = p.title || p.url
-    lines.push(`${indent}[${p.depth}] ${head} (${p.chars} 字符)  ${p.url}`)
-    if (p.text) lines.push(`${indent}   ${p.text.slice(0, 80)}`)
+    lines.push(`${indent}[${p.depth}] ${head} (${p.chars} 字符)  ${p.url}`)
+    if (p.spaHint) lines.push(`${indent}提示: ${p.spaHint}`)
+    if (p.text) lines.push(`${indent}   ${p.text}`)
   }
   for (const f of v.failures) lines.push(`[失败] ${f.url} — ${f.error}`)
   return lines.join('\n')
@@ -2364,21 +2519,25 @@ function readUrlSiteTool(ctx, cfg) {
       const url = String((args && args.url) || '').trim()
       if (!/^https?:\/\//i.test(url)) return { error: 'Only http/https URLs are supported' }
       const maxPages = Math.floor(Math.max(2, Math.min(50, Number(args.maxPages) || 15)))
+      const signal = exec && exec.signal
+      if (signal && signal.aborted) return { error: 'cancelled' }
       const maxDepth = Math.max(1, Math.min(5, Number(args.maxDepth) || 2))
       const includeContent = args.includeContent === true
       const perMax = Math.max(200, Math.min(2000, Number(args.maxCharsPerPage) || 500))
       const { host, pages, failures } = await crawlSite(
-        url, cfg, { maxPages, maxDepth, includeContent, perMax }, exec && exec.signal, ctx,
+        url, cfg, { maxPages, maxDepth, includeContent, perMax }, signal, ctx,
       )
+      if (signal && signal.aborted) return { error: 'cancelled' }
       return {
         host,
         total: pages.length + failures.length,
         succeeded: pages.length,
         failed: failures.length,
         pages: pages.map((p) => {
-          const o = { url: p.url, depth: p.depth, title: p.title || '', chars: p.chars }
-          if (p.text) o.text = p.text
-          return o
+          const o = { url: p.url, depth: p.depth, title: p.title || '', chars: p.chars }
+          if (p.text) o.text = p.text
+          if (p.spaHint) o.spaHint = p.spaHint
+          return o
         }),
         failures,
       }
@@ -2405,7 +2564,8 @@ export function apply(ctx, config) {
   cfg.paginateMax = clamp(cfg.paginateMax, DEFAULTS.paginateMax, 1, 10)
   if (typeof cfg.userAgent !== 'string' || !cfg.userAgent.trim()) cfg.userAgent = DEFAULTS.userAgent
   if (typeof cfg.spaRender !== 'boolean') cfg.spaRender = DEFAULTS.spaRender
-  if (typeof cfg.paginate !== 'boolean') cfg.paginate = DEFAULTS.paginate
+  if (typeof cfg.paginate !== 'boolean') cfg.paginate = DEFAULTS.paginate
+  cfg.directFetchOrigins = directOrigins(cfg.directFetchOrigins)
 
   // Temporal composability: unload must fully revert side effects.
   ctx.effect(() => () => {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-read-url",
-  "version": "1.8.1",
+  "version": "1.9.1",
   "description": "DeepSeek Harness URL reader: fetch any webpage (HTML/JSON/RSS/Atom), auto-detect encoding (UTF-16 BOM/GBK/Big5/Shift-JIS), extract clean main content with auto-pagination, output compact text/Markdown to save tokens. Zero runtime dependencies, no API key, drop-in. Follows meta-refresh shells, reads JSON-LD metadata, honors base href.",
   "type": "module",
   "main": "index.js",