dsh-read-url 1.9.3 → 1.9.4

package/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## [1.9.4] - 2026-11-15
+
+### 修复
+
+- Windows 系统代理映射按请求的 HTTP/HTTPS 协议选择，缓存保留完整映射。环境变量同时配置两种代理时优先选择目标协议，单一代理配置保留兼容回退。
+- 链接扫描和站点读取保留 HTML 提供方的原文截断提示，刷新至完整目标后清除旧提示，刷新失败时保留当前页面提示。
+- 短前言或过早的句子边界不会耗尽整次正文读取，干净正文优先使用窗口后半段的段落或句子边界，并保留准确续读坐标。
+- 含原始 HTML 的窗口保留早期对齐，避免扩展短首段时引入更多超长属性残留，字面代码示例按原文续读。
+- 双语文档更新 DSH 0.2.2 HTTP 提供方的内网访问行为。
+
+### 验证
+
+- 435 个单元断言、15 个真实 Chromium SPA 断言、70 项对抗探针通过，新增 22 条单元检查。分协议代理选择、截断提示及短前言问题在 v1.9.2/v1.9.3 同输入复现。
+- 本机 DSH 桌面 0.2.2-rc.1 的 Electron 44.7.0 / Node 24.21.0 通过 17 项服务和 3 项作用域检查。官方 0.2.2-rc.3 通过 17 项服务、9 项 DeepSeek 提供方与 5 项 HTTP 提供方受控检查，发布版 patch 通过加载器 4 项和临时 profile 2 项检查。
+- 官方 npm latest 为 0.2.2-rc.1、next 为 0.2.2-rc.3。八个接触包、Cordis/Loader 与四个桌面加载文件完成字节差异审计，交叉核对截至 0.2.2-alpha.3 的社区升级卡。新版工具执行器的调整已纳入集成测试，插件消费接口保持兼容。
+- 干净正文随机续读及 emoji 边界检查通过，固定 800 字符窗口的截断耗时不随全文大小增长。原始 HTML 的长属性扫描限制继续保留，部分站点仍有既有残留标签边界。
+- 核心零运行时依赖，四工具 description/schema 为 1,159/1,876 字符。受控服务检查未覆盖真实模型会话或外部 DeepSeek API 服务。
+- 最终 152 站为 104 OK / 21 THIN / 4 EMPTY / 23 ERR / 0 THREW，OK 站零噪声，三个已知 NOISE 站点同输入对照 v1.9.3 一致。错误为 7 个 HTTP 错误、3 个预期类型拒绝、13 个连接/DNS 失败。
+
 ## [1.9.3] - 2026-10-19
 
 ### 修复

package/README.md

@@ -204,7 +204,7 @@ Values are coerced and clamped to sane ranges at load — quoted numbers in YAML
       - https://raw.githubusercontent.com
 ```
 
-Listed origins use direct fetch immediately, bypassing the host web provider **and its SSRF checks**. Add only origins you trust; this also permits their DNS/hosts mappings to local or private addresses. The option is not a tool argument. Unlisted initial URLs keep the existing provider selection and fallback behavior; a provider refusal does not enable this exception automatically.
+Listed origins use direct fetch immediately, bypassing the host web provider and its access policy. Add only origins you trust, including their DNS/hosts mappings to local or private addresses. DSH 0.2.2's HTTP provider already supports local and private destinations. Other providers may apply different access policies. The option is configured in the profile. Unlisted initial URLs retain provider selection and fallback behavior. A provider refusal does not automatically enable explicit direct retrieval.
 
 Matching is exact after URL normalization: scheme, hostname and port must match; subdomains are separate. At most 32 entries of 2,048 characters each are accepted. Only an HTTP(S) origin with an optional trailing `/` is valid: credentials, paths, queries, fragments, wildcards and ambiguous syntax cause a configuration error. Unlike numeric options, invalid entries do not fall back to defaults.
 
@@ -350,6 +350,14 @@ Plus a **15-item DSH acceptance round** (a real agent driving every read_url too
 - The DeepSeek Fetch provider returns Markdown as `body.kind=text`. Single-page reading preserves that text; links/site operations report that HTML navigation was not scanned, rather than interpreting HTML examples as links, redirects or SPA scripts. The provider does not expose the target's final URL or status, so HTML metadata, automatic pagination and site discovery are not equivalent to raw HTML retrieval.
 - Nested Markdown blocks retain paragraph boundaries, list nesting, parent numbering and separate code/quote endings from following text. Attribute scans and the 100-level recursion limit remain bounded. Proxy retrieval decompresses supported HTTP encodings and checks the decoded byte budget even with older curl versions. Core runtime dependencies remain zero; fixed model input remains **1,159 description characters / 1,876 schema characters**.
 
+### v1.9.4 Maintenance validation (2026-11-15)
+
+- **435 unit assertions, 15 real Chromium SPA assertions and 70 adversarial probes** passed. The 22 new unit checks cover protocol-specific proxy selection, HTML-provider truncation hints and continuation after short prose prefixes. Core runtime dependencies remain zero and fixed tool input remains **1,159 description characters / 1,876 schema characters**.
+- Official npm channels report `latest=0.2.2-rc.1` and `next=0.2.2-rc.3`, both release candidates. The [November 13 release](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.2.2-rc.3) was audited using byte differences across eight contact packages, Cordis/Loader and four Desktop loading files. Community upgrade cards currently reach 0.2.2-alpha.3. The checked plugin interfaces remain compatible.
+- The installed Desktop is **0.2.2-rc.1 / Electron 44.7.0 / Node 24.21.0**. Its bundled runtime passed **17 service checks and 3 tool-scope checks**, alongside the 15 SPA assertions. Official 0.2.2-rc.3 passed **17 service checks, 9 DeepSeek provider checks and 5 HTTP-provider checks** with controlled responses. The published plugin patch also passed 4 loader and 2 temporary-profile checks. Full model conversations and Desktop UI interactions were outside these checks.
+- Clean prose uses a paragraph or sentence boundary in the latter half of the requested window, avoiding tiny heading-only responses. Windows containing raw HTML retain earlier alignment. Literal examples remain available through exact-offset continuation. The 1,000-character attribute limit remains deliberate, so some pages with long attributes still expose residual markup or return a short initial window.
+- The final 152-site sweep completed with **104 OK / 21 THIN / 4 EMPTY / 23 ERR / 0 THREW**, with zero noise in OK results. The three known NOISE pages matched v1.9.3 on identical input. The 23 errors comprised 7 HTTP errors, 3 expected content-type rejections and 13 connection/DNS failures. Continuation and the five-page crawl completed successfully.
+
 ## Roadmap
 
 - [x] Single-page continuation (`offset` parameter)

package/README.zh.md

@@ -204,7 +204,7 @@ chars 800+800/12398 · cached
       - https://raw.githubusercontent.com
 ```
 
-命中的来源立即直连，跳过宿主 web provider **及其 SSRF 检查**。仅添加你信任的来源；其 DNS/hosts 映射到本机或私有地址也会被允许。此选项不开放为工具参数。未列出的初始 URL 保留既有 provider 选择与回退行为；provider 拒绝不会自动启用此例外。
+命中的来源立即直连，跳过宿主 web provider 及其访问策略。仅添加你信任的来源，包括其 DNS/hosts 到本机或私有地址的映射。DSH 0.2.2 的 HTTP 提供方已支持本机和内网地址，其他提供方可采用不同访问策略。此选项由 profile 配置。未列出的初始 URL 保留既有 provider 选择与回退行为，provider 拒绝不会自动启用显式直连。
 
 按 URL 规范化后的协议、域名、端口精确匹配，子域名需单独配置。最多 32 项，每项不超过 2,048 字符。仅接受 HTTP(S) origin，可带一个结尾 `/`；凭据、路径、查询、片段、通配符和歧义写法都会触发配置错误。此选项与数字选项不同，非法项不会静默回退默认值。
 
@@ -350,6 +350,14 @@ v1.4.0 复验（2026-08-28，代理环境）：python-docs stdtypes.html 锚点
 - DeepSeek Fetch 提供方以 `body.kind=text` 返回 Markdown。单页读取保留原文；链接/站点工具明确提示未扫描 HTML 导航，避免把 HTML 示例当作链接、跳转或 SPA 脚本。该提供方不暴露目标最终 URL 和状态，因此 HTML 元数据、自动分页和站点发现不能视为与原始 HTML 抓取等价。
 - Markdown 嵌套块保留段落边界、列表层级、父级编号，代码块/引用与后文正确分隔；属性扫描和 100 层递归上限继续有界。代理读取解压受支持的 HTTP 编码，并对旧版 curl 同样执行解压后字节限额。核心仍为零运行时依赖，模型固定输入仍为 **description 1,159 字符 / schema 1,876 字符**。
 
+### v1.9.4 维护验证（2026-11-15）
+
+- **435 个单元断言、15 个真实 Chromium SPA 断言、70 项对抗探针**通过。新增 22 条单元检查覆盖分协议代理选择、HTML 提供方截断提示及短前言后的续读。核心零运行时依赖，工具固定输入仍为 **description 1,159 字符 / schema 1,876 字符**。
+- 官方 npm 频道为 `latest=0.2.2-rc.1`、`next=0.2.2-rc.3`，均为候选版本。[11 月 13 日发布的 rc.3](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.2.2-rc.3) 已通过八个接触包、Cordis/Loader 及四个桌面加载文件的字节差异审计。社区升级卡目前到 0.2.2-alpha.3，已核查的插件接口保持兼容。
+- 本机桌面为 **0.2.2-rc.1 / Electron 44.7.0 / Node 24.21.0**，内置运行时通过 **17 项服务集成、3 项工具作用域**检查，并通过上述 15 项 SPA 断言。官方 0.2.2-rc.3 通过 **17 项服务、9 项 DeepSeek 提供方和 5 项 HTTP 提供方**受控检查，已发布插件 patch 另通过 4 项加载器和 2 项临时 profile 检查。验证范围未覆盖完整模型会话和桌面 UI 交互。
+- 干净正文在请求窗口的后半段选择段落或句子边界，避免只返回极短标题。含原始 HTML 的窗口保留早期对齐，字面示例可按精确 offset 继续读取。属性长度继续限制为 1,000 字符，部分超长属性页面仍可能残留标签或返回较短的首段。
+- 最终 152 站为 **104 OK / 21 THIN / 4 EMPTY / 23 ERR / 0 THREW**，OK 站零噪声。三个已知 NOISE 页面在同一输入下与 v1.9.3 一致。23 个错误包括 7 个 HTTP 错误、3 个预期类型拒绝及 13 个连接/DNS 失败，续读和五页站点抓取完成。
+
 ## Roadmap
 
 - [x] 单页多段续读（`offset` 参数）

package/index.js

@@ -99,6 +99,7 @@ function allowedDirectUrl(url, origins) {
 
 const DIRECT_BOUNDARY_ERROR = 'Direct fetch stopped: target is outside directFetchOrigins or contains credentials'
 const DIRECT_SPA_HINT = '已配置直连仅支持静态读取，未启用浏览器渲染'
+const SOURCE_TRUNCATED_HINT = '提供方已截断原文，当前内容不是完整页面'
 
 function providerTextHint(page) {
   return '提供方返回纯文本，未扫描 HTML 链接或跳转' + (page.sourceTruncated ? '；提供方已截断原文，当前内容不是完整页面' : '')
@@ -1495,15 +1496,21 @@ export function smartTruncate(text, maxChars, offset = 0) {
   }
   let end = Math.min(total, start + maxChars)
   if (end < total && maxChars > 0) {
+    // Prefer useful prose boundaries; raw HTML permits earlier boundaries
+    // to avoid returning unparsed attributes.
+    const rawTag = /<\/?[a-z][a-z0-9:-]{0,63}(?=[\s/>])/i.test(text.slice(start, end))
+    const minEnd = start + (rawTag ? 1 : Math.ceil(maxChars / 2))
     const window = text.slice(start, end + 2)
     let boundary = window.lastIndexOf('\n\n', end - start)
     if (boundary >= 0) boundary += start
     while (boundary > start && text[boundary - 1] === '\n') boundary--
-    if (boundary > start) end = boundary
+    if (boundary >= minEnd) end = boundary
     else {
       // Prefer the last complete sentence inside the current window.
-      for (let i = end - 1; i >= start; i--) {
-        if ('.!?\n。！？'.includes(text[i])) {
+      for (let i = end - 1; i >= minEnd - 1; i--) {
+        const ch = text[i]
+        // Rejected paragraph separators are not sentence boundaries either.
+        if ('.!?。！？'.includes(ch) || (ch === '\n' && text[i - 1] !== '\n' && text[i + 1] !== '\n')) {
           end = i + 1
           break
         }
@@ -1974,7 +1981,7 @@ export async function readUrl(args, ctx, externalSignal, cfg = DEFAULTS) {
   if (externalSignal && externalSignal.aborted) return { error: 'cancelled' }
   if (payload.error) return failWith(payload.error)
   const resultUrl = payload.finalUrl || url
-  const truncatedHint = '提供方已截断原文，当前内容不是完整页面'
+  const truncatedHint = SOURCE_TRUNCATED_HINT
   let sourceHint = payload.sourceTruncated ? truncatedHint : ''
 
   if (payload.kind === 'text') {
@@ -2279,6 +2286,7 @@ function readLinksTool(ctx, cfg) {
       if (signal && signal.aborted) return { error: 'cancelled' }
       html = followed.html
       finalUrl = followed.finalUrl
+      let sourceTruncated = followed.sourceTruncated ?? page.sourceTruncated
       let links = extractLinks(html, limit, finalUrl)
       // SPA fallback: a JS-only page yields no links from its static HTML;
       // render it and re-extract when the static result looks empty. Any
@@ -2295,10 +2303,11 @@ function readLinksTool(ctx, cfg) {
           if (rl.length > links.length) {
             links = rl
             finalUrl = rr.finalUrl || finalUrl
+            sourceTruncated = false
           }
         }
       }
-      const spaHint = [route.hint, route.restricted && needsRender ? DIRECT_SPA_HINT : ''].filter(Boolean).join('；')
+      const spaHint = [sourceTruncated ? SOURCE_TRUNCATED_HINT : '', route.hint, route.restricted && needsRender ? DIRECT_SPA_HINT : ''].filter(Boolean).join('；')
       return { url: finalUrl, count: links.length, links, ...(spaHint ? { spaHint } : {}) }
     },
   }
@@ -2512,12 +2521,14 @@ async function crawlSite(entryUrl, cfg, opts, externalSignal, ctx) {
         if (externalSignal && externalSignal.aborted) return null
         html = followed.html
         pageUrl = followed.finalUrl
+        const sourceTruncated = followed.sourceTruncated ?? page.sourceTruncated
         if (depth === 0) host = hostOf(pageUrl)
         else if (!sameHost(pageUrl, host)) {
           return { url, depth, error: `Redirected outside the site: ${pageUrl}` }
         }
         const ex = extract(html, 'text')
         const links = extractLinks(html, 100, pageUrl)
+        const spaHint = [sourceTruncated ? SOURCE_TRUNCATED_HINT : '', route.hint].filter(Boolean).join('；')
         return {
           url: pageUrl,
           depth,
@@ -2526,7 +2537,7 @@ async function crawlSite(entryUrl, cfg, opts, externalSignal, ctx) {
           text: includeContent ? smartTruncate(ex.text, perMax).text : undefined,
           links,
           route,
-          ...(route.hint ? { spaHint: route.hint } : {}),
+          ...(spaHint ? { spaHint } : {}),
         }
       }),
     )

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dsh-read-url",
-  "version": "1.9.3",
+  "version": "1.9.4",
   "description": "DeepSeek Harness URL reader: fetch any webpage (HTML/JSON/RSS/Atom), auto-detect encoding (UTF-16 BOM/GBK/Big5/Shift-JIS), extract clean main content with auto-pagination, output compact text/Markdown to save tokens. Zero runtime dependencies, no API key, drop-in. Follows meta-refresh shells, reads JSON-LD metadata, honors base href.",
   "type": "module",
   "main": "index.js",

package/proxy-fallback.js

@@ -92,9 +92,8 @@ function readWindowsSystemProxy() {
     )
     const [enable, server] = out.trim().split(/\r?\n/).map((s) => s.trim())
     if (enable !== '1' || !server) return sysProxyCache
-    // ProxyServer may be "host:port" or "http=...;https=host:port"
-    const httpsPart = server.split(';').map((s) => s.trim()).find((s) => /^https=/i.test(s))
-    sysProxyCache = (httpsPart ? httpsPart.replace(/^https=/i, '') : server) || ''
+    // Cache the complete map so HTTP and HTTPS share discovery, not selection.
+    sysProxyCache = server
     return sysProxyCache
   } catch {
     sysProxyCache = ''
@@ -168,13 +167,24 @@ function bypassesProxy(url) {
 // The user's proxy, in priority order: env vars, then Windows system proxy.
 export async function detectProxy(url) {
   if (bypassesProxy(url)) return ''
-  const env =
-    process.env.HTTPS_PROXY ||
-    process.env.https_proxy ||
-    process.env.HTTP_PROXY ||
-    process.env.http_proxy
+  let protocol = ''
+  try { protocol = new URL(url).protocol.slice(0, -1) } catch {}
+  const httpProxy = process.env.HTTP_PROXY || process.env.http_proxy
+  const httpsProxy = process.env.HTTPS_PROXY || process.env.https_proxy
+  // Prefer the requested protocol; retain a sole configured proxy as fallback.
+  const env = protocol === 'http' ? httpProxy || httpsProxy : httpsProxy || httpProxy
   if (env) return env
-  if (process.platform === 'win32') return readWindowsSystemProxy()
+  if (process.platform === 'win32') {
+    const server = readWindowsSystemProxy()
+    const entries = server.split(';').map(entry => entry.trim())
+    if (!entries.some(entry => /^[a-z]{1,16}\s{0,16}=/i.test(entry))) return server
+    const proxies = new Map()
+    for (const entry of entries) {
+      const equal = entry.indexOf('=')
+      if (equal > 0) proxies.set(entry.slice(0, equal).trim().toLowerCase(), entry.slice(equal + 1).trim())
+    }
+    return protocol ? proxies.get(protocol) || '' : proxies.get('https') || proxies.get('http') || ''
+  }
   return ''
 }