dsh-read-url 1.9.1 → 1.9.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +17 -0
- package/README.md +17 -1
- package/README.zh.md +17 -1
- package/index.js +41 -20
- package/package.json +1 -1
- package/proxy-fallback.js +19 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,22 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## [1.9.2] - 2026-10-03
|
|
4
|
+
|
|
5
|
+
### 修复
|
|
6
|
+
|
|
7
|
+
- Windows 代理回退优先使用系统目录中的 PowerShell/curl,避免桌面进程的精简 PATH 导致代理发现或请求失败;系统工具不可用时保留原有回退与取消行为。
|
|
8
|
+
- JSON-LD 按真实 `type` 属性读取,支持等号空格、无引号值及引号内的 `>`;忽略 `data-type`、注释、普通脚本及 raw-text 元素中的伪 JSON-LD。
|
|
9
|
+
- JSON-LD 作者解析限制递归深度,正文数组仅拼接字符串项,避免深层数组触发栈溢出或非文本值污染正文;保留标签长度、JSON 大小及递归上限。
|
|
10
|
+
- 双语安装说明补充桌面版插件入口、独立 profile、内置运行时及本地链接代码重启加载方式。
|
|
11
|
+
|
|
12
|
+
### 验证
|
|
13
|
+
|
|
14
|
+
- 375 个零依赖单元断言、15 个真实 Chromium SPA 断言、64 项对抗探针通过;新增 23 条单元检查和 5 条 JSON-LD 探针。JSON-LD 原始缺陷在 v1.9.0/v1.9.1 同输入均复现;Windows 空 PATH 的真实代理请求对照 v1.9.1 确认修复。
|
|
15
|
+
- DSH 桌面版 0.2.0-rc.2 的实际 Electron 44.0.0 / Node 24.18.1 通过 17 项官方服务集成、3 项工具作用域、2 项本地代理 A/B 及 15 项 Chromium SPA 检查。极简 preset 继承全局插件工具;服务响应使用受控数据,未覆盖完整模型会话。
|
|
16
|
+
- DSH 0.1.7-rc.2 → 0.2.0-rc.2 的 8 个接触包与 4 个桌面加载链文件完成字节差异审计,并交叉核对社区 0.2.0-rc.1 升级卡;已核查接口无需适配修改。0.2.1-alpha.1 未审计。
|
|
17
|
+
- 2026-10-03 的 152 站回归:104 OK / 20 THIN / 4 EMPTY / 24 ERR / 0 THREW,OK 站零噪声。已知 NOISE 与状态波动站使用相同 HTML/DOM 和可选依赖对照 v1.9.1,全文续读与 800 字输出一致;新增订阅站 HTTP 403 在新旧版本均复现。
|
|
18
|
+
- 核心仍为零运行时依赖;四工具 description/schema 为 1,159/1,876 字符。
|
|
19
|
+
|
|
3
20
|
## [1.9.1] - 2026-09-28
|
|
4
21
|
|
|
5
22
|
### 修复
|
package/README.md
CHANGED
|
@@ -55,8 +55,16 @@ Implemented per official docs (`docs/capability-seams.md`, `docs/cordis-primer.m
|
|
|
55
55
|
|
|
56
56
|
## Install
|
|
57
57
|
|
|
58
|
+
### DSH Desktop
|
|
59
|
+
|
|
60
|
+
Open **Plugins → Add plugin**, enter `dsh-read-url` (npm package), the GitHub repository URL, or the local project directory, and install it. In the plugin details, check the displayed version and that the component is **running**.
|
|
61
|
+
|
|
62
|
+
Desktop uses its own `desktop` profile and bundled DSH runtime. CLI profile installations and global CLI upgrades are separate from Desktop. When configuring this plugin manually, use the Desktop profile's `cordis.patch.yml`. A local directory installation links to that source directory; restart Desktop after changing its code to load a fresh module.
|
|
63
|
+
|
|
64
|
+
### CLI / local development
|
|
65
|
+
|
|
58
66
|
```bash
|
|
59
|
-
# From GitHub
|
|
67
|
+
# From GitHub in the web profile
|
|
60
68
|
npx @deepseek-ai/dsh plugin --profile web add github:2672243194/dsh-read-url
|
|
61
69
|
|
|
62
70
|
# Local development
|
|
@@ -323,6 +331,14 @@ Plus a **15-item DSH acceptance round** (a real agent driving every read_url too
|
|
|
323
331
|
- [DSH 0.1.7-rc.2](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.7-rc.2) compatibility was checked against official npm package byte differences and community upgrade notes (available through rc.1). Both published v1.9.0 and this maintenance code passed **17 actual official-service integration checks each**; the published patch passed **4 loader checks** covering loading, execution, configuration remount and removal. Web-provider/runtime contracts remain compatible; no adaptation change is needed for the checked interfaces. These checks used controlled provider responses, not a complete host UI/model session.
|
|
324
332
|
- Core runtime dependencies remain zero. Model input stays at **1,159 description characters / 1,876 schema characters**; site body summaries remain opt-in and respect `maxCharsPerPage` without an extra 80-character render cutoff.
|
|
325
333
|
|
|
334
|
+
### v1.9.2 Desktop maintenance validation (2026-10-03)
|
|
335
|
+
|
|
336
|
+
- **375 zero-dependency unit assertions, 15 real Chromium SPA assertions and 64 adversarial probes** passed. Coverage includes Windows proxy fallback with a minimal PATH, real JSON-LD attributes, comment/raw-text isolation, bounded scans and deeply nested data.
|
|
337
|
+
- The installed **DSH Desktop 0.2.0-rc.2** runtime (Electron 44.0.0 / Node 24.18.1) passed **17 official-service integration checks, 3 tool-scope checks, 2 local proxy A/B checks and 15 Chromium SPA checks**. The minimal preset inherits the four globally registered plugin tools. Service responses were controlled fixtures; these checks do not cover a full model conversation.
|
|
338
|
+
- [DSH 0.2.0-rc.2](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.2.0-rc.2) was audited against 0.1.7-rc.2 using byte differences across eight contact packages and four Desktop loading files, with the [community 0.2.0-rc.1 upgrade card](https://github.com/oh-my-dsh/dsh-plugin-upgrade-skill/blob/main/skills/plugin-upgrade/references/v0.2.0-rc.1.md) as a cross-check. The verified interfaces require no adaptation. The newer 0.2.1-alpha.1 was not audited.
|
|
339
|
+
- The 152-site sweep returned **104 OK / 20 THIN / 4 EMPTY / 24 ERR / 0 THREW**, with no noise in OK results. Known NOISE and fluctuating sites matched v1.9.1 on identical HTML/DOM and optional dependencies, including full continuation and 800-character output. The newly failing feed returned HTTP 403 in both versions.
|
|
340
|
+
- JSON-LD metadata/body extraction accepts valid attribute variants and rejects lookalike attributes. Author recursion is bounded; body arrays retain only strings. Core runtime dependencies remain zero, with fixed model input of **1,159 description characters / 1,876 schema characters**.
|
|
341
|
+
|
|
326
342
|
## Roadmap
|
|
327
343
|
|
|
328
344
|
- [x] Single-page continuation (`offset` parameter)
|
package/README.zh.md
CHANGED
|
@@ -55,8 +55,16 @@ DSH 的 Agent 能搜索(返回链接和片段),但缺"把 URL 读成干净
|
|
|
55
55
|
|
|
56
56
|
## 安装
|
|
57
57
|
|
|
58
|
+
### DSH 桌面版
|
|
59
|
+
|
|
60
|
+
打开 **插件 → 添加插件**,输入 `dsh-read-url`(npm 包名)、GitHub 仓库地址或本地项目目录后安装。在插件详情检查版本,以及组件是否显示 **运行中**。
|
|
61
|
+
|
|
62
|
+
桌面版使用独立的 `desktop` profile 和内置 DSH 运行时。CLI 各 profile 的安装及全局 CLI 升级与桌面版分别管理。手动配置本插件时,应使用桌面 profile 的 `cordis.patch.yml`。本地目录安装会链接源目录,修改代码后应重启桌面版,以加载新的模块。
|
|
63
|
+
|
|
64
|
+
### CLI / 本地开发
|
|
65
|
+
|
|
58
66
|
```bash
|
|
59
|
-
# 从 GitHub
|
|
67
|
+
# 从 GitHub 安装到 web profile
|
|
60
68
|
npx @deepseek-ai/dsh plugin --profile web add github:2672243194/dsh-read-url
|
|
61
69
|
|
|
62
70
|
# 本地开发
|
|
@@ -323,6 +331,14 @@ v1.4.0 复验(2026-08-28,代理环境):python-docs stdtypes.html 锚点
|
|
|
323
331
|
- [DSH 0.1.7-rc.2](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.1.7-rc.2) 已完成官方 npm 接触包字节差异核查,并与社区升级卡交叉验证(卡片目前到 rc.1)。发布版 v1.9.0 与当前维护代码各通过 **17 项真实官方服务集成检查**;发布版 patch 另通过 **4 项加载器检查**,覆盖加载、调用、配置重挂和卸载。Web provider/runtime 契约兼容,已核查接口无需适配修改;测试使用受控 provider 响应,未覆盖完整宿主 UI/模型会话。
|
|
324
332
|
- 核心仍为零运行时依赖,模型固定输入仍为 **description 1,159 字符 / schema 1,876 字符**;站点正文摘要仍默认关闭,启用后遵循 `maxCharsPerPage`,不再额外截成 80 字符。
|
|
325
333
|
|
|
334
|
+
### v1.9.2 桌面版维护验证(2026-10-03)
|
|
335
|
+
|
|
336
|
+
- **375 个零依赖单元断言、15 个真实 Chromium SPA 断言、64 项对抗探针**通过。覆盖 Windows 精简 PATH 下的代理回退、JSON-LD 真实属性、注释/raw-text 隔离、有界扫描与深层数据。
|
|
337
|
+
- 已安装的 **DSH 桌面版 0.2.0-rc.2** 运行时(Electron 44.0.0 / Node 24.18.1)通过 **17 项官方服务集成、3 项工具作用域、2 项本地代理 A/B 及 15 项 Chromium SPA 检查**。极简 preset 继承全局注册的四个插件工具;服务响应使用受控数据,未覆盖完整模型会话。
|
|
338
|
+
- [DSH 0.2.0-rc.2](https://github.com/deepseek-ai/deepseek-harness/releases/tag/dsh-v0.2.0-rc.2) 已对照 0.1.7-rc.2 完成八个接触包与四个桌面加载链文件的字节差异审计,并交叉核对[社区 0.2.0-rc.1 升级卡](https://github.com/oh-my-dsh/dsh-plugin-upgrade-skill/blob/main/skills/plugin-upgrade/references/v0.2.0-rc.1.md)。已核查接口无需适配修改;更新的 0.2.1-alpha.1 未审计。
|
|
339
|
+
- 152 站结果为 **104 OK / 20 THIN / 4 EMPTY / 24 ERR / 0 THREW**,OK 站零噪声。已知 NOISE 与状态波动站使用相同 HTML/DOM 和可选依赖对照 v1.9.1,全文续读及 800 字输出一致;新增失败的订阅站在新旧版本均返回 HTTP 403。
|
|
340
|
+
- JSON-LD 元数据/正文支持合法属性变体并排除伪属性;作者递归有界,正文数组仅保留字符串。核心仍为零运行时依赖,模型固定输入为 **description 1,159 字符 / schema 1,876 字符**。
|
|
341
|
+
|
|
326
342
|
## Roadmap
|
|
327
343
|
|
|
328
344
|
- [x] 单页多段续读(`offset` 参数)
|
package/index.js
CHANGED
|
@@ -1116,7 +1116,39 @@ function extractMeta(html) {
|
|
|
1116
1116
|
// Bounded block size keeps a hostile giant script tag from dragging the scan.
|
|
1117
1117
|
// 100k covers real article-list JSON-LD (news sites often embed dozens of
|
|
1118
1118
|
// items); metadata sits in the first node anyway.
|
|
1119
|
-
|
|
1119
|
+
function* jsonLdBlocks(html) {
|
|
1120
|
+
const open = /<!--|<([a-z][a-z0-9:-]{0,63})(?=[\s>])/gi
|
|
1121
|
+
const rawNames = new Set(['script', 'style', 'textarea', 'title', 'xmp', 'iframe', 'noembed', 'noframes', 'noscript', 'plaintext'])
|
|
1122
|
+
let match
|
|
1123
|
+
while ((match = open.exec(html))) {
|
|
1124
|
+
if (!match[1]) {
|
|
1125
|
+
const end = html.indexOf('-->', open.lastIndex)
|
|
1126
|
+
if (end < 0) break
|
|
1127
|
+
open.lastIndex = end + 3
|
|
1128
|
+
continue
|
|
1129
|
+
}
|
|
1130
|
+
const tag = navigationTags(html.slice(match.index, match.index + 1002), /^<[a-z][a-z0-9:-]{0,63}(?=[\s>])/gi).next().value
|
|
1131
|
+
if (!tag) continue
|
|
1132
|
+
const end = match.index + tag.end
|
|
1133
|
+
open.lastIndex = end
|
|
1134
|
+
const name = match[1].toLowerCase()
|
|
1135
|
+
if (!rawNames.has(name)) continue
|
|
1136
|
+
if (name === 'plaintext') break
|
|
1137
|
+
const close = new RegExp(`</${name}\\s{0,1000}>`, 'gi')
|
|
1138
|
+
close.lastIndex = end
|
|
1139
|
+
const closing = close.exec(html)
|
|
1140
|
+
if (!closing) break
|
|
1141
|
+
// Raw-text bodies and quoted attributes cannot introduce script elements.
|
|
1142
|
+
open.lastIndex = close.lastIndex
|
|
1143
|
+
if (name !== 'script' || tag.attrs.type?.trim().toLowerCase() !== 'application/ld+json' || closing.index - end > 100000) continue
|
|
1144
|
+
try {
|
|
1145
|
+
yield JSON.parse(html.slice(end, closing.index))
|
|
1146
|
+
} catch {
|
|
1147
|
+
// Invalid data blocks do not prevent reading later JSON-LD blocks.
|
|
1148
|
+
}
|
|
1149
|
+
}
|
|
1150
|
+
}
|
|
1151
|
+
|
|
1120
1152
|
function jsonLdMeta(html) {
|
|
1121
1153
|
// Recursively collect every node: nested structures are common (ItemList →
|
|
1122
1154
|
// mainEntity → Article, WebPage → mainEntityOfPage), so a flat top-level
|
|
@@ -1136,20 +1168,15 @@ function jsonLdMeta(html) {
|
|
|
1136
1168
|
if (v && typeof v === 'object') collect(v, out, depth + 1)
|
|
1137
1169
|
}
|
|
1138
1170
|
}
|
|
1139
|
-
const authorOf = (a) => {
|
|
1171
|
+
const authorOf = (a, depth = 0) => {
|
|
1172
|
+
if (depth > 20) return ''
|
|
1140
1173
|
if (typeof a === 'string') return a
|
|
1141
|
-
if (Array.isArray(a)) return authorOf(a[0])
|
|
1142
|
-
if (a && typeof a === 'object') return typeof a.name === 'string' ? a.name : authorOf(a.name)
|
|
1174
|
+
if (Array.isArray(a)) return authorOf(a[0], depth + 1)
|
|
1175
|
+
if (a && typeof a === 'object') return typeof a.name === 'string' ? a.name : authorOf(a.name, depth + 1)
|
|
1143
1176
|
return ''
|
|
1144
1177
|
}
|
|
1145
1178
|
const clean = (s) => (s ? decodeTextEntities(s).slice(0, 40) : '')
|
|
1146
|
-
for (const
|
|
1147
|
-
let json
|
|
1148
|
-
try {
|
|
1149
|
-
json = JSON.parse(m[1])
|
|
1150
|
-
} catch {
|
|
1151
|
-
continue
|
|
1152
|
-
}
|
|
1179
|
+
for (const json of jsonLdBlocks(html)) {
|
|
1153
1180
|
const nodes = []
|
|
1154
1181
|
collect(json, nodes)
|
|
1155
1182
|
for (const n of nodes) {
|
|
@@ -1165,15 +1192,9 @@ function jsonLdMeta(html) {
|
|
|
1165
1192
|
// (articleBody) while the visible HTML is an anti-bot shell or a thin
|
|
1166
1193
|
// fragment — a fallback body source the DOM extractors cannot reach. Returns
|
|
1167
1194
|
// the first string articleBody found (array values joined), stripped to
|
|
1168
|
-
// paragraphs via textLines.
|
|
1195
|
+
// paragraphs via textLines. Metadata and body share the bounded block scan.
|
|
1169
1196
|
function jsonLdArticleBody(html) {
|
|
1170
|
-
for (const
|
|
1171
|
-
let json
|
|
1172
|
-
try {
|
|
1173
|
-
json = JSON.parse(m[1])
|
|
1174
|
-
} catch {
|
|
1175
|
-
continue
|
|
1176
|
-
}
|
|
1197
|
+
for (const json of jsonLdBlocks(html)) {
|
|
1177
1198
|
const nodes = []
|
|
1178
1199
|
const collect = (node, depth = 0) => {
|
|
1179
1200
|
if (!node || typeof node !== 'object' || depth > 20) return
|
|
@@ -1189,7 +1210,7 @@ function jsonLdArticleBody(html) {
|
|
|
1189
1210
|
}
|
|
1190
1211
|
collect(json)
|
|
1191
1212
|
for (const n of nodes) {
|
|
1192
|
-
const raw = Array.isArray(n.articleBody) ? n.articleBody.join('\n\n') : n.articleBody
|
|
1213
|
+
const raw = Array.isArray(n.articleBody) ? n.articleBody.filter(part => typeof part === 'string').join('\n\n') : n.articleBody
|
|
1193
1214
|
if (typeof raw === 'string' && raw.trim()) return textLines(raw).slice(0, 200000)
|
|
1194
1215
|
}
|
|
1195
1216
|
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-read-url",
|
|
3
|
-
"version": "1.9.
|
|
3
|
+
"version": "1.9.2",
|
|
4
4
|
"description": "DeepSeek Harness URL reader: fetch any webpage (HTML/JSON/RSS/Atom), auto-detect encoding (UTF-16 BOM/GBK/Big5/Shift-JIS), extract clean main content with auto-pagination, output compact text/Markdown to save tokens. Zero runtime dependencies, no API key, drop-in. Follows meta-refresh shells, reads JSON-LD metadata, honors base href.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "index.js",
|
package/proxy-fallback.js
CHANGED
|
@@ -14,6 +14,23 @@
|
|
|
14
14
|
// caller reports the ORIGINAL direct-connect error with a clear note, so the
|
|
15
15
|
// model/user can act (turn the proxy on, or accept the network boundary).
|
|
16
16
|
import { execFile, execFileSync } from 'node:child_process'
|
|
17
|
+
import { existsSync } from 'node:fs'
|
|
18
|
+
import { win32 } from 'node:path'
|
|
19
|
+
|
|
20
|
+
// Desktop launches can inherit a reduced PATH. Prefer the Windows-shipped
|
|
21
|
+
// utilities when present; other platforms and custom installations use PATH.
|
|
22
|
+
function systemExecutable(name) {
|
|
23
|
+
if (process.platform !== 'win32') return name
|
|
24
|
+
const parts = name === 'powershell'
|
|
25
|
+
? ['System32', 'WindowsPowerShell', 'v1.0', 'powershell.exe']
|
|
26
|
+
: ['System32', 'curl.exe']
|
|
27
|
+
for (const root of [process.env.SystemRoot, process.env.WINDIR]) {
|
|
28
|
+
if (typeof root !== 'string' || !win32.isAbsolute(root)) continue
|
|
29
|
+
const executable = win32.join(root, ...parts)
|
|
30
|
+
if (existsSync(executable)) return executable
|
|
31
|
+
}
|
|
32
|
+
return name
|
|
33
|
+
}
|
|
17
34
|
|
|
18
35
|
// Some servers omit the Content-Type header entirely; the content-type gate
|
|
19
36
|
// then has nothing to test and a binary body (PDF/PNG/zip) would flow into
|
|
@@ -71,7 +88,7 @@ function readWindowsSystemProxy() {
|
|
|
71
88
|
sysProxyCheckedAt = now
|
|
72
89
|
try {
|
|
73
90
|
const out = execFileSync(
|
|
74
|
-
'powershell',
|
|
91
|
+
systemExecutable('powershell'),
|
|
75
92
|
[
|
|
76
93
|
'-NoProfile', '-Command',
|
|
77
94
|
"(Get-ItemProperty 'HKCU:\\Software\\Microsoft\\Windows\\CurrentVersion\\Internet Settings').ProxyEnable, (Get-ItemProperty 'HKCU:\\Software\\Microsoft\\Windows\\CurrentVersion\\Internet Settings').ProxyServer",
|
|
@@ -127,7 +144,7 @@ export async function fetchViaCurlProxy(url, cfg, externalSignal, proxyOverride)
|
|
|
127
144
|
]
|
|
128
145
|
try {
|
|
129
146
|
const buf = await new Promise((resolve, reject) => {
|
|
130
|
-
execFile('curl', args, { maxBuffer: cfg.maxBytes + 8192, encoding: 'buffer', windowsHide: true, signal: externalSignal || undefined }, (err, stdout) => {
|
|
147
|
+
execFile(systemExecutable('curl'), args, { maxBuffer: cfg.maxBytes + 8192, encoding: 'buffer', windowsHide: true, signal: externalSignal || undefined }, (err, stdout) => {
|
|
131
148
|
if (err) return reject(err)
|
|
132
149
|
resolve(stdout)
|
|
133
150
|
})
|