dsh-rss 0.2.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +20 -1
- package/README.md +21 -3
- package/lib/config.d.ts +4 -0
- package/lib/config.js +48 -16
- package/lib/execution.d.ts +11 -0
- package/lib/execution.js +1 -0
- package/lib/index.d.ts +20 -6
- package/lib/index.js +21 -17
- package/lib/network.d.ts +20 -0
- package/lib/network.js +232 -0
- package/lib/settings.d.ts +3 -0
- package/lib/settings.js +1 -0
- package/lib/tools.d.ts +20 -10
- package/lib/tools.js +290 -70
- package/lib/workspace.d.ts +6 -0
- package/lib/workspace.js +42 -0
- package/package.json +65 -66
package/README.en.md
CHANGED
|
@@ -1,9 +1,19 @@
|
|
|
1
1
|
# dsh-rss
|
|
2
2
|
|
|
3
|
+
   
|
|
4
|
+
|
|
3
5
|
[](https://awesome-dsh-plugin.com)
|
|
4
6
|
|
|
5
7
|
DSH (DeepSeek Harness) plugin for RSS/Atom subscriptions: manage feeds, fetch and parse RSS 0.9x / 1.0 / 2.0 and Atom, with OPML bulk import/export, exposing seven model-facing tools.
|
|
6
8
|
|
|
9
|
+
## Compatibility
|
|
10
|
+
|
|
11
|
+
Adapted to the tool execution contracts in `@deepseek-ai/dsh@0.1.3-alpha.1`. Built for the cordis patch-bundle plugin model (`cordis.patch.yml` + `dsh.bundle.patch`). No runtime imports of `@deepseek-ai/*` internals.
|
|
12
|
+
|
|
13
|
+
Fetches honor the Harness cancellation signal through DNS, response streaming, and cross-feed searches. Hostname preflight checks reject loopback, private, and link-local addresses by default, including redirect destinations. Set `allowPrivateNetwork: true` for trusted internal feeds.
|
|
14
|
+
|
|
15
|
+
`rss_opml_export.path` is relative to the calling session's workspace, with an existing parent directory. Absolute paths, traversal, and links outside the workspace are rejected; the result returns the absolute written path. File writes request Harness approval by default (`opmlWriteApproval: false` disables this gate). Omit `path` to return only OPML text.
|
|
16
|
+
|
|
7
17
|
## Installation
|
|
8
18
|
|
|
9
19
|
```bash
|
|
@@ -12,6 +22,15 @@ dsh plugin --profile web add dsh-rss
|
|
|
12
22
|
|
|
13
23
|
Restart the web service after installing.
|
|
14
24
|
|
|
25
|
+
## Uninstall
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
dsh plugin --profile web remove dsh-rss
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Then restart the web service. To clean up fully, also remove the plugin entry from your profile `cordis.patch.yml` if you overrode it.
|
|
32
|
+
|
|
33
|
+
|
|
15
34
|
## Configuration
|
|
16
35
|
|
|
17
36
|
Override the plugin row in your profile's `cordis.patch.yml` (the plugin also loads with all defaults when absent):
|
|
@@ -73,7 +92,7 @@ Most feeds are reachable directly; a few require a special proxy from your netwo
|
|
|
73
92
|
|
|
74
93
|
```bash
|
|
75
94
|
pnpm install
|
|
76
|
-
pnpm test #
|
|
95
|
+
pnpm test # build + offline tests with explicit fetch and DNS fixtures
|
|
77
96
|
```
|
|
78
97
|
|
|
79
98
|
## License
|
package/README.md
CHANGED
|
@@ -9,7 +9,15 @@
|
|
|
9
9
|
[](https://awesome-dsh-plugin.com)
|
|
10
10
|
|
|
11
11
|
|
|
12
|
-
DSH(DeepSeek Harness)的 RSS/Atom 订阅工具插件:管理订阅源,抓取并解析 RSS 0.9x / 1.0 / 2.0 与 Atom,支持 OPML
|
|
12
|
+
DSH(DeepSeek Harness)的 RSS/Atom 订阅工具插件:管理订阅源,抓取并解析 RSS 0.9x / 1.0 / 2.0 与 Atom,支持 OPML 批量导入导出,给模型提供九个可直接调用的工具。
|
|
13
|
+
|
|
14
|
+
## 兼容性
|
|
15
|
+
|
|
16
|
+
按 `@deepseek-ai/dsh@0.1.3-alpha.1` 的工具执行接口适配。遵循 cordis 组合包补丁模型(`cordis.patch.yml` + `dsh.bundle.patch`),运行时不 import 任何 `@deepseek-ai/*` 内部模块。
|
|
17
|
+
|
|
18
|
+
抓取接收 Harness 的取消信号,取消 DNS、网络读取或跨源搜索时保留原始取消原因。默认预检域名解析结果并拒绝回环、私网和链路本地地址;每次重定向都重新校验。可信内网源可显式配置 `allowPrivateNetwork: true`。
|
|
19
|
+
|
|
20
|
+
`rss_opml_export.path` 必须相对本次调用的会话工作区,父目录须已存在;工具拒绝绝对路径、目录穿越和指向工作区外的链接,并返回实际写入文件的绝对路径。文件写入默认通过 Harness 审批门,`opmlWriteApproval: false` 可关闭该门;省略 `path` 时仅返回 OPML 文本。
|
|
13
21
|
|
|
14
22
|
## 安装
|
|
15
23
|
|
|
@@ -19,6 +27,15 @@ dsh plugin --profile web add dsh-rss
|
|
|
19
27
|
|
|
20
28
|
安装后重启 Web 服务即可。
|
|
21
29
|
|
|
30
|
+
## 卸载
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
dsh plugin --profile web remove dsh-rss
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
卸载后重启 Web 服务。如需彻底清理,可再手动删除自己 profile `cordis.patch.yml` 中覆盖的插件行。
|
|
37
|
+
|
|
38
|
+
|
|
22
39
|
## 配置
|
|
23
40
|
|
|
24
41
|
在你自己的 profile 的 `cordis.patch.yml` 里覆盖本插件行(缺省时插件也能加载,只是全部用默认值):
|
|
@@ -48,6 +65,8 @@ dsh plugin --profile web add dsh-rss
|
|
|
48
65
|
| `rss_check` | 校验地址是否为可解析的订阅源 | `url` 必填 |
|
|
49
66
|
| `rss_opml_export` | 导出订阅列表为 OPML 2.0 文本(可写入文件) | `path` 可选 |
|
|
50
67
|
| `rss_opml_import` | 从 OPML 2.0 文本批量导入订阅源 | `opml` 必填 |
|
|
68
|
+
| `rss_search` | 跨订阅源关键词搜索条目(标题/摘要/正文/作者) | `query` 必填;`name`/`url`/`since`/`limit` 可选 |
|
|
69
|
+
| `rss_health` | 插件自检(配置/订阅/游标/解析器,离线) | 无 |
|
|
51
70
|
|
|
52
71
|
### 示例
|
|
53
72
|
|
|
@@ -80,7 +99,7 @@ rss_opml_import { opml: "<?xml version=\"1.0\"?>..." }
|
|
|
80
99
|
|
|
81
100
|
```bash
|
|
82
101
|
pnpm install
|
|
83
|
-
pnpm test # 构建 +
|
|
102
|
+
pnpm test # 构建 + 离线测试(fetch 与 DNS 均使用夹具)
|
|
84
103
|
```
|
|
85
104
|
|
|
86
105
|
发布前门禁:危险模式扫描、manifest 自检、`pnpm audit --prod`、全量测试,以及全新 profile 的真实启动冒烟测试。
|
|
@@ -88,4 +107,3 @@ pnpm test # 构建 + 55 个测试
|
|
|
88
107
|
## License
|
|
89
108
|
|
|
90
109
|
MIT
|
|
91
|
-
|
package/lib/config.d.ts
CHANGED
|
@@ -11,6 +11,8 @@ export interface RssConfig {
|
|
|
11
11
|
maxBodyBytes?: number;
|
|
12
12
|
userAgent?: string;
|
|
13
13
|
feedsYaml?: string;
|
|
14
|
+
allowPrivateNetwork?: boolean;
|
|
15
|
+
opmlWriteApproval?: boolean;
|
|
14
16
|
}
|
|
15
17
|
/** 解析后的配置:所有字段都有值。 */
|
|
16
18
|
export interface ResolvedRssConfig {
|
|
@@ -19,6 +21,8 @@ export interface ResolvedRssConfig {
|
|
|
19
21
|
maxBodyBytes: number;
|
|
20
22
|
userAgent: string;
|
|
21
23
|
feedsYaml: string;
|
|
24
|
+
allowPrivateNetwork: boolean;
|
|
25
|
+
opmlWriteApproval: boolean;
|
|
22
26
|
}
|
|
23
27
|
/**
|
|
24
28
|
* 解析并校验配置。
|
package/lib/config.js
CHANGED
|
@@ -6,9 +6,30 @@
|
|
|
6
6
|
*/
|
|
7
7
|
const DEFAULT_TIMEOUT_MS = 15000;
|
|
8
8
|
const DEFAULT_MAX_BODY_BYTES = 5 * 1024 * 1024;
|
|
9
|
-
const DEFAULT_USER_AGENT = 'dsh-rss/0.
|
|
10
|
-
function
|
|
11
|
-
|
|
9
|
+
const DEFAULT_USER_AGENT = 'dsh-rss/0.3.1 (DeepSeek Harness RSS plugin)';
|
|
10
|
+
function configRecord(config) {
|
|
11
|
+
if (config === undefined || config === null)
|
|
12
|
+
return {};
|
|
13
|
+
if (typeof config !== 'object' || Array.isArray(config)) {
|
|
14
|
+
throw new Error('dsh-rss config 必须是对象。');
|
|
15
|
+
}
|
|
16
|
+
return config;
|
|
17
|
+
}
|
|
18
|
+
function optionalString(config, key) {
|
|
19
|
+
const value = config[key];
|
|
20
|
+
if (value === undefined)
|
|
21
|
+
return undefined;
|
|
22
|
+
if (typeof value !== 'string')
|
|
23
|
+
throw new Error(key + ' 必须是字符串。');
|
|
24
|
+
return value;
|
|
25
|
+
}
|
|
26
|
+
function optionalBoolean(config, key, fallback) {
|
|
27
|
+
const value = config[key];
|
|
28
|
+
if (value === undefined)
|
|
29
|
+
return fallback;
|
|
30
|
+
if (typeof value !== 'boolean')
|
|
31
|
+
throw new Error(key + ' 必须是 true 或 false。');
|
|
32
|
+
return value;
|
|
12
33
|
}
|
|
13
34
|
/**
|
|
14
35
|
* 解析并校验配置。
|
|
@@ -16,26 +37,37 @@ function optionalString(value) {
|
|
|
16
37
|
* @throws 配置值非法时抛出中文错误。
|
|
17
38
|
*/
|
|
18
39
|
export function resolveConfig(config) {
|
|
19
|
-
const cfg = config
|
|
20
|
-
const proxyUrl = (optionalString(cfg
|
|
21
|
-
if (proxyUrl !== ''
|
|
22
|
-
|
|
40
|
+
const cfg = configRecord(config);
|
|
41
|
+
const proxyUrl = (optionalString(cfg, 'proxyUrl') ?? '').trim();
|
|
42
|
+
if (proxyUrl !== '') {
|
|
43
|
+
try {
|
|
44
|
+
const parsed = new URL(proxyUrl);
|
|
45
|
+
if ((parsed.protocol !== 'http:' && parsed.protocol !== 'https:') || parsed.hostname === '')
|
|
46
|
+
throw new Error();
|
|
47
|
+
}
|
|
48
|
+
catch {
|
|
49
|
+
throw new Error('proxyUrl 必须是有效的 http(s) 地址,例如 http://127.0.0.1:7890。');
|
|
50
|
+
}
|
|
23
51
|
}
|
|
24
52
|
let timeoutMs = DEFAULT_TIMEOUT_MS;
|
|
25
53
|
if (cfg.timeoutMs !== undefined) {
|
|
26
|
-
if (typeof cfg.timeoutMs !== 'number' || !Number.
|
|
27
|
-
throw new Error('timeoutMs
|
|
54
|
+
if (typeof cfg.timeoutMs !== 'number' || !Number.isInteger(cfg.timeoutMs) || cfg.timeoutMs < 1000 || cfg.timeoutMs > 120000) {
|
|
55
|
+
throw new Error('timeoutMs 必须是 1000 到 120000 之间的整数(毫秒),例如 15000。');
|
|
28
56
|
}
|
|
29
|
-
timeoutMs =
|
|
57
|
+
timeoutMs = cfg.timeoutMs;
|
|
30
58
|
}
|
|
31
59
|
let maxBodyBytes = DEFAULT_MAX_BODY_BYTES;
|
|
32
60
|
if (cfg.maxBodyBytes !== undefined) {
|
|
33
|
-
if (typeof cfg.maxBodyBytes !== 'number' || !Number.
|
|
34
|
-
throw new Error('maxBodyBytes
|
|
61
|
+
if (typeof cfg.maxBodyBytes !== 'number' || !Number.isInteger(cfg.maxBodyBytes) || cfg.maxBodyBytes < 1024 * 1024 || cfg.maxBodyBytes > 50 * 1024 * 1024) {
|
|
62
|
+
throw new Error('maxBodyBytes 必须是 1048576 到 52428800 之间的整数(字节),例如 5242880。');
|
|
35
63
|
}
|
|
36
|
-
maxBodyBytes =
|
|
64
|
+
maxBodyBytes = cfg.maxBodyBytes;
|
|
37
65
|
}
|
|
38
|
-
const userAgent = optionalString(cfg
|
|
39
|
-
|
|
40
|
-
|
|
66
|
+
const userAgent = (optionalString(cfg, 'userAgent') ?? DEFAULT_USER_AGENT).trim();
|
|
67
|
+
if (userAgent === '')
|
|
68
|
+
throw new Error('userAgent 不能为空字符串。');
|
|
69
|
+
const feedsYaml = optionalString(cfg, 'feedsYaml') ?? '';
|
|
70
|
+
const allowPrivateNetwork = optionalBoolean(cfg, 'allowPrivateNetwork', false);
|
|
71
|
+
const opmlWriteApproval = optionalBoolean(cfg, 'opmlWriteApproval', true);
|
|
72
|
+
return { proxyUrl, timeoutMs, maxBodyBytes, userAgent, feedsYaml, allowPrivateNetwork, opmlWriteApproval };
|
|
41
73
|
}
|
package/lib/execution.js
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export {};
|
package/lib/index.d.ts
CHANGED
|
@@ -1,17 +1,30 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* dsh-rss —— RSS/Atom 订阅工具插件(node 半身,配置走 cordis.patch.yml)。
|
|
3
3
|
*
|
|
4
|
-
* 插件导出 apply(ctx, config)
|
|
5
|
-
*
|
|
6
|
-
* 命名空间(dsh-rss.feedsYaml)。配置缺失时插件照常加载,工具在 execute 时才抛出
|
|
7
|
-
* 带中文指引的错误。
|
|
4
|
+
* 插件导出 apply(ctx, config):把九个面向模型的 RSS 工具注册进宿主进程的工具注册表,
|
|
5
|
+
* 并把订阅列表接入 settings 命名空间(dsh-rss.feedsYaml)。
|
|
8
6
|
*
|
|
9
7
|
* @module dsh-rss
|
|
10
8
|
*/
|
|
11
9
|
import { type RssConfig } from './config.js';
|
|
12
10
|
import { type RssSettingsScope, type RssToolDefinition } from './tools.js';
|
|
13
11
|
/** cordis 服务注入:apply 里要用 ctx.settings 与 ctx.tools,必须显式声明,否则宿主会抛 cannot get property without inject。 */
|
|
12
|
+
export declare const name = "rss";
|
|
14
13
|
export declare const inject: string[];
|
|
14
|
+
type RssPreToolDecision = {
|
|
15
|
+
kind: 'allow';
|
|
16
|
+
} | {
|
|
17
|
+
kind: 'deny';
|
|
18
|
+
reason: string;
|
|
19
|
+
} | {
|
|
20
|
+
kind: 'ask';
|
|
21
|
+
reason?: string;
|
|
22
|
+
};
|
|
23
|
+
interface RssPendingToolExecution {
|
|
24
|
+
readonly name: string;
|
|
25
|
+
readonly arguments: unknown;
|
|
26
|
+
}
|
|
27
|
+
type RssPreExecuteListener = (exec: RssPendingToolExecution, next: () => Promise<RssPreToolDecision>) => Promise<RssPreToolDecision>;
|
|
15
28
|
/** 插件所需的最小 ctx 面(社区插件不依赖宿主内部类型)。 */
|
|
16
29
|
export interface RssPluginContext {
|
|
17
30
|
settings: {
|
|
@@ -23,10 +36,11 @@ export interface RssPluginContext {
|
|
|
23
36
|
tools: {
|
|
24
37
|
register(definition: RssToolDefinition): () => void;
|
|
25
38
|
};
|
|
26
|
-
on
|
|
39
|
+
on(event: 'tools/pre-execute', listener: RssPreExecuteListener): () => void;
|
|
40
|
+
on(event: 'dispose', listener: () => void): () => void;
|
|
27
41
|
}
|
|
28
42
|
/**
|
|
29
|
-
*
|
|
43
|
+
* 插件入口:严格解析配置、注册 settings 命名空间、九个 RSS 工具与 OPML 写入审批门。
|
|
30
44
|
* @param ctx - 宿主上下文(至少含 settings.register 与 tools.register)。
|
|
31
45
|
* @param config - 插件配置(可缺省)。
|
|
32
46
|
*/
|
package/lib/index.js
CHANGED
|
@@ -1,10 +1,8 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* dsh-rss —— RSS/Atom 订阅工具插件(node 半身,配置走 cordis.patch.yml)。
|
|
3
3
|
*
|
|
4
|
-
* 插件导出 apply(ctx, config)
|
|
5
|
-
*
|
|
6
|
-
* 命名空间(dsh-rss.feedsYaml)。配置缺失时插件照常加载,工具在 execute 时才抛出
|
|
7
|
-
* 带中文指引的错误。
|
|
4
|
+
* 插件导出 apply(ctx, config):把九个面向模型的 RSS 工具注册进宿主进程的工具注册表,
|
|
5
|
+
* 并把订阅列表接入 settings 命名空间(dsh-rss.feedsYaml)。
|
|
8
6
|
*
|
|
9
7
|
* @module dsh-rss
|
|
10
8
|
*/
|
|
@@ -12,21 +10,15 @@ import { resolveConfig } from './config.js';
|
|
|
12
10
|
import { RSS_SETTINGS_NAMESPACE, RssSettingsSchema } from './settings.js';
|
|
13
11
|
import { buildRssTools } from './tools.js';
|
|
14
12
|
/** cordis 服务注入:apply 里要用 ctx.settings 与 ctx.tools,必须显式声明,否则宿主会抛 cannot get property without inject。 */
|
|
13
|
+
export const name = 'rss';
|
|
15
14
|
export const inject = ['settings', 'tools'];
|
|
16
15
|
/**
|
|
17
|
-
*
|
|
16
|
+
* 插件入口:严格解析配置、注册 settings 命名空间、九个 RSS 工具与 OPML 写入审批门。
|
|
18
17
|
* @param ctx - 宿主上下文(至少含 settings.register 与 tools.register)。
|
|
19
18
|
* @param config - 插件配置(可缺省)。
|
|
20
19
|
*/
|
|
21
20
|
export function apply(ctx, config) {
|
|
22
|
-
|
|
23
|
-
try {
|
|
24
|
-
cfg = resolveConfig(config);
|
|
25
|
-
}
|
|
26
|
-
catch (error) {
|
|
27
|
-
console.warn('[dsh-rss] ' + (error instanceof Error ? error.message : String(error)));
|
|
28
|
-
cfg = resolveConfig(null);
|
|
29
|
-
}
|
|
21
|
+
const cfg = resolveConfig(config);
|
|
30
22
|
const settingsScope = ctx.settings.register(RSS_SETTINGS_NAMESPACE, RssSettingsSchema, {
|
|
31
23
|
base: { feedsYaml: cfg.feedsYaml },
|
|
32
24
|
applies: 'live',
|
|
@@ -35,12 +27,24 @@ export function apply(ctx, config) {
|
|
|
35
27
|
for (const definition of buildRssTools(cfg, settingsScope)) {
|
|
36
28
|
disposers.push(ctx.tools.register(definition));
|
|
37
29
|
}
|
|
38
|
-
if (
|
|
39
|
-
ctx.on('
|
|
40
|
-
|
|
41
|
-
|
|
30
|
+
if (cfg.opmlWriteApproval) {
|
|
31
|
+
ctx.on('tools/pre-execute', async (exec, next) => {
|
|
32
|
+
if (exec.name !== 'rss_opml_export')
|
|
33
|
+
return next();
|
|
34
|
+
const args = typeof exec.arguments === 'object' && exec.arguments !== null
|
|
35
|
+
? exec.arguments
|
|
36
|
+
: {};
|
|
37
|
+
const path = typeof args.path === 'string' ? args.path.trim() : '';
|
|
38
|
+
if (path === '')
|
|
39
|
+
return next();
|
|
40
|
+
const safeLabel = path.replace(/[\r\n]/g, ' ').slice(0, 200);
|
|
41
|
+
return { kind: 'ask', reason: '将 RSS 订阅列表导出并写入当前会话工作区文件:' + safeLabel };
|
|
42
42
|
});
|
|
43
43
|
}
|
|
44
|
+
ctx.on('dispose', () => {
|
|
45
|
+
for (const dispose of disposers)
|
|
46
|
+
dispose();
|
|
47
|
+
});
|
|
44
48
|
}
|
|
45
49
|
export * from './config.js';
|
|
46
50
|
export * from './feeds.js';
|
package/lib/network.d.ts
ADDED
|
@@ -0,0 +1,20 @@
|
|
|
1
|
+
import type { ResolvedRssConfig } from './config.js';
|
|
2
|
+
import type { RssToolExecution } from './execution.js';
|
|
3
|
+
/** 可注入的 fetch 实现(测试用假 fetch 替换真网络)。 */
|
|
4
|
+
export type FetchLike = (url: string, init?: {
|
|
5
|
+
headers?: Record<string, string>;
|
|
6
|
+
redirect?: string;
|
|
7
|
+
signal?: AbortSignal;
|
|
8
|
+
}) => Promise<Response>;
|
|
9
|
+
/** 可注入的 DNS 查询(测试不依赖公网 DNS)。 */
|
|
10
|
+
export type DnsLookupLike = (hostname: string) => Promise<readonly {
|
|
11
|
+
address: string;
|
|
12
|
+
family: number;
|
|
13
|
+
}[]>;
|
|
14
|
+
export declare function assertHttpUrl(url: string): void;
|
|
15
|
+
/** 仅允许可公开路由的地址,避免回环、私网、链路本地和保留地址。 */
|
|
16
|
+
export declare function isBlockedNetworkAddress(address: string): boolean;
|
|
17
|
+
export declare function fetchFeedXml(url: string, cfg: ResolvedRssConfig, exec: RssToolExecution | undefined, fetchImpl?: FetchLike, lookupImpl?: DnsLookupLike): Promise<{
|
|
18
|
+
xml: string;
|
|
19
|
+
finalUrl: string;
|
|
20
|
+
}>;
|
package/lib/network.js
ADDED
|
@@ -0,0 +1,232 @@
|
|
|
1
|
+
/** RSS 网络边界:地址校验、重定向、响应大小与调用取消。 */
|
|
2
|
+
import { lookup } from 'node:dns/promises';
|
|
3
|
+
import { isIP } from 'node:net';
|
|
4
|
+
import { createProxyFetch } from './proxy-fetch.js';
|
|
5
|
+
const MAX_REDIRECTS = 5;
|
|
6
|
+
const PROXY_HINT = '。若该订阅源需要特殊代理(梯子)才能访问,请在 cordis.patch.yml 里给 dsh-rss 配置 proxyUrl(如 http://127.0.0.1:7890)后重启。';
|
|
7
|
+
function parseHttpUrl(url) {
|
|
8
|
+
let parsed;
|
|
9
|
+
try {
|
|
10
|
+
parsed = new URL(url);
|
|
11
|
+
}
|
|
12
|
+
catch {
|
|
13
|
+
throw new Error('订阅源地址必须是 http(s):// 开头的完整地址,例如 https://example.com/feed.xml。');
|
|
14
|
+
}
|
|
15
|
+
if ((parsed.protocol !== 'http:' && parsed.protocol !== 'https:') || parsed.hostname === '') {
|
|
16
|
+
throw new Error('订阅源地址必须是 http(s):// 开头的完整地址,例如 https://example.com/feed.xml。');
|
|
17
|
+
}
|
|
18
|
+
if (parsed.username !== '' || parsed.password !== '') {
|
|
19
|
+
throw new Error('订阅源地址不能包含用户名或密码。');
|
|
20
|
+
}
|
|
21
|
+
return parsed;
|
|
22
|
+
}
|
|
23
|
+
export function assertHttpUrl(url) {
|
|
24
|
+
parseHttpUrl(url);
|
|
25
|
+
}
|
|
26
|
+
function ipv4Bytes(address) {
|
|
27
|
+
if (isIP(address) !== 4)
|
|
28
|
+
return null;
|
|
29
|
+
return address.split('.').map((part) => Number(part));
|
|
30
|
+
}
|
|
31
|
+
function ipv6Words(address) {
|
|
32
|
+
let input = address.toLowerCase().replace(/^\[|\]$/g, '').split('%', 1)[0];
|
|
33
|
+
if (isIP(input) !== 6)
|
|
34
|
+
return null;
|
|
35
|
+
if (input.includes('.')) {
|
|
36
|
+
const lastColon = input.lastIndexOf(':');
|
|
37
|
+
const v4 = ipv4Bytes(input.slice(lastColon + 1));
|
|
38
|
+
if (v4 === null)
|
|
39
|
+
return null;
|
|
40
|
+
input = input.slice(0, lastColon) + ':' + ((v4[0] << 8) | v4[1]).toString(16) + ':' + ((v4[2] << 8) | v4[3]).toString(16);
|
|
41
|
+
}
|
|
42
|
+
const halves = input.split('::');
|
|
43
|
+
if (halves.length > 2)
|
|
44
|
+
return null;
|
|
45
|
+
const left = halves[0] === '' ? [] : halves[0].split(':');
|
|
46
|
+
const right = halves.length === 1 || halves[1] === '' ? [] : halves[1].split(':');
|
|
47
|
+
const zeroCount = halves.length === 2 ? 8 - left.length - right.length : 0;
|
|
48
|
+
if (zeroCount < 0 || (halves.length === 1 && left.length !== 8))
|
|
49
|
+
return null;
|
|
50
|
+
const words = [...left, ...Array.from({ length: zeroCount }, () => '0'), ...right].map((word) => Number.parseInt(word, 16));
|
|
51
|
+
return words.length === 8 && words.every((word) => Number.isInteger(word) && word >= 0 && word <= 0xffff) ? words : null;
|
|
52
|
+
}
|
|
53
|
+
/** 仅允许可公开路由的地址,避免回环、私网、链路本地和保留地址。 */
|
|
54
|
+
export function isBlockedNetworkAddress(address) {
|
|
55
|
+
const v4 = ipv4Bytes(address);
|
|
56
|
+
if (v4 !== null) {
|
|
57
|
+
const [a, b] = v4;
|
|
58
|
+
return a === 0
|
|
59
|
+
|| a === 10
|
|
60
|
+
|| a === 127
|
|
61
|
+
|| (a === 100 && b >= 64 && b <= 127)
|
|
62
|
+
|| (a === 169 && b === 254)
|
|
63
|
+
|| (a === 172 && b >= 16 && b <= 31)
|
|
64
|
+
|| (a === 192 && b === 168)
|
|
65
|
+
|| (a === 198 && (b === 18 || b === 19))
|
|
66
|
+
|| a >= 224;
|
|
67
|
+
}
|
|
68
|
+
const v6 = ipv6Words(address);
|
|
69
|
+
if (v6 === null)
|
|
70
|
+
return true;
|
|
71
|
+
const isUnspecified = v6.every((word) => word === 0);
|
|
72
|
+
const isLoopback = v6.slice(0, 7).every((word) => word === 0) && v6[7] === 1;
|
|
73
|
+
const isUniqueLocal = (v6[0] & 0xfe00) === 0xfc00;
|
|
74
|
+
const isLinkLocal = (v6[0] & 0xffc0) === 0xfe80;
|
|
75
|
+
const isMulticast = (v6[0] & 0xff00) === 0xff00;
|
|
76
|
+
if (isUnspecified || isLoopback || isUniqueLocal || isLinkLocal || isMulticast)
|
|
77
|
+
return true;
|
|
78
|
+
const isMappedV4 = v6.slice(0, 5).every((word) => word === 0) && v6[5] === 0xffff;
|
|
79
|
+
const isCompatibleV4 = v6.slice(0, 6).every((word) => word === 0);
|
|
80
|
+
if (isMappedV4 || isCompatibleV4) {
|
|
81
|
+
const mapped = [v6[6] >> 8, v6[6] & 0xff, v6[7] >> 8, v6[7] & 0xff].join('.');
|
|
82
|
+
return isBlockedNetworkAddress(mapped);
|
|
83
|
+
}
|
|
84
|
+
return false;
|
|
85
|
+
}
|
|
86
|
+
function mergedSignal(timeoutMs, callerSignal) {
|
|
87
|
+
const timeoutSignal = AbortSignal.timeout(timeoutMs);
|
|
88
|
+
return callerSignal === undefined ? timeoutSignal : AbortSignal.any([callerSignal, timeoutSignal]);
|
|
89
|
+
}
|
|
90
|
+
async function lookupWithSignal(hostname, lookupImpl, signal) {
|
|
91
|
+
signal.throwIfAborted();
|
|
92
|
+
return await new Promise((resolveLookup, reject) => {
|
|
93
|
+
const onAbort = () => reject(signal.reason);
|
|
94
|
+
signal.addEventListener('abort', onAbort, { once: true });
|
|
95
|
+
void Promise.resolve().then(() => lookupImpl(hostname)).then((addresses) => {
|
|
96
|
+
signal.removeEventListener('abort', onAbort);
|
|
97
|
+
resolveLookup(addresses);
|
|
98
|
+
}, (error) => {
|
|
99
|
+
signal.removeEventListener('abort', onAbort);
|
|
100
|
+
reject(error);
|
|
101
|
+
});
|
|
102
|
+
});
|
|
103
|
+
}
|
|
104
|
+
async function assertPublicUrl(url, cfg, lookupImpl, signal) {
|
|
105
|
+
if (cfg.allowPrivateNetwork)
|
|
106
|
+
return;
|
|
107
|
+
const hostname = url.hostname.replace(/^\[|\]$/g, '').toLowerCase();
|
|
108
|
+
if (hostname === 'localhost' || hostname.endsWith('.localhost')) {
|
|
109
|
+
throw new Error('出于安全原因,默认禁止访问 localhost。若确需访问可信内网源,请显式配置 allowPrivateNetwork: true。');
|
|
110
|
+
}
|
|
111
|
+
if (isIP(hostname) !== 0) {
|
|
112
|
+
if (isBlockedNetworkAddress(hostname)) {
|
|
113
|
+
throw new Error('出于安全原因,默认禁止访问回环、私网、链路本地或保留地址:' + hostname + '。若确需访问可信内网源,请显式配置 allowPrivateNetwork: true。');
|
|
114
|
+
}
|
|
115
|
+
return;
|
|
116
|
+
}
|
|
117
|
+
let addresses;
|
|
118
|
+
try {
|
|
119
|
+
addresses = await lookupWithSignal(hostname, lookupImpl, signal);
|
|
120
|
+
}
|
|
121
|
+
catch (error) {
|
|
122
|
+
signal.throwIfAborted();
|
|
123
|
+
throw new Error('无法安全解析订阅源域名 ' + hostname + ':' + (error instanceof Error ? error.message : String(error)));
|
|
124
|
+
}
|
|
125
|
+
if (addresses.length === 0)
|
|
126
|
+
throw new Error('无法安全解析订阅源域名 ' + hostname + ':DNS 未返回地址。');
|
|
127
|
+
const blocked = addresses.find((entry) => isBlockedNetworkAddress(entry.address));
|
|
128
|
+
if (blocked !== undefined) {
|
|
129
|
+
throw new Error('出于安全原因,域名 ' + hostname + ' 解析到了非公网地址 ' + blocked.address + ',已拒绝访问。若确需访问可信内网源,请显式配置 allowPrivateNetwork: true。');
|
|
130
|
+
}
|
|
131
|
+
}
|
|
132
|
+
/** 构造默认 fetch:配置了 proxyUrl 时走插件级代理。 */
|
|
133
|
+
function makeFetch(cfg) {
|
|
134
|
+
if (cfg.proxyUrl === '')
|
|
135
|
+
return globalThis.fetch;
|
|
136
|
+
return createProxyFetch(cfg.proxyUrl);
|
|
137
|
+
}
|
|
138
|
+
/**
|
|
139
|
+
* 抓取订阅源 XML:校验地址、限时、限制体积、剥 BOM。
|
|
140
|
+
*/
|
|
141
|
+
async function readResponseText(response, maxBodyBytes, signal) {
|
|
142
|
+
signal.throwIfAborted();
|
|
143
|
+
const declaredLength = Number(response.headers.get('content-length'));
|
|
144
|
+
if (Number.isFinite(declaredLength) && declaredLength > maxBodyBytes) {
|
|
145
|
+
await response.body?.cancel('response body exceeds maxBodyBytes');
|
|
146
|
+
throw new Error('订阅源内容超过 ' + maxBodyBytes + ' 字节上限,已停止读取。');
|
|
147
|
+
}
|
|
148
|
+
if (response.body === null)
|
|
149
|
+
return '';
|
|
150
|
+
const reader = response.body.getReader();
|
|
151
|
+
const onAbort = () => { void reader.cancel(signal.reason).catch(() => { }); };
|
|
152
|
+
signal.addEventListener('abort', onAbort, { once: true });
|
|
153
|
+
const decoder = new TextDecoder('utf-8', { fatal: false });
|
|
154
|
+
let byteLength = 0;
|
|
155
|
+
let text = '';
|
|
156
|
+
try {
|
|
157
|
+
while (true) {
|
|
158
|
+
signal.throwIfAborted();
|
|
159
|
+
const chunk = await reader.read();
|
|
160
|
+
signal.throwIfAborted();
|
|
161
|
+
if (chunk.done)
|
|
162
|
+
break;
|
|
163
|
+
byteLength += chunk.value.byteLength;
|
|
164
|
+
if (byteLength > maxBodyBytes) {
|
|
165
|
+
await reader.cancel('response body exceeds maxBodyBytes');
|
|
166
|
+
throw new Error('订阅源内容超过 ' + maxBodyBytes + ' 字节上限,已停止读取。');
|
|
167
|
+
}
|
|
168
|
+
text += decoder.decode(chunk.value, { stream: true });
|
|
169
|
+
}
|
|
170
|
+
return text + decoder.decode();
|
|
171
|
+
}
|
|
172
|
+
finally {
|
|
173
|
+
signal.removeEventListener('abort', onAbort);
|
|
174
|
+
reader.releaseLock();
|
|
175
|
+
}
|
|
176
|
+
}
|
|
177
|
+
export async function fetchFeedXml(url, cfg, exec, fetchImpl, lookupImpl = async (hostname) => await lookup(hostname, { all: true, verbatim: true })) {
|
|
178
|
+
let currentUrl = parseHttpUrl(url);
|
|
179
|
+
const fetcher = fetchImpl ?? makeFetch(cfg);
|
|
180
|
+
const signal = mergedSignal(cfg.timeoutMs, exec?.signal);
|
|
181
|
+
let response;
|
|
182
|
+
try {
|
|
183
|
+
for (let redirectCount = 0;; redirectCount++) {
|
|
184
|
+
signal.throwIfAborted();
|
|
185
|
+
await assertPublicUrl(currentUrl, cfg, lookupImpl, signal);
|
|
186
|
+
response = await fetcher(currentUrl.href, {
|
|
187
|
+
headers: {
|
|
188
|
+
'user-agent': cfg.userAgent,
|
|
189
|
+
accept: 'application/rss+xml, application/atom+xml, application/xml, text/xml, */*',
|
|
190
|
+
},
|
|
191
|
+
redirect: 'manual',
|
|
192
|
+
signal,
|
|
193
|
+
});
|
|
194
|
+
signal.throwIfAborted();
|
|
195
|
+
if (![301, 302, 303, 307, 308].includes(response.status))
|
|
196
|
+
break;
|
|
197
|
+
const location = response.headers.get('location');
|
|
198
|
+
if (location === null)
|
|
199
|
+
break;
|
|
200
|
+
await response.body?.cancel('following validated redirect');
|
|
201
|
+
if (redirectCount >= MAX_REDIRECTS)
|
|
202
|
+
throw new Error('抓取失败:重定向次数超过 ' + MAX_REDIRECTS + ' 次上限。');
|
|
203
|
+
currentUrl = parseHttpUrl(new URL(location, currentUrl).href);
|
|
204
|
+
}
|
|
205
|
+
}
|
|
206
|
+
catch (error) {
|
|
207
|
+
signal.throwIfAborted();
|
|
208
|
+
if (error instanceof Error && (error.message.includes('出于安全原因') || error.message.includes('无法安全解析') || error.message.includes('重定向次数超过')))
|
|
209
|
+
throw error;
|
|
210
|
+
throw new Error('抓取失败:' + (error instanceof Error ? error.message : String(error)) + PROXY_HINT);
|
|
211
|
+
}
|
|
212
|
+
if (response === undefined)
|
|
213
|
+
throw new Error('抓取失败:未收到服务器响应。');
|
|
214
|
+
if (!response.ok) {
|
|
215
|
+
await response.body?.cancel('unsuccessful feed response');
|
|
216
|
+
throw new Error('抓取失败:服务器返回 HTTP ' + response.status + '。');
|
|
217
|
+
}
|
|
218
|
+
const reportedUrl = response.url === '' ? currentUrl : parseHttpUrl(response.url);
|
|
219
|
+
await assertPublicUrl(reportedUrl, cfg, lookupImpl, signal);
|
|
220
|
+
let xml;
|
|
221
|
+
try {
|
|
222
|
+
xml = await readResponseText(response, cfg.maxBodyBytes, signal);
|
|
223
|
+
}
|
|
224
|
+
catch (error) {
|
|
225
|
+
signal.throwIfAborted();
|
|
226
|
+
if (error instanceof Error && error.message.includes('字节上限'))
|
|
227
|
+
throw error;
|
|
228
|
+
throw new Error('读取订阅源内容失败:' + (error instanceof Error ? error.message : String(error)));
|
|
229
|
+
}
|
|
230
|
+
xml = xml.replace(/^\uFEFF/, '');
|
|
231
|
+
return { xml, finalUrl: reportedUrl.href };
|
|
232
|
+
}
|
package/lib/settings.d.ts
CHANGED
|
@@ -9,10 +9,13 @@ export declare const RSS_SETTINGS_NAMESPACE = "dsh-rss";
|
|
|
9
9
|
/** settings 页形状:目前只有订阅列表(YAML 文本)。 */
|
|
10
10
|
export declare const RssSettingsSchema: z<Schemastery.ObjectS<{
|
|
11
11
|
feedsYaml: z<string, string>;
|
|
12
|
+
cursorsJson: z<string, string>;
|
|
12
13
|
}>, Schemastery.ObjectT<{
|
|
13
14
|
feedsYaml: z<string, string>;
|
|
15
|
+
cursorsJson: z<string, string>;
|
|
14
16
|
}>>;
|
|
15
17
|
/** settings 解析后的值。 */
|
|
16
18
|
export interface RssSettingsValue {
|
|
17
19
|
feedsYaml: string;
|
|
20
|
+
cursorsJson: string;
|
|
18
21
|
}
|