koishi-plugin-share-links-analysis 0.13.3 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +71 -71
- package/lib/index.js +439 -69
- package/lib/types.d.ts +14 -0
- package/lib/utils.d.ts +6 -0
- package/lib/utils.js +73 -9
- package/package.json +1 -1
package/LICENSE
CHANGED
|
@@ -1,21 +1,21 @@
|
|
|
1
|
-
MIT License
|
|
2
|
-
|
|
3
|
-
Copyright (c) 2024 shangxue
|
|
4
|
-
|
|
5
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
-
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
-
in the Software without restriction, including without limitation the rights
|
|
8
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
-
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
-
furnished to do so, subject to the following conditions:
|
|
11
|
-
|
|
12
|
-
The above copyright notice and this permission notice shall be included in all
|
|
13
|
-
copies or substantial portions of the Software.
|
|
14
|
-
|
|
15
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
-
SOFTWARE.
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2024 shangxue
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,71 +1,71 @@
|
|
|
1
|
-
# koishi-plugin-share-links-analysis
|
|
2
|
-
|
|
3
|
-
# 视频链接解析插件说明 📺✨
|
|
4
|
-
|
|
5
|
-
本插件为用户提供便捷的分享链接链接解析服务,让聊天体验更加丰富多彩。
|
|
6
|
-
|
|
7
|
-
---
|
|
8
|
-
|
|
9
|
-
## 鸣谢 💖
|
|
10
|
-
|
|
11
|
-
本插件的解析能力与稳定运行,离不开以下开源项目及社区文档的强大支撑。特此向这些项目及其维护者致以最诚挚的感谢:
|
|
12
|
-
|
|
13
|
-
- [koishi-plugin-bilibili-videolink-analysis](https://github.com/shangxueink/koishi-shangxue-apps/tree/main/plugins/bilibili-videolink-analysis)
|
|
14
|
-
- [koishi-plugin-bili-parser](https://github.com/summonhim/koishi-plugin-bili-parser)
|
|
15
|
-
- [bilibili-API-collect](https://github.com/SocialSisterYi/bilibili-API-collect)
|
|
16
|
-
- [koishi-plugin-xiaohongshu](https://www.npmjs.com/package/koishi-plugin-xiaohongshu)
|
|
17
|
-
以及解析方式原作者:[@MuJie](https://mu-jie.cc/)
|
|
18
|
-
- [BetterTwitFix](https://github.com/dylanpdx/BetterTwitFix)
|
|
19
|
-
- [yt-dlp](https://github.com/yt-dlp/yt-dlp)
|
|
20
|
-
- [NeteaseCloudMusicApiEnhanced](https://github.com/NeteaseCloudMusicApiEnhanced/api-enhanced)
|
|
21
|
-
|
|
22
|
-
---
|
|
23
|
-
|
|
24
|
-
## 🚀 YouTube 解析后端部署指南 (必看)
|
|
25
|
-
|
|
26
|
-
由于 YouTube 引入了极其严苛的反爬机制,传统的直连解析已全部失效。为了保证解析的稳定,本插件采用了 **“前端 Koishi + 后端 Python 微服务”** 的分离架构。
|
|
27
|
-
|
|
28
|
-
Python 后端利用 `yt-dlp` 的最新特性,通过调用本地 Node.js 实时计算 YouTube 签名,并已内置自动热更新。
|
|
29
|
-
|
|
30
|
-
### 部署环境要求
|
|
31
|
-
- **Python 3.8+**
|
|
32
|
-
- **Node.js** (⚠️ 必须安装!`yt-dlp` 需要调用 Node 环境来执行 JS 脚本破解签名)
|
|
33
|
-
- **一个能够正常访问 YouTube 的网络代理** (强烈建议使用流媒体解锁节点)
|
|
34
|
-
|
|
35
|
-
### 部署步骤
|
|
36
|
-
|
|
37
|
-
**1. 下载后端脚本**
|
|
38
|
-
将本仓库内的 `yt_server.py` 下载到你的服务器上:
|
|
39
|
-
```bash
|
|
40
|
-
wget https://raw.githubusercontent.com/furryaxw/share-links-analysis/Master/yt_server.py
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
**2. 安装 Python 依赖**
|
|
44
|
-
```bash
|
|
45
|
-
pip install httpx fastapi uvicorn yt-dlp
|
|
46
|
-
```
|
|
47
|
-
|
|
48
|
-
**3. 配置网络代理 (重要)**
|
|
49
|
-
请使用编辑器(如 vim 或 nano)打开 `yt_server.py`,在文件顶部找到代理配置区域,**将地址修改为你自己的真实代理**:
|
|
50
|
-
```python
|
|
51
|
-
# =====================================
|
|
52
|
-
# 配置代理
|
|
53
|
-
proxy = "http://127.0.0.1:7890" # 👈 在这里填入你的代理IP和端口
|
|
54
|
-
# =====================================
|
|
55
|
-
```
|
|
56
|
-
|
|
57
|
-
**4. 启动微服务**
|
|
58
|
-
```bash
|
|
59
|
-
python yt_server.py
|
|
60
|
-
```
|
|
61
|
-
*(注:由于脚本内置了自动更新热重启逻辑,强烈建议直接使用上述命令启动,或者使用 `pm2 start yt_server.py --name yt-api --interpreter python3` 进行后台进程守护。)*
|
|
62
|
-
|
|
63
|
-
**5. 在 Koishi 中配置**
|
|
64
|
-
确保 Python 服务成功运行(默认监听 `12001` 端口)后,回到 Koishi 控制台的本插件配置页。
|
|
65
|
-
在 `youtube_pythonApiUrl` 配置项中填入你的后端地址:
|
|
66
|
-
```text
|
|
67
|
-
http://127.0.0.1:12001/api/parse
|
|
68
|
-
```
|
|
69
|
-
*(如果你的 Python 服务部署在其他服务器上,请将 `127.0.0.1` 替换为对应的服务器 IP)*
|
|
70
|
-
|
|
71
|
-
🎉 配置完成!现在你可以尽情享受丝滑的 YouTube 视频与 Shorts 解析了!
|
|
1
|
+
# koishi-plugin-share-links-analysis
|
|
2
|
+
|
|
3
|
+
# 视频链接解析插件说明 📺✨
|
|
4
|
+
|
|
5
|
+
本插件为用户提供便捷的分享链接链接解析服务,让聊天体验更加丰富多彩。
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 鸣谢 💖
|
|
10
|
+
|
|
11
|
+
本插件的解析能力与稳定运行,离不开以下开源项目及社区文档的强大支撑。特此向这些项目及其维护者致以最诚挚的感谢:
|
|
12
|
+
|
|
13
|
+
- [koishi-plugin-bilibili-videolink-analysis](https://github.com/shangxueink/koishi-shangxue-apps/tree/main/plugins/bilibili-videolink-analysis)
|
|
14
|
+
- [koishi-plugin-bili-parser](https://github.com/summonhim/koishi-plugin-bili-parser)
|
|
15
|
+
- [bilibili-API-collect](https://github.com/SocialSisterYi/bilibili-API-collect)
|
|
16
|
+
- [koishi-plugin-xiaohongshu](https://www.npmjs.com/package/koishi-plugin-xiaohongshu)
|
|
17
|
+
以及解析方式原作者:[@MuJie](https://mu-jie.cc/)
|
|
18
|
+
- [BetterTwitFix](https://github.com/dylanpdx/BetterTwitFix)
|
|
19
|
+
- [yt-dlp](https://github.com/yt-dlp/yt-dlp)
|
|
20
|
+
- [NeteaseCloudMusicApiEnhanced](https://github.com/NeteaseCloudMusicApiEnhanced/api-enhanced)
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 🚀 YouTube 解析后端部署指南 (必看)
|
|
25
|
+
|
|
26
|
+
由于 YouTube 引入了极其严苛的反爬机制,传统的直连解析已全部失效。为了保证解析的稳定,本插件采用了 **“前端 Koishi + 后端 Python 微服务”** 的分离架构。
|
|
27
|
+
|
|
28
|
+
Python 后端利用 `yt-dlp` 的最新特性,通过调用本地 Node.js 实时计算 YouTube 签名,并已内置自动热更新。
|
|
29
|
+
|
|
30
|
+
### 部署环境要求
|
|
31
|
+
- **Python 3.8+**
|
|
32
|
+
- **Node.js** (⚠️ 必须安装!`yt-dlp` 需要调用 Node 环境来执行 JS 脚本破解签名)
|
|
33
|
+
- **一个能够正常访问 YouTube 的网络代理** (强烈建议使用流媒体解锁节点)
|
|
34
|
+
|
|
35
|
+
### 部署步骤
|
|
36
|
+
|
|
37
|
+
**1. 下载后端脚本**
|
|
38
|
+
将本仓库内的 `yt_server.py` 下载到你的服务器上:
|
|
39
|
+
```bash
|
|
40
|
+
wget https://raw.githubusercontent.com/furryaxw/share-links-analysis/Master/yt_server.py
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
**2. 安装 Python 依赖**
|
|
44
|
+
```bash
|
|
45
|
+
pip install httpx fastapi uvicorn yt-dlp
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
**3. 配置网络代理 (重要)**
|
|
49
|
+
请使用编辑器(如 vim 或 nano)打开 `yt_server.py`,在文件顶部找到代理配置区域,**将地址修改为你自己的真实代理**:
|
|
50
|
+
```python
|
|
51
|
+
# =====================================
|
|
52
|
+
# 配置代理
|
|
53
|
+
proxy = "http://127.0.0.1:7890" # 👈 在这里填入你的代理IP和端口
|
|
54
|
+
# =====================================
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
**4. 启动微服务**
|
|
58
|
+
```bash
|
|
59
|
+
python yt_server.py
|
|
60
|
+
```
|
|
61
|
+
*(注:由于脚本内置了自动更新热重启逻辑,强烈建议直接使用上述命令启动,或者使用 `pm2 start yt_server.py --name yt-api --interpreter python3` 进行后台进程守护。)*
|
|
62
|
+
|
|
63
|
+
**5. 在 Koishi 中配置**
|
|
64
|
+
确保 Python 服务成功运行(默认监听 `12001` 端口)后,回到 Koishi 控制台的本插件配置页。
|
|
65
|
+
在 `youtube_pythonApiUrl` 配置项中填入你的后端地址:
|
|
66
|
+
```text
|
|
67
|
+
http://127.0.0.1:12001/api/parse
|
|
68
|
+
```
|
|
69
|
+
*(如果你的 Python 服务部署在其他服务器上,请将 `127.0.0.1` 替换为对应的服务器 IP)*
|
|
70
|
+
|
|
71
|
+
🎉 配置完成!现在你可以尽情享受丝滑的 YouTube 视频与 Shorts 解析了!
|
package/lib/index.js
CHANGED
|
@@ -90,7 +90,8 @@ exports.Config = koishi_1.Schema.intersect([
|
|
|
90
90
|
----------
|
|
91
91
|
{mainbody}
|
|
92
92
|
----------
|
|
93
|
-
{
|
|
93
|
+
{cache}
|
|
94
|
+
{sourceUrl}`).description('图文/视频输出格式。<br/>可用占位符: `{title}`, `{cover}`, `{authorName}`, `{mainbody}`, `{stats}`, `{cache}`, `{sourceUrl}`'),
|
|
94
95
|
}).description("格式化模板"),
|
|
95
96
|
koishi_1.Schema.object({
|
|
96
97
|
parseLimit: koishi_1.Schema.number().default(3).description("单对话多链接解析上限"),
|
|
@@ -151,6 +152,258 @@ async function reportMetric(ctx, config, payload) {
|
|
|
151
152
|
}
|
|
152
153
|
});
|
|
153
154
|
}
|
|
155
|
+
function getCombinedRetentionHours(config) {
|
|
156
|
+
const expirations = [config.cacheExpiration, config.optimisticExpiration];
|
|
157
|
+
if (expirations.some(value => value === 0))
|
|
158
|
+
return 0;
|
|
159
|
+
return Math.max(...expirations);
|
|
160
|
+
}
|
|
161
|
+
function getRetentionThreshold(hours) {
|
|
162
|
+
if (hours === 0)
|
|
163
|
+
return null;
|
|
164
|
+
return Date.now() - hours * 60 * 60 * 1000;
|
|
165
|
+
}
|
|
166
|
+
function normalizeForCompare(value) {
|
|
167
|
+
if (Array.isArray(value))
|
|
168
|
+
return value.map(normalizeForCompare);
|
|
169
|
+
if (value && typeof value === 'object') {
|
|
170
|
+
const output = {};
|
|
171
|
+
for (const key of Object.keys(value).sort()) {
|
|
172
|
+
if (key.startsWith('_'))
|
|
173
|
+
continue;
|
|
174
|
+
output[key] = normalizeForCompare(value[key]);
|
|
175
|
+
}
|
|
176
|
+
return output;
|
|
177
|
+
}
|
|
178
|
+
return value;
|
|
179
|
+
}
|
|
180
|
+
function stableStringify(value) {
|
|
181
|
+
return JSON.stringify(normalizeForCompare(value));
|
|
182
|
+
}
|
|
183
|
+
function isSameParsedInfo(a, b) {
|
|
184
|
+
return stableStringify(a) === stableStringify(b);
|
|
185
|
+
}
|
|
186
|
+
function formatDiffValue(value) {
|
|
187
|
+
if (value === undefined)
|
|
188
|
+
return 'undefined';
|
|
189
|
+
if (value === null)
|
|
190
|
+
return 'null';
|
|
191
|
+
if (typeof value === 'string')
|
|
192
|
+
return value.length > 160 ? `${value.slice(0, 157)}...` : value;
|
|
193
|
+
const text = JSON.stringify(value);
|
|
194
|
+
return text.length > 160 ? `${text.slice(0, 157)}...` : text;
|
|
195
|
+
}
|
|
196
|
+
function collectDiffLines(before, after, path = '') {
|
|
197
|
+
if (stableStringify(before) === stableStringify(after))
|
|
198
|
+
return [];
|
|
199
|
+
if (Array.isArray(before) || Array.isArray(after)) {
|
|
200
|
+
return [
|
|
201
|
+
`- ${path || 'root'}: ${formatDiffValue(before)}`,
|
|
202
|
+
`+ ${path || 'root'}: ${formatDiffValue(after)}`
|
|
203
|
+
];
|
|
204
|
+
}
|
|
205
|
+
if (before && after &&
|
|
206
|
+
typeof before === 'object' &&
|
|
207
|
+
typeof after === 'object') {
|
|
208
|
+
const keys = new Set([...Object.keys(before), ...Object.keys(after)]);
|
|
209
|
+
const lines = [];
|
|
210
|
+
for (const key of [...keys].sort()) {
|
|
211
|
+
if (key.startsWith('_'))
|
|
212
|
+
continue;
|
|
213
|
+
const nextPath = path ? `${path}.${key}` : key;
|
|
214
|
+
lines.push(...collectDiffLines(before[key], after[key], nextPath));
|
|
215
|
+
}
|
|
216
|
+
return lines;
|
|
217
|
+
}
|
|
218
|
+
return [
|
|
219
|
+
`- ${path || 'root'}: ${formatDiffValue(before)}`,
|
|
220
|
+
`+ ${path || 'root'}: ${formatDiffValue(after)}`
|
|
221
|
+
];
|
|
222
|
+
}
|
|
223
|
+
function buildParsedInfoDelta(previous, current) {
|
|
224
|
+
if (!previous)
|
|
225
|
+
return 'base';
|
|
226
|
+
const lines = collectDiffLines(normalizeForCompare(previous), normalizeForCompare(current));
|
|
227
|
+
return lines.length > 0 ? lines.join('\n') : 'no changes';
|
|
228
|
+
}
|
|
229
|
+
function summarizeParsedInfo(data) {
|
|
230
|
+
const parts = [
|
|
231
|
+
`标题: ${data.title || '无标题'}`,
|
|
232
|
+
`作者: ${data.authorName || '未知'}`,
|
|
233
|
+
`文件: ${Array.isArray(data.files) ? data.files.length : 0}`,
|
|
234
|
+
];
|
|
235
|
+
if (data.stats)
|
|
236
|
+
parts.push(`统计: ${data.stats}`);
|
|
237
|
+
return parts.join('\n');
|
|
238
|
+
}
|
|
239
|
+
function summarizeDelta(delta) {
|
|
240
|
+
const lines = delta.split('\n');
|
|
241
|
+
const text = lines.length > 8
|
|
242
|
+
? `${lines.slice(0, 8).join('\n')}\n...`
|
|
243
|
+
: delta;
|
|
244
|
+
return text.length > 800 ? `${text.slice(0, 797)}...` : text;
|
|
245
|
+
}
|
|
246
|
+
function formatTime(timestamp) {
|
|
247
|
+
return new Date(timestamp).toLocaleString('zh-CN', { hour12: false });
|
|
248
|
+
}
|
|
249
|
+
function createTimelineNodeId(cacheKey, createdAt = Date.now()) {
|
|
250
|
+
const safeKey = cacheKey.replace(/[^a-zA-Z0-9_-]/g, '_');
|
|
251
|
+
return `${safeKey}:${createdAt}:${Math.random().toString(36).slice(2, 8)}`;
|
|
252
|
+
}
|
|
253
|
+
async function getTimelineNodes(ctx, cacheKey) {
|
|
254
|
+
const nodes = await ctx.database.get('sla_parse_timeline', { cache_key: cacheKey });
|
|
255
|
+
return nodes.sort((a, b) => a.created_at - b.created_at);
|
|
256
|
+
}
|
|
257
|
+
function isTimelineNodeActive(config, node) {
|
|
258
|
+
const threshold = getRetentionThreshold(getCombinedRetentionHours(config));
|
|
259
|
+
if (threshold === null)
|
|
260
|
+
return true;
|
|
261
|
+
return (node.last_checked_at || node.created_at) >= threshold;
|
|
262
|
+
}
|
|
263
|
+
async function getActiveTimelineNodes(ctx, config, cacheKey) {
|
|
264
|
+
const nodes = await getTimelineNodes(ctx, cacheKey);
|
|
265
|
+
return nodes.filter(node => isTimelineNodeActive(config, node));
|
|
266
|
+
}
|
|
267
|
+
async function touchTimelineNode(ctx, node, checkedAt = Date.now()) {
|
|
268
|
+
const touched = { ...node, last_checked_at: checkedAt };
|
|
269
|
+
await ctx.database.upsert('sla_parse_timeline', [touched]);
|
|
270
|
+
return touched;
|
|
271
|
+
}
|
|
272
|
+
async function createTimelineNodeFromCache(ctx, cacheKey, result, createdAt, delta = 'base') {
|
|
273
|
+
const node = {
|
|
274
|
+
node_id: createTimelineNodeId(cacheKey, createdAt),
|
|
275
|
+
cache_key: cacheKey,
|
|
276
|
+
data: result,
|
|
277
|
+
delta,
|
|
278
|
+
created_at: createdAt,
|
|
279
|
+
last_checked_at: createdAt,
|
|
280
|
+
};
|
|
281
|
+
await ctx.database.upsert('sla_parse_timeline', [node]);
|
|
282
|
+
return node;
|
|
283
|
+
}
|
|
284
|
+
async function upsertTimelineIfChanged(ctx, config, cacheKey, result) {
|
|
285
|
+
const nodes = await getActiveTimelineNodes(ctx, config, cacheKey);
|
|
286
|
+
const latest = nodes[nodes.length - 1];
|
|
287
|
+
if (latest && isSameParsedInfo(latest.data, result)) {
|
|
288
|
+
return touchTimelineNode(ctx, latest);
|
|
289
|
+
}
|
|
290
|
+
const now = Date.now();
|
|
291
|
+
const node = {
|
|
292
|
+
node_id: createTimelineNodeId(cacheKey, now),
|
|
293
|
+
cache_key: cacheKey,
|
|
294
|
+
data: result,
|
|
295
|
+
delta: buildParsedInfoDelta(latest?.data ?? null, result),
|
|
296
|
+
created_at: now,
|
|
297
|
+
last_checked_at: now,
|
|
298
|
+
};
|
|
299
|
+
await ctx.database.upsert('sla_parse_timeline', [node]);
|
|
300
|
+
return node;
|
|
301
|
+
}
|
|
302
|
+
async function removeExpiredTimelineNodes(ctx, threshold) {
|
|
303
|
+
if (threshold === null)
|
|
304
|
+
return;
|
|
305
|
+
await ctx.database.remove('sla_parse_timeline', {
|
|
306
|
+
last_checked_at: { $lt: threshold }
|
|
307
|
+
});
|
|
308
|
+
}
|
|
309
|
+
async function pruneExpiredTimelineNodes(ctx, config) {
|
|
310
|
+
await removeExpiredTimelineNodes(ctx, getRetentionThreshold(getCombinedRetentionHours(config)));
|
|
311
|
+
}
|
|
312
|
+
async function getLatestTimelineNode(ctx, config, cacheKey) {
|
|
313
|
+
const nodes = await getActiveTimelineNodes(ctx, config, cacheKey);
|
|
314
|
+
return nodes[nodes.length - 1] || null;
|
|
315
|
+
}
|
|
316
|
+
function registerLegacyParseCacheModel(ctx) {
|
|
317
|
+
ctx.model.extend('sla_parse_cache', {
|
|
318
|
+
key: 'string',
|
|
319
|
+
data: 'json',
|
|
320
|
+
created_at: 'double',
|
|
321
|
+
}, { primary: 'key' });
|
|
322
|
+
}
|
|
323
|
+
async function getLegacyParseCacheEntries(ctx) {
|
|
324
|
+
try {
|
|
325
|
+
registerLegacyParseCacheModel(ctx);
|
|
326
|
+
return await ctx.database.get('sla_parse_cache', {});
|
|
327
|
+
}
|
|
328
|
+
catch {
|
|
329
|
+
return [];
|
|
330
|
+
}
|
|
331
|
+
}
|
|
332
|
+
async function migrateLegacyParseCache(ctx) {
|
|
333
|
+
const legacyEntries = await getLegacyParseCacheEntries(ctx);
|
|
334
|
+
let migrated = 0;
|
|
335
|
+
let skipped = 0;
|
|
336
|
+
let invalid = 0;
|
|
337
|
+
for (const entry of legacyEntries) {
|
|
338
|
+
if (!entry?.key || !entry.data || !entry.created_at) {
|
|
339
|
+
invalid++;
|
|
340
|
+
continue;
|
|
341
|
+
}
|
|
342
|
+
const nodes = await getTimelineNodes(ctx, entry.key);
|
|
343
|
+
const sameNode = nodes.find(node => isSameParsedInfo(node.data, entry.data));
|
|
344
|
+
if (sameNode) {
|
|
345
|
+
await touchTimelineNode(ctx, sameNode, Math.max(sameNode.last_checked_at || 0, entry.created_at));
|
|
346
|
+
skipped++;
|
|
347
|
+
continue;
|
|
348
|
+
}
|
|
349
|
+
const latest = nodes[nodes.length - 1];
|
|
350
|
+
const delta = latest ? buildParsedInfoDelta(latest.data, entry.data) : 'base';
|
|
351
|
+
await createTimelineNodeFromCache(ctx, entry.key, entry.data, entry.created_at, delta);
|
|
352
|
+
migrated++;
|
|
353
|
+
}
|
|
354
|
+
let mismatched = 0;
|
|
355
|
+
for (const entry of legacyEntries) {
|
|
356
|
+
if (!entry?.key || !entry.data)
|
|
357
|
+
continue;
|
|
358
|
+
const nodes = await getTimelineNodes(ctx, entry.key);
|
|
359
|
+
if (!nodes.some(node => isSameParsedInfo(node.data, entry.data))) {
|
|
360
|
+
mismatched++;
|
|
361
|
+
}
|
|
362
|
+
}
|
|
363
|
+
return {
|
|
364
|
+
legacyParse: legacyEntries.length,
|
|
365
|
+
migrated,
|
|
366
|
+
skipped,
|
|
367
|
+
invalid,
|
|
368
|
+
mismatched,
|
|
369
|
+
};
|
|
370
|
+
}
|
|
371
|
+
async function dropLegacyParseCache(ctx) {
|
|
372
|
+
const messages = [];
|
|
373
|
+
try {
|
|
374
|
+
registerLegacyParseCacheModel(ctx);
|
|
375
|
+
await ctx.database.remove('sla_parse_cache', {});
|
|
376
|
+
messages.push('旧 sla_parse_cache 数据已清空。');
|
|
377
|
+
}
|
|
378
|
+
catch (e) {
|
|
379
|
+
messages.push(`清空旧 sla_parse_cache 失败: ${e.message || String(e)}`);
|
|
380
|
+
}
|
|
381
|
+
const database = ctx.database;
|
|
382
|
+
const candidates = [
|
|
383
|
+
{ fn: database.drop, args: ['sla_parse_cache'] },
|
|
384
|
+
{ fn: database.dropTable, args: ['sla_parse_cache'] },
|
|
385
|
+
{ fn: database.dropTables, args: [['sla_parse_cache']] },
|
|
386
|
+
];
|
|
387
|
+
let dropped = false;
|
|
388
|
+
for (const candidate of candidates) {
|
|
389
|
+
if (typeof candidate.fn !== 'function')
|
|
390
|
+
continue;
|
|
391
|
+
try {
|
|
392
|
+
const result = candidate.fn.apply(database, candidate.args);
|
|
393
|
+
if (result && typeof result.then === 'function')
|
|
394
|
+
await result;
|
|
395
|
+
messages.push('旧表 sla_parse_cache 已尝试删除。');
|
|
396
|
+
dropped = true;
|
|
397
|
+
break;
|
|
398
|
+
}
|
|
399
|
+
catch {
|
|
400
|
+
}
|
|
401
|
+
}
|
|
402
|
+
if (!dropped) {
|
|
403
|
+
messages.push('当前数据库适配器未暴露删除 sla_parse_cache 的表 API,请按需手动 drop。');
|
|
404
|
+
}
|
|
405
|
+
return messages.join('\n');
|
|
406
|
+
}
|
|
154
407
|
function apply(ctx, config) {
|
|
155
408
|
// 数据库模型定义
|
|
156
409
|
ctx.model.extend('sla_cookie_cache', {
|
|
@@ -166,12 +419,6 @@ function apply(ctx, config) {
|
|
|
166
419
|
}, {
|
|
167
420
|
primary: 'guildId',
|
|
168
421
|
});
|
|
169
|
-
// 解析结果缓存
|
|
170
|
-
ctx.model.extend('sla_parse_cache', {
|
|
171
|
-
key: 'string', // platform + ':' + id
|
|
172
|
-
data: 'json',
|
|
173
|
-
created_at: 'double',
|
|
174
|
-
}, { primary: 'key' });
|
|
175
422
|
// 资源文件缓存 (hash)
|
|
176
423
|
ctx.model.extend('sla_file_cache', {
|
|
177
424
|
hash: 'string', // URL MD5
|
|
@@ -179,6 +426,15 @@ function apply(ctx, config) {
|
|
|
179
426
|
url: 'string',
|
|
180
427
|
created_at: 'double',
|
|
181
428
|
}, { primary: 'hash' });
|
|
429
|
+
// 解析结果时间线
|
|
430
|
+
ctx.model.extend('sla_parse_timeline', {
|
|
431
|
+
node_id: 'string',
|
|
432
|
+
cache_key: 'string',
|
|
433
|
+
data: 'json',
|
|
434
|
+
delta: 'text',
|
|
435
|
+
created_at: 'double',
|
|
436
|
+
last_checked_at: 'double',
|
|
437
|
+
}, { primary: 'node_id' });
|
|
182
438
|
const logger = ctx.logger('share-links-analysis');
|
|
183
439
|
// 根据配置设置日志等级
|
|
184
440
|
if (config.debug) {
|
|
@@ -189,32 +445,13 @@ function apply(ctx, config) {
|
|
|
189
445
|
const cleanExpiredCache = async () => {
|
|
190
446
|
if (!config.enableCache)
|
|
191
447
|
return;
|
|
192
|
-
const
|
|
193
|
-
let maxExpiration = 0;
|
|
194
|
-
if (config.optimisticCache) {
|
|
195
|
-
// 如果开启了乐观缓存,且 L1 或 L2 中有任何一个设为 0(永不过期),则直接跳过定时清理
|
|
196
|
-
if (config.cacheExpiration === 0 || config.optimisticExpiration === 0) {
|
|
197
|
-
return;
|
|
198
|
-
}
|
|
199
|
-
// 只有两者都不为 0 时,才取它们的最大值作为物理清理时间
|
|
200
|
-
maxExpiration = Math.max(config.cacheExpiration, config.optimisticExpiration);
|
|
201
|
-
}
|
|
202
|
-
else {
|
|
203
|
-
// 如果没开启乐观缓存,且 L1 设为 0(永不过期),直接跳过
|
|
204
|
-
if (config.cacheExpiration === 0) {
|
|
205
|
-
return;
|
|
206
|
-
}
|
|
207
|
-
maxExpiration = config.cacheExpiration;
|
|
208
|
-
}
|
|
209
|
-
const threshold = now - maxExpiration * 60 * 60 * 1000;
|
|
210
|
-
// 清理解析缓存
|
|
211
|
-
await ctx.database.remove('sla_parse_cache', {
|
|
212
|
-
created_at: { $lt: threshold }
|
|
213
|
-
});
|
|
448
|
+
const fileAndTimelineThreshold = getRetentionThreshold(getCombinedRetentionHours(config));
|
|
214
449
|
// 清理文件缓存
|
|
215
|
-
const expiredFiles =
|
|
216
|
-
|
|
217
|
-
|
|
450
|
+
const expiredFiles = fileAndTimelineThreshold === null
|
|
451
|
+
? []
|
|
452
|
+
: await ctx.database.get('sla_file_cache', {
|
|
453
|
+
created_at: { $lt: fileAndTimelineThreshold }
|
|
454
|
+
});
|
|
218
455
|
for (const file of expiredFiles) {
|
|
219
456
|
try {
|
|
220
457
|
if (fs.existsSync(file.path)) {
|
|
@@ -225,9 +462,12 @@ function apply(ctx, config) {
|
|
|
225
462
|
logger.warn(`删除过期文件失败 ${file.path}: ${e}`);
|
|
226
463
|
}
|
|
227
464
|
}
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
465
|
+
if (fileAndTimelineThreshold !== null) {
|
|
466
|
+
await ctx.database.remove('sla_file_cache', {
|
|
467
|
+
created_at: { $lt: fileAndTimelineThreshold }
|
|
468
|
+
});
|
|
469
|
+
await pruneExpiredTimelineNodes(ctx, config);
|
|
470
|
+
}
|
|
231
471
|
if (expiredFiles.length > 0) {
|
|
232
472
|
logger.info(`已自动清理 ${expiredFiles.length} 个过期文件。`);
|
|
233
473
|
}
|
|
@@ -284,10 +524,27 @@ function apply(ctx, config) {
|
|
|
284
524
|
await ctx.database.remove('sla_group_settings', { guildId: session.guildId });
|
|
285
525
|
return '已重置为全局默认设置。';
|
|
286
526
|
});
|
|
527
|
+
cmd.subcommand('.migratecache', '迁移旧解析缓存到新时间线并校验', { authority: 4 })
|
|
528
|
+
.action(async () => {
|
|
529
|
+
const result = await migrateLegacyParseCache(ctx);
|
|
530
|
+
const status = result.mismatched === 0 ? '校验通过' : `校验失败 ${result.mismatched} 条`;
|
|
531
|
+
return [
|
|
532
|
+
'旧解析缓存迁移完成。',
|
|
533
|
+
`旧解析缓存记录: ${result.legacyParse}`,
|
|
534
|
+
`新增节点: ${result.migrated}`,
|
|
535
|
+
`已存在/续期: ${result.skipped}`,
|
|
536
|
+
`无效记录: ${result.invalid}`,
|
|
537
|
+
status,
|
|
538
|
+
].join('\n');
|
|
539
|
+
});
|
|
540
|
+
cmd.subcommand('.dropoldcache', '删除旧 sla_parse_cache 数据/表', { authority: 4 })
|
|
541
|
+
.action(async () => {
|
|
542
|
+
return await dropLegacyParseCache(ctx);
|
|
543
|
+
});
|
|
287
544
|
// 清除缓存指令
|
|
288
545
|
cmd.subcommand('.clean', '清除所有缓存和文件', { authority: 3 })
|
|
289
546
|
.action(async () => {
|
|
290
|
-
await ctx.database.remove('
|
|
547
|
+
await ctx.database.remove('sla_parse_timeline', {});
|
|
291
548
|
const allFiles = await ctx.database.get('sla_file_cache', {});
|
|
292
549
|
for (const file of allFiles) {
|
|
293
550
|
try {
|
|
@@ -301,6 +558,115 @@ function apply(ctx, config) {
|
|
|
301
558
|
await ctx.database.remove('sla_file_cache', {});
|
|
302
559
|
return '缓存及对应文件已清理。';
|
|
303
560
|
});
|
|
561
|
+
cmd.subcommand('.checkcache <url:string> [index:string]', '查看缓存时间线/读取指定节点', { authority: 1 })
|
|
562
|
+
.action(async ({ session }, url, index) => {
|
|
563
|
+
if (!session)
|
|
564
|
+
return '会话不可用。';
|
|
565
|
+
if (!config.enableCache)
|
|
566
|
+
return '缓存功能未启用。';
|
|
567
|
+
const links = await (0, core_1.resolveLinks)(url, ctx, config);
|
|
568
|
+
if (links.length === 0)
|
|
569
|
+
return '未在该链接中识别到支持的内容。';
|
|
570
|
+
const link = links[0];
|
|
571
|
+
const cacheKey = `${link.platform}:${link.id}`;
|
|
572
|
+
const nodes = await getTimelineNodes(ctx, cacheKey);
|
|
573
|
+
if (!index) {
|
|
574
|
+
if (nodes.length === 0)
|
|
575
|
+
return `未找到历史节点: ${cacheKey}`;
|
|
576
|
+
const latest = nodes[nodes.length - 1];
|
|
577
|
+
const lines = [
|
|
578
|
+
`历史节点: ${cacheKey}`,
|
|
579
|
+
summarizeParsedInfo(latest.data),
|
|
580
|
+
''
|
|
581
|
+
];
|
|
582
|
+
nodes.forEach((node, nodeIndex) => {
|
|
583
|
+
lines.push(`#${nodeIndex + 1} | ${formatTime(node.created_at)}`);
|
|
584
|
+
lines.push(summarizeDelta(node.delta));
|
|
585
|
+
});
|
|
586
|
+
return lines.join('\n');
|
|
587
|
+
}
|
|
588
|
+
let selectedIndex;
|
|
589
|
+
const normalizedIndex = index.trim().toLowerCase();
|
|
590
|
+
if (normalizedIndex === 'l' || normalizedIndex === 'latest') {
|
|
591
|
+
if (nodes.length === 0)
|
|
592
|
+
return `未找到历史节点: ${cacheKey}`;
|
|
593
|
+
selectedIndex = nodes.length;
|
|
594
|
+
}
|
|
595
|
+
else {
|
|
596
|
+
selectedIndex = Number.parseInt(normalizedIndex, 10);
|
|
597
|
+
if (!Number.isInteger(selectedIndex) || `${selectedIndex}` !== normalizedIndex || selectedIndex < 1) {
|
|
598
|
+
return '节点编号必须是从 1 开始的整数,或 l/latest。';
|
|
599
|
+
}
|
|
600
|
+
}
|
|
601
|
+
const node = nodes[selectedIndex - 1];
|
|
602
|
+
if (!node)
|
|
603
|
+
return `节点不存在: #${selectedIndex}`;
|
|
604
|
+
const result = { ...node.data };
|
|
605
|
+
result._cache = {
|
|
606
|
+
...result._cache,
|
|
607
|
+
parse: `Timeline #${selectedIndex} | 节点时间: ${formatTime(node.created_at)}`
|
|
608
|
+
};
|
|
609
|
+
const sendStats = { downloadTime: 0, sendTime: 0, errors: [] };
|
|
610
|
+
await (0, utils_1.sendResult)(ctx, session, config, result, logger, sendStats);
|
|
611
|
+
return;
|
|
612
|
+
});
|
|
613
|
+
cmd.subcommand('.forceparse <url:string>', '忽略缓存强制解析链接', { authority: 2 })
|
|
614
|
+
.action(async ({ session }, url) => {
|
|
615
|
+
if (!session)
|
|
616
|
+
return '会话不可用。';
|
|
617
|
+
const links = await (0, core_1.resolveLinks)(url, ctx, config);
|
|
618
|
+
if (links.length === 0)
|
|
619
|
+
return '未在该链接中识别到支持的内容。';
|
|
620
|
+
const link = links[0];
|
|
621
|
+
if (config.waitTip_Switch)
|
|
622
|
+
await session.send(config.waitTip_Switch);
|
|
623
|
+
const result = await (0, core_1.processLink)(ctx, config, link, session);
|
|
624
|
+
if (!result)
|
|
625
|
+
return `解析失败: ${link.platform}/${link.type}:${link.id}`;
|
|
626
|
+
const cacheKey = `${link.platform}:${link.id}`;
|
|
627
|
+
if (config.enableCache) {
|
|
628
|
+
await upsertTimelineIfChanged(ctx, config, cacheKey, result);
|
|
629
|
+
result._cache = { ...result._cache, parse: `强制刷新 | 缓存时间: ${formatTime(Date.now())}` };
|
|
630
|
+
}
|
|
631
|
+
const sendStats = { downloadTime: 0, sendTime: 0, errors: [] };
|
|
632
|
+
await (0, utils_1.sendResult)(ctx, session, config, result, logger, sendStats);
|
|
633
|
+
return;
|
|
634
|
+
});
|
|
635
|
+
cmd.subcommand('.directlink <url:string> [quality:string]', '获取视频/音频直链', { authority: 1 })
|
|
636
|
+
.action(async ({ session }, url, quality) => {
|
|
637
|
+
if (!session)
|
|
638
|
+
return '会话不可用。';
|
|
639
|
+
const links = await (0, core_1.resolveLinks)(url, ctx, config);
|
|
640
|
+
if (links.length === 0)
|
|
641
|
+
return '未在该链接中识别到支持的内容。';
|
|
642
|
+
const link = links[0];
|
|
643
|
+
let clarity = config.Video_ClarityPriority;
|
|
644
|
+
if (quality) {
|
|
645
|
+
const q = quality.trim().toLowerCase();
|
|
646
|
+
if (q === 'high' || q === 'h' || q === '高' || q === '高清晰度') {
|
|
647
|
+
clarity = '2';
|
|
648
|
+
}
|
|
649
|
+
else if (q === 'low' || q === 'l' || q === '低' || q === '低清晰度') {
|
|
650
|
+
clarity = '1';
|
|
651
|
+
}
|
|
652
|
+
else {
|
|
653
|
+
return `无效的画质参数: ${quality}。可用: high/h/高清晰度, low/l/低清晰度`;
|
|
654
|
+
}
|
|
655
|
+
}
|
|
656
|
+
const modifiedConfig = { ...config, Video_ClarityPriority: clarity, Max_size: Number.MAX_SAFE_INTEGER };
|
|
657
|
+
const result = await (0, core_1.processLink)(ctx, modifiedConfig, link, session);
|
|
658
|
+
if (!result)
|
|
659
|
+
return `解析失败: ${link.platform}/${link.type}:${link.id}`;
|
|
660
|
+
if (result.files.length === 0) {
|
|
661
|
+
return `未获取到直链: ${link.platform}/${link.type}:${link.id}\n该内容类型可能不支持直链获取。`;
|
|
662
|
+
}
|
|
663
|
+
const clarityLabel = clarity === '2' ? '高清晰度优先' : '低清晰度优先';
|
|
664
|
+
let output = `直链结果 (${link.platform}/${link.type}:${link.id}):\n画质策略: ${clarityLabel}\n标题: ${result.title}\n`;
|
|
665
|
+
for (const [i, file] of result.files.entries()) {
|
|
666
|
+
output += `\n[${i + 1}] ${file.type}: ${file.url}`;
|
|
667
|
+
}
|
|
668
|
+
return output;
|
|
669
|
+
});
|
|
304
670
|
cmd.subcommand('.refresh', '强制刷新所有 Cookie', { authority: 3 })
|
|
305
671
|
.action(async ({ session }) => {
|
|
306
672
|
await session?.send('🔄 开始执行 Cookie 全量刷新流程...');
|
|
@@ -476,9 +842,12 @@ function apply(ctx, config) {
|
|
|
476
842
|
}
|
|
477
843
|
// === 性能统计变量 ===
|
|
478
844
|
const startTotal = Date.now();
|
|
479
|
-
const sendStats = {
|
|
845
|
+
const sendStats = {
|
|
846
|
+
downloadTime: 0,
|
|
847
|
+
sendTime: 0,
|
|
848
|
+
errors: []
|
|
849
|
+
};
|
|
480
850
|
let parseTime = 0;
|
|
481
|
-
let isCache = false;
|
|
482
851
|
let status = "success";
|
|
483
852
|
let errorMsg = "";
|
|
484
853
|
let errorStack = "";
|
|
@@ -488,31 +857,25 @@ function apply(ctx, config) {
|
|
|
488
857
|
let optimisticData = null; // 用于暂存乐观缓存数据
|
|
489
858
|
let optimisticTime = 0; // 记录乐观缓存的生成时间
|
|
490
859
|
try {
|
|
491
|
-
// 1.
|
|
860
|
+
// 1. 查解析时间线的最新节点
|
|
492
861
|
if (config.enableCache) {
|
|
493
|
-
const
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
const
|
|
497
|
-
const ageMs = Date.now() - entry.created_at;
|
|
862
|
+
const latestNode = await getLatestTimelineNode(ctx, config, cacheKey);
|
|
863
|
+
if (latestNode) {
|
|
864
|
+
const checkedAt = latestNode.last_checked_at || latestNode.created_at;
|
|
865
|
+
const ageMs = Date.now() - checkedAt;
|
|
498
866
|
// 分别判断是否超过 L1 和 乐观缓存 时效
|
|
499
867
|
const isExpiredL1 = config.cacheExpiration > 0 && (ageMs > config.cacheExpiration * 60 * 60 * 1000);
|
|
500
868
|
const isExpiredL2 = config.optimisticExpiration > 0 && (ageMs > config.optimisticExpiration * 60 * 60 * 1000);
|
|
501
869
|
if (!isExpiredL1) {
|
|
502
870
|
logger.debug(`使用 L1 缓存解析结果: ${cacheKey}`);
|
|
503
|
-
result = { ...
|
|
504
|
-
const cacheTimeStr =
|
|
505
|
-
result.
|
|
506
|
-
isCache = true;
|
|
871
|
+
result = { ...latestNode.data }; // 浅拷贝,避免修改污染原缓存对象
|
|
872
|
+
const cacheTimeStr = formatTime(checkedAt);
|
|
873
|
+
result._cache = { ...result._cache, parse: `L1 命中 | 缓存时间: ${cacheTimeStr}` };
|
|
507
874
|
}
|
|
508
875
|
else if (config.optimisticCache && !isExpiredL2) {
|
|
509
876
|
logger.debug(`L1 缓存已过期,暂存乐观缓存备用: ${cacheKey}`);
|
|
510
|
-
optimisticData = { ...
|
|
511
|
-
optimisticTime =
|
|
512
|
-
}
|
|
513
|
-
else {
|
|
514
|
-
// 彻底过期,删除
|
|
515
|
-
await ctx.database.remove('sla_parse_cache', { key: cacheKey });
|
|
877
|
+
optimisticData = { ...latestNode.data }; // 浅拷贝备用
|
|
878
|
+
optimisticTime = checkedAt;
|
|
516
879
|
}
|
|
517
880
|
}
|
|
518
881
|
}
|
|
@@ -525,7 +888,6 @@ function apply(ctx, config) {
|
|
|
525
888
|
result = await pendingChecks.get(cacheKey) || null;
|
|
526
889
|
// 若合并任务返回的结果带有乐观缓存标记
|
|
527
890
|
if (result && result._isOptimisticFallback) {
|
|
528
|
-
isCache = true;
|
|
529
891
|
status = "optimistic_fallback";
|
|
530
892
|
}
|
|
531
893
|
}
|
|
@@ -540,10 +902,12 @@ function apply(ctx, config) {
|
|
|
540
902
|
if (optimisticData) {
|
|
541
903
|
logger.warn(`合并任务失败,触发乐观缓存回退: ${cacheKey} | Error: ${mergeErrDetail}`);
|
|
542
904
|
result = optimisticData;
|
|
543
|
-
const cacheTimeStr =
|
|
544
|
-
result.
|
|
905
|
+
const cacheTimeStr = formatTime(optimisticTime);
|
|
906
|
+
result._cache = {
|
|
907
|
+
...result._cache,
|
|
908
|
+
parse: `L2 回退: 并发请求异常 | 缓存时间: ${cacheTimeStr}`
|
|
909
|
+
};
|
|
545
910
|
result._isOptimisticFallback = true;
|
|
546
|
-
isCache = true;
|
|
547
911
|
status = "optimistic_fallback";
|
|
548
912
|
}
|
|
549
913
|
else {
|
|
@@ -561,8 +925,11 @@ function apply(ctx, config) {
|
|
|
561
925
|
if (!res) {
|
|
562
926
|
if (optimisticData) {
|
|
563
927
|
logger.warn(`API 返回 null,触发乐观缓存回退: ${cacheKey}`);
|
|
564
|
-
const cacheTimeStr =
|
|
565
|
-
optimisticData.
|
|
928
|
+
const cacheTimeStr = formatTime(optimisticTime);
|
|
929
|
+
optimisticData._cache = {
|
|
930
|
+
...optimisticData._cache,
|
|
931
|
+
parse: `L2 回退: 解析数据为空 | 缓存时间: ${cacheTimeStr}`
|
|
932
|
+
};
|
|
566
933
|
optimisticData._isOptimisticFallback = true;
|
|
567
934
|
return optimisticData;
|
|
568
935
|
}
|
|
@@ -570,11 +937,12 @@ function apply(ctx, config) {
|
|
|
570
937
|
}
|
|
571
938
|
// 解析成功且开启缓存,则写入 DB,刷新缓存时间戳
|
|
572
939
|
if (config.enableCache) {
|
|
573
|
-
|
|
574
|
-
|
|
575
|
-
|
|
576
|
-
|
|
577
|
-
}
|
|
940
|
+
const cacheTime = Date.now();
|
|
941
|
+
await upsertTimelineIfChanged(ctx, config, cacheKey, res);
|
|
942
|
+
res._cache = {
|
|
943
|
+
...res._cache,
|
|
944
|
+
parse: `未命中,已刷新 | 缓存时间: ${formatTime(cacheTime)}`
|
|
945
|
+
};
|
|
578
946
|
}
|
|
579
947
|
return res;
|
|
580
948
|
}
|
|
@@ -589,8 +957,11 @@ function apply(ctx, config) {
|
|
|
589
957
|
// 抛出异常(网络错误/封禁)时触发乐观回退
|
|
590
958
|
if (optimisticData) {
|
|
591
959
|
logger.warn(`解析抛出异常,触发乐观缓存回退: ${cacheKey} | Error: ${errDetail}`);
|
|
592
|
-
const cacheTimeStr =
|
|
593
|
-
optimisticData.
|
|
960
|
+
const cacheTimeStr = formatTime(optimisticTime);
|
|
961
|
+
optimisticData._cache = {
|
|
962
|
+
...optimisticData._cache,
|
|
963
|
+
parse: `L2 回退: 接口异常 | 缓存时间: ${cacheTimeStr}`
|
|
964
|
+
};
|
|
594
965
|
optimisticData._isOptimisticFallback = true;
|
|
595
966
|
return optimisticData;
|
|
596
967
|
}
|
|
@@ -609,7 +980,6 @@ function apply(ctx, config) {
|
|
|
609
980
|
result = await task;
|
|
610
981
|
// 识别最终是否使用了乐观缓存回退
|
|
611
982
|
if (result && result._isOptimisticFallback) {
|
|
612
|
-
isCache = true;
|
|
613
983
|
status = "optimistic_fallback";
|
|
614
984
|
}
|
|
615
985
|
}
|
|
@@ -678,7 +1048,7 @@ function apply(ctx, config) {
|
|
|
678
1048
|
raw_content_len: session.content ? session.content.length : 0, // 查明是否因长文本混排导致误触
|
|
679
1049
|
// === 3. 执行状态与缓存命中 ===
|
|
680
1050
|
status: status, // success, failed, error, optimistic_fallback
|
|
681
|
-
is_cache:
|
|
1051
|
+
is_cache: (sendStats.fileCacheHits || 0) > 0,
|
|
682
1052
|
// === 4. 关键排障配置状态 ===
|
|
683
1053
|
// 记录当时的运行配置,排查是否是特定模式下才报错(如本地下载+混合发送)
|
|
684
1054
|
cfg_using_local: config.usingLocal,
|
package/lib/types.d.ts
CHANGED
|
@@ -13,11 +13,16 @@ export interface ParsedInfo {
|
|
|
13
13
|
files: FileInfo[];
|
|
14
14
|
sourceUrl: string;
|
|
15
15
|
stats: string;
|
|
16
|
+
_cache?: ParsedInfoCacheStatus;
|
|
16
17
|
}
|
|
17
18
|
export interface FileInfo {
|
|
18
19
|
type: 'video' | 'audio' | 'generic';
|
|
19
20
|
url: string;
|
|
20
21
|
}
|
|
22
|
+
export interface ParsedInfoCacheStatus {
|
|
23
|
+
parse?: string;
|
|
24
|
+
file?: string;
|
|
25
|
+
}
|
|
21
26
|
export interface PluginConfig {
|
|
22
27
|
Video_ClarityPriority: '1' | '2';
|
|
23
28
|
Max_size: number;
|
|
@@ -100,6 +105,7 @@ declare module 'koishi' {
|
|
|
100
105
|
sla_parse_cache: SlaParseCache;
|
|
101
106
|
sla_file_cache: SlaFileCache;
|
|
102
107
|
sla_cookie_cache: SlaCookieCache;
|
|
108
|
+
sla_parse_timeline: SlaParseTimelineNode;
|
|
103
109
|
sla_group_settings: SlaGroupSettings;
|
|
104
110
|
}
|
|
105
111
|
}
|
|
@@ -119,6 +125,14 @@ export interface SlaCookieCache {
|
|
|
119
125
|
cookie: string;
|
|
120
126
|
updated_at: number;
|
|
121
127
|
}
|
|
128
|
+
export interface SlaParseTimelineNode {
|
|
129
|
+
node_id: string;
|
|
130
|
+
cache_key: string;
|
|
131
|
+
data: ParsedInfo;
|
|
132
|
+
delta: string;
|
|
133
|
+
created_at: number;
|
|
134
|
+
last_checked_at: number;
|
|
135
|
+
}
|
|
122
136
|
export interface SlaGroupSettings {
|
|
123
137
|
guildId: string;
|
|
124
138
|
custom_parsers: Record<string, boolean>;
|
package/lib/utils.d.ts
CHANGED
|
@@ -44,14 +44,20 @@ export declare function sendResult(ctx: Context, session: Session, config: Plugi
|
|
|
44
44
|
downloadTime: number;
|
|
45
45
|
sendTime: number;
|
|
46
46
|
errors: string[];
|
|
47
|
+
fileCacheHits?: number;
|
|
48
|
+
fileCacheMisses?: number;
|
|
47
49
|
}): Promise<void>;
|
|
48
50
|
export declare function sendResult_plain(ctx: Context, session: Session, config: PluginConfig, result: ParsedInfo, logger: Logger, statsRef: {
|
|
49
51
|
downloadTime: number;
|
|
50
52
|
sendTime: number;
|
|
51
53
|
errors: string[];
|
|
54
|
+
fileCacheHits?: number;
|
|
55
|
+
fileCacheMisses?: number;
|
|
52
56
|
}): Promise<void>;
|
|
53
57
|
export declare function sendResult_forward(ctx: Context, session: Session, config: PluginConfig, result: ParsedInfo, logger: Logger, mixed_sending: boolean | undefined, statsRef: {
|
|
54
58
|
downloadTime: number;
|
|
55
59
|
sendTime: number;
|
|
56
60
|
errors: string[];
|
|
61
|
+
fileCacheHits?: number;
|
|
62
|
+
fileCacheMisses?: number;
|
|
57
63
|
}): Promise<void>;
|
package/lib/utils.js
CHANGED
|
@@ -385,7 +385,7 @@ function parseHtmlToSegments(html) {
|
|
|
385
385
|
}
|
|
386
386
|
return segments;
|
|
387
387
|
}
|
|
388
|
-
async function downloadAndMapUrl(ctx, url, proxy, userAgent, localDownloadDir, onebotReadDir, logger, enableCache) {
|
|
388
|
+
async function downloadAndMapUrl(ctx, url, proxy, userAgent, localDownloadDir, onebotReadDir, logger, enableCache, statsRef) {
|
|
389
389
|
await fs.promises.mkdir(localDownloadDir, { recursive: true });
|
|
390
390
|
// 1. 计算 Hash
|
|
391
391
|
const hash = (0, crypto_1.createHash)('md5').update(url).digest('hex');
|
|
@@ -400,6 +400,14 @@ async function downloadAndMapUrl(ctx, url, proxy, userAgent, localDownloadDir, o
|
|
|
400
400
|
if (fs.existsSync(cachedPath)) {
|
|
401
401
|
const filename = path_1.default.basename(cachedPath);
|
|
402
402
|
const onebotPath = path_1.default.posix.join(onebotReadDir, filename);
|
|
403
|
+
await ctx.database.upsert('sla_file_cache', [{
|
|
404
|
+
hash,
|
|
405
|
+
path: cachedPath,
|
|
406
|
+
url,
|
|
407
|
+
created_at: Date.now()
|
|
408
|
+
}]);
|
|
409
|
+
if (statsRef)
|
|
410
|
+
statsRef.fileCacheHits = (statsRef.fileCacheHits || 0) + 1;
|
|
403
411
|
logger.debug(`缓存命中: ${url} -> ${cachedPath}`);
|
|
404
412
|
return `file://${onebotPath}`;
|
|
405
413
|
}
|
|
@@ -427,6 +435,8 @@ async function downloadAndMapUrl(ctx, url, proxy, userAgent, localDownloadDir, o
|
|
|
427
435
|
url,
|
|
428
436
|
created_at: Date.now()
|
|
429
437
|
}]);
|
|
438
|
+
if (statsRef)
|
|
439
|
+
statsRef.fileCacheHits = (statsRef.fileCacheHits || 0) + 1;
|
|
430
440
|
return fileUrl;
|
|
431
441
|
}
|
|
432
442
|
const attemptDownload = () => new Promise((resolve, reject) => {
|
|
@@ -443,7 +453,7 @@ async function downloadAndMapUrl(ctx, url, proxy, userAgent, localDownloadDir, o
|
|
|
443
453
|
if (res.statusCode && res.statusCode >= 300 && res.statusCode < 400 && res.headers.location) {
|
|
444
454
|
req.destroy();
|
|
445
455
|
logger.debug(`重定向: ${url} -> ${res.headers.location}`);
|
|
446
|
-
resolve(downloadAndMapUrl(ctx, res.headers.location, proxy, userAgent, localDownloadDir, onebotReadDir, logger, enableCache));
|
|
456
|
+
resolve(downloadAndMapUrl(ctx, res.headers.location, proxy, userAgent, localDownloadDir, onebotReadDir, logger, enableCache, statsRef));
|
|
447
457
|
return;
|
|
448
458
|
}
|
|
449
459
|
if (res.statusCode !== 200) {
|
|
@@ -479,6 +489,8 @@ async function downloadAndMapUrl(ctx, url, proxy, userAgent, localDownloadDir, o
|
|
|
479
489
|
logger.warn(`写入文件缓存数据库失败: ${dbErr}`);
|
|
480
490
|
}
|
|
481
491
|
}
|
|
492
|
+
if (statsRef)
|
|
493
|
+
statsRef.fileCacheMisses = (statsRef.fileCacheMisses || 0) + 1;
|
|
482
494
|
resolve(fileUrl);
|
|
483
495
|
})
|
|
484
496
|
.catch((err) => {
|
|
@@ -641,7 +653,7 @@ async function isUserAdmin(session, userId) {
|
|
|
641
653
|
if (!memberInfo)
|
|
642
654
|
return false;
|
|
643
655
|
const adminRoles = ["owner", "admin", "administrator"];
|
|
644
|
-
const memberRoles =
|
|
656
|
+
const memberRoles = (memberInfo.roles || []).map(r => r.name).filter((n) => !!n);
|
|
645
657
|
for (const role of memberRoles) {
|
|
646
658
|
if (adminRoles.includes(role.toLowerCase()))
|
|
647
659
|
return true;
|
|
@@ -653,6 +665,15 @@ async function isUserAdmin(session, userId) {
|
|
|
653
665
|
}
|
|
654
666
|
}
|
|
655
667
|
async function sendResult(ctx, session, config, result, logger, statsRef) {
|
|
668
|
+
if (config.enableCache && config.usingLocal) {
|
|
669
|
+
const availability = await getLocalCacheAvailability(ctx, result);
|
|
670
|
+
if (availability.total > 0) {
|
|
671
|
+
result._cache = {
|
|
672
|
+
...result._cache,
|
|
673
|
+
file: `命中 ${availability.hits}/${availability.total}`
|
|
674
|
+
};
|
|
675
|
+
}
|
|
676
|
+
}
|
|
656
677
|
if (!session.channel) {
|
|
657
678
|
await sendResult_plain(ctx, session, config, result, logger, statsRef);
|
|
658
679
|
return;
|
|
@@ -669,6 +690,47 @@ async function sendResult(ctx, session, config, result, logger, statsRef) {
|
|
|
669
690
|
break;
|
|
670
691
|
}
|
|
671
692
|
}
|
|
693
|
+
function collectResultRemoteUrls(result) {
|
|
694
|
+
const urls = new Set();
|
|
695
|
+
if (result.coverUrl)
|
|
696
|
+
urls.add(result.coverUrl);
|
|
697
|
+
if (result.mainbody) {
|
|
698
|
+
for (const match of result.mainbody.matchAll(/<img\s[^>]*src\s*=\s*["']?([^"'>\s]+)["']?/gi)) {
|
|
699
|
+
urls.add(match[1]);
|
|
700
|
+
}
|
|
701
|
+
}
|
|
702
|
+
if (Array.isArray(result.files)) {
|
|
703
|
+
for (const file of result.files) {
|
|
704
|
+
if (file?.url)
|
|
705
|
+
urls.add(file.url);
|
|
706
|
+
}
|
|
707
|
+
}
|
|
708
|
+
return [...urls].filter(url => /^https?:\/\//i.test(url));
|
|
709
|
+
}
|
|
710
|
+
async function getLocalCacheAvailability(ctx, result) {
|
|
711
|
+
const urls = collectResultRemoteUrls(result);
|
|
712
|
+
let hits = 0;
|
|
713
|
+
for (const url of urls) {
|
|
714
|
+
const hash = (0, crypto_1.createHash)('md5').update(url).digest('hex');
|
|
715
|
+
const cached = await ctx.database.get('sla_file_cache', hash);
|
|
716
|
+
if (cached.length > 0 && cached[0].path && fs.existsSync(cached[0].path)) {
|
|
717
|
+
hits++;
|
|
718
|
+
}
|
|
719
|
+
}
|
|
720
|
+
return { total: urls.length, hits };
|
|
721
|
+
}
|
|
722
|
+
function buildCachePlaceholder(result, statsRef) {
|
|
723
|
+
const lines = [];
|
|
724
|
+
if (result._cache?.parse)
|
|
725
|
+
lines.push(`解析缓存: ${result._cache.parse}`);
|
|
726
|
+
if (result._cache?.file) {
|
|
727
|
+
lines.push(`文件缓存: ${result._cache.file}`);
|
|
728
|
+
}
|
|
729
|
+
else if ((statsRef.fileCacheHits || 0) > 0 || (statsRef.fileCacheMisses || 0) > 0) {
|
|
730
|
+
lines.push(`文件缓存: 命中 ${statsRef.fileCacheHits || 0} / 新下载 ${statsRef.fileCacheMisses || 0}`);
|
|
731
|
+
}
|
|
732
|
+
return lines.join('\n');
|
|
733
|
+
}
|
|
672
734
|
async function sendResult_plain(ctx, session, config, result, logger, statsRef) {
|
|
673
735
|
logger.debug('进入普通发送');
|
|
674
736
|
const localDownloadDir = config.localDownloadDir;
|
|
@@ -685,7 +747,7 @@ async function sendResult_plain(ctx, session, config, result, logger, statsRef)
|
|
|
685
747
|
if (config.usingLocal) {
|
|
686
748
|
const t = Date.now(); // 计时开始
|
|
687
749
|
try {
|
|
688
|
-
mediaCoverUrl = await downloadAndMapUrl(ctx, result.coverUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache);
|
|
750
|
+
mediaCoverUrl = await downloadAndMapUrl(ctx, result.coverUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache, statsRef);
|
|
689
751
|
logger.debug(`封面已下载: ${mediaCoverUrl}`);
|
|
690
752
|
}
|
|
691
753
|
catch (e) {
|
|
@@ -708,7 +770,7 @@ async function sendResult_plain(ctx, session, config, result, logger, statsRef)
|
|
|
708
770
|
const remoteUrl = match[1];
|
|
709
771
|
if (config.usingLocal) {
|
|
710
772
|
try {
|
|
711
|
-
const localUrl = await downloadAndMapUrl(ctx, remoteUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache);
|
|
773
|
+
const localUrl = await downloadAndMapUrl(ctx, remoteUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache, statsRef);
|
|
712
774
|
urlMap[remoteUrl] = localUrl;
|
|
713
775
|
logger.debug(`正文图片已下载: ${localUrl}`);
|
|
714
776
|
}
|
|
@@ -736,6 +798,7 @@ async function sendResult_plain(ctx, session, config, result, logger, statsRef)
|
|
|
736
798
|
message = message.replace(/{sourceUrl}/g, escapeHtml(result.sourceUrl || ''));
|
|
737
799
|
message = message.replace(/{cover}/g, mediaCoverUrl ? koishi_1.h.image(mediaCoverUrl).toString() : '');
|
|
738
800
|
message = message.replace(/{stats}/g, escapeHtml(result.stats || ''));
|
|
801
|
+
message = message.replace(/{cache}/g, escapeHtml(buildCachePlaceholder(result, statsRef)));
|
|
739
802
|
// 清理空行
|
|
740
803
|
const cleanMessage = message.split('\n').filter(line => line.trim() !== '' || line.includes('<')).join('\n');
|
|
741
804
|
logger.debug(`解析结果: \n ${JSON.stringify(result, null, 2)}`);
|
|
@@ -783,7 +846,7 @@ async function sendResult_plain(ctx, session, config, result, logger, statsRef)
|
|
|
783
846
|
let localUrl = remoteUrl;
|
|
784
847
|
if (config.usingLocal) {
|
|
785
848
|
const t = Date.now();
|
|
786
|
-
localUrl = await downloadAndMapUrl(ctx, remoteUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache);
|
|
849
|
+
localUrl = await downloadAndMapUrl(ctx, remoteUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache, statsRef);
|
|
787
850
|
statsRef.downloadTime += Date.now() - t;
|
|
788
851
|
}
|
|
789
852
|
if (!localUrl)
|
|
@@ -841,7 +904,7 @@ async function sendResult_forward(ctx, session, config, result, logger, mixed_se
|
|
|
841
904
|
if (config.usingLocal) {
|
|
842
905
|
const t = Date.now();
|
|
843
906
|
try {
|
|
844
|
-
mediaCoverUrl = await downloadAndMapUrl(ctx, result.coverUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache);
|
|
907
|
+
mediaCoverUrl = await downloadAndMapUrl(ctx, result.coverUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache, statsRef);
|
|
845
908
|
}
|
|
846
909
|
catch (e) {
|
|
847
910
|
logger.warn('封面下载失败', e);
|
|
@@ -864,7 +927,7 @@ async function sendResult_forward(ctx, session, config, result, logger, mixed_se
|
|
|
864
927
|
try {
|
|
865
928
|
// 去重下载
|
|
866
929
|
if (!urlMap[url]) {
|
|
867
|
-
urlMap[url] = await downloadAndMapUrl(ctx, url, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache);
|
|
930
|
+
urlMap[url] = await downloadAndMapUrl(ctx, url, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache, statsRef);
|
|
868
931
|
}
|
|
869
932
|
}
|
|
870
933
|
catch (e) {
|
|
@@ -888,6 +951,7 @@ async function sendResult_forward(ctx, session, config, result, logger, mixed_se
|
|
|
888
951
|
message = message.replace(/{authorName}/g, result.authorName || '');
|
|
889
952
|
message = message.replace(/{sourceUrl}/g, result.sourceUrl || '');
|
|
890
953
|
message = message.replace(/{stats}/g, result.stats || '');
|
|
954
|
+
message = message.replace(/{cache}/g, buildCachePlaceholder(result, statsRef));
|
|
891
955
|
const lines = message.split('\n').filter(line => line.trim() !== '');
|
|
892
956
|
const mainSegments = [];
|
|
893
957
|
for (let i = 0; i < lines.length; i++) {
|
|
@@ -983,7 +1047,7 @@ async function sendResult_forward(ctx, session, config, result, logger, mixed_se
|
|
|
983
1047
|
let localUrl = remoteUrl;
|
|
984
1048
|
if (config.usingLocal) {
|
|
985
1049
|
const t = Date.now();
|
|
986
|
-
localUrl = await downloadAndMapUrl(ctx, remoteUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache);
|
|
1050
|
+
localUrl = await downloadAndMapUrl(ctx, remoteUrl, proxy, config.userAgent, localDownloadDir, onebotReadDir, logger, config.enableCache, statsRef);
|
|
987
1051
|
statsRef.downloadTime += Date.now() - t;
|
|
988
1052
|
}
|
|
989
1053
|
if (!localUrl)
|