dsh-vidlens 1.2.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +52 -2
- package/RELEASING.md +25 -10
- package/bridge/vidlens_bridge.py +38 -2
- package/index.js +247 -71
- package/package.json +2 -2
- package/tools/selftest.mjs +34 -3
package/README.md
CHANGED
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
# VidLens for DeepSeek Harness
|
|
2
2
|
|
|
3
3
|
[](https://github.com/cctv2333/dsh-vidlens/actions/workflows/ci.yml)
|
|
4
|
+
[](https://www.npmjs.com/package/dsh-vidlens)
|
|
5
|
+
[](https://pypi.org/project/vidlens-mcp/)
|
|
6
|
+
[](https://github.com/topics/dsh-plugin)
|
|
4
7
|
[](./LICENSE)
|
|
5
8
|
|
|
6
9
|
把 [vidlens](https://github.com/cctv2333/vidlens) 从一个 stdio MCP 外壳变成 **DSH 原生插件**:18 个 `vidlens_*` 工具、图片直接进对话、按工具设超时、外加三个零下载的缓存工具。
|
|
@@ -99,8 +102,9 @@ dsh plugin --profile desktop add dsh-vidlens
|
|
|
99
102
|
|
|
100
103
|
| 工具 | 下载量 | 用途 |
|
|
101
104
|
|---|---|---|
|
|
105
|
+
| `vidlens_watch(url, minutes=5, cursor="next")` | 每段该窗音频(或缓存) | **新增**:一段一段看(默认 5 分钟、约 0.5k 字),返回该段要点 + 「已看 X / 共 Y(%)」,并要求先汇报、再问用户要不要继续 |
|
|
102
106
|
| `vidlens_window(url, start, end)` | 0(有缓存/字幕)或该窗音频 | 某时间窗的台词与停顿 |
|
|
103
|
-
| `vidlens_brief(url,
|
|
107
|
+
| `vidlens_brief(url, ..., level="digest")` | 平台字幕 0;无字幕才下音频 | **摘要**(默认约 1.5k 字:章节 + 5 句关键台词 + 全文路径);`level="full"` 才给 6–9k 字完整报告(结构/停顿/低置信词/广告段落/风险) |
|
|
104
108
|
| `vidlens_seek(query, url='')` | 0(**新增**) | 在缓存转写里按关键词查原话 → 时间码 |
|
|
105
109
|
| `vidlens_audio(url, start, end)` | 该段音频 | SenseVoice 语种/情感/事件 + 节拍与能量 |
|
|
106
110
|
| `vidlens_soundscape(url, start, end)` | 该段音频 | **新增**:AudioSet 527 类声音事件(吃饭/敲击/水声/碰撞…)+ 概率 + 时间码 |
|
|
@@ -112,6 +116,39 @@ dsh plugin --profile desktop add dsh-vidlens
|
|
|
112
116
|
| `vidlens_sheet(url, start, end, cols, rows)` | 该段(20s≈0.9MB) | 带绝对时间码的拼图 |
|
|
113
117
|
| `vidlens_frames(url, at=[...])` | 该段 | 最多 6 张精确帧(读小字用) |
|
|
114
118
|
| `vidlens_scan(url, n=24)` | 最低画质整片(7min≈14-21MB,用完即删) | 全片抽样拼图(无音频视频) |
|
|
119
|
+
| `vidlens_ocr(url, start, end, n=12)` | 该段最低画质 | **把画面上的字读成文本**(本地 OCR,离线);无人声、无字幕的视频靠它,输出带时间码 |
|
|
120
|
+
|
|
121
|
+
`vidlens_ocr` 需要一次性装 OCR 依赖(RapidOCR,Apache-2.0,PP-OCRv6 模型随包分发,约 30MB,装完离线):
|
|
122
|
+
|
|
123
|
+
```
|
|
124
|
+
"D:\vidlens\vidlens\.venv\Scripts\python.exe" -m pip install rapidocr
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
没装也不会坏:`vidlens_ocr` 会返回一句「装它只要一步 + pip 命令」,其他工具不受影响。
|
|
128
|
+
|
|
129
|
+
### 上下文预算(为什么默认给摘要)
|
|
130
|
+
|
|
131
|
+
一次「看视频」如果按老路子走,实测是这么多字符:
|
|
132
|
+
|
|
133
|
+
| 调用 | 字符 | 说明 |
|
|
134
|
+
|---|---|---|
|
|
135
|
+
| `meta` + `plan` | ~730 | 便宜 |
|
|
136
|
+
| `window` × 3(各 1 分钟) | ~2,800 | ~0.9k/分钟 |
|
|
137
|
+
| `soundscape`(60 秒) | ~1,500 | 12 窗 × 8 标签 |
|
|
138
|
+
| `brief`(旧版全量) | **~9,000** | 6–9k 字 ≈ 4–6k tokens |
|
|
139
|
+
| 合计 | **~14,200** | 还没算图片 |
|
|
140
|
+
|
|
141
|
+
关键不在单次有多大,而在于**这些字符之后每一轮都在上下文里**。一个 64k 窗口,
|
|
142
|
+
两三个视频 + 正常对话就会顶到上限,于是出现「看完视频就超上下文」。
|
|
143
|
+
|
|
144
|
+
所以默认改成三档,按需升级:
|
|
145
|
+
|
|
146
|
+
1. `vidlens_brief` → **摘要**(约 0.7–1.5k 字:标题/时长/语音覆盖/章节/5 句均匀取样的关键台词/全文路径);
|
|
147
|
+
2. `vidlens_window` → 只读某一段(约 0.8k 字/分钟),读细节用它,别用 brief 代;
|
|
148
|
+
3. `vidlens_brief(level="full")` → 完整报告(6–9k 字),**只在用户明确要全文时**用一次。
|
|
149
|
+
|
|
150
|
+
全文始终写在 `out/<id>/brief.md`,不会丢;摘要里直接给路径和下一次该调什么。
|
|
151
|
+
`vidlens_window` 另有 80 行上限,超了会明说「还有 N 行,缩小范围」而不是无限倾倒。
|
|
115
152
|
|
|
116
153
|
**判断 / 长视频 / 自检**
|
|
117
154
|
|
|
@@ -205,7 +242,7 @@ git tag v1.2.0 && git push origin v1.2.0
|
|
|
205
242
|
|
|
206
243
|
`release.yml` 会 `npm pack`、`npm publish --access public`,走 **Trusted Publishing(OIDC)**——
|
|
207
244
|
CI 里没有任何长期 token。npm 的 publisher 只能在包存在后配置,所以**首次发布要在本地做一次**
|
|
208
|
-
(`npm
|
|
245
|
+
(`npm stage publish` → `npm stage approve`,全程不需要把 token 交给 CI,见 `RELEASING.md`),之后打 tag 即可。
|
|
209
246
|
|
|
210
247
|
> 注意:PowerShell 5.1 不支持 `&&`,用 `;` 或分行。
|
|
211
248
|
|
|
@@ -228,6 +265,19 @@ CI 里没有任何长期 token。npm 的 publisher 只能在包存在后配置
|
|
|
228
265
|
没有在浏览器里验证(无头浏览器对 `file:` / `data:` 导航超时),这一项留待用户目视确认。
|
|
229
266
|
|
|
230
267
|
|
|
268
|
+
## 路线图
|
|
269
|
+
|
|
270
|
+
产品级路线图(已做 / 未做 / 验收标准 / 不做清单)在工程仓库:
|
|
271
|
+
[vidlens/ROADMAP.md](https://github.com/cctv2333/vidlens/blob/main/ROADMAP.md)。
|
|
272
|
+
|
|
273
|
+
插件专属待办,按工作量从小到大:
|
|
274
|
+
|
|
275
|
+
1. `Config` schema(现在 `status: absent`,只能手改 YAML)
|
|
276
|
+
2. `/vl <url>` 斜杠命令
|
|
277
|
+
3. `present` 交付物卡片(brief.md / 拼图)
|
|
278
|
+
4. 右侧栏 Web 面板(缓存列表 + brief 预览 + 时间轴)
|
|
279
|
+
5. 设置页(模型路径 / 代理 / cookie)
|
|
280
|
+
|
|
231
281
|
## 边界与已知限制
|
|
232
282
|
|
|
233
283
|
- **job 状态在进程内存**:`vidlens_job_start` 起的任务,桥进程重启(超时/取消/宿主重启)后会消失;已下的缓存不丢,重跑即可。
|
package/RELEASING.md
CHANGED
|
@@ -1,22 +1,37 @@
|
|
|
1
1
|
# 发布(维护者)
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
3
|
+
首次发布用 **staged publishing**(`npm stage publish` **不需要 2FA**,批准时才要);
|
|
4
|
+
之后全部走 **OIDC Trusted Publishing**,CI 里没有任何长期 token。
|
|
5
5
|
|
|
6
|
-
|
|
6
|
+
> 别用 `npm publish` 做首次发布:npm 现在对它强制 2FA,CLI 的 `--otp` 也常被同样拒绝
|
|
7
|
+
> (`403 ... bypass 2fa enabled is required`)。
|
|
8
|
+
|
|
9
|
+
## 1. 首次发布:stage(不需要 2FA)
|
|
7
10
|
|
|
8
11
|
```powershell
|
|
12
|
+
npm --version # 需要 >= 11.15.0,低了就 npm i -g npm@latest
|
|
9
13
|
cd D:\vidlens\vidlens-mcp-dsh
|
|
10
|
-
npm
|
|
11
|
-
npm
|
|
14
|
+
npm stage publish --access public
|
|
15
|
+
npm stage list # 记下 id,等 status 变成 staged
|
|
12
16
|
```
|
|
13
17
|
|
|
14
|
-
|
|
18
|
+
新包会先在 registry 上生成一个 `0.0.0-stage` 占位版本(公开可见),
|
|
19
|
+
真正要发的版本留在 staging 区,**批准后才上线**。
|
|
20
|
+
|
|
21
|
+
## 2. 批准(要 2FA,只有你能做)
|
|
22
|
+
|
|
23
|
+
**用命令行,最可靠**(网页的 Staged Packages 入口时有时无,`/package/<name>/staged` 可能 404):
|
|
24
|
+
|
|
25
|
+
```powershell
|
|
26
|
+
npm stage approve <stage-id> # 会提示输入 6 位 OTP
|
|
27
|
+
npm view dsh-vidlens versions # 批准后应能看到目标版本
|
|
28
|
+
```
|
|
15
29
|
|
|
16
|
-
>
|
|
17
|
-
>
|
|
30
|
+
> OTP 只有 30 秒有效,输慢了就重来。
|
|
31
|
+
> 如果 2FA 是「安全密钥(WebAuthn)」而不是验证器 App,CLI 无法输入 OTP——
|
|
32
|
+
> 先把 npm 账号的 2FA 改成 TOTP(Authenticator app)再重试。
|
|
18
33
|
|
|
19
|
-
##
|
|
34
|
+
## 3. 配置 Trusted Publisher(一次性)
|
|
20
35
|
|
|
21
36
|
1. 打开 <https://www.npmjs.com/package/dsh-vidlens/access>
|
|
22
37
|
(或包页 → **Settings** → **Trusted Publisher**)
|
|
@@ -24,7 +39,7 @@ npm publish --access public # 账号开了 2FA 会要一次 OTP;这是交互
|
|
|
24
39
|
|
|
25
40
|
| 字段 | 值 |
|
|
26
41
|
|---|---|
|
|
27
|
-
| Organization or user | `cctv2333` |
|
|
42
|
+
| Organization or user | `cctv2333` ← **GitHub** 用户名(不是 npm 的 `ghost120`) |
|
|
28
43
|
| Repository | `dsh-vidlens` |
|
|
29
44
|
| Workflow filename | `release.yml`(只写文件名,带 `.yml`) |
|
|
30
45
|
| Environment name | 留空 |
|
package/bridge/vidlens_bridge.py
CHANGED
|
@@ -86,6 +86,10 @@ if str(PROJECT) not in sys.path:
|
|
|
86
86
|
sys.path.insert(0, str(PROJECT))
|
|
87
87
|
os.environ.setdefault("PYTHONIOENCODING", "utf-8")
|
|
88
88
|
os.environ.setdefault("HF_HUB_DISABLE_XET", "1")
|
|
89
|
+
# The bridge turns `Image` objects into file paths (see `_image_entry`), so a
|
|
90
|
+
# large sheet costs the plugin nothing. Say so, or vl_mcp's `auto` policy would
|
|
91
|
+
# drop the picture here and the tool would return a path with no attachment.
|
|
92
|
+
os.environ.setdefault("VIDLENS_INLINE_IMAGES", "always")
|
|
89
93
|
|
|
90
94
|
# ---------------------------------------------------------------------- imports
|
|
91
95
|
IMPORT_ERROR = None
|
|
@@ -102,6 +106,36 @@ except BaseException as exc: # noqa: BLE001 - report, never crash the loop
|
|
|
102
106
|
IMPORT_ERROR = f"{type(exc).__name__}: {exc}"
|
|
103
107
|
traceback.print_exc(file=sys.stderr)
|
|
104
108
|
|
|
109
|
+
_INSTALL_HINT = (
|
|
110
|
+
"vidlens 依赖没法导入。两种修法(任选一种):\n"
|
|
111
|
+
" 1) pip install vidlens-mcp —— 然后让插件用同一个解释器:\n"
|
|
112
|
+
" 插件配置项 pythonPath,或环境变量 VIDLENS_PYTHON。\n"
|
|
113
|
+
" 2) 用已有的 checkout:把 VIDLENS_PROJECT 设成含 vidlens.py 的目录。\n"
|
|
114
|
+
" 本进程已经找过: {project}"
|
|
115
|
+
)
|
|
116
|
+
|
|
117
|
+
|
|
118
|
+
def _import_error_text() -> str:
|
|
119
|
+
"""Turn a bare ImportError into something the user can act on.
|
|
120
|
+
|
|
121
|
+
The raw traceback goes to stderr (visible in the bridge log); what reaches the
|
|
122
|
+
model is the failure plus the two ways to fix it. Without this the whole plugin
|
|
123
|
+
answers every call with ``No module named 'vidlens'`` and no lead on why.
|
|
124
|
+
"""
|
|
125
|
+
err = IMPORT_ERROR or ""
|
|
126
|
+
hint = _INSTALL_HINT.format(project=PROJECT)
|
|
127
|
+
match = re.search(r"No module named '([^']+)'", err)
|
|
128
|
+
if not match:
|
|
129
|
+
return f"{err}\n\n{hint}"
|
|
130
|
+
missing = match.group(1).split(".")[0]
|
|
131
|
+
if missing == "mcp":
|
|
132
|
+
pip = "pip install mcp"
|
|
133
|
+
elif missing in ("vl_mcp",) or missing.startswith("vidlens"):
|
|
134
|
+
pip = "pip install vidlens-mcp"
|
|
135
|
+
else:
|
|
136
|
+
pip = f"pip install {missing}"
|
|
137
|
+
return f"{err}\n\n缺 Python 包 {missing}。先试:{pip}\n{hint}"
|
|
138
|
+
|
|
105
139
|
|
|
106
140
|
# ---------------------------------------------------------------- result shaping
|
|
107
141
|
_EXT_MEDIA = {
|
|
@@ -399,11 +433,13 @@ def _install_dispatch() -> None:
|
|
|
399
433
|
"subs": vl.video_subs,
|
|
400
434
|
"brief": vl.video_brief,
|
|
401
435
|
"window": vl.video_window,
|
|
436
|
+
"watch": vl.video_watch,
|
|
402
437
|
"audio": vl.video_audio,
|
|
403
438
|
"soundscape": vl.video_soundscape,
|
|
404
439
|
"sheet": vl.video_sheet,
|
|
405
440
|
"frames": vl.video_frames,
|
|
406
441
|
"scan": vl.video_visual_scan,
|
|
442
|
+
"ocr": vl.video_ocr,
|
|
407
443
|
"comments": vl.video_comments,
|
|
408
444
|
"job_start": vl.video_job_start,
|
|
409
445
|
"job": vl.video_job,
|
|
@@ -422,7 +458,7 @@ _install_dispatch()
|
|
|
422
458
|
|
|
423
459
|
def _handle(req: dict) -> dict:
|
|
424
460
|
if IMPORT_ERROR:
|
|
425
|
-
raise RuntimeError("vidlens 依赖导入失败: " +
|
|
461
|
+
raise RuntimeError("vidlens 依赖导入失败: " + _import_error_text())
|
|
426
462
|
tool = str(req.get("tool") or "")
|
|
427
463
|
entry = DISPATCH.get(tool)
|
|
428
464
|
if entry is None:
|
|
@@ -457,7 +493,7 @@ def main() -> int:
|
|
|
457
493
|
"project": str(PROJECT),
|
|
458
494
|
"python": sys.version.split()[0],
|
|
459
495
|
"tools": sorted(DISPATCH),
|
|
460
|
-
"error": IMPORT_ERROR,
|
|
496
|
+
"error": _import_error_text() if IMPORT_ERROR else None,
|
|
461
497
|
})
|
|
462
498
|
for line in sys.stdin:
|
|
463
499
|
line = line.strip()
|
package/index.js
CHANGED
|
@@ -348,23 +348,24 @@ const num = (description) => ({ type: 'number', description })
|
|
|
348
348
|
const int = (description) => ({ type: 'integer', description })
|
|
349
349
|
const bool = (description) => ({ type: 'boolean', description })
|
|
350
350
|
|
|
351
|
-
const URL_ARG = str('
|
|
351
|
+
const URL_ARG = str('视频链接(公开可看)')
|
|
352
|
+
const LANG_ARG = str('zh|yue|ja|en')
|
|
353
|
+
const MODEL_ARG = str('whisper 模型,默认 small')
|
|
354
|
+
const PROMPT_ARG = str('ASR 提示词(方言/术语)')
|
|
352
355
|
|
|
353
356
|
const TOOLS = [
|
|
354
357
|
{
|
|
355
358
|
name: 'vidlens_meta',
|
|
356
359
|
bridge: 'meta',
|
|
357
360
|
timeoutMs: 120_000,
|
|
358
|
-
description: '
|
|
359
|
-
+ '用来判断「这个视频值不值得看」,也是后续所有工具的第一步。',
|
|
361
|
+
description: '0 下载:标题/UP主/时长/播放/字幕情况。先看值不值得看。',
|
|
360
362
|
parameters: P({ url: URL_ARG }, ['url']),
|
|
361
363
|
},
|
|
362
364
|
{
|
|
363
365
|
name: 'vidlens_plan',
|
|
364
366
|
bridge: 'plan',
|
|
365
367
|
timeoutMs: 180_000,
|
|
366
|
-
description: '
|
|
367
|
-
+ '长视频先调它,再挑 2-3 段用 vidlens_window(读台词)与 vidlens_sheet(看画面)。',
|
|
368
|
+
description: '0 下载:按切点/静音分章节。长视频先调它,再挑 2-3 段。',
|
|
368
369
|
parameters: P({
|
|
369
370
|
url: URL_ARG,
|
|
370
371
|
window: num('章节窗口秒数,默认 300(夹在 60–3600 之间)'),
|
|
@@ -374,8 +375,7 @@ const TOOLS = [
|
|
|
374
375
|
name: 'vidlens_subs',
|
|
375
376
|
bridge: 'subs',
|
|
376
377
|
timeoutMs: 180_000,
|
|
377
|
-
description: '
|
|
378
|
-
+ '方言/口音重、本地 ASR 不确定时,先看这里有没有原字幕;没有字幕再看画面硬字幕。',
|
|
378
|
+
description: '0 下载:平台字幕(人工优先)。有字幕就别做 ASR。',
|
|
379
379
|
parameters: P({
|
|
380
380
|
url: URL_ARG,
|
|
381
381
|
lang: str('语言偏好,默认 zh;粤语可传 yue'),
|
|
@@ -386,43 +386,54 @@ const TOOLS = [
|
|
|
386
386
|
name: 'vidlens_brief',
|
|
387
387
|
bridge: 'brief',
|
|
388
388
|
timeoutMs: 900_000,
|
|
389
|
-
description: '
|
|
390
|
-
+ '超过 20 分钟且无平台字幕时默认只给章节建议;确实要全量逐字才传 full=true。只看画面用 vidlens_sheet / vidlens_frames,不要下整片。',
|
|
389
|
+
description: '看视频:默认摘要约 1.5k 字(章节+5 句关键台词+全文路径),直接回答;要 6-9k 字全文才传 level="full"。无字幕才下音频做 ASR。',
|
|
391
390
|
parameters: P({
|
|
392
391
|
url: URL_ARG,
|
|
393
|
-
lang:
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
392
|
+
lang: LANG_ARG,
|
|
393
|
+
level: str('digest|full'),
|
|
394
|
+
model: MODEL_ARG,
|
|
395
|
+
scan_cuts: bool('扫镜头切点(下14-21MB)'),
|
|
396
|
+
refresh: bool('忽略缓存'),
|
|
397
|
+
full: bool('全量转写'),
|
|
398
|
+
prompt: PROMPT_ARG,
|
|
399
|
+
}, ['url']),
|
|
400
|
+
},
|
|
401
|
+
{
|
|
402
|
+
name: 'vidlens_watch',
|
|
403
|
+
bridge: 'watch',
|
|
404
|
+
timeoutMs: 600_000,
|
|
405
|
+
description: '分段看:一次一段(默认 5 分钟、约 0.5k 字),回进度与下一步。只在用户要「细看/逐段」时用。',
|
|
406
|
+
parameters: P({
|
|
407
|
+
url: URL_ARG,
|
|
408
|
+
minutes: num('每段分钟,默认5'),
|
|
409
|
+
cursor: str('next|restart|stop'),
|
|
410
|
+
lang: LANG_ARG,
|
|
411
|
+
model: MODEL_ARG,
|
|
399
412
|
}, ['url']),
|
|
400
413
|
},
|
|
401
414
|
{
|
|
402
415
|
name: 'vidlens_window',
|
|
403
416
|
bridge: 'window',
|
|
404
417
|
timeoutMs: 300_000,
|
|
405
|
-
description: '
|
|
406
|
-
+ '适合「刚才那 30 秒他说了什么」,是长视频抽样的主力工具。',
|
|
418
|
+
description: '读某时间窗的台词与停顿(约 0.8k 字/分钟)。有缓存则 0 下载。',
|
|
407
419
|
parameters: P({
|
|
408
420
|
url: URL_ARG,
|
|
409
|
-
start: str('
|
|
410
|
-
end: str('
|
|
411
|
-
lang:
|
|
412
|
-
model:
|
|
413
|
-
prompt:
|
|
421
|
+
start: str('起点(秒/MM:SS)'),
|
|
422
|
+
end: str('终点(秒/MM:SS)'),
|
|
423
|
+
lang: LANG_ARG,
|
|
424
|
+
model: MODEL_ARG,
|
|
425
|
+
prompt: PROMPT_ARG,
|
|
414
426
|
}, ['url']),
|
|
415
427
|
},
|
|
416
428
|
{
|
|
417
429
|
name: 'vidlens_audio',
|
|
418
430
|
bridge: 'audio',
|
|
419
431
|
timeoutMs: 300_000,
|
|
420
|
-
description: '
|
|
421
|
-
+ '只下这一段音频(20 秒约 0.5MB)。情感、事件、BPM 都是近似信号,必须结合画面与平台元数据核对。',
|
|
432
|
+
description: '听一段:语种/情感/音频事件 + 节拍能量(本地 SenseVoice,启发式)。',
|
|
422
433
|
parameters: P({
|
|
423
434
|
url: URL_ARG,
|
|
424
|
-
start: str('
|
|
425
|
-
end: str('
|
|
435
|
+
start: str('起点(秒/MM:SS)'),
|
|
436
|
+
end: str('终点(秒/MM:SS)'),
|
|
426
437
|
lang: str('语言提示,默认 zh'),
|
|
427
438
|
}, ['url']),
|
|
428
439
|
},
|
|
@@ -430,14 +441,11 @@ const TOOLS = [
|
|
|
430
441
|
name: 'vidlens_soundscape',
|
|
431
442
|
bridge: 'soundscape',
|
|
432
443
|
timeoutMs: 300_000,
|
|
433
|
-
description: '
|
|
434
|
-
+ '能覆盖吃饭/咀嚼(Chewing)、敲桌子/拍(Knock / Tap / Wood block)、水声(Water / Water tap)、'
|
|
435
|
-
+ '杯子碰撞、拖拽、塑料袋等;**喝水/吞咽没有对应类别**,只能靠水声+画面推断。'
|
|
436
|
-
+ '只下这一段音频(60 秒约 1MB)。概率是模型输出,不是事实断言。',
|
|
444
|
+
description: '声音事件:AudioSet 527 类(敲击/吃饭/水声…)+ 概率 + 时间码。识别不了喝水。',
|
|
437
445
|
parameters: P({
|
|
438
446
|
url: URL_ARG,
|
|
439
|
-
start: str('
|
|
440
|
-
end: str('
|
|
447
|
+
start: str('起点(秒/MM:SS)'),
|
|
448
|
+
end: str('终点(秒/MM:SS)'),
|
|
441
449
|
top_k: int('每个窗口保留几个标签,默认 8(上限 20)'),
|
|
442
450
|
min_prob: num('低于这个概率的标签不列出,默认 0.15'),
|
|
443
451
|
window: num('分析窗长秒数,默认 10(2–30)'),
|
|
@@ -449,14 +457,13 @@ const TOOLS = [
|
|
|
449
457
|
bridge: 'sheet',
|
|
450
458
|
images: true,
|
|
451
459
|
timeoutMs: 300_000,
|
|
452
|
-
description: '
|
|
453
|
-
+ '「看一眼这段画面在发生什么」就用它;比 vidlens_frames 便宜得多。',
|
|
460
|
+
description: '看某段画面:拼图带绝对时间码,图片直接进对话(20 秒 ≈ 0.9MB)。',
|
|
454
461
|
parameters: P({
|
|
455
462
|
url: URL_ARG,
|
|
456
|
-
start: str('
|
|
457
|
-
end: str('
|
|
458
|
-
cols: int('
|
|
459
|
-
rows: int('
|
|
463
|
+
start: str('起点(秒/MM:SS)'),
|
|
464
|
+
end: str('终点(秒/MM:SS)'),
|
|
465
|
+
cols: int('列数'),
|
|
466
|
+
rows: int('行数'),
|
|
460
467
|
height: int('片段下载高度,默认 360'),
|
|
461
468
|
}, ['url']),
|
|
462
469
|
},
|
|
@@ -465,11 +472,11 @@ const TOOLS = [
|
|
|
465
472
|
bridge: 'frames',
|
|
466
473
|
images: true,
|
|
467
474
|
timeoutMs: 300_000,
|
|
468
|
-
description: '
|
|
475
|
+
description: '按秒取帧(最多 6 张,读小字用)。比 sheet 贵,先试 sheet。',
|
|
469
476
|
parameters: P({
|
|
470
477
|
url: URL_ARG,
|
|
471
478
|
at: { type: 'array', items: { type: 'number' }, description: '要取帧的秒数列表,如 [19.5, 21.0],最多 6 个' },
|
|
472
|
-
width: int('
|
|
479
|
+
width: int('宽度px'),
|
|
473
480
|
}, ['url', 'at']),
|
|
474
481
|
},
|
|
475
482
|
{
|
|
@@ -477,8 +484,7 @@ const TOOLS = [
|
|
|
477
484
|
bridge: 'scan',
|
|
478
485
|
images: true,
|
|
479
486
|
timeoutMs: 900_000,
|
|
480
|
-
description: '
|
|
481
|
-
+ '用于无音频或长时间无语音的视频;代价是下最低画质视频(7 分钟约 14-21MB),用完即删。',
|
|
487
|
+
description: '全片抽样拼图(无音频视频用)。要下最低画质整片,用完即删。',
|
|
482
488
|
parameters: P({
|
|
483
489
|
url: URL_ARG,
|
|
484
490
|
n: int('抽样帧数,默认 24(4–48)'),
|
|
@@ -486,13 +492,25 @@ const TOOLS = [
|
|
|
486
492
|
rows: int('拼图行数,默认 4'),
|
|
487
493
|
}, ['url']),
|
|
488
494
|
},
|
|
495
|
+
{
|
|
496
|
+
name: 'vidlens_ocr',
|
|
497
|
+
bridge: 'ocr',
|
|
498
|
+
timeoutMs: 600_000,
|
|
499
|
+
description: '读画面上的字(本地 OCR,离线):无人声/无字幕视频靠它,输出带时间码。',
|
|
500
|
+
parameters: P({
|
|
501
|
+
url: URL_ARG,
|
|
502
|
+
start: str('起点,默认 0(秒或 MM:SS)'),
|
|
503
|
+
end: str('终点,默认整片末尾'),
|
|
504
|
+
n: int('取帧数,默认 12(1–24)'),
|
|
505
|
+
width: int('取帧宽度,默认 1280'),
|
|
506
|
+
}, ['url']),
|
|
507
|
+
},
|
|
489
508
|
{
|
|
490
509
|
name: 'vidlens_comments',
|
|
491
510
|
bridge: 'comments',
|
|
492
511
|
images: true,
|
|
493
512
|
timeoutMs: 180_000,
|
|
494
|
-
description: '
|
|
495
|
-
+ '评论是最敌对的文本来源,已做清洗与注入扫描;它是「观众怎么看」,不是「视频讲了什么」。',
|
|
513
|
+
description: '评论区信号(B站免登录):互动比/争议度/高赞。默认不带配图。',
|
|
496
514
|
parameters: P({
|
|
497
515
|
url: URL_ARG,
|
|
498
516
|
limit: int('抓多少条评论,默认 40'),
|
|
@@ -502,23 +520,22 @@ const TOOLS = [
|
|
|
502
520
|
name: 'vidlens_job_start',
|
|
503
521
|
bridge: 'job_start',
|
|
504
522
|
timeoutMs: 90_000,
|
|
505
|
-
description: '
|
|
506
|
-
+ '普通分析先用 vidlens_brief。任务状态保存在工作进程内存里,进程重启后需重跑(缓存仍在)。',
|
|
523
|
+
description: '起后台任务(长视频防超时),拿 job_id 再轮询 vidlens_job。',
|
|
507
524
|
parameters: P({
|
|
508
525
|
url: URL_ARG,
|
|
509
|
-
lang:
|
|
510
|
-
model:
|
|
511
|
-
scan_cuts: bool('
|
|
526
|
+
lang: LANG_ARG,
|
|
527
|
+
model: MODEL_ARG,
|
|
528
|
+
scan_cuts: bool('扫镜头切点(下14-21MB)'),
|
|
512
529
|
comments: bool('顺带附上评论区分析'),
|
|
513
530
|
full: bool('全量逐字转写'),
|
|
514
|
-
prompt:
|
|
531
|
+
prompt: PROMPT_ARG,
|
|
515
532
|
}, ['url']),
|
|
516
533
|
},
|
|
517
534
|
{
|
|
518
535
|
name: 'vidlens_job',
|
|
519
536
|
bridge: 'job',
|
|
520
537
|
timeoutMs: 120_000,
|
|
521
|
-
description: '
|
|
538
|
+
description: '查后台任务:运行中给阶段与百分比,完成给结果。',
|
|
522
539
|
parameters: P({
|
|
523
540
|
job_id: str('vidlens_job_start 返回的 job_id'),
|
|
524
541
|
wait_seconds: num('最多等多少秒,默认 20(夹在 0–60)'),
|
|
@@ -528,8 +545,7 @@ const TOOLS = [
|
|
|
528
545
|
name: 'vidlens_library',
|
|
529
546
|
bridge: 'library',
|
|
530
547
|
timeoutMs: 60_000,
|
|
531
|
-
description: '
|
|
532
|
-
+ '开工前先看它,能避免重复下载;它也是 vidlens_seek 的索引。',
|
|
548
|
+
description: '0 网络:列本地已缓存视频,避免重复下载。',
|
|
533
549
|
parameters: P({
|
|
534
550
|
limit: int('最多列多少个,默认 20(上限 100)'),
|
|
535
551
|
query: str('按标题或 id 过滤(可选)'),
|
|
@@ -539,8 +555,7 @@ const TOOLS = [
|
|
|
539
555
|
name: 'vidlens_seek',
|
|
540
556
|
bridge: 'seek',
|
|
541
557
|
timeoutMs: 60_000,
|
|
542
|
-
description: '
|
|
543
|
-
+ '给定 url 时只查那一个视频;没命中就先用 vidlens_brief 建立转写。',
|
|
558
|
+
description: '0 网络:在缓存转写里按关键词查原话→时间码。',
|
|
544
559
|
parameters: P({
|
|
545
560
|
query: str('要查的关键词或原话片段'),
|
|
546
561
|
url: str('可选:只在这个视频里查'),
|
|
@@ -551,8 +566,7 @@ const TOOLS = [
|
|
|
551
566
|
name: 'vidlens_compare',
|
|
552
567
|
bridge: 'compare',
|
|
553
568
|
timeoutMs: 600_000,
|
|
554
|
-
description: '
|
|
555
|
-
+ '用来快速筛选「这几个视频先看哪个」。最多 8 个链接。',
|
|
569
|
+
description: '0 媒体下载:多个链接的时长/字幕/播放量对比,先决定看哪个。',
|
|
556
570
|
parameters: P({
|
|
557
571
|
urls: { type: 'array', items: { type: 'string' }, description: '要对比的链接列表(最多 8 个)' },
|
|
558
572
|
}, ['urls']),
|
|
@@ -561,8 +575,7 @@ const TOOLS = [
|
|
|
561
575
|
name: 'vidlens_models',
|
|
562
576
|
bridge: 'models',
|
|
563
577
|
timeoutMs: 30_000,
|
|
564
|
-
description: '
|
|
565
|
-
+ '模型缺失时先调它,再问用户是否安装。',
|
|
578
|
+
description: '本地模型目录与硬件建议(只读,不自动下载)。',
|
|
566
579
|
parameters: P({
|
|
567
580
|
action: str('list(默认)看目录;recommend 看本机建议'),
|
|
568
581
|
}),
|
|
@@ -571,24 +584,150 @@ const TOOLS = [
|
|
|
571
584
|
name: 'vidlens_doctor',
|
|
572
585
|
bridge: 'doctor',
|
|
573
586
|
timeoutMs: 180_000,
|
|
574
|
-
description: '自检:python
|
|
587
|
+
description: '自检:python/ffmpeg/yt-dlp/依赖/模型/磁盘/网络,断了直接说。',
|
|
588
|
+
parameters: P({}),
|
|
589
|
+
},
|
|
590
|
+
{
|
|
591
|
+
name: 'vidlens_budget',
|
|
592
|
+
bridge: 'budget',
|
|
593
|
+
local: true,
|
|
594
|
+
timeoutMs: 5_000,
|
|
595
|
+
description: '查本对话已用多少上下文预算,是否已进强制摘要模式。',
|
|
575
596
|
parameters: P({}),
|
|
576
597
|
},
|
|
577
598
|
]
|
|
578
599
|
|
|
579
|
-
//
|
|
600
|
+
// --------------------------------------------------------------- context budget
|
|
601
|
+
// DSH hands a tool no session id, but every dispatch carries the Agent it runs
|
|
602
|
+
// for, and there is one Agent per conversation. Keying the ledger on it gives
|
|
603
|
+
// per-chat accounting without inventing ids. The ledger is advisory for the
|
|
604
|
+
// model and a hard clamp for the expensive arguments: past the ceiling, brief is
|
|
605
|
+
// forced to digest, watch cannot widen its window, comments cannot ship base64.
|
|
606
|
+
const BUDGET_WARN = 12_000
|
|
607
|
+
const BUDGET_HARD = 24_000
|
|
608
|
+
const MEMO_TTL_MS = 20 * 60 * 1000
|
|
609
|
+
const ledgers = new WeakMap()
|
|
610
|
+
let nextAgentId = 0
|
|
611
|
+
const fallbackLedger = { chars: 0, calls: 0, sessionId: 'shared', memo: new Map() }
|
|
612
|
+
|
|
613
|
+
function ledgerFor(exec) {
|
|
614
|
+
const agent = exec && exec.agent
|
|
615
|
+
if (!agent || typeof agent !== 'object') return fallbackLedger
|
|
616
|
+
let ledger = ledgers.get(agent)
|
|
617
|
+
if (!ledger) {
|
|
618
|
+
nextAgentId += 1
|
|
619
|
+
ledger = { chars: 0, calls: 0, sessionId: `s${nextAgentId}`, memo: new Map() }
|
|
620
|
+
ledgers.set(agent, ledger)
|
|
621
|
+
}
|
|
622
|
+
return ledger
|
|
623
|
+
}
|
|
580
624
|
|
|
581
|
-
|
|
625
|
+
function memoKey(tool, args) {
|
|
626
|
+
// Volatile arguments must not collapse into the same key.
|
|
627
|
+
const stable = {}
|
|
628
|
+
for (const k of Object.keys(args || {}).sort()) {
|
|
629
|
+
if (k === 'refresh' || k === 'force' || k === 'session') continue
|
|
630
|
+
stable[k] = args[k]
|
|
631
|
+
}
|
|
632
|
+
try {
|
|
633
|
+
return `${tool.bridge}|${JSON.stringify(stable)}`
|
|
634
|
+
} catch {
|
|
635
|
+
return ''
|
|
636
|
+
}
|
|
637
|
+
}
|
|
638
|
+
|
|
639
|
+
function repeatStub(hit) {
|
|
640
|
+
const head = String(hit.head || '').replace(/\n/g, ' ').slice(0, 120)
|
|
641
|
+
return ['[重复调用,未重跑] 这次调用的参数与上文那次完全相同,结果没有变化,所以不再重传'
|
|
642
|
+
+ `(上次回传 ${hit.chars} 字)。`,
|
|
643
|
+
head ? `上次开头:${head}` : '',
|
|
644
|
+
'要看新东西:改参数(换区间/换 level,或传 refresh=true)——原样重试只会再占一遍上下文。']
|
|
645
|
+
.filter(Boolean).join('\n')
|
|
646
|
+
}
|
|
647
|
+
|
|
648
|
+
function failureStub(hit) {
|
|
649
|
+
return ['[上次同样的调用失败过,未重试]', String(hit.error || '').slice(0, 240),
|
|
650
|
+
'原样重试只会再烧一次上下文;换 URL/参数,或先 vidlens_doctor 看环境。'].join('\n')
|
|
651
|
+
}
|
|
652
|
+
|
|
653
|
+
function noteSpend(ledger, text) {
|
|
654
|
+
ledger.chars += String(text || '').length
|
|
655
|
+
ledger.calls += 1
|
|
656
|
+
}
|
|
657
|
+
|
|
658
|
+
function overBudget(ledger) {
|
|
659
|
+
return ledger.chars >= BUDGET_HARD
|
|
660
|
+
}
|
|
582
661
|
|
|
583
|
-
|
|
662
|
+
function enforceBudget(ledger, tool, args) {
|
|
663
|
+
const shaped = { ...(args && typeof args === 'object' ? args : {}) }
|
|
664
|
+
if (!overBudget(ledger)) return shaped
|
|
665
|
+
if (tool.bridge === 'brief') shaped.level = 'digest'
|
|
666
|
+
if (tool.bridge === 'watch') {
|
|
667
|
+
const minutes = Number(shaped.minutes)
|
|
668
|
+
shaped.minutes = Number.isFinite(minutes) && minutes > 0 ? Math.min(minutes, 5) : 5
|
|
669
|
+
}
|
|
670
|
+
if (tool.bridge === 'comments') shaped.with_images = false
|
|
671
|
+
return shaped
|
|
672
|
+
}
|
|
584
673
|
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
674
|
+
function budgetNote(ledger) {
|
|
675
|
+
if (ledger.chars < BUDGET_WARN) return ''
|
|
676
|
+
const kb = (ledger.chars / 1000).toFixed(1)
|
|
677
|
+
const pct = Math.round((ledger.chars / BUDGET_HARD) * 100)
|
|
678
|
+
const head = `\n\n[预算] 本对话已回传约 ${kb}k 字(${ledger.calls} 次,占软上限 ${pct}%)。`
|
|
679
|
+
if (overBudget(ledger)) {
|
|
680
|
+
return head + '已超软上限:后续只能摘要/分段,level=full 与评论配图已被自动关掉。'
|
|
681
|
+
}
|
|
682
|
+
return head + '再加一份全文会明显挤占上下文;能回答就别再加料。'
|
|
683
|
+
}
|
|
590
684
|
|
|
685
|
+
function budgetReport(ledger) {
|
|
686
|
+
const tokens = Math.round(ledger.chars / 1.4)
|
|
687
|
+
const state = overBudget(ledger)
|
|
688
|
+
? '已超软上限(brief 强制 digest、watch 每段 ≤5 分钟、评论图关闭)'
|
|
689
|
+
: ledger.chars >= BUDGET_WARN ? '接近软上限(每次回执会带预算提示)' : '正常'
|
|
690
|
+
return ['# 上下文预算台账(本对话)',
|
|
691
|
+
`- 已回传文字: ${ledger.chars} 字(中文约 ${tokens} tokens)`,
|
|
692
|
+
`- 工具调用: ${ledger.calls} 次`,
|
|
693
|
+
`- 软上限 ${BUDGET_WARN} 字 / 硬上限 ${BUDGET_HARD} 字`,
|
|
694
|
+
`- 状态: ${state}`,
|
|
695
|
+
'',
|
|
696
|
+
'口径:只统计本插件返回的文字长度,不含图片和你自己的推理;重载插件或新会话会重新计数。'].join('\n')
|
|
697
|
+
}
|
|
698
|
+
|
|
699
|
+
// ----------------------------------------------------------------------- apply
|
|
700
|
+
|
|
701
|
+
const PROMPT = `## vidlens(本地视频取证)
|
|
702
|
+
|
|
703
|
+
拿到公开视频链接时,默认「一次答完」,别让用户陪你看片:
|
|
704
|
+
|
|
705
|
+
1. \`vidlens_meta\`(0 下载)确认时长/字幕;
|
|
706
|
+
2. \`vidlens_brief\`(默认 digest,约 1.5k 字:章节 + 5 句关键台词 + 全文路径);
|
|
707
|
+
3. **直接给结论**:讲了什么、立场/结构、值不值得看、哪里可疑。
|
|
708
|
+
不要问「要继续看吗」,不要一段一段汇报——用户要的是结论,不是陪你走流程。
|
|
709
|
+
4. 摘要只抽样看了几个点,所以结论要说「粗看」;不要把它说成逐字看完。
|
|
710
|
+
**结论前先写一行覆盖范围**:音频 ✓ / 画面 ✗(或 ✓)/ 评论 ✗——工具结果里已经是这个格式,
|
|
711
|
+
照抄不要改。没看画面就说没看,别让「我读完了转写」读成「我看完了视频」。
|
|
712
|
+
要更硬的证据再补一次 \`vidlens_window\`(某段台词)或 \`vidlens_soundscape\`(声音),仍然**一次答完**。
|
|
713
|
+
|
|
714
|
+
**只有用户明确要「细看 / 详细 / 逐段 / 每段都说」时**才分段:
|
|
715
|
+
\`vidlens_watch\` 一段一段看(默认 5 分钟、约 0.5k 字),每段先把这一段讲给用户、
|
|
716
|
+
报进度(已看 05:00 / 40:45),再问「继续还是先讲这段」。用户说继续才调下一段。
|
|
717
|
+
|
|
718
|
+
工具顺序(需要更多证据时才往下走):
|
|
719
|
+
1. \`vidlens_plan\`(0 下载)拿章节,挑 2–3 段;
|
|
720
|
+
2. \`vidlens_window\` 读台词,\`vidlens_sheet\` 看画面(图片直接出现在结果里);
|
|
721
|
+
3. 平台字幕优先:\`vidlens_subs\`(0 下载);没有字幕才让 \`vidlens_brief\` 走本地 ASR;
|
|
722
|
+
4. 超过 20 分钟且无字幕时,\`vidlens_brief\` 只给章节是正常行为;要全量逐字用 \`vidlens_job_start(full=true)\` → \`vidlens_job\` 轮询。
|
|
723
|
+
|
|
724
|
+
- **细看才分段**:只有用户要「细看 / 逐段」时才用 \`vidlens_watch\`;默认模式不要主动分段、也不要说「要不要继续看」。
|
|
725
|
+
分段时每段先把这一段讲给用户(内容 + 初步判断),报进度(「已看 05:00 / 40:45,12%」),再问一次要不要继续。
|
|
591
726
|
- 开工前先 \`vidlens_library\`,已处理过的视频不要重复下载;问「他哪分钟说过 X」用 \`vidlens_seek\`。
|
|
727
|
+
- **上下文预算**:\`vidlens_brief\` 默认只给摘要(约 1.5k 字);\`level="full"\` 是 6–9k 字,只在用户明确要全文时才用,
|
|
728
|
+
而且同一个视频**不要重复调用**(结果已经在上文里了)。看细节用 \`vidlens_window\`(约 0.8k 字/分钟),
|
|
729
|
+
不要用 brief 代替 window。图片不要连发:一次 1 张,\`vidlens_frames\` 一次最多 2 张,除非用户要求更多;
|
|
730
|
+
\`vidlens_comments\` 默认不返回配图(base64 很贵),要看评论图才传 with_images=true。
|
|
592
731
|
- 问「有没有敲桌子/吃饭/水声/杂音」用 \`vidlens_soundscape\`(AudioSet 527 类 + 概率 + 时间码);
|
|
593
732
|
它识别不了「喝水/吞咽」这类动作,别把概率当事实。
|
|
594
733
|
- 一次评估多个链接用 \`vidlens_compare\`(0 媒体下载)。
|
|
@@ -618,14 +757,47 @@ export function apply(ctx, config) {
|
|
|
618
757
|
isConcurrencySafe: () => true,
|
|
619
758
|
timeoutMs: tool.timeoutMs || DEFAULT_TIMEOUT_MS,
|
|
620
759
|
async execute(args, exec) {
|
|
621
|
-
const
|
|
622
|
-
|
|
760
|
+
const ledger = ledgerFor(exec)
|
|
761
|
+
if (tool.bridge === 'budget') return { text: budgetReport(ledger) }
|
|
762
|
+
const shaped = enforceBudget(ledger, tool, args)
|
|
763
|
+
if (tool.bridge === 'watch') shaped.session = ledger.sessionId
|
|
764
|
+
if (shaped.refresh === true || shaped.force === true) {
|
|
765
|
+
const stale = memoKey(tool, shaped)
|
|
766
|
+
if (stale) ledger.memo.delete(stale)
|
|
767
|
+
}
|
|
768
|
+
const key = memoKey(tool, shaped)
|
|
769
|
+
const hit = key ? ledger.memo.get(key) : null
|
|
770
|
+
if (hit && Date.now() - hit.at < MEMO_TTL_MS) {
|
|
771
|
+
return { text: hit.ok ? repeatStub(hit) : failureStub(hit) }
|
|
772
|
+
}
|
|
773
|
+
let res
|
|
774
|
+
try {
|
|
775
|
+
res = await bridge.call({ tool: tool.bridge, args: shaped },
|
|
776
|
+
{ signal: exec?.signal, timeoutMs: tool.timeoutMs })
|
|
777
|
+
} catch (err) {
|
|
778
|
+
if (key) {
|
|
779
|
+
ledger.memo.set(key, {
|
|
780
|
+
ok: false, at: Date.now(), error: String(err?.message || err),
|
|
781
|
+
})
|
|
782
|
+
}
|
|
783
|
+
throw err
|
|
784
|
+
}
|
|
623
785
|
const value = { text: res.text || '' }
|
|
624
786
|
if (tool.images && res.images.length) {
|
|
625
787
|
const { refs, notes } = await attachImages(ctx, res.images)
|
|
626
788
|
if (refs.length) value.images = refs
|
|
627
789
|
if (notes.length) value.text += '\n\n' + notes.map((n) => `(${n})`).join('\n')
|
|
628
790
|
}
|
|
791
|
+
if (key) {
|
|
792
|
+
ledger.memo.set(key, {
|
|
793
|
+
ok: true,
|
|
794
|
+
at: Date.now(),
|
|
795
|
+
chars: value.text.length,
|
|
796
|
+
head: value.text.slice(0, 160),
|
|
797
|
+
})
|
|
798
|
+
}
|
|
799
|
+
noteSpend(ledger, value.text)
|
|
800
|
+
value.text += budgetNote(ledger)
|
|
629
801
|
return value
|
|
630
802
|
},
|
|
631
803
|
}
|
|
@@ -638,4 +810,8 @@ export function apply(ctx, config) {
|
|
|
638
810
|
}
|
|
639
811
|
}
|
|
640
812
|
|
|
641
|
-
export const __internals = {
|
|
813
|
+
export const __internals = {
|
|
814
|
+
TOOLS, Bridge, resolveProjectDir, resolvePython, PROMPT, attachImages,
|
|
815
|
+
BUDGET_WARN, BUDGET_HARD, enforceBudget, budgetNote, budgetReport, noteSpend,
|
|
816
|
+
memoKey, repeatStub, failureStub,
|
|
817
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
{
|
|
1
|
+
{
|
|
2
2
|
"name": "dsh-vidlens",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.3.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "VidLens for DeepSeek Harness: turn a public video into an evidence table (timecoded transcript, sampling sheets, sound events, comments) without downloading the whole file. 18 native tools backed by the local vidlens Python core.",
|
|
6
6
|
"keywords": [
|
package/tools/selftest.mjs
CHANGED
|
@@ -40,13 +40,44 @@ for (const t of tools) {
|
|
|
40
40
|
assert(`${t.name}: parameters is an object schema`,
|
|
41
41
|
t.parameters && t.parameters.type === 'object' && t.parameters.properties,
|
|
42
42
|
JSON.stringify(t.parameters || {}).slice(0, 120))
|
|
43
|
-
|
|
43
|
+
// Schemas cost tokens on every request, so "short but specific" is the goal.
|
|
44
|
+
// 16 chars still rules out placeholders; the old 30-char bar just forced padding.
|
|
45
|
+
assert(`${t.name}: description >= 16 chars`, String(t.description || '').length >= 16,
|
|
46
|
+
String(t.description || ''))
|
|
44
47
|
assert(`${t.name}: has a deadline`, Number(t.timeoutMs) > 0, `${t.timeoutMs}ms`)
|
|
45
48
|
for (const req of t.parameters.required || []) {
|
|
46
49
|
assert(`${t.name}: required '${req}' is declared`, !!t.parameters.properties[req])
|
|
47
50
|
}
|
|
48
51
|
}
|
|
49
52
|
assert('prompt section mentions the sampling order', /vidlens_meta/.test(mod.__internals.PROMPT))
|
|
53
|
+
assert('prompt defaults to one coarse answer', /直接给结论/.test(mod.__internals.PROMPT))
|
|
54
|
+
|
|
55
|
+
// ------------------------------------------------------------- 1b. budget ledger
|
|
56
|
+
{
|
|
57
|
+
const I = mod.__internals
|
|
58
|
+
const ledger = { chars: 0, calls: 0 }
|
|
59
|
+
assert('budget: no note below the warn line', I.budgetNote(ledger) === '')
|
|
60
|
+
I.noteSpend(ledger, 'x'.repeat(I.BUDGET_WARN + 10))
|
|
61
|
+
assert('budget: note appears past the warn line', /预算/.test(I.budgetNote(ledger)))
|
|
62
|
+
const brief = I.TOOLS.find((t) => t.bridge === 'brief')
|
|
63
|
+
const watch = I.TOOLS.find((t) => t.bridge === 'watch')
|
|
64
|
+
const comments = I.TOOLS.find((t) => t.bridge === 'comments')
|
|
65
|
+
assert('budget: full brief is allowed below the ceiling', I.enforceBudget(ledger, brief, { level: 'full' }).level === 'full')
|
|
66
|
+
ledger.chars = I.BUDGET_HARD + 1
|
|
67
|
+
assert('budget: brief is clamped to digest past the ceiling', I.enforceBudget(ledger, brief, { level: 'full' }).level === 'digest')
|
|
68
|
+
assert('budget: watch is clamped to 5 minutes', I.enforceBudget(ledger, watch, { minutes: 30 }).minutes === 5)
|
|
69
|
+
assert('budget: comment images are forced off', I.enforceBudget(ledger, comments, { with_images: true }).with_images === false)
|
|
70
|
+
assert('budget: report renders', /台账/.test(I.budgetReport(ledger)))
|
|
71
|
+
|
|
72
|
+
const briefTool = I.TOOLS.find((t) => t.bridge === 'brief')
|
|
73
|
+
const k1 = I.memoKey(briefTool, { url: 'u', level: 'digest' })
|
|
74
|
+
const k2 = I.memoKey(briefTool, { url: 'u', level: 'full' })
|
|
75
|
+
const k3 = I.memoKey(briefTool, { url: 'u', level: 'digest', refresh: true })
|
|
76
|
+
assert('memo: different args give different keys', k1 !== k2)
|
|
77
|
+
assert('memo: refresh does not change the key (it clears instead)', k1 === k3)
|
|
78
|
+
assert('memo: repeat stub names the cost', /未重跑/.test(I.repeatStub({ chars: 1234, head: 'x' })))
|
|
79
|
+
assert('memo: failure stub refuses a blind retry', /未重试/.test(I.failureStub({ error: 'boom' })))
|
|
80
|
+
}
|
|
50
81
|
|
|
51
82
|
// -------------------------------------------------------------- 2. python side
|
|
52
83
|
const project = process.env.VIDLENS_PROJECT || path.resolve(ROOT, '..', 'vidlens')
|
|
@@ -111,8 +142,8 @@ while (!ready && child.exitCode === null && Date.now() < bootDeadline) {
|
|
|
111
142
|
assert('bridge boot handshake', !!ready, ready ? `py ${ready.python} @ ${ready.project}` : stderr.slice(-500))
|
|
112
143
|
assert('bridge imported vidlens', !!ready?.ok, ready?.error || '')
|
|
113
144
|
assert('bridge exposes every tool the Host registers',
|
|
114
|
-
tools.every((t) => (ready?.tools || []).includes(t.bridge)),
|
|
115
|
-
`missing: ${tools.map((t) => t.bridge).filter((b) => !(ready?.tools || []).includes(b)).join(',')}`)
|
|
145
|
+
tools.filter((t) => !t.local).every((t) => (ready?.tools || []).includes(t.bridge)),
|
|
146
|
+
`missing: ${tools.filter((t) => !t.local).map((t) => t.bridge).filter((b) => !(ready?.tools || []).includes(b)).join(',')}`)
|
|
116
147
|
|
|
117
148
|
try {
|
|
118
149
|
const library = await call('library', { limit: 3 })
|