dsh-pf-stream 0.1.3 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +26 -4
- package/README.md +28 -5
- package/lib/entry/adapter.d.ts +2 -0
- package/lib/entry/adapter.js +4 -1
- package/lib/entry/adapter.js.map +1 -1
- package/lib/entry/legacyNormalizer.d.ts +6 -2
- package/lib/entry/legacyNormalizer.js +19 -9
- package/lib/entry/legacyNormalizer.js.map +1 -1
- package/lib/index.d.ts +40 -5
- package/lib/index.js +81 -6
- package/lib/index.js.map +1 -1
- package/lib/session/state.js +5 -23
- package/lib/session/state.js.map +1 -1
- package/lib/stage0/decoder.d.ts +2 -0
- package/lib/stage0/decoder.js +23 -3
- package/lib/stage0/decoder.js.map +1 -1
- package/lib/stage2/stateMachine.js +0 -3
- package/lib/stage2/stateMachine.js.map +1 -1
- package/lib/stage3/deterministicRouter.d.ts +4 -2
- package/lib/stage3/deterministicRouter.js +129 -80
- package/lib/stage3/deterministicRouter.js.map +1 -1
- package/lib/stage3/streamSampler.d.ts +2 -0
- package/lib/stage3/streamSampler.js +33 -26
- package/lib/stage3/streamSampler.js.map +1 -1
- package/lib/stage4/viewport.d.ts +2 -1
- package/lib/stage4/viewport.js +7 -3
- package/lib/stage4/viewport.js.map +1 -1
- package/lib/stage5/controller.js +1 -1
- package/lib/stage5/intent.d.ts +2 -0
- package/lib/stage5/intent.js +23 -10
- package/lib/stage5/intent.js.map +1 -1
- package/lib/stage6/precheck.d.ts +4 -2
- package/lib/stage6/precheck.js +5 -5
- package/lib/stage6/precheck.js.map +1 -1
- package/lib/tool/handler.d.ts +34 -0
- package/lib/tool/handler.js +29 -9
- package/lib/tool/handler.js.map +1 -1
- package/lib/tool/schema.d.ts +4 -1
- package/lib/tool/schema.js +15 -1
- package/lib/tool/schema.js.map +1 -1
- package/lib/tool/streamCall.d.ts +24 -2
- package/lib/tool/streamCall.js +29 -7
- package/lib/tool/streamCall.js.map +1 -1
- package/lib/types.d.ts +2 -4
- package/package.json +1 -1
package/README.en.md
CHANGED
|
@@ -22,12 +22,34 @@ One tool, `dsh-pf-stream`, routes through `intent.action`:
|
|
|
22
22
|
|
|
23
23
|
## Install
|
|
24
24
|
|
|
25
|
-
|
|
25
|
+
### 1. Install the plugin
|
|
26
26
|
|
|
27
|
-
- DSH
|
|
28
|
-
-
|
|
27
|
+
- Open the DSH plugin manager, search for `dsh-pf-stream`, and install the latest version.
|
|
28
|
+
- Or tell the agent "install the plugin dsh-pf-stream"; the agent runs the same flow through plugin_manager.
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
The host performs the install: it adds the package through pnpm and mounts the plugin from `cordis.patch.yml`. Peer dependencies `@deepseek-ai/cordis` and `@deepseek-ai/schemastery` resolve in the host runtime. The only runtime dependency is `pngjs`; canvas PNG encoding uses fixed parameters, so output bytes are deterministic.
|
|
31
|
+
|
|
32
|
+
### 2. Prepare ffmpeg and ffprobe
|
|
33
|
+
|
|
34
|
+
Pick one install method:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
winget install Gyan.FFmpeg # Windows
|
|
38
|
+
brew install ffmpeg # macOS
|
|
39
|
+
sudo apt install ffmpeg # Debian / Ubuntu
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
You can also download from [ffmpeg.org](https://ffmpeg.org/download.html), unpack, and point an environment variable at the `bin` directory. Executable search order: plugin config `ffmpegPath` / `ffprobePath` → `DSH_FFMPEG_PATH` / `FFMPEG_PATH` → `DSH_FFMPEG_DIR` → `~/.dsh/tools/ffmpeg/bin` → `~/.dsh/bin` → `PATH`.
|
|
43
|
+
|
|
44
|
+
### 3. Verify the installation
|
|
45
|
+
|
|
46
|
+
After installation the model-side tool `dsh-pf-stream` appears. Probe any local video:
|
|
47
|
+
|
|
48
|
+
```json
|
|
49
|
+
{ "video_path": "D:/videos/demo.mp4", "intent": { "action": "probe" } }
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
A response with display size, duration, and frame rate confirms a working installation.
|
|
31
53
|
|
|
32
54
|
## Usage
|
|
33
55
|
|
package/README.md
CHANGED
|
@@ -22,12 +22,34 @@ DeepSeek Harness（DSH）视频感知插件。模型用一个工具即可「看
|
|
|
22
22
|
|
|
23
23
|
## 安装
|
|
24
24
|
|
|
25
|
-
|
|
25
|
+
### 1. 安装插件
|
|
26
26
|
|
|
27
|
-
- DSH
|
|
28
|
-
-
|
|
27
|
+
- 在 DSH 插件管理器中搜索 `dsh-pf-stream`，安装最新版本。
|
|
28
|
+
- 或直接对智能体说「安装插件 dsh-pf-stream」，智能体经 plugin_manager 完成同一流程。
|
|
29
29
|
|
|
30
|
-
|
|
30
|
+
安装由宿主执行：宿主经 pnpm 安装包，读取 `cordis.patch.yml` 挂载插件。peer 依赖 `@deepseek-ai/cordis` 与 `@deepseek-ai/schemastery` 由宿主运行时解析；运行时直接依赖仅 `pngjs`，画布 PNG 编码参数固定，输出字节幂等。
|
|
31
|
+
|
|
32
|
+
### 2. 准备 ffmpeg 与 ffprobe
|
|
33
|
+
|
|
34
|
+
任选一种安装方式：
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
winget install Gyan.FFmpeg # Windows
|
|
38
|
+
brew install ffmpeg # macOS
|
|
39
|
+
sudo apt install ffmpeg # Debian / Ubuntu
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
也可从 [ffmpeg.org](https://ffmpeg.org/download.html) 下载解压，用环境变量指向 `bin` 目录。可执行文件定位顺序：插件配置 `ffmpegPath` / `ffprobePath` → `DSH_FFMPEG_PATH` / `FFMPEG_PATH` → `DSH_FFMPEG_DIR` → `~/.dsh/tools/ffmpeg/bin` → `~/.dsh/bin` → `PATH`。
|
|
43
|
+
|
|
44
|
+
### 3. 验证安装
|
|
45
|
+
|
|
46
|
+
安装成功后，模型侧出现工具 `dsh-pf-stream`。用任一本地视频探测元数据：
|
|
47
|
+
|
|
48
|
+
```json
|
|
49
|
+
{ "video_path": "D:/videos/demo.mp4", "intent": { "action": "probe" } }
|
|
50
|
+
```
|
|
51
|
+
|
|
52
|
+
返回显示宽高、时长、帧率，即安装成功。
|
|
31
53
|
|
|
32
54
|
## 使用
|
|
33
55
|
|
|
@@ -59,7 +81,7 @@ DeepSeek Harness（DSH）视频感知插件。模型用一个工具即可「看
|
|
|
59
81
|
|
|
60
82
|
每次调用返回一个或多个画布包：
|
|
61
83
|
|
|
62
|
-
- **PNG
|
|
84
|
+
- **PNG 画布**：四形态之一（见下表）。单元间双色隔离线阻断相邻边缘特征串扰；可选超像素角标渲染单元编号。画布 PNG 同时作为图像内容块进入模型上下文（单次调用至多 `maxImagesPerCall` 张）。
|
|
63
85
|
- **结构化元数据**：`packet_id`、每单元 PTS、token 网格 bbox、源采样 bbox、双载体 SHA-256。
|
|
64
86
|
- **时序提示行**：模型可读的 P/F 单元时间轴（如 `[Packet #42 | Anchor: #105 | #101(P, 12.1s, Left-Top) → …]`）。
|
|
65
87
|
|
|
@@ -85,6 +107,7 @@ DeepSeek Harness（DSH）视频感知插件。模型用一个工具即可「看
|
|
|
85
107
|
| --- | --- | --- |
|
|
86
108
|
| `renderVisualId` | `true` | 单元左上角超像素反色角标 |
|
|
87
109
|
| `maxPacketsPerCall` | `64` | 单次调用返回包数上限 |
|
|
110
|
+
| `maxImagesPerCall` | `20` | 单次调用回传图像上限（宿主单条消息图像上限 20） |
|
|
88
111
|
| `maxTokensBudgetPerCall` | `65536` | 单次调用视觉 token 预算 |
|
|
89
112
|
| `maxRequestBytes` | `56 MiB` | 请求字节软水位（Base64 后口径） |
|
|
90
113
|
| `maxColdRefs` | `4` | 冷区引用上限 |
|
package/lib/entry/adapter.d.ts
CHANGED
package/lib/entry/adapter.js
CHANGED
|
@@ -27,7 +27,10 @@ export function adaptCallArgs(args, limits) {
|
|
|
27
27
|
if (typeof videoPath !== 'string' || videoPath.length === 0) {
|
|
28
28
|
throw new PfStreamError('PFSTREAM_ARGS_VIDEO_PATH', 'video_path 必须为非空字符串');
|
|
29
29
|
}
|
|
30
|
-
const legacy = normalizeLegacyArgs(args, limits.lastPtsMs
|
|
30
|
+
const legacy = normalizeLegacyArgs(args, limits.lastPtsMs, {
|
|
31
|
+
width: limits.displayWidth,
|
|
32
|
+
height: limits.displayHeight,
|
|
33
|
+
});
|
|
31
34
|
const rawIntent = isRecord(args['intent']) ? args['intent'] : {};
|
|
32
35
|
const intent = {};
|
|
33
36
|
const action = (rawIntent['action'] ?? legacy.intent.action);
|
package/lib/entry/adapter.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"adapter.js","sourceRoot":"","sources":["../../src/entry/adapter.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAE7C,OAAO,EAAE,mBAAmB,EAAyB,MAAM,uBAAuB,CAAC;AAEnF,MAAM,OAAO,GAAwB,IAAI,GAAG,CAAC,CAAC,WAAW,EAAE,YAAY,EAAE,SAAS,EAAE,OAAO,CAAC,CAAC,CAAC;AAC9F,MAAM,KAAK,GAAwB,IAAI,GAAG,CAAC,CAAC,OAAO,EAAE,UAAU,EAAE,QAAQ,CAAC,CAAC,CAAC;AAC5E,MAAM,KAAK,GAAwB,IAAI,GAAG,CAAC,CAAC,QAAQ,EAAE,QAAQ,EAAE,UAAU,CAAC,CAAC,CAAC;
|
|
1
|
+
{"version":3,"file":"adapter.js","sourceRoot":"","sources":["../../src/entry/adapter.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAE7C,OAAO,EAAE,mBAAmB,EAAyB,MAAM,uBAAuB,CAAC;AAEnF,MAAM,OAAO,GAAwB,IAAI,GAAG,CAAC,CAAC,WAAW,EAAE,YAAY,EAAE,SAAS,EAAE,OAAO,CAAC,CAAC,CAAC;AAC9F,MAAM,KAAK,GAAwB,IAAI,GAAG,CAAC,CAAC,OAAO,EAAE,UAAU,EAAE,QAAQ,CAAC,CAAC,CAAC;AAC5E,MAAM,KAAK,GAAwB,IAAI,GAAG,CAAC,CAAC,QAAQ,EAAE,QAAQ,EAAE,UAAU,CAAC,CAAC,CAAC;AAgB7E,SAAS,QAAQ,CAAC,CAAU;IAC1B,OAAO,OAAO,CAAC,KAAK,QAAQ,IAAI,CAAC,KAAK,IAAI,CAAC;AAC7C,CAAC;AAED,SAAS,QAAQ,CAAC,GAAY;IAC5B,MAAM,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAC1C,MAAM,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAC1C,MAAM,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC;IACjD,MAAM,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC;IACjD,OAAO,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC;AACxB,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,aAAa,CAC3B,IAA6B,EAC7B,MAAqB;IAErB,MAAM,SAAS,GAAG,IAAI,CAAC,YAAY,CAAC,CAAC;IACrC,IAAI,OAAO,SAAS,KAAK,QAAQ,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC5D,MAAM,IAAI,aAAa,CAAC,0BAA0B,EAAE,qBAAqB,CAAC,CAAC;IAC7E,CAAC;IAED,MAAM,MAAM,GAAqB,mBAAmB,CAAC,IAAI,EAAE,MAAM,CAAC,SAAS,EAAE;QAC3E,KAAK,EAAE,MAAM,CAAC,YAAY;QAC1B,MAAM,EAAE,MAAM,CAAC,aAAa;KAC7B,CAAC,CAAC;IACH,MAAM,SAAS,GAAG,QAAQ,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAE,IAAI,CAAC,QAAQ,CAA6B,CAAC,CAAC,CAAC,EAAE,CAAC;IAE9F,MAAM,MAAM,GAAoB,EAAE,CAAC;IAEnC,MAAM,MAAM,GAAG,CAAC,SAAS,CAAC,QAAQ,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,MAAM,CAA6B,CAAC;IACzF,IAAI,MAAM,IAAI,IAAI,EAAE,CAAC;QACnB,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC;YACzB,MAAM,IAAI,aAAa,CAAC,sBAAsB,EAAE,eAAe,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;QACnF,CAAC;QACD,MAAM,CAAC,MAAM,GAAG,MAAM,CAAC;IACzB,CAAC;IAED,MAAM,IAAI,GAAG,SAAS,CAAC,MAAM,CAA4B,CAAC;IAC1D,IAAI,IAAI,IAAI,IAAI,EAAE,CAAC;QACjB,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC;YAAE,MAAM,IAAI,aAAa,CAAC,oBAAoB,EAAE,aAAa,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACjG,MAAM,CAAC,IAAI,GAAG,IAAI,CAAC;IACrB,CAAC;IACD,MAAM,KAAK,GAAG,SAAS,CAAC,OAAO,CAA0B,CAAC;IAC1D,IAAI,KAAK,IAAI,IAAI,EAAE,CAAC;QAClB,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,KAAK,CAAC;YAAE,MAAM,IAAI,aAAa,CAAC,qBAAqB,EAAE,cAAc,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;QACrG,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC;IACvB,CAAC;IAED,MAAM,MAAM,GAAG,CAAC,SAAS,CAAC,UAAU,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,QAAQ,CAAwB,CAAC;IACxF,IAAI,MAAM,IAAI,IAAI,EAAE,CAAC;QACnB,MAAM,CAAC,QAAQ,GAAG,QAAQ,CAAC,MAAM,CAAC,CAAC;IACrC,CAAC;IACD,IAAI,SAAS,CAAC,YAAY,CAAC,KAAK,IAAI;QAAE,MAAM,CAAC,UAAU,GAAG,IAAI,CAAC;IAE/D,MAAM,SAAS,GAAG,CAAC,SAAS,CAAC,YAAY,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,UAAU,CAAiC,CAAC;IACxG,IAAI,SAAS,IAAI,IAAI,EAAE,CAAC;QACtB,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YACxD,MAAM,IAAI,aAAa,CAAC,sBAAsB,EAAE,mCAAmC,CAAC,CAAC;QACvF,CAAC;QACD,MAAM,EAAE,GAAG,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;QACnD,MAAM,EAAE,GAAG,OAAO,CAAC,SAAS,CAAC,CAAC,CAAC,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;QACnD,MAAM,CAAC,UAAU,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC,CAAC;IAC3D,CAAC;IAED,MAAM,KAAK,GAAG,CAAC,SAAS,CAAC,eAAe,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,aAAa,CAAyB,CAAC;IAClG,IAAI,KAAK,IAAI,IAAI,EAAE,CAAC;QAClB,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC9C,MAAM,IAAI,aAAa,CAAC,0BAA0B,EAAE,uBAAuB,CAAC,CAAC;QAC/E,CAAC;QACD,IAAI,MAAM,CAAC,MAAM,IAAI,IAAI;YAAE,MAAM,CAAC,MAAM,GAAG,YAAY,CAAC;QACxD,MAAM,CAAC,aAAa,GAAG,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,OAAO,CAAC,CAAC,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC;IACxE,CAAC;IAED,MAAM,WAAW,GAAG,CAAC,SAAS,CAAC,aAAa,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,WAAW,CAAuB,CAAC;IAClG,IAAI,WAAW,IAAI,IAAI,EAAE,CAAC;QACxB,IAAI,OAAO,WAAW,KAAK,QAAQ,IAAI,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC,EAAE,CAAC;YAC1D,MAAM,IAAI,aAAa,CAAC,wBAAwB,EAAE,mBAAmB,CAAC,CAAC;QACzE,CAAC;QACD,IAAI,MAAM,CAAC,MAAM,IAAI,IAAI;YAAE,MAAM,CAAC,MAAM,GAAG,SAAS,CAAC;QACrD,MAAM,CAAC,WAAW,GAAG,WAAW,CAAC;IACnC,CAAC;IAED,MAAM,KAAK,GAAG,CAAC,SAAS,CAAC,aAAa,CAAC,IAAI,MAAM,CAAC,MAAM,CAAC,WAAW,CAAuB,CAAC;IAC5F,IAAI,KAAK,IAAI,IAAI,EAAE,CAAC;QAClB,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,EAAE,CAAC;YAC/C,MAAM,IAAI,aAAa,CAAC,2BAA2B,EAAE,uBAAuB,CAAC,CAAC;QAChF,CAAC;QACD,IAAI,MAAM,CAAC,MAAM,IAAI,IAAI;YAAE,MAAM,CAAC,MAAM,GAAG,SAAS,CAAC;QACrD,MAAM,CAAC,WAAW,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,GAAG,CAAC,CAAC;IACxD,CAAC;IAED,MAAM,KAAK,GAAG,SAAS,CAAC,aAAa,CAAuB,CAAC;IAC7D,IAAI,KAAK,IAAI,IAAI,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC7D,MAAM,CAAC,WAAW,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC,UAAU,CAAC,CAAC;IACtE,CAAC;IACD,MAAM,MAAM,GAAG,SAAS,CAAC,mBAAmB,CAAuB,CAAC;IACpE,IAAI,MAAM,IAAI,IAAI,IAAI,OAAO,MAAM,KAAK,QAAQ,IAAI,MAAM,IAAI,GAAG,EAAE,CAAC;QAClE,MAAM,CAAC,iBAAiB,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC,eAAe,CAAC,CAAC;IAClF,CAAC;IACD,IAAI,SAAS,CAAC,OAAO,CAAC,KAAK,IAAI;QAAE,MAAM,CAAC,KAAK,GAAG,IAAI,CAAC;IAErD,OAAO,EAAE,MAAM,EAAE,cAAc,EAAE,MAAM,CAAC,cAAc,EAAE,SAAS,EAAE,CAAC;AACtE,CAAC;AAED,SAAS,OAAO,CAAC,CAAS,EAAE,SAAiB;IAC3C,MAAM,CAAC,GAAG,OAAO,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC;IAChD,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;QAAE,MAAM,IAAI,aAAa,CAAC,mBAAmB,EAAE,WAAW,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;IAC9F,OAAO,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,SAAS,CAAC,CAAC;AACzD,CAAC"}
|
|
@@ -3,7 +3,8 @@
|
|
|
3
3
|
*
|
|
4
4
|
* 输入便利层：start_time/end_time/time_range/timestamps/fps/interval/num_frames/
|
|
5
5
|
* frame_count/crop/bbox/roi/probe_only → 标准 IntentDirective 字段。
|
|
6
|
-
*
|
|
6
|
+
* 判秒规则（《技术设计方案》§2.4）：无后缀数值 ≤ 时长（秒）×2 判秒，超过判毫秒；
|
|
7
|
+
* 归一化来源记入 normalizedFrom。
|
|
7
8
|
*/
|
|
8
9
|
import type { IntentDirective } from '../types.js';
|
|
9
10
|
export interface NormalizedIntent {
|
|
@@ -13,4 +14,7 @@ export interface NormalizedIntent {
|
|
|
13
14
|
}
|
|
14
15
|
/** 数值或字符串时间 → 毫秒（判秒规则见模块注释）。 */
|
|
15
16
|
export declare function toMs(value: number | string, durationMs: number): number;
|
|
16
|
-
export declare function normalizeLegacyArgs(args: Record<string, unknown>, durationMs: number
|
|
17
|
+
export declare function normalizeLegacyArgs(args: Record<string, unknown>, durationMs: number, display?: {
|
|
18
|
+
width: number;
|
|
19
|
+
height: number;
|
|
20
|
+
}): NormalizedIntent;
|
|
@@ -3,27 +3,37 @@
|
|
|
3
3
|
*
|
|
4
4
|
* 输入便利层：start_time/end_time/time_range/timestamps/fps/interval/num_frames/
|
|
5
5
|
* frame_count/crop/bbox/roi/probe_only → 标准 IntentDirective 字段。
|
|
6
|
-
*
|
|
6
|
+
* 判秒规则（《技术设计方案》§2.4）：无后缀数值 ≤ 时长（秒）×2 判秒，超过判毫秒；
|
|
7
|
+
* 归一化来源记入 normalizedFrom。
|
|
7
8
|
*/
|
|
8
9
|
/** 数值或字符串时间 → 毫秒（判秒规则见模块注释）。 */
|
|
9
10
|
export function toMs(value, durationMs) {
|
|
10
11
|
const n = typeof value === 'string' ? Number.parseFloat(value) : value;
|
|
11
12
|
if (!Number.isFinite(n))
|
|
12
13
|
throw new Error(`时间值无法解析：${String(value)}`);
|
|
13
|
-
const secThreshold = (durationMs / 1000) *
|
|
14
|
+
const secThreshold = (durationMs / 1000) * 2;
|
|
14
15
|
const ms = n <= secThreshold ? n * 1000 : n;
|
|
15
16
|
return Math.round(ms);
|
|
16
17
|
}
|
|
17
|
-
|
|
18
|
+
/**
|
|
19
|
+
* ROI 归一化：归一化坐标直接钳制 [0,1]；像素坐标（任一分量 >1）
|
|
20
|
+
* 按显示宽高换算为归一化坐标（规格《五》§1.1 crop/bbox/roi 规则）。
|
|
21
|
+
*/
|
|
22
|
+
function normalizeRoi(raw, displayWidth, displayHeight) {
|
|
18
23
|
const clamp01 = (v) => Math.min(Math.max(v, 0), 1);
|
|
24
|
+
const pixelMode = raw.x > 1 || raw.y > 1 || raw.w > 1 || raw.h > 1;
|
|
25
|
+
const x = pixelMode ? raw.x / displayWidth : raw.x;
|
|
26
|
+
const y = pixelMode ? raw.y / displayHeight : raw.y;
|
|
27
|
+
const w = pixelMode ? raw.w / displayWidth : raw.w;
|
|
28
|
+
const h = pixelMode ? raw.h / displayHeight : raw.h;
|
|
19
29
|
return {
|
|
20
|
-
x: clamp01(
|
|
21
|
-
y: clamp01(
|
|
22
|
-
w: Math.min(Math.max(
|
|
23
|
-
h: Math.min(Math.max(
|
|
30
|
+
x: clamp01(x),
|
|
31
|
+
y: clamp01(y),
|
|
32
|
+
w: Math.min(Math.max(w, 0), 1 - clamp01(x)),
|
|
33
|
+
h: Math.min(Math.max(h, 0), 1 - clamp01(y)),
|
|
24
34
|
};
|
|
25
35
|
}
|
|
26
|
-
export function normalizeLegacyArgs(args, durationMs) {
|
|
36
|
+
export function normalizeLegacyArgs(args, durationMs, display = { width: 1, height: 1 }) {
|
|
27
37
|
const intent = {};
|
|
28
38
|
const normalizedFrom = [];
|
|
29
39
|
const has = (k) => Object.prototype.hasOwnProperty.call(args, k) && args[k] !== undefined;
|
|
@@ -74,7 +84,7 @@ export function normalizeLegacyArgs(args, durationMs) {
|
|
|
74
84
|
}
|
|
75
85
|
const roiRaw = (args['crop'] ?? args['bbox'] ?? args['roi']);
|
|
76
86
|
if (roiRaw && typeof roiRaw === 'object') {
|
|
77
|
-
intent.roi_hint = normalizeRoi(roiRaw);
|
|
87
|
+
intent.roi_hint = normalizeRoi(roiRaw, display.width, display.height);
|
|
78
88
|
normalizedFrom.push('crop/bbox/roi');
|
|
79
89
|
}
|
|
80
90
|
if (args['probe_only'] === true) {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"legacyNormalizer.js","sourceRoot":"","sources":["../../src/entry/legacyNormalizer.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"legacyNormalizer.js","sourceRoot":"","sources":["../../src/entry/legacyNormalizer.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAUH,gCAAgC;AAChC,MAAM,UAAU,IAAI,CAAC,KAAsB,EAAE,UAAkB;IAC7D,MAAM,CAAC,GAAG,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,UAAU,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC;IACvE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,WAAW,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;IACrE,MAAM,YAAY,GAAG,CAAC,UAAU,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC;IAC7C,MAAM,EAAE,GAAG,CAAC,IAAI,YAAY,CAAC,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;IAC5C,OAAO,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;AACxB,CAAC;AAED;;;GAGG;AACH,SAAS,YAAY,CACnB,GAAmD,EACnD,YAAoB,EACpB,aAAqB;IAErB,MAAM,OAAO,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAC3D,MAAM,SAAS,GAAG,GAAG,CAAC,CAAC,GAAG,CAAC,IAAI,GAAG,CAAC,CAAC,GAAG,CAAC,IAAI,GAAG,CAAC,CAAC,GAAG,CAAC,IAAI,GAAG,CAAC,CAAC,GAAG,CAAC,CAAC;IACnE,MAAM,CAAC,GAAG,SAAS,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,YAAY,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IACnD,MAAM,CAAC,GAAG,SAAS,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,aAAa,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IACpD,MAAM,CAAC,GAAG,SAAS,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,YAAY,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IACnD,MAAM,CAAC,GAAG,SAAS,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,aAAa,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;IACpD,OAAO;QACL,CAAC,EAAE,OAAO,CAAC,CAAC,CAAC;QACb,CAAC,EAAE,OAAO,CAAC,CAAC,CAAC;QACb,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC,CAAC;QAC3C,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,CAAC,GAAG,OAAO,CAAC,CAAC,CAAC,CAAC;KAC5C,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,mBAAmB,CACjC,IAA6B,EAC7B,UAAkB,EAClB,UAA6C,EAAE,KAAK,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE;IAEpE,MAAM,MAAM,GAAoB,EAAE,CAAC;IACnC,MAAM,cAAc,GAAa,EAAE,CAAC;IACpC,MAAM,GAAG,GAAG,CAAC,CAAS,EAAE,EAAE,CAAC,MAAM,CAAC,SAAS,CAAC,cAAc,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC,CAAC,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,SAAS,CAAC;IAElG,IAAI,GAAG,CAAC,YAAY,CAAC,IAAI,GAAG,CAAC,UAAU,CAAC,IAAI,GAAG,CAAC,YAAY,CAAC,EAAE,CAAC;QAC9D,MAAM,KAAK,GAAG,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,YAAY,CAAoB,EAAE,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QAC9F,MAAM,GAAG,GAAG,GAAG,CAAC,UAAU,CAAC;YACzB,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,UAAU,CAAoB,EAAE,UAAU,CAAC;YACvD,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,KAAK,EAAE,UAAU,CAAC,CAAC;QAChC,MAAM,CAAC,UAAU,GAAG,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC;QACjC,cAAc,CAAC,IAAI,CAAC,qBAAqB,CAAC,CAAC;IAC7C,CAAC;IACD,IAAI,GAAG,CAAC,YAAY,CAAC,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC,IAAI,IAAI,CAAC,YAAY,CAAC,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC9F,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,GAAG,IAAI,CAAC,YAAY,CAA2B,CAAC;QAC5D,IAAI,CAAC,KAAK,SAAS,IAAI,CAAC,KAAK,SAAS,EAAE,CAAC;YACvC,MAAM,CAAC,UAAU,GAAG,CAAC,IAAI,CAAC,CAAC,EAAE,UAAU,CAAC,EAAE,IAAI,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC;YAC/D,cAAc,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;QACpC,CAAC;IACH,CAAC;IACD,IAAI,GAAG,CAAC,YAAY,CAAC,IAAI,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC,EAAE,CAAC;QAC3D,MAAM,CAAC,aAAa,GAAI,IAAI,CAAC,YAAY,CAA4B,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC;QACtG,MAAM,CAAC,MAAM,GAAG,YAAY,CAAC;QAC7B,cAAc,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;IACpC,CAAC;IACD,IAAI,GAAG,CAAC,KAAK,CAAC,EAAE,CAAC;QACf,MAAM,GAAG,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC;QAChC,IAAI,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC,IAAI,GAAG,GAAG,CAAC,EAAE,CAAC;YACpC,MAAM,CAAC,WAAW,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,GAAG,GAAG,CAAC,CAAC;YAC5C,MAAM,CAAC,MAAM,GAAG,SAAS,CAAC;YAC1B,cAAc,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;QAC7B,CAAC;IACH,CAAC;IACD,IAAI,GAAG,CAAC,UAAU,CAAC,EAAE,CAAC;QACpB,MAAM,EAAE,GAAG,MAAM,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC;QACpC,IAAI,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,IAAI,EAAE,GAAG,CAAC,EAAE,CAAC;YAClC,MAAM,CAAC,WAAW,GAAG,IAAI,CAAC,EAAE,EAAE,UAAU,CAAC,CAAC;YAC1C,MAAM,CAAC,MAAM,GAAG,SAAS,CAAC;YAC1B,cAAc,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;QAClC,CAAC;IACH,CAAC;IACD,IAAI,GAAG,CAAC,YAAY,CAAC,IAAI,GAAG,CAAC,aAAa,CAAC,EAAE,CAAC;QAC5C,MAAM,EAAE,GAAG,MAAM,CAAC,IAAI,CAAC,YAAY,CAAC,IAAI,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC;QAC7D,IAAI,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,IAAI,EAAE,IAAI,CAAC,EAAE,CAAC;YACnC,MAAM,CAAC,WAAW,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,EAAE,GAAG,CAAC,CAAC;YACnD,IAAI,MAAM,CAAC,MAAM,IAAI,IAAI;gBAAE,MAAM,CAAC,MAAM,GAAG,SAAS,CAAC;YACrD,cAAc,CAAC,IAAI,CAAC,wBAAwB,CAAC,CAAC;QAChD,CAAC;IACH,CAAC;IACD,MAAM,MAAM,GAAG,CAAC,IAAI,CAAC,MAAM,CAAC,IAAI,IAAI,CAAC,MAAM,CAAC,IAAI,IAAI,CAAC,KAAK,CAAC,CAE9C,CAAC;IACd,IAAI,MAAM,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;QACzC,MAAM,CAAC,QAAQ,GAAG,YAAY,CAAC,MAAM,EAAE,OAAO,CAAC,KAAK,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC;QACtE,cAAc,CAAC,IAAI,CAAC,eAAe,CAAC,CAAC;IACvC,CAAC;IACD,IAAI,IAAI,CAAC,YAAY,CAAC,KAAK,IAAI,EAAE,CAAC;QAChC,MAAM,CAAC,MAAM,GAAG,OAAO,CAAC;QACxB,cAAc,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;IACpC,CAAC;IACD,OAAO,EAAE,MAAM,EAAE,cAAc,EAAE,CAAC;AACpC,CAAC"}
|
package/lib/index.d.ts
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
import Schema from '@deepseek-ai/schemastery';
|
|
11
11
|
import type { Context } from '@deepseek-ai/cordis';
|
|
12
12
|
import type { PfStreamPluginConfig } from './types.js';
|
|
13
|
-
import { type ToolCallArgs, type ToolExec, type ToolResult } from './tool/handler.js';
|
|
13
|
+
import { type AttachedImage, type ToolCallArgs, type ToolExec, type ToolResult } from './tool/handler.js';
|
|
14
14
|
export declare const name = "dsh-pf-stream";
|
|
15
15
|
export declare const inject: string[];
|
|
16
16
|
/** 56 MiB = 58,720,256 字节（含 8 MiB Base64 编码膨胀与协议结构预留）。 */
|
|
@@ -18,6 +18,7 @@ export declare const DEFAULT_MAX_REQUEST_BYTES: number;
|
|
|
18
18
|
export declare const Config: Schema<Schemastery.ObjectS<NoInfer<{
|
|
19
19
|
renderVisualId: Schema<boolean, boolean, "defined">;
|
|
20
20
|
maxPacketsPerCall: Schema<number, number, "defined">;
|
|
21
|
+
maxImagesPerCall: Schema<number, number, "defined">;
|
|
21
22
|
maxTokensBudgetPerCall: Schema<number, number, "defined">;
|
|
22
23
|
maxRequestBytes: Schema<number, number, "defined">;
|
|
23
24
|
maxColdRefs: Schema<number, number, "defined">;
|
|
@@ -47,6 +48,7 @@ export declare const Config: Schema<Schemastery.ObjectS<NoInfer<{
|
|
|
47
48
|
}>>, Schemastery.ObjectT<NoInfer<{
|
|
48
49
|
renderVisualId: Schema<boolean, boolean, "defined">;
|
|
49
50
|
maxPacketsPerCall: Schema<number, number, "defined">;
|
|
51
|
+
maxImagesPerCall: Schema<number, number, "defined">;
|
|
50
52
|
maxTokensBudgetPerCall: Schema<number, number, "defined">;
|
|
51
53
|
maxRequestBytes: Schema<number, number, "defined">;
|
|
52
54
|
maxColdRefs: Schema<number, number, "defined">;
|
|
@@ -81,10 +83,45 @@ export declare const SELF_MANAGED_BLOCK_CEILING_MS: number;
|
|
|
81
83
|
export interface ToolsService {
|
|
82
84
|
register(definition: unknown): () => void;
|
|
83
85
|
}
|
|
86
|
+
/** 宿主附件服务返回的引用（@deepseek-ai/dsh-attachment 口径）。 */
|
|
87
|
+
export interface AttachmentServiceRef {
|
|
88
|
+
attachmentId: string;
|
|
89
|
+
mediaType: string;
|
|
90
|
+
width: number;
|
|
91
|
+
height: number;
|
|
92
|
+
bytes: number;
|
|
93
|
+
}
|
|
94
|
+
/**
|
|
95
|
+
* 宿主附件服务；运行时由 inject=['attachments'] 保证存在。
|
|
96
|
+
*
|
|
97
|
+
* 提交原始 PNG 字节，服务负责校验、规范化与落盘，返回可直接放进
|
|
98
|
+
* 图像内容块 `{type:'image', attachment}` 的引用。
|
|
99
|
+
*/
|
|
100
|
+
export interface AttachmentsService {
|
|
101
|
+
saveImages(inputs: Array<{
|
|
102
|
+
data: Uint8Array;
|
|
103
|
+
mediaType: string;
|
|
104
|
+
name?: string;
|
|
105
|
+
}>): Promise<AttachmentServiceRef[]>;
|
|
106
|
+
}
|
|
84
107
|
/** 结构化宿主上下文：cordis Context 加 tools 服务。 */
|
|
85
108
|
export type HostContext = Context & {
|
|
86
109
|
tools: ToolsService;
|
|
87
110
|
};
|
|
111
|
+
type ContentBlock = {
|
|
112
|
+
type: 'text';
|
|
113
|
+
text: string;
|
|
114
|
+
} | {
|
|
115
|
+
type: 'image';
|
|
116
|
+
attachment: AttachedImage;
|
|
117
|
+
};
|
|
118
|
+
/**
|
|
119
|
+
* 工具结果 → 模型可见内容块：一段完整 JSON 文本 + 每张画布一张真图。
|
|
120
|
+
*
|
|
121
|
+
* 宿主把 render 的返回值直接当 content 数组使用，并调用 content.some(...)，
|
|
122
|
+
* 因此必须返回内容块数组（单个块对象会在消费侧抛 TypeError）。
|
|
123
|
+
*/
|
|
124
|
+
export declare function renderContent(value: unknown): ContentBlock[];
|
|
88
125
|
export declare function buildToolDefinition(_deps: {
|
|
89
126
|
ctx: HostContext;
|
|
90
127
|
config: PfStreamPluginConfig;
|
|
@@ -96,13 +133,11 @@ export declare function buildToolDefinition(_deps: {
|
|
|
96
133
|
schema: {
|
|
97
134
|
readonly type: "object";
|
|
98
135
|
};
|
|
99
|
-
render: (_args: unknown, value: unknown) =>
|
|
100
|
-
type: "text";
|
|
101
|
-
text: string;
|
|
102
|
-
};
|
|
136
|
+
render: (_args: unknown, value: unknown) => ContentBlock[];
|
|
103
137
|
};
|
|
104
138
|
timeoutMs: number;
|
|
105
139
|
isConcurrencySafe: () => boolean;
|
|
106
140
|
execute(args: ToolCallArgs, exec: ToolExec | undefined): Promise<ToolResult>;
|
|
107
141
|
};
|
|
108
142
|
export declare function apply(ctx: Context, config: PfStreamPluginConfig): void;
|
|
143
|
+
export {};
|
package/lib/index.js
CHANGED
|
@@ -15,12 +15,13 @@ import { runDeterministicInspection } from './stage3/deterministicRouter.js';
|
|
|
15
15
|
import { runStreamCall } from './tool/streamCall.js';
|
|
16
16
|
import { resolveBinPaths } from './utils/ffmpegBin.js';
|
|
17
17
|
export const name = 'dsh-pf-stream';
|
|
18
|
-
export const inject = ['tools', 'subprocess'];
|
|
18
|
+
export const inject = ['tools', 'subprocess', 'attachments'];
|
|
19
19
|
/** 56 MiB = 58,720,256 字节（含 8 MiB Base64 编码膨胀与协议结构预留）。 */
|
|
20
20
|
export const DEFAULT_MAX_REQUEST_BYTES = 56 * 1024 * 1024;
|
|
21
21
|
export const Config = Schema.object({
|
|
22
22
|
renderVisualId: Schema.boolean().default(true).description('渲染超像素微型反色角标'),
|
|
23
23
|
maxPacketsPerCall: Schema.number().default(64).description('单次调用画布包数上限'),
|
|
24
|
+
maxImagesPerCall: Schema.number().default(20).description('单次调用回传图像上限（宿主单条消息图像上限 20）'),
|
|
24
25
|
maxTokensBudgetPerCall: Schema.number().default(65536).description('单次调用视觉 Token 预算上限（intent.max_tokens_budget 受此钳制）'),
|
|
25
26
|
maxRequestBytes: Schema.number().default(DEFAULT_MAX_REQUEST_BYTES).description('请求字节软水位'),
|
|
26
27
|
maxColdRefs: Schema.number().default(4).description('冷区引用数量上限'),
|
|
@@ -44,12 +45,77 @@ export const Config = Schema.object({
|
|
|
44
45
|
/** 流式单次调用阻塞上限：宿主死线减 40s 余量（超时协作式，signal 绝不硬终止）。 */
|
|
45
46
|
export const TOOL_TIMEOUT_MS = 900_000;
|
|
46
47
|
export const SELF_MANAGED_BLOCK_CEILING_MS = TOOL_TIMEOUT_MS - 40_000;
|
|
48
|
+
/** 从 cordis 上下文取附件服务（测试替身可能没有 get）。 */
|
|
49
|
+
function resolveAttachments(ctx) {
|
|
50
|
+
const get = ctx.get;
|
|
51
|
+
if (typeof get !== 'function')
|
|
52
|
+
return undefined;
|
|
53
|
+
const service = get.call(ctx, 'attachments');
|
|
54
|
+
if (service === undefined || service === null)
|
|
55
|
+
return undefined;
|
|
56
|
+
return service;
|
|
57
|
+
}
|
|
58
|
+
/**
|
|
59
|
+
* 画布包 PNG 提交给宿主附件服务。
|
|
60
|
+
*
|
|
61
|
+
* 宿主单条消息有图像张数上限（缺省 20），超出时保留最新包，旧包只回元数据。
|
|
62
|
+
* 返回值与 artifacts 按索引对齐，未提交的槽位为 undefined。
|
|
63
|
+
*/
|
|
64
|
+
async function savePacketImages(attachments, artifacts, limit) {
|
|
65
|
+
if (attachments === undefined || artifacts.length === 0)
|
|
66
|
+
return [];
|
|
67
|
+
const keep = Math.max(1, Math.min(Math.trunc(limit) || 1, artifacts.length));
|
|
68
|
+
const start = artifacts.length - keep;
|
|
69
|
+
const inputs = artifacts.slice(start).map((artifact) => ({
|
|
70
|
+
data: artifact.imageBytes,
|
|
71
|
+
mediaType: 'image/png',
|
|
72
|
+
name: `pf-packet-${artifact.packet.packetId}.png`,
|
|
73
|
+
}));
|
|
74
|
+
const refs = await attachments.saveImages(inputs);
|
|
75
|
+
if (refs.length !== inputs.length) {
|
|
76
|
+
throw new Error(`attachment service returned ${refs.length} references for ${inputs.length} canvases; ` +
|
|
77
|
+
'refusing to return a partial result');
|
|
78
|
+
}
|
|
79
|
+
const aligned = new Array(artifacts.length);
|
|
80
|
+
refs.forEach((ref, index) => {
|
|
81
|
+
aligned[start + index] = {
|
|
82
|
+
attachmentId: String(ref.attachmentId),
|
|
83
|
+
mediaType: String(ref.mediaType),
|
|
84
|
+
bytes: Math.round(Number(ref.bytes) || 0),
|
|
85
|
+
width: Math.round(Number(ref.width) || 0),
|
|
86
|
+
height: Math.round(Number(ref.height) || 0),
|
|
87
|
+
};
|
|
88
|
+
});
|
|
89
|
+
return aligned;
|
|
90
|
+
}
|
|
91
|
+
/**
|
|
92
|
+
* 工具结果 → 模型可见内容块：一段完整 JSON 文本 + 每张画布一张真图。
|
|
93
|
+
*
|
|
94
|
+
* 宿主把 render 的返回值直接当 content 数组使用，并调用 content.some(...)，
|
|
95
|
+
* 因此必须返回内容块数组（单个块对象会在消费侧抛 TypeError）。
|
|
96
|
+
*/
|
|
97
|
+
export function renderContent(value) {
|
|
98
|
+
const blocks = [{ type: 'text', text: JSON.stringify(value, null, 2) }];
|
|
99
|
+
const packets = value.packets;
|
|
100
|
+
if (!Array.isArray(packets))
|
|
101
|
+
return blocks;
|
|
102
|
+
for (const packet of packets) {
|
|
103
|
+
const image = packet.image;
|
|
104
|
+
if (image !== undefined && image !== null && typeof image.attachmentId === 'string') {
|
|
105
|
+
blocks.push({ type: 'image', attachment: image });
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
return blocks;
|
|
109
|
+
}
|
|
47
110
|
export function buildToolDefinition(_deps) {
|
|
48
111
|
const { config } = _deps;
|
|
49
112
|
const bins = resolveBinPaths({ ffmpegPath: config.ffmpegPath, ffprobePath: config.ffprobePath });
|
|
113
|
+
const attachments = resolveAttachments(_deps.ctx);
|
|
114
|
+
const attachImages = (artifacts) => savePacketImages(attachments, artifacts, config.maxImagesPerCall);
|
|
50
115
|
const probe = (videoPath, signal) => probeVideo(videoPath, { ffprobePath: bins.ffprobe, signal });
|
|
51
116
|
const inspect = (videoPath, probeResult, intent, signal) => runDeterministicInspection(videoPath, probeResult, intent, {
|
|
52
117
|
ffmpegPath: bins.ffmpeg,
|
|
118
|
+
ffprobePath: bins.ffprobe,
|
|
53
119
|
renderVisualId: config.renderVisualId,
|
|
54
120
|
packetIdStart: 1,
|
|
55
121
|
globalIdStart: 1,
|
|
@@ -60,6 +126,7 @@ export function buildToolDefinition(_deps) {
|
|
|
60
126
|
});
|
|
61
127
|
const stream = (videoPath, probeResult, intent, sessionId, signal) => runStreamCall(videoPath, probeResult, intent, sessionId ?? 'default', {
|
|
62
128
|
ffmpegPath: bins.ffmpeg,
|
|
129
|
+
ffprobePath: bins.ffprobe,
|
|
63
130
|
renderVisualId: config.renderVisualId,
|
|
64
131
|
defaultStreamWindowMs: config.defaultStreamWindowMs,
|
|
65
132
|
streamWorkMaxEdge: config.streamWorkMaxEdge,
|
|
@@ -78,17 +145,25 @@ export function buildToolDefinition(_deps) {
|
|
|
78
145
|
parameters: toolParameters,
|
|
79
146
|
output: {
|
|
80
147
|
schema: toolOutputSchema,
|
|
81
|
-
render
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
}),
|
|
148
|
+
// 宿主把 render 的返回值直接当 content 使用，并调用 content.some(...)，
|
|
149
|
+
// 因此必须返回内容块数组（单个块对象会在消费侧抛 TypeError）。
|
|
150
|
+
render: (_args, value) => renderContent(value),
|
|
85
151
|
},
|
|
86
152
|
timeoutMs: TOOL_TIMEOUT_MS,
|
|
87
153
|
// 同一 (sessionId, videoKey) 的调用经 per-key promise 队列串行化，
|
|
88
154
|
// 会话级状态使并发不安全。
|
|
89
155
|
isConcurrencySafe: () => false,
|
|
90
156
|
async execute(args, exec) {
|
|
91
|
-
return handleToolCall(args, exec, {
|
|
157
|
+
return handleToolCall(args, exec, {
|
|
158
|
+
probe,
|
|
159
|
+
inspect,
|
|
160
|
+
stream,
|
|
161
|
+
attachImages,
|
|
162
|
+
limits: {
|
|
163
|
+
maxPackets: config.maxPacketsPerCall,
|
|
164
|
+
maxTokensBudget: config.maxTokensBudgetPerCall,
|
|
165
|
+
},
|
|
166
|
+
});
|
|
92
167
|
},
|
|
93
168
|
};
|
|
94
169
|
}
|
package/lib/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,MAAM,MAAM,0BAA0B,CAAC;AAG9C,OAAO,EACL,cAAc,
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;;GAQG;AAEH,OAAO,MAAM,MAAM,0BAA0B,CAAC;AAG9C,OAAO,EACL,cAAc,GASf,MAAM,mBAAmB,CAAC;AAE3B,OAAO,EAAE,SAAS,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,cAAc,EAAE,MAAM,kBAAkB,CAAC;AACjG,OAAO,EAAE,UAAU,EAAE,MAAM,mBAAmB,CAAC;AAC/C,OAAO,EAAE,0BAA0B,EAAE,MAAM,iCAAiC,CAAC;AAC7E,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,EAAE,eAAe,EAAE,MAAM,sBAAsB,CAAC;AAEvD,MAAM,CAAC,MAAM,IAAI,GAAG,eAAe,CAAC;AAEpC,MAAM,CAAC,MAAM,MAAM,GAAG,CAAC,OAAO,EAAE,YAAY,EAAE,aAAa,CAAC,CAAC;AAE7D,0DAA0D;AAC1D,MAAM,CAAC,MAAM,yBAAyB,GAAG,EAAE,GAAG,IAAI,GAAG,IAAI,CAAC;AAE1D,MAAM,CAAC,MAAM,MAAM,GAAG,MAAM,CAAC,MAAM,CAAC;IAClC,cAAc,EAAE,MAAM,CAAC,OAAO,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,WAAW,CAAC,aAAa,CAAC;IACzE,iBAAiB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,WAAW,CAAC,YAAY,CAAC;IACxE,gBAAgB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,WAAW,CAAC,2BAA2B,CAAC;IACtF,sBAAsB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,WAAW,CAAC,kDAAkD,CAAC;IACtH,eAAe,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,yBAAyB,CAAC,CAAC,WAAW,CAAC,SAAS,CAAC;IAC1F,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,UAAU,CAAC;IAC/D,gBAAgB,EAAE,MAAM,CAAC,OAAO,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,WAAW,CAAC,MAAM,CAAC;IACpE,UAAU,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,WAAW,CAAC,wBAAwB,CAAC;IACjE,WAAW,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,WAAW,CAAC,yBAAyB,CAAC;IACnE,OAAO,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,WAAW,CAAC,gBAAgB,CAAC;IACtD,mBAAmB,EAAE,MAAM,CAAC,OAAO,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,WAAW,CAAC,WAAW,CAAC;IAC7E,qBAAqB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,WAAW,CAAC,cAAc,CAAC;IACjF,iBAAiB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,WAAW,CAAC,eAAe,CAAC;IAC7E,qBAAqB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,eAAe,CAAC;IAC9E,eAAe,EAAE,MAAM,CAAC,MAAM,CAAC;QAC7B,YAAY,EAAE,MAAM,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,WAAW,CAAC,WAAW,CAAC;QACvD,QAAQ,EAAE,MAAM,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,WAAW,CAAC,YAAY,CAAC;QACxD,iBAAiB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,WAAW,CAAC,8CAA8C,CAAC;QAC5G,mBAAmB,EAAE,MAAM,CAAC,MAAM,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,WAAW,CAAC,kBAAkB,CAAC;QACjF,sBAAsB,EAAE,MAAM,CAAC,OAAO,EAAE,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,WAAW,CAAC,qBAAqB,CAAC;QAC1F,WAAW,EAAE,MAAM,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,WAAW,CAAC,gBAAgB,CAAC;KAC5E,CAAC;CACH,CAAC,CAAC;AAEH,mDAAmD;AACnD,MAAM,CAAC,MAAM,eAAe,GAAG,OAAO,CAAC;AACvC,MAAM,CAAC,MAAM,6BAA6B,GAAG,eAAe,GAAG,MAAM,CAAC;AAiCtE,uCAAuC;AACvC,SAAS,kBAAkB,CAAC,GAAgB;IAC1C,MAAM,GAAG,GAAI,GAAsD,CAAC,GAAG,CAAC;IACxE,IAAI,OAAO,GAAG,KAAK,UAAU;QAAE,OAAO,SAAS,CAAC;IAChD,MAAM,OAAO,GAAG,GAAG,CAAC,IAAI,CAAC,GAAG,EAAE,aAAa,CAAC,CAAC;IAC7C,IAAI,OAAO,KAAK,SAAS,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,SAAS,CAAC;IAChE,OAAO,OAA6B,CAAC;AACvC,CAAC;AAED;;;;;GAKG;AACH,KAAK,UAAU,gBAAgB,CAC7B,WAA2C,EAC3C,SAA2B,EAC3B,KAAa;IAEb,IAAI,WAAW,KAAK,SAAS,IAAI,SAAS,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,EAAE,CAAC;IACnE,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,SAAS,CAAC,MAAM,CAAC,CAAC,CAAC;IAC7E,MAAM,KAAK,GAAG,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC;IACtC,MAAM,MAAM,GAAG,SAAS,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC;QACvD,IAAI,EAAE,QAAQ,CAAC,UAAU;QACzB,SAAS,EAAE,WAAW;QACtB,IAAI,EAAE,aAAa,QAAQ,CAAC,MAAM,CAAC,QAAQ,MAAM;KAClD,CAAC,CAAC,CAAC;IACJ,MAAM,IAAI,GAAG,MAAM,WAAW,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC;IAClD,IAAI,IAAI,CAAC,MAAM,KAAK,MAAM,CAAC,MAAM,EAAE,CAAC;QAClC,MAAM,IAAI,KAAK,CACb,+BAA+B,IAAI,CAAC,MAAM,mBAAmB,MAAM,CAAC,MAAM,aAAa;YACrF,qCAAqC,CACxC,CAAC;IACJ,CAAC;IACD,MAAM,OAAO,GAAqC,IAAI,KAAK,CAA4B,SAAS,CAAC,MAAM,CAAC,CAAC;IACzG,IAAI,CAAC,OAAO,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE;QAC1B,OAAO,CAAC,KAAK,GAAG,KAAK,CAAC,GAAG;YACvB,YAAY,EAAE,MAAM,CAAC,GAAG,CAAC,YAAY,CAAC;YACtC,SAAS,EAAE,MAAM,CAAC,GAAG,CAAC,SAAS,CAAC;YAChC,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;YACzC,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;YACzC,MAAM,EAAE,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;SAC5C,CAAC;IACJ,CAAC,CAAC,CAAC;IACH,OAAO,OAAO,CAAC;AACjB,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,aAAa,CAAC,KAAc;IAC1C,MAAM,MAAM,GAAmB,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC;IACxF,MAAM,OAAO,GAAI,KAA+B,CAAC,OAAO,CAAC;IACzD,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,OAAO,CAAC;QAAE,OAAO,MAAM,CAAC;IAC3C,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;QAC7B,MAAM,KAAK,GAAI,MAA8B,CAAC,KAAkC,CAAC;QACjF,IAAI,KAAK,KAAK,SAAS,IAAI,KAAK,KAAK,IAAI,IAAI,OAAO,KAAK,CAAC,YAAY,KAAK,QAAQ,EAAE,CAAC;YACpF,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,OAAO,EAAE,UAAU,EAAE,KAAK,EAAE,CAAC,CAAC;QACpD,CAAC;IACH,CAAC;IACD,OAAO,MAAM,CAAC;AAChB,CAAC;AAED,MAAM,UAAU,mBAAmB,CAAC,KAAyD;IAC3F,MAAM,EAAE,MAAM,EAAE,GAAG,KAAK,CAAC;IACzB,MAAM,IAAI,GAAG,eAAe,CAAC,EAAE,UAAU,EAAE,MAAM,CAAC,UAAU,EAAE,WAAW,EAAE,MAAM,CAAC,WAAW,EAAE,CAAC,CAAC;IACjG,MAAM,WAAW,GAAG,kBAAkB,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IAClD,MAAM,YAAY,GAAmB,CAAC,SAAS,EAAE,EAAE,CACjD,gBAAgB,CAAC,WAAW,EAAE,SAAS,EAAE,MAAM,CAAC,gBAAgB,CAAC,CAAC;IACpE,MAAM,KAAK,GAAY,CAAC,SAAS,EAAE,MAAM,EAAE,EAAE,CAC3C,UAAU,CAAC,SAAS,EAAE,EAAE,WAAW,EAAE,IAAI,CAAC,OAAO,EAAE,MAAM,EAAE,CAAC,CAAC;IAC/D,MAAM,OAAO,GAAc,CAAC,SAAS,EAAE,WAAW,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE,CACpE,0BAA0B,CAAC,SAAS,EAAE,WAAW,EAAE,MAAM,EAAE;QACzD,UAAU,EAAE,IAAI,CAAC,MAAM;QACvB,WAAW,EAAE,IAAI,CAAC,OAAO;QACzB,cAAc,EAAE,MAAM,CAAC,cAAc;QACrC,aAAa,EAAE,CAAC;QAChB,aAAa,EAAE,CAAC;QAChB,eAAe,EAAE,UAAU;QAC3B,eAAe,EAAE,MAAM,CAAC,eAAe;QACvC,iBAAiB,EAAE,MAAM,CAAC,eAAe,CAAC,iBAAiB;QAC3D,MAAM;KACP,CAAC,CAAC;IACL,MAAM,MAAM,GAAa,CAAC,SAAS,EAAE,WAAW,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,EAAE,CAC7E,aAAa,CAAC,SAAS,EAAE,WAAW,EAAE,MAAM,EAAE,SAAS,IAAI,SAAS,EAAE;QACpE,UAAU,EAAE,IAAI,CAAC,MAAM;QACvB,WAAW,EAAE,IAAI,CAAC,OAAO;QACzB,cAAc,EAAE,MAAM,CAAC,cAAc;QACrC,qBAAqB,EAAE,MAAM,CAAC,qBAAqB;QACnD,iBAAiB,EAAE,MAAM,CAAC,iBAAiB;QAC3C,iBAAiB,EAAE,MAAM,CAAC,iBAAiB;QAC3C,sBAAsB,EAAE,MAAM,CAAC,sBAAsB;QACrD,eAAe,EAAE,MAAM,CAAC,eAAe;QACvC,WAAW,EAAE,MAAM,CAAC,WAAW;QAC/B,qBAAqB,EAAE,MAAM,CAAC,qBAAqB;QACnD,sBAAsB,EAAE,MAAM,CAAC,eAAe,CAAC,sBAAsB;QACrE,iBAAiB,EAAE,MAAM,CAAC,eAAe,CAAC,iBAAiB;QAC3D,MAAM;KACP,CAAC,CAAC;IACL,OAAO;QACL,IAAI,EAAE,SAAS;QACf,WAAW,EAAE,gBAAgB;QAC7B,UAAU,EAAE,cAAc;QAC1B,MAAM,EAAE;YACN,MAAM,EAAE,gBAAgB;YACxB,uDAAuD;YACvD,sCAAsC;YACtC,MAAM,EAAE,CAAC,KAAc,EAAE,KAAc,EAAE,EAAE,CAAC,aAAa,CAAC,KAAK,CAAC;SACjE;QACD,SAAS,EAAE,eAAe;QAC1B,uDAAuD;QACvD,eAAe;QACf,iBAAiB,EAAE,GAAG,EAAE,CAAC,KAAK;QAC9B,KAAK,CAAC,OAAO,CAAC,IAAkB,EAAE,IAA0B;YAC1D,OAAO,cAAc,CAAC,IAAI,EAAE,IAAI,EAAE;gBAChC,KAAK;gBACL,OAAO;gBACP,MAAM;gBACN,YAAY;gBACZ,MAAM,EAAE;oBACN,UAAU,EAAE,MAAM,CAAC,iBAAiB;oBACpC,eAAe,EAAE,MAAM,CAAC,sBAAsB;iBAC/C;aACF,CAAC,CAAC;QACL,CAAC;KACF,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,KAAK,CAAC,GAAY,EAAE,MAA4B;IAC9D,MAAM,IAAI,GAAG,GAAkB,CAAC;IAChC,MAAM,OAAO,GAAG,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,mBAAmB,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC;IAChF,+CAA+C;IAC/C,yCAAyC;IACzC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,SAAS,EAAE,GAAG,EAAE;QAC5B,IAAI,CAAC;YACH,OAAO,EAAE,CAAC;QACZ,CAAC;QAAC,MAAM,CAAC;YACP,cAAc;QAChB,CAAC;IACH,CAAC,CAAC,CAAC;AACL,CAAC"}
|
package/lib/session/state.js
CHANGED
|
@@ -10,8 +10,11 @@ import { statSync } from 'node:fs';
|
|
|
10
10
|
import { resolve } from 'node:path';
|
|
11
11
|
/** 视频指纹：规范化路径 + size + mtime 联合哈希前 16 hex。 */
|
|
12
12
|
export function fingerprintOf(videoPath) {
|
|
13
|
-
|
|
14
|
-
|
|
13
|
+
// stat 使用原始路径（Linux/macOS 大小写敏感，小写化会指向不存在的文件）；
|
|
14
|
+
// 小写化仅用于指纹哈希输入，保证 Windows 路径大小写差异不产生重复槽位。
|
|
15
|
+
const resolvedPath = resolve(videoPath);
|
|
16
|
+
const st = statSync(resolvedPath);
|
|
17
|
+
const normalizedPath = resolvedPath.replaceAll('\\', '/').toLowerCase();
|
|
15
18
|
const digest = createHash('sha256')
|
|
16
19
|
.update(`${normalizedPath}|${st.size}|${Math.round(st.mtimeMs)}`)
|
|
17
20
|
.digest('hex');
|
|
@@ -68,9 +71,6 @@ export function getVideoSlot(state, fingerprint) {
|
|
|
68
71
|
gEma: 0,
|
|
69
72
|
emaWarmupLeft: 0,
|
|
70
73
|
settleStartPtsMs: null,
|
|
71
|
-
lastKCandidates: null,
|
|
72
|
-
lastMotionVec: null,
|
|
73
|
-
arena: null,
|
|
74
74
|
},
|
|
75
75
|
pendingPUnits: [],
|
|
76
76
|
lastAnchorF: null,
|
|
@@ -112,15 +112,6 @@ export function exportSessionSnapshot(state) {
|
|
|
112
112
|
lastEmittedPacketPtsMs: c.lastEmittedPacketPtsMs,
|
|
113
113
|
snapshot: {
|
|
114
114
|
...c.snapshot,
|
|
115
|
-
arena: c.snapshot.arena == null
|
|
116
|
-
? null
|
|
117
|
-
: {
|
|
118
|
-
ptsMs: c.snapshot.arena.ptsMs,
|
|
119
|
-
rho: c.snapshot.arena.rho,
|
|
120
|
-
g: c.snapshot.arena.g,
|
|
121
|
-
image672: c.snapshot.arena.image672.toString('base64'),
|
|
122
|
-
sourceCropBbox: c.snapshot.arena.sourceCropBbox,
|
|
123
|
-
},
|
|
124
115
|
},
|
|
125
116
|
pendingPUnits: c.pendingPUnits.map((p) => ({
|
|
126
117
|
ptsMs: p.ptsMs,
|
|
@@ -209,15 +200,6 @@ export function importSessionSnapshot(json) {
|
|
|
209
200
|
lastEmittedPacketPtsMs: v.lastEmittedPacketPtsMs,
|
|
210
201
|
snapshot: {
|
|
211
202
|
...v.snapshot,
|
|
212
|
-
arena: v.snapshot.arena == null
|
|
213
|
-
? null
|
|
214
|
-
: {
|
|
215
|
-
ptsMs: v.snapshot.arena.ptsMs,
|
|
216
|
-
rho: v.snapshot.arena.rho,
|
|
217
|
-
g: v.snapshot.arena.g,
|
|
218
|
-
image672: Buffer.from(v.snapshot.arena.image672, 'base64'),
|
|
219
|
-
sourceCropBbox: v.snapshot.arena.sourceCropBbox,
|
|
220
|
-
},
|
|
221
203
|
},
|
|
222
204
|
pendingPUnits: v.pendingPUnits.map((p) => ({
|
|
223
205
|
ptsMs: p.ptsMs,
|
package/lib/session/state.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"state.js","sourceRoot":"","sources":["../../src/session/state.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,QAAQ,EAAE,MAAM,SAAS,CAAC;AACnC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAapC,8CAA8C;AAC9C,MAAM,UAAU,aAAa,CAAC,SAAiB;IAC7C,MAAM,
|
|
1
|
+
{"version":3,"file":"state.js","sourceRoot":"","sources":["../../src/session/state.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH,OAAO,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACzC,OAAO,EAAE,QAAQ,EAAE,MAAM,SAAS,CAAC;AACnC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAapC,8CAA8C;AAC9C,MAAM,UAAU,aAAa,CAAC,SAAiB;IAC7C,+CAA+C;IAC/C,0CAA0C;IAC1C,MAAM,YAAY,GAAG,OAAO,CAAC,SAAS,CAAC,CAAC;IACxC,MAAM,EAAE,GAAG,QAAQ,CAAC,YAAY,CAAC,CAAC;IAClC,MAAM,cAAc,GAAG,YAAY,CAAC,UAAU,CAAC,IAAI,EAAE,GAAG,CAAC,CAAC,WAAW,EAAE,CAAC;IACxE,MAAM,MAAM,GAAG,UAAU,CAAC,QAAQ,CAAC;SAChC,MAAM,CAAC,GAAG,cAAc,IAAI,EAAE,CAAC,IAAI,IAAI,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,OAAO,CAAC,EAAE,CAAC;SAChE,MAAM,CAAC,KAAK,CAAC,CAAC;IACjB,OAAO;QACL,cAAc;QACd,SAAS,EAAE,EAAE,CAAC,IAAI;QAClB,OAAO,EAAE,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,OAAO,CAAC;QAC/B,WAAW,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC;KACjC,CAAC;AACJ,CAAC;AAED,MAAM,UAAU,oBAAoB;IAClC,OAAO,EAAE,QAAQ,EAAE,UAAU,EAAE,IAAI,EAAE,UAAU,EAAE,KAAK,EAAE,UAAU,EAAE,SAAS,EAAE,KAAK,EAAE,CAAC;AACzF,CAAC;AAED,MAAM,UAAU,kBAAkB,CAAC,SAAiB;IAClD,OAAO;QACL,SAAS;QACT,MAAM,EAAE,oBAAoB,EAAE;QAC9B,MAAM,EAAE,IAAI,GAAG,EAAE;QACjB,UAAU,EAAE,IAAI,GAAG,EAAE;QACrB,aAAa,EAAE,CAAC;QAChB,SAAS,EAAE,CAAC;QACZ,SAAS,EAAE,CAAC;QACZ,OAAO,EAAE,IAAI,GAAG,EAAE;QAClB,YAAY,EAAE,CAAC;KAChB,CAAC;AACJ,CAAC;AAED,sBAAsB;AACtB,MAAM,CAAC,MAAM,eAAe,GAAG,CAAC,CAAC;AAEjC,mCAAmC;AACnC,MAAM,UAAU,YAAY,CAAC,KAAmB,EAAE,WAA6B;IAC7E,MAAM,GAAG,GAAG,WAAW,CAAC,WAAW,CAAC;IACpC,MAAM,QAAQ,GAAG,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IACvC,IAAI,QAAQ,EAAE,CAAC;QACb,uBAAuB;QACvB,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;QACzB,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;QAChC,OAAO,QAAQ,CAAC;IAClB,CAAC;IACD,OAAO,KAAK,CAAC,MAAM,CAAC,IAAI,IAAI,eAAe,EAAE,CAAC;QAC5C,MAAM,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC;QAChD,IAAI,MAAM,IAAI,IAAI;YAAE,MAAM;QAC1B,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IAC9B,CAAC;IACD,MAAM,IAAI,GAAG;QACX,MAAM,EAAE;YACN,WAAW;YACX,iBAAiB,EAAE,CAAC;YACpB,sBAAsB,EAAE,IAAI;YAC5B,QAAQ,EAAE;gBACR,KAAK,EAAE,QAAiB;gBACxB,cAAc,EAAE,CAAC;gBACjB,WAAW,EAAE,CAAC;gBACd,IAAI,EAAE,CAAC;gBACP,aAAa,EAAE,CAAC;gBAChB,gBAAgB,EAAE,IAAI;aACvB;YACD,aAAa,EAAE,EAAE;YACjB,WAAW,EAAE,IAAI;SAClB;QACD,OAAO,EAAE,EAAE,OAAO,EAAE,EAAE,EAAE;QACxB,KAAK,EAAE,EAAE,UAAU,EAAE,CAAC,EAAE,QAAQ,EAAE,EAAE,EAAE;KACvC,CAAC;IACF,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;IAC5B,OAAO,IAAI,CAAC;AACd,CAAC;AAED,0DAA0D;AAC1D,MAAM,cAAc,GAAG,IAAI,GAAG,EAAwB,CAAC;AAEvD,MAAM,UAAU,eAAe,CAAC,SAAiB;IAC/C,IAAI,KAAK,GAAG,cAAc,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC;IAC1C,IAAI,CAAC,KAAK,EAAE,CAAC;QACX,KAAK,GAAG,kBAAkB,CAAC,SAAS,CAAC,CAAC;QACtC,cAAc,CAAC,GAAG,CAAC,SAAS,EAAE,KAAK,CAAC,CAAC;IACvC,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,sBAAsB;AACtB,MAAM,UAAU,mBAAmB;IACjC,cAAc,CAAC,KAAK,EAAE,CAAC;AACzB,CAAC;AAED,sDAAsD;AACtD,MAAM,UAAU,gBAAgB,CAAC,KAAmB;IAClD,cAAc,CAAC,GAAG,CAAC,KAAK,CAAC,SAAS,EAAE,KAAK,CAAC,CAAC;AAC7C,CAAC;AAkDD,2CAA2C;AAC3C,MAAM,UAAU,qBAAqB,CAAC,KAAmB;IACvD,MAAM,MAAM,GAAmC,EAAE,CAAC;IAClD,KAAK,MAAM,CAAC,GAAG,EAAE,IAAI,CAAC,IAAI,KAAK,CAAC,MAAM,EAAE,CAAC;QACvC,MAAM,CAAC,GAAG,IAAI,CAAC,MAAM,CAAC;QACtB,MAAM,CAAC,IAAI,CAAC;YACV,GAAG;YACH;gBACE,WAAW,EAAE,CAAC,CAAC,WAAW;gBAC1B,iBAAiB,EAAE,CAAC,CAAC,iBAAiB;gBACtC,sBAAsB,EAAE,CAAC,CAAC,sBAAsB;gBAChD,QAAQ,EAAE;oBACR,GAAG,CAAC,CAAC,QAAQ;iBACd;gBACD,aAAa,EAAE,CAAC,CAAC,aAAa,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;oBACzC,KAAK,EAAE,CAAC,CAAC,KAAK;oBACd,QAAQ,EAAE,CAAC,CAAC,QAAQ,CAAC,QAAQ,CAAC,QAAQ,CAAC;oBACvC,cAAc,EAAE,CAAC,CAAC,cAAc;iBACjC,CAAC,CAAC;gBACH,WAAW,EAAE,CAAC,CAAC,WAAW;gBAC1B,OAAO,EAAE;oBACP,OAAO,EAAE,IAAI,CAAC,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;wBACxC,MAAM,EAAE,CAAC,CAAC,MAAM;wBAChB,UAAU,EAAE,CAAC,CAAC,UAAU,CAAC,QAAQ,CAAC,QAAQ,CAAC;wBAC3C,WAAW,EAAE,CAAC,CAAC,WAAW,CAAC,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI;wBACpE,kBAAkB,EAAE,CAAC,CAAC,kBAAkB;qBACzC,CAAC,CAAC;iBACJ;gBACD,KAAK,EAAE;oBACL,UAAU,EAAE,IAAI,CAAC,KAAK,CAAC,UAAU;oBACjC,QAAQ,EAAE,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;wBACxC,WAAW,EAAE,CAAC,CAAC,WAAW;wBAC1B,cAAc,EAAE,CAAC,CAAC,cAAc;wBAChC,SAAS,EAAE,CAAC,CAAC,SAAS;wBACtB,aAAa,EAAE,CAAC,CAAC,aAAa;wBAC9B,KAAK,EAAE,CAAC,CAAC,KAAK;wBACd,SAAS,EAAE,CAAC,CAAC,SAAS;wBACtB,cAAc,EAAE,CAAC,CAAC,cAAc;wBAChC,KAAK,EAAE,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,QAAQ,CAAC;wBACjC,WAAW,EAAE,CAAC,CAAC,WAAW;qBAC3B,CAAC,CAAC;iBACJ;aACF;SACF,CAAC,CAAC;IACL,CAAC;IACD,MAAM,GAAG,GAAgB;QACvB,OAAO,EAAE,CAAC;QACV,SAAS,EAAE,KAAK,CAAC,SAAS;QAC1B,MAAM,EAAE,KAAK,CAAC,MAAM;QACpB,aAAa,EAAE,KAAK,CAAC,aAAa;QAClC,SAAS,EAAE,KAAK,CAAC,SAAS;QAC1B,SAAS,EAAE,KAAK,CAAC,SAAS;QAC1B,YAAY,EAAE,KAAK,CAAC,YAAY;QAChC,OAAO,EAAE,CAAC,GAAG,KAAK,CAAC,OAAO,CAAC,OAAO,EAAE,CAAC;QACrC,UAAU,EAAE,CAAC,GAAG,KAAK,CAAC,UAAU,CAAC,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;YACrD,SAAS,EAAE,CAAC,CAAC,SAAS;YACtB,QAAQ,EAAE,CAAC,CAAC,QAAQ;YACpB,KAAK,EAAE,CAAC,CAAC,KAAK;YACd,gBAAgB,EAAE,CAAC,CAAC,gBAAgB,CAAC,QAAQ,CAAC,QAAQ,CAAC;YACvD,aAAa,EAAE,CAAC,CAAC,aAAa;YAC9B,UAAU,EAAE,CAAC,CAAC,UAAU,EAAE,QAAQ,CAAC,QAAQ,CAAC;YAC5C,WAAW,EAAE,CAAC,CAAC,WAAW;YAC1B,WAAW,EAAE,CAAC,CAAC,WAAW;SAC3B,CAAC,CAAC;QACH,MAAM;KACP,CAAC;IACF,OAAO,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC;AAC7B,CAAC;AAED,6CAA6C;AAC7C,MAAM,UAAU,qBAAqB,CAAC,IAAY;IAChD,MAAM,GAAG,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAgB,CAAC;IAC5C,IAAI,GAAG,CAAC,OAAO,KAAK,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,cAAc,GAAG,CAAC,OAAO,EAAE,CAAC,CAAC;IACpE,MAAM,KAAK,GAAG,kBAAkB,CAAC,GAAG,CAAC,SAAS,CAAC,CAAC;IAChD,KAAK,CAAC,MAAM,GAAG,GAAG,CAAC,MAAM,CAAC;IAC1B,KAAK,CAAC,aAAa,GAAG,GAAG,CAAC,aAAa,CAAC;IACxC,KAAK,CAAC,SAAS,GAAG,GAAG,CAAC,SAAS,CAAC;IAChC,KAAK,CAAC,SAAS,GAAG,GAAG,CAAC,SAAS,CAAC;IAChC,KAAK,CAAC,YAAY,GAAG,GAAG,CAAC,YAAY,CAAC;IACtC,KAAK,CAAC,OAAO,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;IACrC,KAAK,CAAC,UAAU,GAAG,IAAI,GAAG,CACxB,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC;QACxB,CAAC,CAAC,SAAS;QACX;YACE,SAAS,EAAE,CAAC,CAAC,SAAS;YACtB,QAAQ,EAAE,CAAC,CAAC,QAAQ;YACpB,KAAK,EAAE,CAAC,CAAC,KAAK;YACd,gBAAgB,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,gBAAgB,EAAE,QAAQ,CAAC;YAC3D,aAAa,EAAE,CAAC,CAAC,aAAa;YAC9B,UAAU,EAAE,CAAC,CAAC,UAAU,IAAI,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,EAAE,QAAQ,CAAC,CAAC,CAAC,CAAC,SAAS;YAClF,WAAW,EAAE,CAAC,CAAC,WAAW;YAC1B,WAAW,EAAE,CAAC,CAAC,WAAW;SAC3B;KACF,CAAC,CACH,CAAC;IACF,KAAK,MAAM,CAAC,GAAG,EAAE,CAAC,CAAC,IAAI,GAAG,CAAC,MAAM,EAAE,CAAC;QAClC,KAAK,CAAC,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE;YACpB,MAAM,EAAE;gBACN,WAAW,EAAE,CAAC,CAAC,WAAW;gBAC1B,iBAAiB,EAAE,CAAC,CAAC,iBAAiB;gBACtC,sBAAsB,EAAE,CAAC,CAAC,sBAAsB;gBAChD,QAAQ,EAAE;oBACR,GAAG,CAAC,CAAC,QAAQ;iBACd;gBACD,aAAa,EAAE,CAAC,CAAC,aAAa,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;oBACzC,KAAK,EAAE,CAAC,CAAC,KAAK;oBACd,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,QAAQ,EAAE,QAAQ,CAAC;oBAC3C,cAAc,EAAE,CAAC,CAAC,cAAc;iBACjC,CAAC,CAAC;gBACH,WAAW,EAAE,CAAC,CAAC,WAAW;aAC3B;YACD,OAAO,EAAE;gBACP,OAAO,EAAE,CAAC,CAAC,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;oBACrC,MAAM,EAAE,CAAC,CAAC,MAAM;oBAChB,UAAU,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,UAAU,EAAE,QAAQ,CAAC;oBAC/C,WAAW,EAAE,CAAC,CAAC,WAAW,IAAI,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,WAAW,EAAE,QAAQ,CAAC,CAAC,CAAC,CAAC,IAAI;oBAChF,kBAAkB,EAAE,CAAC,CAAC,kBAAkB;iBACzC,CAAC,CAAC;aACJ;YACD,KAAK,EAAE;gBACL,UAAU,EAAE,CAAC,CAAC,KAAK,CAAC,UAAU;gBAC9B,QAAQ,EAAE,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;oBACrC,WAAW,EAAE,CAAC,CAAC,WAAW;oBAC1B,cAAc,EAAE,CAAC,CAAC,cAAc;oBAChC,SAAS,EAAE,CAAC,CAAC,SAAS;oBACtB,aAAa,EAAE,CAAC,CAAC,aAAa;oBAC9B,KAAK,EAAE,CAAC,CAAC,KAAK;oBACd,SAAS,EAAE,CAAC,CAAC,SAAS;oBACtB,cAAc,EAAE,CAAC,CAAC,cAAc;oBAChC,KAAK,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,KAAK,EAAE,QAAQ,CAAC;oBACrC,WAAW,EAAE,CAAC,CAAC,WAAW;iBAC3B,CAAC,CAAC;aACJ;SACF,CAAC,CAAC;IACL,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC"}
|
package/lib/stage0/decoder.d.ts
CHANGED
|
@@ -13,6 +13,8 @@
|
|
|
13
13
|
import type { RgbImage } from '../utils/image.js';
|
|
14
14
|
export interface DecodePointOptions {
|
|
15
15
|
ffmpegPath: string;
|
|
16
|
+
/** ffprobe 路径（缺省时由 ffmpegPath 同目录派生）。 */
|
|
17
|
+
ffprobePath?: string | undefined;
|
|
16
18
|
displayWidth: number;
|
|
17
19
|
displayHeight: number;
|
|
18
20
|
signal?: AbortSignal | undefined;
|