dsh-pf-stream 0.2.2 → 0.2.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.en.md +33 -121
- package/README.md +33 -122
- package/lib/index.d.ts +1 -1
- package/lib/tool/handler.js +6 -2
- package/lib/tool/handler.js.map +1 -1
- package/lib/tool/schema.d.ts +18 -6
- package/lib/tool/schema.js +27 -10
- package/lib/tool/schema.js.map +1 -1
- package/package.json +1 -1
package/README.en.md
CHANGED
|
@@ -2,145 +2,57 @@
|
|
|
2
2
|
|
|
3
3
|
[中文](README.md) | **English**
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
- **Continuous events** → adaptive streaming sampling. During motion, the sampler emits low-token process units by displacement integration. On settle or sharpness peaks, it freezes a high-resolution focus unit.
|
|
8
|
-
- **Suspicious time points** → deterministic inspection: metadata probe, frame extraction at given timestamps, uniform slicing, and local RoI zoom-in.
|
|
9
|
-
|
|
10
|
-
Every canvas aligns with the DeepSeek-V4.1-Flash vision pipeline: 42-px superpixel grid, billing `r×(c+1)+2`. All four forms bill ≤530 tokens, within the 1024-token per-image limit. The server encodes each canvas at its original resolution.
|
|
11
|
-
|
|
12
|
-
## Capability routing
|
|
13
|
-
|
|
14
|
-
One tool, `dsh-pf-stream`, routes through `intent.action`:
|
|
15
|
-
|
|
16
|
-
| action | engine | use |
|
|
17
|
-
| --- | --- | --- |
|
|
18
|
-
| `pf_stream` (default) | adaptive streaming | 336² process units (P) during motion; 672² focus units (F) on settle/peak. Schmitt four-state machine + two-layer adaptation (intent baseline + content feedback R1-R4) |
|
|
19
|
-
| `timestamps` | deterministic inspection | high-resolution frames at given timestamps; `roi_hint` zoom-in |
|
|
20
|
-
| `uniform` | deterministic inspection | uniform slicing by step or target frame count |
|
|
21
|
-
| `probe` | deterministic inspection | metadata only (display size, rotation, last-frame PTS, stream summary); zero vision tokens |
|
|
5
|
+
A video plugin for DeepSeek Harness (DSH). The agent uses it to watch local videos. During motion, the plugin records the process. At settle, it captures detail frames. Exact timestamps support full-resolution inspection.
|
|
22
6
|
|
|
23
7
|
## Install
|
|
24
8
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
- Open the DSH plugin manager, search for `dsh-pf-stream`, and install the latest version.
|
|
28
|
-
- Or tell the agent "install the plugin dsh-pf-stream"; the agent runs the same flow through plugin_manager.
|
|
9
|
+
Prerequisite: ffmpeg and ffprobe must be available.
|
|
29
10
|
|
|
30
|
-
|
|
11
|
+
### Option 1: delegate to the agent
|
|
31
12
|
|
|
32
|
-
|
|
13
|
+
Give this prompt to the agent:
|
|
33
14
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
15
|
+
```text
|
|
16
|
+
Install the DSH plugin dsh-pf-stream. Follow these steps:
|
|
17
|
+
1. Install the latest dsh-pf-stream from npm with the plugin manager. The result must be "applied".
|
|
18
|
+
2. Check ffmpeg and ffprobe. If they are missing, install them:
|
|
19
|
+
Windows: winget install Gyan.FFmpeg;
|
|
20
|
+
macOS: brew install ffmpeg;
|
|
21
|
+
Linux: sudo apt install ffmpeg.
|
|
22
|
+
3. Call probe on a local video. A returned duration and resolution means success.
|
|
23
|
+
4. Report the plugin version, the ffmpeg location, and the check result.
|
|
40
24
|
```
|
|
41
25
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
### 3. Verify the installation
|
|
26
|
+
### Option 2: manual install
|
|
45
27
|
|
|
46
|
-
|
|
28
|
+
1. Install ffmpeg and ffprobe:
|
|
47
29
|
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
|
|
30
|
+
```bash
|
|
31
|
+
winget install Gyan.FFmpeg # Windows
|
|
32
|
+
brew install ffmpeg # macOS
|
|
33
|
+
sudo apt install ffmpeg # Debian / Ubuntu
|
|
34
|
+
```
|
|
51
35
|
|
|
52
|
-
|
|
36
|
+
2. If the executables are outside PATH, set `ffmpegPath` and `ffprobePath` in the plugin config.
|
|
37
|
+
3. Add the package to the host: `pnpm add dsh-pf-stream`.
|
|
38
|
+
4. Mount the plugin as declared in `cordis.patch.yml` inside the package.
|
|
39
|
+
5. Verify: start a new DSH session. The tool `dsh-pf-stream` appears in the tool list.
|
|
53
40
|
|
|
54
41
|
## Usage
|
|
55
42
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
```json
|
|
59
|
-
{
|
|
60
|
-
"video_path": "D:/videos/demo.mp4",
|
|
61
|
-
"intent": { "action": "pf_stream", "focus": "detail", "mode": "dense" }
|
|
62
|
-
}
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
Point inspection (text found at 12.9 s in a streaming result):
|
|
66
|
-
|
|
67
|
-
```json
|
|
68
|
-
{
|
|
69
|
-
"video_path": "D:/videos/demo.mp4",
|
|
70
|
-
"intent": {
|
|
71
|
-
"action": "timestamps",
|
|
72
|
-
"timestamps_ms": [12900],
|
|
73
|
-
"roi_hint": { "x": 0.2, "y": 0.3, "w": 0.4, "h": 0.4 }
|
|
74
|
-
}
|
|
75
|
-
}
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
The top level also accepts common convenience fields from the video-inspection domain: `start_time`, `end_time`, `timestamps`, `fps`, `interval`, `num_frames`, `crop`, `probe_only`. The entry adapter normalizes them into the standard `intent` structure.
|
|
79
|
-
|
|
80
|
-
## Returned content
|
|
81
|
-
|
|
82
|
-
Each call returns one or more canvas packets:
|
|
43
|
+
Ask the agent directly:
|
|
83
44
|
|
|
84
|
-
-
|
|
85
|
-
-
|
|
86
|
-
-
|
|
45
|
+
- "What happens in this video?"
|
|
46
|
+
- "What text is shown at 12.9 seconds?"
|
|
47
|
+
- "Sample one frame every 5 seconds and describe the change."
|
|
87
48
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
- Intent persists across calls.
|
|
91
|
-
- Streaming cursor resume: the next call on the same video continues from the last position.
|
|
92
|
-
- Two-tier history reference: the hot tier reuses the latest 2 full strips; the cold tier reuses up to 4 independent F-anchor carriers.
|
|
93
|
-
- Epoch compaction: hot-to-cold conversion in batches of 6 packets, keeping the multi-turn Prefix KV-Cache prefix stable.
|
|
94
|
-
|
|
95
|
-
## Canvas and billing
|
|
96
|
-
|
|
97
|
-
| form | layout | tokens |
|
|
98
|
-
| --- | --- | --- |
|
|
99
|
-
| `FORM_1_STANDARD_STRIP` | 4P + 1F, 1344×672 | 530 |
|
|
100
|
-
| `FORM_2_COMPACT_4P` | 4P, 672×672 | 274 |
|
|
101
|
-
| `FORM_3_COMPACT_1F` | 1F, 672×672 | 274 |
|
|
102
|
-
| `FORM_4_PARTIAL_STRIP` | 1-3P + 1F, 1344×672 | 530 |
|
|
103
|
-
|
|
104
|
-
## Canvas samples
|
|
105
|
-
|
|
106
|
-
`FORM_1` standard strip: four process units (P) on the left, one focus unit (F) on the right.
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
`FORM_2` process-only: four process units.
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
`FORM_3` single focus: RoI zoom review via `roi_hint`.
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
## Configuration
|
|
119
|
-
|
|
120
|
-
| option | default | description |
|
|
121
|
-
| --- | --- | --- |
|
|
122
|
-
| `renderVisualId` | `true` | superpixel inverse badge at each unit corner |
|
|
123
|
-
| `maxPacketsPerCall` | `64` | packet count limit per call |
|
|
124
|
-
| `maxFramesPerCall` | `128` | deterministic frame target limit per call (clamps `frame_count`) |
|
|
125
|
-
| `maxTokensBudgetPerCall` | `65536` | vision-token budget per call |
|
|
126
|
-
| `maxRequestBytes` | `56 MiB` | request byte watermark, Base64-inclusive |
|
|
127
|
-
| `maxColdRefs` | `4` | cold-tier reference limit |
|
|
128
|
-
| `defaultStreamWindowMs` | `10000` | window length in ms when `pts_window` is absent |
|
|
129
|
-
| `streamWorkMaxEdge` | `2048` | working-resolution long edge for streaming decode; deterministic inspection always uses full resolution |
|
|
130
|
-
| `epochCompactionWindow` | `6` | hot-to-cold batch window, in packets |
|
|
131
|
-
| `meteringProfile.imageCountTiersEnabled` | `false` | gate for the ≥15-image 4096-px rule |
|
|
132
|
-
|
|
133
|
-
## Testing
|
|
134
|
-
|
|
135
|
-
```bash
|
|
136
|
-
pnpm test # boundary (pure logic) + e2e (fixture-driven), 20 boundary cases
|
|
137
|
-
pnpm run build # TypeScript compile
|
|
138
|
-
pnpm verify:host # host-contract self-check (schema shape, keyword subset, dependency boundary)
|
|
139
|
-
```
|
|
49
|
+
For parameters and return content, see the [usage guide](docs/usage-guide.md).
|
|
140
50
|
|
|
141
|
-
##
|
|
51
|
+
## Documentation
|
|
142
52
|
|
|
143
|
-
|
|
53
|
+
- [使用指南](docs/使用指南.md) (Chinese)
|
|
54
|
+
- [Usage Guide](docs/usage-guide.md) (English)
|
|
55
|
+
- [发布规范](docs/发布规范.md) (release process)
|
|
144
56
|
|
|
145
57
|
## License
|
|
146
58
|
|
package/README.md
CHANGED
|
@@ -2,146 +2,57 @@
|
|
|
2
2
|
|
|
3
3
|
**中文** | [English](README.en.md)
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
- **连续事件** → 自适应流式采样。运动时按位移积分抽低 token 过程单元；停稳与顿挫波峰时定格高清焦点单元。
|
|
8
|
-
- **可疑时间点** → 确定性检视。能力：元数据探测、指定时间戳抽帧、均匀切片、局部 RoI 放大复核。
|
|
9
|
-
|
|
10
|
-
全部画布对齐 DeepSeek-V4.1-Flash 视觉编码管线：42px 超像素网格，计费 `r×(c+1)+2`。四种形态计费均 ≤530 token，满足单图 1024 token 上限。服务端按画布原始分辨率计量。
|
|
11
|
-
|
|
12
|
-
## 能力路由
|
|
13
|
-
|
|
14
|
-
一个主工具 `dsh-pf-stream`，统一 `intent.action` 路由：
|
|
15
|
-
|
|
16
|
-
| action | 引擎 | 用途 |
|
|
17
|
-
| --- | --- | --- |
|
|
18
|
-
| `pf_stream`（默认） | 动态流式采样 | 运动中抽 336² 过程单元（P）；停稳/波峰定格 672² 焦点单元（F）。施密特四状态机 + 双层自适应调节（意图基准 + 内容自闭环 R1-R4） |
|
|
19
|
-
| `timestamps` | 确定性检视 | 指定时间戳单帧/组合高清检视，支持 `roi_hint` 局部放大 |
|
|
20
|
-
| `uniform` | 确定性检视 | 固定步长或目标帧数均匀切片 |
|
|
21
|
-
| `probe` | 确定性检视 | 仅元数据（显示宽高/旋转/末帧 PTS/流摘要），零视觉 token |
|
|
5
|
+
DSH 视频感知插件。智能体用它观看本地视频。画面运动时，插件记录过程。画面停稳时，插件定格细节。指定时间点可以高清检视。
|
|
22
6
|
|
|
23
7
|
## 安装
|
|
24
8
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
- 在 DSH 插件管理器中搜索 `dsh-pf-stream`，安装最新版本。
|
|
28
|
-
- 或直接对智能体说「安装插件 dsh-pf-stream」，智能体经 plugin_manager 完成同一流程。
|
|
9
|
+
前置条件：ffmpeg 与 ffprobe 必须可用。
|
|
29
10
|
|
|
30
|
-
|
|
11
|
+
### 方式一：交给智能体
|
|
31
12
|
|
|
32
|
-
|
|
13
|
+
把下面的提示词交给智能体：
|
|
33
14
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
15
|
+
```text
|
|
16
|
+
安装 DSH 插件 dsh-pf-stream，按以下步骤执行：
|
|
17
|
+
1. 用插件管理器从 npm 安装 dsh-pf-stream 最新版本。安装结果为 applied 才算成功。
|
|
18
|
+
2. 检查 ffmpeg 与 ffprobe 是否可用。缺失时安装：
|
|
19
|
+
Windows 用 winget install Gyan.FFmpeg；
|
|
20
|
+
macOS 用 brew install ffmpeg；
|
|
21
|
+
Linux 用 sudo apt install ffmpeg。
|
|
22
|
+
3. 对一个本地视频调用 probe。返回时长与分辨率即成功。
|
|
23
|
+
4. 汇报插件版本、ffmpeg 位置、验证结果。
|
|
40
24
|
```
|
|
41
25
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
### 3. 验证安装
|
|
26
|
+
### 方式二：手动安装
|
|
45
27
|
|
|
46
|
-
|
|
28
|
+
1. 安装 ffmpeg 与 ffprobe：
|
|
47
29
|
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
|
|
30
|
+
```bash
|
|
31
|
+
winget install Gyan.FFmpeg # Windows
|
|
32
|
+
brew install ffmpeg # macOS
|
|
33
|
+
sudo apt install ffmpeg # Debian / Ubuntu
|
|
34
|
+
```
|
|
51
35
|
|
|
52
|
-
|
|
36
|
+
2. 可执行文件不在 PATH 时，在插件配置里写 `ffmpegPath` 与 `ffprobePath`。
|
|
37
|
+
3. 在宿主环境安装插件包：`pnpm add dsh-pf-stream`。
|
|
38
|
+
4. 按包内 `cordis.patch.yml` 写宿主装载配置。
|
|
39
|
+
5. 验证：新开 DSH 会话。工具列表出现 `dsh-pf-stream` 即成功。
|
|
53
40
|
|
|
54
41
|
## 使用
|
|
55
42
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
```json
|
|
59
|
-
{
|
|
60
|
-
"video_path": "D:/videos/demo.mp4",
|
|
61
|
-
"intent": { "action": "pf_stream", "focus": "detail", "mode": "dense" }
|
|
62
|
-
}
|
|
63
|
-
```
|
|
64
|
-
|
|
65
|
-
定点复核（流式结果中 12.9s 处发现文字）：
|
|
66
|
-
|
|
67
|
-
```json
|
|
68
|
-
{
|
|
69
|
-
"video_path": "D:/videos/demo.mp4",
|
|
70
|
-
"intent": {
|
|
71
|
-
"action": "timestamps",
|
|
72
|
-
"timestamps_ms": [12900],
|
|
73
|
-
"roi_hint": { "x": 0.2, "y": 0.3, "w": 0.4, "h": 0.4 }
|
|
74
|
-
}
|
|
75
|
-
}
|
|
76
|
-
```
|
|
77
|
-
|
|
78
|
-
顶层同时接受视频检视领域常见便捷字段：`start_time` / `end_time` / `timestamps` / `fps` / `interval` / `num_frames` / `crop` / `probe_only`。入口适配器将其归一化为标准 `intent` 结构。
|
|
79
|
-
|
|
80
|
-
## 返回内容
|
|
81
|
-
|
|
82
|
-
每次调用返回一个或多个画布包：
|
|
43
|
+
对智能体直接提问：
|
|
83
44
|
|
|
84
|
-
-
|
|
85
|
-
-
|
|
86
|
-
-
|
|
45
|
+
- 「这个视频里发生了什么？」
|
|
46
|
+
- 「第 12.9 秒的文字写的是什么？」
|
|
47
|
+
- 「每 5 秒抽一帧，看画面怎么变化。」
|
|
87
48
|
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
- 意图跨调用持久化。
|
|
91
|
-
- 流式游标续窗：同视频下次调用从上次位置继续。
|
|
92
|
-
- 两级历史引用：热区最近 2 包整条带复用；冷区至多 4 个 F 锚点独立载体复用。
|
|
93
|
-
- Epoch Compaction：默认 6 包窗口批量热转冷，保持多轮对话 Prefix KV-Cache 前缀稳定。
|
|
94
|
-
|
|
95
|
-
## 画布与计费
|
|
96
|
-
|
|
97
|
-
| 形态 | 布局 | Token |
|
|
98
|
-
| --- | --- | --- |
|
|
99
|
-
| `FORM_1_STANDARD_STRIP` | 4P + 1F，1344×672 | 530 |
|
|
100
|
-
| `FORM_2_COMPACT_4P` | 4P，672×672 | 274 |
|
|
101
|
-
| `FORM_3_COMPACT_1F` | 1F，672×672 | 274 |
|
|
102
|
-
| `FORM_4_PARTIAL_STRIP` | 1-3P + 1F，1344×672 | 530 |
|
|
103
|
-
|
|
104
|
-
## 画布样例
|
|
105
|
-
|
|
106
|
-
`FORM_1` 标准条带：左半 4 个过程单元（P），右半 1 个焦点单元（F）。
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
`FORM_2` 纯过程：4 个过程单元。
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
`FORM_3` 单焦点：`roi_hint` 局部放大检视。
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
## 配置
|
|
119
|
-
|
|
120
|
-
| 配置项 | 默认 | 说明 |
|
|
121
|
-
| --- | --- | --- |
|
|
122
|
-
| `renderVisualId` | `true` | 单元左上角超像素反色角标 |
|
|
123
|
-
| `maxPacketsPerCall` | `64` | 单次调用返回包数上限 |
|
|
124
|
-
| `maxFramesPerCall` | `128` | 确定性抽帧单次目标帧数上限（`frame_count` 受此钳制） |
|
|
125
|
-
| `maxImagesPerCall` | `20` | 单次调用回传图像上限（宿主单条消息图像上限 20） |
|
|
126
|
-
| `maxTokensBudgetPerCall` | `65536` | 单次调用视觉 token 预算 |
|
|
127
|
-
| `maxRequestBytes` | `56 MiB` | 请求字节软水位（Base64 后口径） |
|
|
128
|
-
| `maxColdRefs` | `4` | 冷区引用上限 |
|
|
129
|
-
| `defaultStreamWindowMs` | `10000` | `pts_window` 缺省时窗口长度（毫秒） |
|
|
130
|
-
| `streamWorkMaxEdge` | `2048` | 流式解码工作分辨率长边上限（定点检视恒全分辨率） |
|
|
131
|
-
| `epochCompactionWindow` | `6` | 热转冷批量收敛窗口（包数） |
|
|
132
|
-
| `meteringProfile.imageCountTiersEnabled` | `false` | ≥15 图单边 4096px 规则门控 |
|
|
133
|
-
|
|
134
|
-
## 测试
|
|
135
|
-
|
|
136
|
-
```bash
|
|
137
|
-
pnpm test # boundary（纯逻辑）+ e2e（fixture 驱动），20 项边界用例
|
|
138
|
-
pnpm run build # TypeScript 编译
|
|
139
|
-
pnpm verify:host # 宿主契约自检（Schema 形状/关键字子集/依赖边界）
|
|
140
|
-
```
|
|
49
|
+
参数与返回内容见[使用指南](docs/使用指南.md)。
|
|
141
50
|
|
|
142
|
-
##
|
|
51
|
+
## 文档
|
|
143
52
|
|
|
144
|
-
|
|
53
|
+
- [使用指南](docs/使用指南.md)
|
|
54
|
+
- [Usage Guide](docs/usage-guide.md)
|
|
55
|
+
- [发布规范](docs/发布规范.md)
|
|
145
56
|
|
|
146
57
|
## 许可证
|
|
147
58
|
|
package/lib/index.d.ts
CHANGED
|
@@ -130,7 +130,7 @@ export declare function buildToolDefinition(_deps: {
|
|
|
130
130
|
}): {
|
|
131
131
|
name: string;
|
|
132
132
|
description: string;
|
|
133
|
-
parameters:
|
|
133
|
+
parameters: import("./tool/schema.js").ParameterRootSchema;
|
|
134
134
|
output: {
|
|
135
135
|
schema: {
|
|
136
136
|
readonly type: "object";
|
package/lib/tool/handler.js
CHANGED
|
@@ -28,8 +28,12 @@ async function withPacketImages(artifacts, attachImages) {
|
|
|
28
28
|
});
|
|
29
29
|
}
|
|
30
30
|
export async function handleToolCall(args, exec, deps) {
|
|
31
|
-
// 统一入口：
|
|
32
|
-
//
|
|
31
|
+
// 统一入口：video_path 前置校验（空实参时给出明确错误码，而非 probe 的
|
|
32
|
+
// 文件未找到）→ probe → 入口适配器归一化（历史便捷字段 + 枚举校验 + 钳制）
|
|
33
|
+
// → 按 intent.action 路由（《五》§1 与《七》§1）。
|
|
34
|
+
if (typeof args.video_path !== 'string' || args.video_path.length === 0) {
|
|
35
|
+
throw new PfStreamError('PFSTREAM_ARGS_VIDEO_PATH', 'video_path 必须为非空字符串');
|
|
36
|
+
}
|
|
33
37
|
const probeResult = await deps.probe(args.video_path, exec?.signal);
|
|
34
38
|
const { intent } = adaptCallArgs(args, {
|
|
35
39
|
lastPtsMs: probeResult.lastPtsMs,
|
package/lib/tool/handler.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"handler.js","sourceRoot":"","sources":["../../src/tool/handler.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAGH,OAAO,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAC7C,OAAO,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AAKpD,OAAO,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAmB7C,mEAAmE;AACnE,MAAM,UAAU,gBAAgB,CAAC,IAA0B;IACzD,OAAO,IAAI,EAAE,SAAS,IAAI,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,EAAE,CAAC;AACrD,CAAC;AAmDD,MAAM,UAAU,aAAa,CAAC,IAAkB;IAC9C,IAAI,IAAI,CAAC,MAAM,EAAE,MAAM;QAAE,OAAO,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC;IACnD,IAAI,IAAI,CAAC,MAAM,EAAE,UAAU,KAAK,IAAI,IAAI,IAAI,CAAC,UAAU,KAAK,IAAI;QAAE,OAAO,OAAO,CAAC;IACjF,OAAO,WAAW,CAAC;AACrB,CAAC;AAED,wCAAwC;AACxC,KAAK,UAAU,gBAAgB,CAC7B,SAA2B,EAC3B,YAAwC;IAExC,MAAM,IAAI,GAAG,YAAY,CAAC,CAAC,CAAC,MAAM,YAAY,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IAC/D,OAAO,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,KAAK,EAAE,EAAE;QACvC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC;QAC1B,OAAO,KAAK,CAAC,CAAC,CAAC,EAAE,GAAG,QAAQ,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,GAAG,QAAQ,CAAC,MAAM,EAAE,CAAC;IACxE,CAAC,CAAC,CAAC;AACL,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,cAAc,CAClC,IAAkB,EAClB,IAA0B,EAC1B,IAAc;IAEd,
|
|
1
|
+
{"version":3,"file":"handler.js","sourceRoot":"","sources":["../../src/tool/handler.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAGH,OAAO,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAC7C,OAAO,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AAKpD,OAAO,EAAE,aAAa,EAAE,MAAM,cAAc,CAAC;AAmB7C,mEAAmE;AACnE,MAAM,UAAU,gBAAgB,CAAC,IAA0B;IACzD,OAAO,IAAI,EAAE,SAAS,IAAI,IAAI,EAAE,KAAK,EAAE,OAAO,EAAE,EAAE,CAAC;AACrD,CAAC;AAmDD,MAAM,UAAU,aAAa,CAAC,IAAkB;IAC9C,IAAI,IAAI,CAAC,MAAM,EAAE,MAAM;QAAE,OAAO,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC;IACnD,IAAI,IAAI,CAAC,MAAM,EAAE,UAAU,KAAK,IAAI,IAAI,IAAI,CAAC,UAAU,KAAK,IAAI;QAAE,OAAO,OAAO,CAAC;IACjF,OAAO,WAAW,CAAC;AACrB,CAAC;AAED,wCAAwC;AACxC,KAAK,UAAU,gBAAgB,CAC7B,SAA2B,EAC3B,YAAwC;IAExC,MAAM,IAAI,GAAG,YAAY,CAAC,CAAC,CAAC,MAAM,YAAY,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IAC/D,OAAO,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,KAAK,EAAE,EAAE;QACvC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC;QAC1B,OAAO,KAAK,CAAC,CAAC,CAAC,EAAE,GAAG,QAAQ,CAAC,MAAM,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,GAAG,QAAQ,CAAC,MAAM,EAAE,CAAC;IACxE,CAAC,CAAC,CAAC;AACL,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,cAAc,CAClC,IAAkB,EAClB,IAA0B,EAC1B,IAAc;IAEd,8CAA8C;IAC9C,+CAA+C;IAC/C,sCAAsC;IACtC,IAAI,OAAO,IAAI,CAAC,UAAU,KAAK,QAAQ,IAAI,IAAI,CAAC,UAAU,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACxE,MAAM,IAAI,aAAa,CAAC,0BAA0B,EAAE,qBAAqB,CAAC,CAAC;IAC7E,CAAC;IACD,MAAM,WAAW,GAAG,MAAM,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,UAAU,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;IACpE,MAAM,EAAE,MAAM,EAAE,GAAG,aAAa,CAAC,IAA0C,EAAE;QAC3E,SAAS,EAAE,WAAW,CAAC,SAAS;QAChC,YAAY,EAAE,WAAW,CAAC,YAAY;QACtC,aAAa,EAAE,WAAW,CAAC,aAAa;QACxC,UAAU,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,IAAI,EAAE;QACzC,eAAe,EAAE,IAAI,CAAC,MAAM,EAAE,eAAe,IAAI,KAAK;QACtD,SAAS,EAAE,IAAI,CAAC,MAAM,EAAE,SAAS,IAAI,GAAG;KACzC,CAAC,CAAC;IACH,MAAM,MAAM,GAAG,MAAM,CAAC,MAAM,IAAI,WAAW,CAAC;IAE5C,IAAI,MAAM,KAAK,OAAO,EAAE,CAAC;QACvB,OAAO,EAAE,MAAM,EAAE,OAAO,EAAE,KAAK,EAAE,WAAW,EAAE,CAAC;IACjD,CAAC;IACD,IAAI,MAAM,KAAK,YAAY,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;QACpD,IAAI,CAAC,IAAI,CAAC,OAAO,EAAE,CAAC;YAClB,MAAM,IAAI,aAAa,CAAC,6BAA6B,EAAE,WAAW,MAAM,SAAS,CAAC,CAAC;QACrF,CAAC;QACD,MAAM,UAAU,GAAG,MAAM,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,UAAU,EAAE,WAAW,EAAE,MAAM,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;QAC1F,MAAM,OAAO,GAAG,MAAM,gBAAgB,CAAC,UAAU,CAAC,OAAO,EAAE,IAAI,CAAC,YAAY,CAAC,CAAC;QAC9E,OAAO;YACL,MAAM;YACN,KAAK,EAAE,WAAW;YAClB,OAAO;YACP,SAAS,EAAE,UAAU,CAAC,SAAS;SAChC,CAAC;IACJ,CAAC;IACD,IAAI,MAAM,KAAK,WAAW,EAAE,CAAC;QAC3B,IAAI,CAAC,IAAI,CAAC,MAAM,EAAE,CAAC;YACjB,MAAM,IAAI,aAAa,CAAC,6BAA6B,EAAE,WAAW,MAAM,SAAS,CAAC,CAAC;QACrF,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,UAAU,EAAE,WAAW,EAAE,MAAM,EAAE,gBAAgB,CAAC,IAAI,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;QAC7G,MAAM,OAAO,GAAG,MAAM,gBAAgB,CAAC,MAAM,CAAC,OAAO,EAAE,IAAI,CAAC,YAAY,CAAC,CAAC;QAC1E,OAAO;YACL,MAAM,EAAE,WAAW;YACnB,KAAK,EAAE,WAAW;YAClB,OAAO;YACP,SAAS,EAAE,MAAM,CAAC,SAAS;YAC3B,UAAU,EAAE,MAAM,CAAC,UAAU;YAC7B,SAAS,EAAE,MAAM,CAAC,SAAS;YAC3B,OAAO,EAAE,EAAE,QAAQ,EAAE,MAAM,CAAC,eAAe,EAAE;SAC9C,CAAC;IACJ,CAAC;IACD,MAAM,IAAI,aAAa,CAAC,6BAA6B,EAAE,WAAW,MAAM,SAAS,CAAC,CAAC;AACrF,CAAC"}
|
package/lib/tool/schema.d.ts
CHANGED
|
@@ -1,10 +1,15 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* 工具参数与输出 Schema
|
|
2
|
+
* 工具参数与输出 Schema。
|
|
3
|
+
*
|
|
4
|
+
* 本插件经 ctx.tools.register() 注册：宿主对该路径的 parameters 不做归一化，
|
|
5
|
+
* 原样投影给模型（dsh-tools schemaOf()）。因此 parameters 必须是标准 JSON
|
|
6
|
+
* Schema 根对象（{type:"object", properties, required}）；「属性名 → 值 schema」
|
|
7
|
+
* 映射形式只在 defineTool() 路径归一化，register() 路径使用会令模型端把工具
|
|
8
|
+
* 视为无参数。
|
|
3
9
|
*
|
|
4
|
-
* parameters 采用「属性名 → 值 schema」映射形式（宿主 dsh-tools 契约）。
|
|
5
10
|
* 数值边界（pts_window 越界、roi_hint 区间、max_packets/max_tokens_budget 上限）
|
|
6
11
|
* 由入口适配器按规格《五、双层自适应调节与统一参数映射》§1.3 钳制，不在
|
|
7
|
-
* Schema 层声明 minimum/maximum
|
|
12
|
+
* Schema 层声明 minimum/maximum。
|
|
8
13
|
*/
|
|
9
14
|
export declare const TOOL_NAME = "dsh-pf-stream";
|
|
10
15
|
export declare const INTENT_DESCRIPTION: string;
|
|
@@ -19,12 +24,19 @@ export interface ValueSchema {
|
|
|
19
24
|
items?: ValueSchema;
|
|
20
25
|
description?: string;
|
|
21
26
|
}
|
|
27
|
+
/** parameters 根对象：register() 路径要求的标准 JSON Schema 根包装。 */
|
|
28
|
+
export interface ParameterRootSchema {
|
|
29
|
+
type: 'object';
|
|
30
|
+
properties: Record<string, ValueSchema>;
|
|
31
|
+
required: string[];
|
|
32
|
+
additionalProperties: boolean;
|
|
33
|
+
}
|
|
22
34
|
export declare const intentValueSchema: ValueSchema;
|
|
23
35
|
/** 历史便捷字段（顶层）：由入口适配器归一化为标准 IntentDirective（《五》§1.1）。 */
|
|
24
36
|
export declare const legacyValueSchemas: Record<string, ValueSchema>;
|
|
25
|
-
/** 工具 parameters
|
|
26
|
-
*
|
|
27
|
-
export declare const toolParameters:
|
|
37
|
+
/** 工具 parameters：标准 JSON Schema 根对象（register() 路径原样投影给模型）。
|
|
38
|
+
* 顶层显式声明历史便捷字段（模型按 properties 生成实参，字段才能到达适配器）。 */
|
|
39
|
+
export declare const toolParameters: ParameterRootSchema;
|
|
28
40
|
export declare const TOOL_DESCRIPTION: string;
|
|
29
41
|
/** 输出 schema：随各里程碑交付逐步收紧，当前为对象根。 */
|
|
30
42
|
export declare const toolOutputSchema: {
|
package/lib/tool/schema.js
CHANGED
|
@@ -1,10 +1,15 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* 工具参数与输出 Schema
|
|
2
|
+
* 工具参数与输出 Schema。
|
|
3
|
+
*
|
|
4
|
+
* 本插件经 ctx.tools.register() 注册：宿主对该路径的 parameters 不做归一化，
|
|
5
|
+
* 原样投影给模型（dsh-tools schemaOf()）。因此 parameters 必须是标准 JSON
|
|
6
|
+
* Schema 根对象（{type:"object", properties, required}）；「属性名 → 值 schema」
|
|
7
|
+
* 映射形式只在 defineTool() 路径归一化，register() 路径使用会令模型端把工具
|
|
8
|
+
* 视为无参数。
|
|
3
9
|
*
|
|
4
|
-
* parameters 采用「属性名 → 值 schema」映射形式（宿主 dsh-tools 契约）。
|
|
5
10
|
* 数值边界（pts_window 越界、roi_hint 区间、max_packets/max_tokens_budget 上限）
|
|
6
11
|
* 由入口适配器按规格《五、双层自适应调节与统一参数映射》§1.3 钳制，不在
|
|
7
|
-
* Schema 层声明 minimum/maximum
|
|
12
|
+
* Schema 层声明 minimum/maximum。
|
|
8
13
|
*/
|
|
9
14
|
export const TOOL_NAME = 'dsh-pf-stream';
|
|
10
15
|
export const INTENT_DESCRIPTION = '统一采样与检视指令：' +
|
|
@@ -54,18 +59,30 @@ export const legacyValueSchemas = {
|
|
|
54
59
|
crop: { type: 'object', description: '局部区域（归一化或像素坐标，归一化为 intent.roi_hint）' },
|
|
55
60
|
probe_only: { type: 'boolean', description: '仅探测视频元数据（等价 action="probe"）' },
|
|
56
61
|
};
|
|
57
|
-
/** 工具 parameters
|
|
58
|
-
*
|
|
62
|
+
/** 工具 parameters：标准 JSON Schema 根对象（register() 路径原样投影给模型）。
|
|
63
|
+
* 顶层显式声明历史便捷字段（模型按 properties 生成实参，字段才能到达适配器）。 */
|
|
59
64
|
export const toolParameters = {
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
65
|
+
type: 'object',
|
|
66
|
+
properties: {
|
|
67
|
+
video_path: { type: 'string', description: '待处理的本地视频文件路径' },
|
|
68
|
+
intent: intentValueSchema,
|
|
69
|
+
...legacyValueSchemas,
|
|
70
|
+
},
|
|
71
|
+
required: ['video_path'],
|
|
72
|
+
additionalProperties: false,
|
|
63
73
|
};
|
|
64
74
|
export const TOOL_DESCRIPTION = 'Look at a local video: adaptive dual-scale streaming perception (low-token P units during motion, ' +
|
|
65
75
|
'high-detail F units at settle) plus deterministic inspection (probe metadata, exact-timestamp frames, ' +
|
|
66
76
|
'uniform slicing, RoI zoom). All canvases are 42px-grid aligned and billed as r*(c+1)+2 vision tokens. ' +
|
|
67
|
-
'
|
|
68
|
-
'
|
|
77
|
+
'Usage: call action="probe" first on a new video. Default action="pf_stream" for continuous content ' +
|
|
78
|
+
'(focus="detail" for text or object details, focus="motion" for actions or camera movement, ' +
|
|
79
|
+
'mode="dense"/"sparse" for more or fewer process frames). ' +
|
|
80
|
+
'Streaming canvases contain low-resolution process units and key freeze frames: before answering ' +
|
|
81
|
+
'questions about text, numbers, or small objects, verify the target time with action="timestamps" ' +
|
|
82
|
+
'and roi_hint. Pass pts_window (ms) to inspect one segment; use action="uniform" with interval_ms ' +
|
|
83
|
+
'or frame_count for an even overview. Repeat streaming calls resume from the last position, so a ' +
|
|
84
|
+
'long video can be viewed in segments. Cite packet IDs, unit IDs, and timestamps from the returned ' +
|
|
85
|
+
'hint lines as evidence.';
|
|
69
86
|
/** 输出 schema：随各里程碑交付逐步收紧，当前为对象根。 */
|
|
70
87
|
export const toolOutputSchema = {
|
|
71
88
|
type: 'object',
|
package/lib/tool/schema.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"schema.js","sourceRoot":"","sources":["../../src/tool/schema.ts"],"names":[],"mappings":"AAAA
|
|
1
|
+
{"version":3,"file":"schema.js","sourceRoot":"","sources":["../../src/tool/schema.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAEH,MAAM,CAAC,MAAM,SAAS,GAAG,eAAe,CAAC;AAEzC,MAAM,CAAC,MAAM,kBAAkB,GAC7B,YAAY;IACZ,wGAAwG;IACxG,sDAAsD;IACtD,0DAA0D;IAC1D,8BAA8B;IAC9B,4BAA4B,CAAC;AAsB/B,MAAM,CAAC,MAAM,iBAAiB,GAAgB;IAC5C,IAAI,EAAE,QAAQ;IACd,WAAW,EAAE,kBAAkB;IAC/B,UAAU,EAAE;QACV,MAAM,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,WAAW,EAAE,YAAY,EAAE,SAAS,EAAE,OAAO,CAAC,EAAE;QACjF,IAAI,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,OAAO,EAAE,UAAU,EAAE,QAAQ,CAAC,EAAE;QAC/D,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,CAAC,QAAQ,EAAE,QAAQ,EAAE,UAAU,CAAC,EAAE;QACjE,UAAU,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,WAAW,EAAE,iCAAiC,EAAE;QACxG,aAAa,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,WAAW,EAAE,gBAAgB,EAAE;QAC1F,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,YAAY,EAAE;QAC1D,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,WAAW,EAAE;QACzD,QAAQ,EAAE;YACR,IAAI,EAAE,QAAQ;YACd,UAAU,EAAE;gBACV,CAAC,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACrB,CAAC,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACrB,CAAC,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACrB,CAAC,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;aACtB;YACD,QAAQ,EAAE,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC;YAC9B,WAAW,EAAE,mBAAmB;SACjC;QACD,UAAU,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,WAAW,EAAE,sBAAsB,EAAE;QACpE,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,cAAc,EAAE;QAC5D,iBAAiB,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,mBAAmB,EAAE;QACvE,KAAK,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,WAAW,EAAE,aAAa,EAAE;KACvD;IACD,oBAAoB,EAAE,KAAK;CAC5B,CAAC;AAEF,wDAAwD;AACxD,MAAM,CAAC,MAAM,kBAAkB,GAAgC;IAC7D,UAAU,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,6CAA6C,EAAE;IAC1F,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,mCAAmC,EAAE;IAC9E,UAAU,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,WAAW,EAAE,yBAAyB,EAAE;IAChG,UAAU,EAAE,EAAE,IAAI,EAAE,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,WAAW,EAAE,gCAAgC,EAAE;IACvG,GAAG,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,uBAAuB,EAAE;IAC7D,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,4BAA4B,EAAE;IACvE,UAAU,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,0BAA0B,EAAE;IACvE,IAAI,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,qCAAqC,EAAE;IAC5E,UAAU,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,WAAW,EAAE,6BAA6B,EAAE;CAC5E,CAAC;AAEF;kDACkD;AAClD,MAAM,CAAC,MAAM,cAAc,GAAwB;IACjD,IAAI,EAAE,QAAQ;IACd,UAAU,EAAE;QACV,UAAU,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,WAAW,EAAE,cAAc,EAAE;QAC3D,MAAM,EAAE,iBAAiB;QACzB,GAAG,kBAAkB;KACtB;IACD,QAAQ,EAAE,CAAC,YAAY,CAAC;IACxB,oBAAoB,EAAE,KAAK;CAC5B,CAAC;AAEF,MAAM,CAAC,MAAM,gBAAgB,GAC3B,oGAAoG;IACpG,wGAAwG;IACxG,wGAAwG;IACxG,qGAAqG;IACrG,6FAA6F;IAC7F,2DAA2D;IAC3D,kGAAkG;IAClG,mGAAmG;IACnG,mGAAmG;IACnG,kGAAkG;IAClG,oGAAoG;IACpG,yBAAyB,CAAC;AAE5B,oCAAoC;AACpC,MAAM,CAAC,MAAM,gBAAgB,GAAG;IAC9B,IAAI,EAAE,QAAQ;CACN,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-pf-stream",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.4",
|
|
4
4
|
"description": "DeepSeek Harness video perception plugin: adaptive dual-scale streaming sampling plus deterministic inspection, aligned to the DeepSeek-V4.1-Flash vision pipeline (42px superpixel grid, <=1024 tokens per image).",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"repository": {
|