cerevox 4.36.0 → 4.37.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/core/ai.d.ts.map +1 -1
- package/dist/core/ai.js +73 -65
- package/dist/core/ai.js.map +1 -1
- package/dist/mcp/servers/prompts/skills//344/270/200/351/224/256/346/210/220/347/211/207.md +776 -80
- package/dist/mcp/servers/prompts/skills//346/227/247/344/270/200/351/224/256/346/210/220/347/211/207.md +103 -0
- package/dist/mcp/servers/zerocut.d.ts.map +1 -1
- package/dist/mcp/servers/zerocut.js +187 -285
- package/dist/mcp/servers/zerocut.js.map +1 -1
- package/dist/utils/pricing.d.ts.map +1 -1
- package/dist/utils/pricing.js +2 -1
- package/dist/utils/pricing.js.map +1 -1
- package/package.json +1 -1
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: 一键成片
|
|
3
|
+
description: 当用户要求创作某个主题的通用视频时,使用该技能,通过Zerocut MCP工具,根据用户需求创作多个分镜并合成通用视频
|
|
4
|
+
---
|
|
5
|
+
## 标准流水线
|
|
6
|
+
|
|
7
|
+
### 全局配置和注意事项
|
|
8
|
+
|
|
9
|
+
1. 音画同步:除非用户明确指定,否则生成视频时一律**不静音**(默认`silent=false`)
|
|
10
|
+
2. 一致性检查
|
|
11
|
+
- ‼️ 除非用户明确要求不传,否则为分镜生成图(generate-image)和视频(generate-video)时,都务必传 `sceneIndex` 参数,从 1 开始编号
|
|
12
|
+
3. 分镜自动化:在第一次创建视频时,根据用户需求,使用`generate-video-outlines`生成分镜草稿 storyboard.json,生成后直接使用,不需要进行任何修改。
|
|
13
|
+
4. 故障排查和自动处理
|
|
14
|
+
- 一般的错误,除非用户明确许可,否则禁止自行跳过一致性检查,而是修改 storyboard.json 中的相关内容,以保持一致性
|
|
15
|
+
5. 优先用 `audio-video-sync` 工具而**不是** `run-ffmpeg-command` 工具合成视频
|
|
16
|
+
6. 调用 `audio-video-sync` 合成视频时,如有对话或旁白,同步合成字幕(参数`addSubtitles=true`)
|
|
17
|
+
7. 在执行任务的过程中,任何工具返回`Not enough credits`错误时,必须暂停任务,等待用户充值后由用户手动触发继续执行
|
|
18
|
+
|
|
19
|
+
### 新建
|
|
20
|
+
|
|
21
|
+
1. 确保项目已正确开启:`project-open` 已被调用
|
|
22
|
+
2. 使用`retrieve-rules-context`,召回规则上下文
|
|
23
|
+
3. 分镜创作:根据用户需求,使用`generate-video-outlines`生成分镜草稿 storyboard.json
|
|
24
|
+
* 使用`generate-video-outlines`时,prompt 请直接转述用户描述,无需自行分析整理或创作
|
|
25
|
+
4. 使用 `generate-image` 生成各分镜图片
|
|
26
|
+
- 生成图片时,除非用户人为指定,否则优先使用`seedream-5l`模型
|
|
27
|
+
- 生成图片时,必须用`referenceImages`参数引用`outline_sheet.png`这张图,这张图已经由`generate-video-outlines`生成
|
|
28
|
+
- 引用 `outline_sheet.png` 时,参考图片的类型(type)为 `normal`
|
|
29
|
+
- 生成图片时,必须用`sceneIndex`参数指定分镜编号,从 1 开始编号
|
|
30
|
+
5. 使用 `generate-video` 生成各分镜视频
|
|
31
|
+
- 生成视频时,必须用`sceneIndex`参数指定分镜编号,从 1 开始编号
|
|
32
|
+
- 生成视频时,一律采用**首帧图生视频**(即参考图的 type 为 `first_frame`)
|
|
33
|
+
- 生成视频时,必须带声音(参数`silent=false`)
|
|
34
|
+
- 如分镜有旁白(narration),生成视频时必须带旁白(参数`narration={text, tone, voice_id}`)
|
|
35
|
+
6. 使用 `generate-music` 生成背景音乐
|
|
36
|
+
- 背景音乐时长为分镜视频总时长,若不足30秒应补足30秒
|
|
37
|
+
7. 合成视频:调用`audio-video-sync`输出视频并根据情况合成字幕(参数`addSubtitles=true`),自动下载到本地
|
|
38
|
+
- 视频如有对话或旁白,务必在调用`audio-video-sync`时设置参数`addSubtitles=true`,否则无需设置
|
|
39
|
+
8. 关闭项目 → `project-close`
|
|
40
|
+
|
|
41
|
+
### 修改
|
|
42
|
+
|
|
43
|
+
1. 确保项目已启动 → `project-open`
|
|
44
|
+
2. 修改脚本 → 按用户要求直接手动修改 storyboard.json (⚠️ 不再使用`generate-video-outlines`重新生成)
|
|
45
|
+
3. 更新素材 → 重新生成需要修改的素材
|
|
46
|
+
4. 重新合成视频 → `audio-video-sync` 输出视频并合成字幕(参数`addSubtitles=true`),自动下载到本地
|
|
47
|
+
5. 关闭项目 → `project-close`
|
|
48
|
+
|
|
49
|
+
---
|
|
50
|
+
|
|
51
|
+
## 主体参考图
|
|
52
|
+
* 如需保持主体形象一致,你可以先用`generate-image`生成主体参考三视图,然后参考三视图创建分镜草稿 storyboard.json
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## 质量建议
|
|
57
|
+
|
|
58
|
+
### materials 资源命名规范
|
|
59
|
+
|
|
60
|
+
- 场景素材:`sc01_first_frame.png`、`sc01_motion.mp4`、`sc01_vo.mp3`
|
|
61
|
+
- 通用素材:`main_bgm_60s.mp3`
|
|
62
|
+
- 合成视频:`<主题名>.v<版本>.mp4`
|
|
63
|
+
|
|
64
|
+
### 工作流管理
|
|
65
|
+
* 规划先行:先分析制定执行计划
|
|
66
|
+
* 工作流顺序:规划→搜索→分镜→图片→视频→BGM→合成视频
|
|
67
|
+
* 视频生成策略:
|
|
68
|
+
- 先使用`generate-image`生成首帧图片
|
|
69
|
+
- 再使用`generate-video`生成动态视频
|
|
70
|
+
* 统一命名:`scXX_*`、`main_bgm_*`、`*_vo.*`
|
|
71
|
+
* 时长控制:单镜头3-16s
|
|
72
|
+
|
|
73
|
+
### 图生视频技巧
|
|
74
|
+
* 运动导向:提示词=主体运动+背景变化+镜头运动
|
|
75
|
+
* 特征定位:突出主体特征(老人、戴墨镜的女人)便于识别
|
|
76
|
+
* 环境一致性:确保场景间环境元素一致
|
|
77
|
+
- 时间:保持时间段一致(白天、夜晚),避免无故突变
|
|
78
|
+
- 天气:保持天气状况一致(晴天、雨天)
|
|
79
|
+
- 地点:场景转换符合空间逻辑
|
|
80
|
+
- 光线:保持光源方向和强度一致
|
|
81
|
+
|
|
82
|
+
### BGM 音量控制
|
|
83
|
+
* 音量:默认BGM音量控制为-15db(`audioVolume=0.177`),通过设置`audio-video-async`的`audioVolume`参数可以调整BGM音量。
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## 规划与搜索规则
|
|
88
|
+
|
|
89
|
+
### 需求分析
|
|
90
|
+
- 理解核心需求:明确视频主题、目标受众、预期效果
|
|
91
|
+
- 确定视频类型:科普解说、产品介绍、故事叙述等
|
|
92
|
+
- 分析技术要求:视频时长、画幅比例、风格偏好
|
|
93
|
+
- 识别素材需求:需要的图片、视频、音频素材
|
|
94
|
+
|
|
95
|
+
### 搜索内容
|
|
96
|
+
- 特定领域知识、热点话题、视觉参考、事实验证
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## 故障排查
|
|
101
|
+
|
|
102
|
+
* `generate-video` 失败且失败原因是内容相关(如包含敏感信息)
|
|
103
|
+
- **禁止**轻易重新生成 outline,应手动编辑 `storyboard.json` 中的 video_prompt 删除或修改可能的敏感信息后重试;若还是失败,停下来询问用户如何处理
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"zerocut.d.ts","sourceRoot":"","sources":["../../../src/mcp/servers/zerocut.ts"],"names":[],"mappings":";
|
|
1
|
+
{"version":3,"file":"zerocut.d.ts","sourceRoot":"","sources":["../../../src/mcp/servers/zerocut.ts"],"names":[],"mappings":";AAs5FA,wBAAsB,GAAG,kBAKxB"}
|
|
@@ -728,9 +728,7 @@ server.registerTool('generate-image', {
|
|
|
728
728
|
.optional()
|
|
729
729
|
.default(9)
|
|
730
730
|
.describe('仅 type=storyboard 时生效,默认9格'),
|
|
731
|
-
prompt: zod_1.z
|
|
732
|
-
.string()
|
|
733
|
-
.describe('生图提示词,需严格忠于用户的具体需求; 如有sceneIndex,一般要严格对应 storyboard 中当前分镜的 first_frame 中的字段描述'),
|
|
731
|
+
prompt: zod_1.z.string().describe('生图提示词,需严格忠于用户的具体需求'),
|
|
734
732
|
sceneIndex: zod_1.z
|
|
735
733
|
.number()
|
|
736
734
|
.min(1)
|
|
@@ -818,7 +816,7 @@ server.registerTool('generate-image', {
|
|
|
818
816
|
},
|
|
819
817
|
}, async ({ model = 'seedream-5l', type = 'default', prompt, sceneIndex, size, imageCount = 1, saveToFileNames, watermark, referenceImages, panelCount, }, context) => {
|
|
820
818
|
try {
|
|
821
|
-
const storyBoardFile = '
|
|
819
|
+
const storyBoardFile = 'scene-bible.md';
|
|
822
820
|
// 验证session状态
|
|
823
821
|
const currentSession = await validateSession('generate-image');
|
|
824
822
|
checkModelEnabled(model);
|
|
@@ -831,90 +829,16 @@ server.registerTool('generate-image', {
|
|
|
831
829
|
size = '1440x2560';
|
|
832
830
|
}
|
|
833
831
|
}
|
|
834
|
-
const storyBoardPath = (0, node_path_1.resolve)(projectLocalDir, storyBoardFile);
|
|
835
|
-
const outlineSheetImagePath = (0, node_path_1.resolve)(projectLocalDir, 'materials', 'outline_sheet.png');
|
|
836
|
-
const hasOutlineSheet = (0, node_fs_1.existsSync)(outlineSheetImagePath);
|
|
832
|
+
const storyBoardPath = (0, node_path_1.resolve)(projectLocalDir, 'materials', storyBoardFile);
|
|
837
833
|
// 校验 prompt 与 storyboard.json 中分镜设定的一致性
|
|
838
834
|
if (sceneIndex) {
|
|
839
835
|
try {
|
|
840
|
-
if ((0, node_fs_1.existsSync)(storyBoardPath)) {
|
|
841
|
-
|
|
842
|
-
// 检查 storyBoard JSON 语法合法性
|
|
843
|
-
let storyBoard;
|
|
844
|
-
try {
|
|
845
|
-
storyBoard = JSON.parse(storyBoardContent);
|
|
846
|
-
}
|
|
847
|
-
catch (jsonError) {
|
|
848
|
-
return createErrorResponse(`The storyboard file ${storyBoardFile} contains invalid JSON. Please fix it and try again. Details: ${jsonError instanceof Error ? jsonError.message : String(jsonError)}`, 'generate-image');
|
|
849
|
-
}
|
|
850
|
-
if (storyBoard.scenes && Array.isArray(storyBoard.scenes)) {
|
|
851
|
-
const scene = storyBoard.scenes[sceneIndex - 1]; // sceneIndex 从1开始,数组从0开始
|
|
852
|
-
if (scene) {
|
|
853
|
-
const startFrame = scene.first_frame;
|
|
854
|
-
// 检查 prompt 是否严格等于 first_frame
|
|
855
|
-
if (prompt.trim() !== startFrame?.trim()) {
|
|
856
|
-
return createErrorResponse('The image prompt must exactly match the storyboard setting, including spaces, newlines, punctuation, and special characters. Ensure the provided prompt matches storyboard.first_frame exactly.', 'generate-image');
|
|
857
|
-
}
|
|
858
|
-
if (hasOutlineSheet &&
|
|
859
|
-
(!referenceImages ||
|
|
860
|
-
!referenceImages.find(item => item.fileName.includes('outline_sheet.png')))) {
|
|
861
|
-
return createErrorResponse('Detected outline_sheet.png in materials (generated by the outline tool). You must pass referenceImages and include outline_sheet.png in it.', 'generate-image');
|
|
862
|
-
}
|
|
863
|
-
// 校验 size 参数与 storyboard 的 orientation 属性一致性
|
|
864
|
-
if (size && storyBoard.orientation) {
|
|
865
|
-
const isLandscapeSize = [
|
|
866
|
-
'1152x864',
|
|
867
|
-
'1280x720',
|
|
868
|
-
'1248x832',
|
|
869
|
-
'1512x648',
|
|
870
|
-
'2304x1728',
|
|
871
|
-
'2560x1440',
|
|
872
|
-
'2496x1664',
|
|
873
|
-
'3024x1456',
|
|
874
|
-
'4096x3072',
|
|
875
|
-
'4096x2304',
|
|
876
|
-
'4096x2731',
|
|
877
|
-
'4096x1968',
|
|
878
|
-
].includes(size);
|
|
879
|
-
const isPortraitSize = [
|
|
880
|
-
'864x1152',
|
|
881
|
-
'720x1280',
|
|
882
|
-
'832x1248',
|
|
883
|
-
'1728x2304',
|
|
884
|
-
'1440x2560',
|
|
885
|
-
'1664x2496',
|
|
886
|
-
'3072x4096',
|
|
887
|
-
'2304x4096',
|
|
888
|
-
'2731x4096',
|
|
889
|
-
].includes(size);
|
|
890
|
-
const isSquareSize = [
|
|
891
|
-
'1024x1024',
|
|
892
|
-
'2048x2048',
|
|
893
|
-
'4096x4096',
|
|
894
|
-
].includes(size);
|
|
895
|
-
if (storyBoard.orientation === 'landscape' &&
|
|
896
|
-
!isLandscapeSize &&
|
|
897
|
-
!isSquareSize) {
|
|
898
|
-
return createErrorResponse(`Storyboard orientation is landscape (orientation: landscape), but the image size ${size} is portrait. Use a landscape size such as 1280x720, 2560x1440, 4096x2304, etc.`, 'generate-image');
|
|
899
|
-
}
|
|
900
|
-
if (storyBoard.orientation === 'portrait' &&
|
|
901
|
-
!isPortraitSize &&
|
|
902
|
-
!isSquareSize) {
|
|
903
|
-
return createErrorResponse(`Storyboard orientation is portrait (orientation: portrait), but the image size ${size} is landscape. Use a portrait size such as 720x1280, 1440x2560, 2304x4096, etc.`, 'generate-image');
|
|
904
|
-
}
|
|
905
|
-
}
|
|
906
|
-
}
|
|
907
|
-
else {
|
|
908
|
-
console.warn(`Scene index ${sceneIndex} not found in storyboard.json`);
|
|
909
|
-
}
|
|
910
|
-
}
|
|
911
|
-
}
|
|
912
|
-
else {
|
|
913
|
-
console.warn(`Story board file not found: ${storyBoardPath}`);
|
|
836
|
+
if (!(0, node_fs_1.existsSync)(storyBoardPath)) {
|
|
837
|
+
return createErrorResponse(`The storyboard file ${storyBoardFile} is missing or not in materials path. Please create it first.`, 'generate-image');
|
|
914
838
|
}
|
|
915
839
|
}
|
|
916
840
|
catch (error) {
|
|
917
|
-
console.error('Failed to validate prompt with
|
|
841
|
+
console.error('Failed to validate prompt with storyboard:', error);
|
|
918
842
|
// 如果读取或解析 storyboard.json 失败,继续执行但记录警告
|
|
919
843
|
}
|
|
920
844
|
}
|
|
@@ -1042,143 +966,180 @@ server.registerTool('generate-image', {
|
|
|
1042
966
|
return createErrorResponse(error, 'generate-image');
|
|
1043
967
|
}
|
|
1044
968
|
});
|
|
1045
|
-
server.registerTool(
|
|
1046
|
-
|
|
1047
|
-
|
|
1048
|
-
|
|
1049
|
-
|
|
1050
|
-
|
|
1051
|
-
|
|
1052
|
-
|
|
1053
|
-
|
|
1054
|
-
|
|
1055
|
-
|
|
1056
|
-
|
|
1057
|
-
|
|
1058
|
-
|
|
1059
|
-
|
|
1060
|
-
|
|
1061
|
-
|
|
1062
|
-
|
|
1063
|
-
|
|
1064
|
-
|
|
1065
|
-
|
|
1066
|
-
|
|
1067
|
-
|
|
1068
|
-
|
|
1069
|
-
|
|
1070
|
-
|
|
1071
|
-
|
|
1072
|
-
|
|
1073
|
-
|
|
1074
|
-
|
|
1075
|
-
|
|
1076
|
-
|
|
1077
|
-
|
|
1078
|
-
|
|
1079
|
-
|
|
1080
|
-
|
|
1081
|
-
|
|
1082
|
-
|
|
1083
|
-
|
|
1084
|
-
|
|
1085
|
-
|
|
1086
|
-
|
|
1087
|
-
|
|
1088
|
-
|
|
1089
|
-
|
|
1090
|
-
|
|
1091
|
-
|
|
1092
|
-
|
|
1093
|
-
|
|
1094
|
-
|
|
1095
|
-
|
|
1096
|
-
|
|
1097
|
-
|
|
1098
|
-
|
|
1099
|
-
|
|
1100
|
-
|
|
1101
|
-
|
|
1102
|
-
|
|
1103
|
-
|
|
1104
|
-
|
|
1105
|
-
|
|
1106
|
-
|
|
1107
|
-
|
|
1108
|
-
|
|
1109
|
-
|
|
1110
|
-
|
|
1111
|
-
|
|
1112
|
-
|
|
1113
|
-
|
|
1114
|
-
|
|
1115
|
-
|
|
1116
|
-
|
|
1117
|
-
|
|
1118
|
-
|
|
1119
|
-
|
|
1120
|
-
|
|
1121
|
-
|
|
1122
|
-
|
|
1123
|
-
|
|
1124
|
-
|
|
1125
|
-
|
|
1126
|
-
|
|
1127
|
-
|
|
1128
|
-
|
|
1129
|
-
|
|
1130
|
-
|
|
1131
|
-
|
|
1132
|
-
|
|
1133
|
-
|
|
1134
|
-
|
|
1135
|
-
|
|
1136
|
-
|
|
1137
|
-
|
|
1138
|
-
|
|
1139
|
-
|
|
1140
|
-
|
|
1141
|
-
|
|
1142
|
-
|
|
1143
|
-
|
|
1144
|
-
|
|
1145
|
-
|
|
1146
|
-
|
|
1147
|
-
|
|
1148
|
-
|
|
1149
|
-
|
|
1150
|
-
|
|
1151
|
-
|
|
1152
|
-
|
|
1153
|
-
|
|
1154
|
-
|
|
1155
|
-
|
|
1156
|
-
|
|
1157
|
-
|
|
1158
|
-
|
|
1159
|
-
|
|
1160
|
-
|
|
1161
|
-
|
|
1162
|
-
|
|
1163
|
-
|
|
1164
|
-
|
|
1165
|
-
|
|
1166
|
-
|
|
1167
|
-
|
|
1168
|
-
|
|
1169
|
-
|
|
1170
|
-
|
|
1171
|
-
|
|
1172
|
-
|
|
1173
|
-
|
|
1174
|
-
|
|
1175
|
-
|
|
1176
|
-
|
|
1177
|
-
|
|
1178
|
-
|
|
1179
|
-
|
|
1180
|
-
|
|
1181
|
-
|
|
969
|
+
// server.registerTool(
|
|
970
|
+
// 'generate-video-outlines',
|
|
971
|
+
// {
|
|
972
|
+
// title: 'Generate Video Outlines',
|
|
973
|
+
// description: `根据用户描述生成短视频的大纲;执行本工具会自动创建两个文件,一个是storyboard.json,一个是outline_sheet.png,前者是分镜设置,后者是分镜宫格图,其中outline_sheet.png生成于materials目录下,storyboard.json生成于materials的上级目录。`,
|
|
974
|
+
// inputSchema: {
|
|
975
|
+
// prompt: z
|
|
976
|
+
// .string()
|
|
977
|
+
// .describe(
|
|
978
|
+
// '⚠️你只须如实转述用户需求中有关分镜大纲的内容以及分镜数量和横竖屏设置,忽略其他无关内容,无需自行分析整理或加工,本工具会智能创作;'
|
|
979
|
+
// ),
|
|
980
|
+
// voiceType: z
|
|
981
|
+
// .enum(['slient', 'voiceover', 'dialogue'])
|
|
982
|
+
// .optional()
|
|
983
|
+
// .default('voiceover')
|
|
984
|
+
// .describe(
|
|
985
|
+
// `语音类型,枚举:
|
|
986
|
+
// - slient:视频只有BGM和音效,无任何语音
|
|
987
|
+
// - voiceover:视频只有画外音,无任何对话
|
|
988
|
+
// - dialogue:视频既有画外音,也有对话`
|
|
989
|
+
// ),
|
|
990
|
+
// language: z.string().optional().default('中文').describe('语言'),
|
|
991
|
+
// referenceImages: z.array(
|
|
992
|
+
// z.object({
|
|
993
|
+
// name: z.string().describe('图片名称,人物名、图片背景名或物品名'),
|
|
994
|
+
// fileName: z.string().describe('Reference image file name'),
|
|
995
|
+
// })
|
|
996
|
+
// ),
|
|
997
|
+
// orientation: z
|
|
998
|
+
// .enum(['portrait', 'landscape'])
|
|
999
|
+
// .optional()
|
|
1000
|
+
// .default('portrait')
|
|
1001
|
+
// .describe('视频方向,枚举:竖屏(portrait)、横屏(landscape)'),
|
|
1002
|
+
// model: z
|
|
1003
|
+
// .enum([
|
|
1004
|
+
// 'seedance-1.5-pro',
|
|
1005
|
+
// 'vidu',
|
|
1006
|
+
// 'viduq2',
|
|
1007
|
+
// 'vidu-turbo',
|
|
1008
|
+
// 'vidu-pro',
|
|
1009
|
+
// 'kling',
|
|
1010
|
+
// 'wan-flash',
|
|
1011
|
+
// ])
|
|
1012
|
+
// .default('seedance-1.5-pro')
|
|
1013
|
+
// .describe(
|
|
1014
|
+
// '除非用户主动提出使用其他模型,否则一律用seedance-1.5-pro模型'
|
|
1015
|
+
// ),
|
|
1016
|
+
// },
|
|
1017
|
+
// },
|
|
1018
|
+
// async (
|
|
1019
|
+
// { prompt, voiceType, language, referenceImages, orientation, model },
|
|
1020
|
+
// context
|
|
1021
|
+
// ) => {
|
|
1022
|
+
// try {
|
|
1023
|
+
// // 验证session状态
|
|
1024
|
+
// const currentSession = await validateSession('generate-video-outlines');
|
|
1025
|
+
// let progress = 0;
|
|
1026
|
+
// const ai = currentSession.ai;
|
|
1027
|
+
// let images;
|
|
1028
|
+
// if (referenceImages) {
|
|
1029
|
+
// images = await Promise.all(
|
|
1030
|
+
// referenceImages.map(
|
|
1031
|
+
// async image =>
|
|
1032
|
+
// await getMaterialUri(currentSession, image.fileName, {
|
|
1033
|
+
// fileSizeLimit: 10,
|
|
1034
|
+
// })
|
|
1035
|
+
// )
|
|
1036
|
+
// );
|
|
1037
|
+
// }
|
|
1038
|
+
// const res = await ai.generateShortVideoOutlines({
|
|
1039
|
+
// prompt,
|
|
1040
|
+
// voiceType,
|
|
1041
|
+
// language,
|
|
1042
|
+
// images,
|
|
1043
|
+
// videoModel: model as any,
|
|
1044
|
+
// aspectRatio: orientation === 'portrait' ? '9:16' : '16:9',
|
|
1045
|
+
// onProgress: async metaData => {
|
|
1046
|
+
// try {
|
|
1047
|
+
// await sendProgress(
|
|
1048
|
+
// context,
|
|
1049
|
+
// ++progress,
|
|
1050
|
+
// undefined,
|
|
1051
|
+
// JSON.stringify(metaData)
|
|
1052
|
+
// );
|
|
1053
|
+
// } catch (progressError) {
|
|
1054
|
+
// console.warn('Failed to send progress update:', progressError);
|
|
1055
|
+
// }
|
|
1056
|
+
// },
|
|
1057
|
+
// });
|
|
1058
|
+
// if (!res) {
|
|
1059
|
+
// throw new Error(
|
|
1060
|
+
// 'Failed to generate short video outlines: no response from AI service'
|
|
1061
|
+
// );
|
|
1062
|
+
// }
|
|
1063
|
+
// if (res.taskUrl) {
|
|
1064
|
+
// return {
|
|
1065
|
+
// content: [
|
|
1066
|
+
// {
|
|
1067
|
+
// type: 'text' as const,
|
|
1068
|
+
// text: JSON.stringify({
|
|
1069
|
+
// success: true,
|
|
1070
|
+
// message:
|
|
1071
|
+
// '该任务正在运行中,它是异步任务,且执行时间较长,你应立即调用工具 wait-for-task-finish (saveFileName=outline_sheet.png) 来等待任务结束,如 wait-for-task-finish 工具调用超时,你应立即再次重新调用直到任务结束。',
|
|
1072
|
+
// taskUrl: res.taskUrl,
|
|
1073
|
+
// }),
|
|
1074
|
+
// },
|
|
1075
|
+
// ],
|
|
1076
|
+
// };
|
|
1077
|
+
// } else if (res.url) {
|
|
1078
|
+
// const url = res.url;
|
|
1079
|
+
// const { savedTo } = await saveMaterial(
|
|
1080
|
+
// currentSession,
|
|
1081
|
+
// url,
|
|
1082
|
+
// 'outline_sheet.png'
|
|
1083
|
+
// );
|
|
1084
|
+
// const {
|
|
1085
|
+
// scenes,
|
|
1086
|
+
// video_type,
|
|
1087
|
+
// voice_type,
|
|
1088
|
+
// voiceover_tone,
|
|
1089
|
+
// voice_id,
|
|
1090
|
+
// bgm_prompt,
|
|
1091
|
+
// video_model,
|
|
1092
|
+
// } = res.data || {};
|
|
1093
|
+
// const seed = getRandomSeed();
|
|
1094
|
+
// const storyboard = {
|
|
1095
|
+
// orientation,
|
|
1096
|
+
// video_type,
|
|
1097
|
+
// outline_sheet: savedTo,
|
|
1098
|
+
// bgm_prompt,
|
|
1099
|
+
// voice_type,
|
|
1100
|
+
// scenes: scenes.map((scene: any) => {
|
|
1101
|
+
// return {
|
|
1102
|
+
// ...scene,
|
|
1103
|
+
// narration_tone: voiceover_tone,
|
|
1104
|
+
// narration_voice_id: voice_id,
|
|
1105
|
+
// use_video_model: video_model,
|
|
1106
|
+
// seed,
|
|
1107
|
+
// };
|
|
1108
|
+
// }),
|
|
1109
|
+
// };
|
|
1110
|
+
// const saveLocalPath = resolve(projectLocalDir, 'storyboard.json');
|
|
1111
|
+
// await writeFile(saveLocalPath, JSON.stringify(storyboard, null, 2));
|
|
1112
|
+
// return {
|
|
1113
|
+
// content: [
|
|
1114
|
+
// {
|
|
1115
|
+
// type: 'text' as const,
|
|
1116
|
+
// text: JSON.stringify({
|
|
1117
|
+
// success: true,
|
|
1118
|
+
// message: '视频大纲生成成功',
|
|
1119
|
+
// storyboard,
|
|
1120
|
+
// }),
|
|
1121
|
+
// },
|
|
1122
|
+
// ],
|
|
1123
|
+
// };
|
|
1124
|
+
// }
|
|
1125
|
+
// return {
|
|
1126
|
+
// content: [
|
|
1127
|
+
// {
|
|
1128
|
+
// type: 'text' as const,
|
|
1129
|
+
// text: JSON.stringify({
|
|
1130
|
+
// success: false,
|
|
1131
|
+
// error: 'No image URL returned from AI service',
|
|
1132
|
+
// response: res,
|
|
1133
|
+
// timestamp: new Date().toISOString(),
|
|
1134
|
+
// }),
|
|
1135
|
+
// },
|
|
1136
|
+
// ],
|
|
1137
|
+
// };
|
|
1138
|
+
// } catch (error) {
|
|
1139
|
+
// return createErrorResponse(error, 'generate-video-outlines');
|
|
1140
|
+
// }
|
|
1141
|
+
// }
|
|
1142
|
+
// );
|
|
1182
1143
|
server.registerTool('generate-music', {
|
|
1183
1144
|
title: '创作音乐(Music)',
|
|
1184
1145
|
description: '生成音乐,包括 BGM 和 歌曲',
|
|
@@ -1381,7 +1342,7 @@ server.registerTool('text-to-speech', {
|
|
|
1381
1342
|
});
|
|
1382
1343
|
server.registerTool('generate-video', {
|
|
1383
1344
|
title: 'Generate Video',
|
|
1384
|
-
description:
|
|
1345
|
+
description: `全能参考视频创作工具`,
|
|
1385
1346
|
inputSchema: {
|
|
1386
1347
|
prompt: zod_1.z
|
|
1387
1348
|
.string()
|
|
@@ -1518,76 +1479,17 @@ server.registerTool('generate-video', {
|
|
|
1518
1479
|
duration > 16) {
|
|
1519
1480
|
return createErrorResponse('Video duration must be no more than 16 seconds except for avatar and mv video', 'generate-video');
|
|
1520
1481
|
}
|
|
1521
|
-
const storyBoardFile = '
|
|
1482
|
+
const storyBoardFile = 'scene-bible.md';
|
|
1522
1483
|
// 校验 prompt 与 storyboard.json 中分镜设定的一致性
|
|
1523
1484
|
if (sceneIndex) {
|
|
1524
1485
|
try {
|
|
1525
|
-
const storyBoardPath = (0, node_path_1.resolve)(projectLocalDir, storyBoardFile);
|
|
1526
|
-
if ((0, node_fs_1.existsSync)(storyBoardPath)) {
|
|
1527
|
-
|
|
1528
|
-
// 检查 storyBoard JSON 语法合法性
|
|
1529
|
-
let storyBoard;
|
|
1530
|
-
try {
|
|
1531
|
-
storyBoard = JSON.parse(storyBoardContent);
|
|
1532
|
-
}
|
|
1533
|
-
catch (jsonError) {
|
|
1534
|
-
return createErrorResponse(`The storyboard file ${storyBoardFile} contains invalid JSON. Please fix it and try again. Details: ${jsonError instanceof Error ? jsonError.message : String(jsonError)}`, 'generate-video');
|
|
1535
|
-
}
|
|
1536
|
-
if (storyBoard.scenes && Array.isArray(storyBoard.scenes)) {
|
|
1537
|
-
const scene = storyBoard.scenes[sceneIndex - 1]; // sceneIndex 从1开始,数组从0开始
|
|
1538
|
-
if (scene) {
|
|
1539
|
-
const videoPrompt = scene.video_prompt;
|
|
1540
|
-
if (videoPrompt && prompt.trim() !== videoPrompt?.trim()) {
|
|
1541
|
-
return createErrorResponse('The video prompt must exactly match the storyboard setting, including spaces, newlines, punctuation, and special characters. Ensure the provided prompt matches storyboard.video_prompt exactly.', 'generate-video');
|
|
1542
|
-
}
|
|
1543
|
-
if (scene.video_duration != null &&
|
|
1544
|
-
duration !== scene.video_duration) {
|
|
1545
|
-
return createErrorResponse(`Video duration must exactly match the storyboard setting. The storyboard duration is ${scene.video_duration} seconds${scene.video_duration === 0 ? '. 0 means the duration will be determined automatically from the prompt.' : ''}`, 'generate-video');
|
|
1546
|
-
}
|
|
1547
|
-
const firstFrame = images.find(img => img.type === 'first_frame');
|
|
1548
|
-
if (!firstFrame) {
|
|
1549
|
-
return createErrorResponse(`This scene must use image-to-video and include only the first frame image (type: first_frame, fileName: sc${sceneIndex.toString().padStart(2, '0')}_first_frame.png). Please update the generate-video parameters and try again.`, 'generate-video');
|
|
1550
|
-
}
|
|
1551
|
-
if (storyBoard.voice_type &&
|
|
1552
|
-
storyBoard.voice_type !== 'slient') {
|
|
1553
|
-
if (silent) {
|
|
1554
|
-
return createErrorResponse('Scenes with dialogue or voiceover cannot be muted. Set silent=false and try again.', 'generate-video');
|
|
1555
|
-
}
|
|
1556
|
-
}
|
|
1557
|
-
if (storyBoard.narration && !narration) {
|
|
1558
|
-
return createErrorResponse('This storyboard requires narration. Please provide the narration parameter.', 'generate-video');
|
|
1559
|
-
}
|
|
1560
|
-
// 检查 storyBoard.orientation 与 size 参数的一致性
|
|
1561
|
-
if (storyBoard.orientation && aspectRatio) {
|
|
1562
|
-
const isLandscapeSize = aspectRatio === '16:9';
|
|
1563
|
-
const isPortraitSize = aspectRatio === '9:16';
|
|
1564
|
-
if (storyBoard.orientation === 'landscape' &&
|
|
1565
|
-
!isLandscapeSize) {
|
|
1566
|
-
return createErrorResponse('Storyboard orientation is landscape (orientation: landscape), but the video aspect ratio is portrait. Use a landscape ratio such as 16:9 (e.g. 1280x720).', 'generate-video');
|
|
1567
|
-
}
|
|
1568
|
-
if (storyBoard.orientation === 'portrait' &&
|
|
1569
|
-
!isPortraitSize) {
|
|
1570
|
-
return createErrorResponse('Storyboard orientation is portrait (orientation: portrait), but the video aspect ratio is landscape. Use a portrait ratio such as 9:16 (e.g. 720x1280).', 'generate-video');
|
|
1571
|
-
}
|
|
1572
|
-
}
|
|
1573
|
-
// 检查 use_video_model 与 type 参数的一致性
|
|
1574
|
-
if (scene.use_video_model &&
|
|
1575
|
-
model &&
|
|
1576
|
-
scene.use_video_model !== model) {
|
|
1577
|
-
return createErrorResponse(`The recommended video model (${scene.use_video_model}) does not match the provided model (${model}). Ensure scene.use_video_model and the model parameter are the same.`, 'generate-video');
|
|
1578
|
-
}
|
|
1579
|
-
}
|
|
1580
|
-
else {
|
|
1581
|
-
console.warn(`Scene index ${sceneIndex} not found in storyboard.json`);
|
|
1582
|
-
}
|
|
1583
|
-
}
|
|
1584
|
-
}
|
|
1585
|
-
else {
|
|
1586
|
-
console.warn(`Story board file not found: ${storyBoardPath}`);
|
|
1486
|
+
const storyBoardPath = (0, node_path_1.resolve)(projectLocalDir, 'materials', storyBoardFile);
|
|
1487
|
+
if (!(0, node_fs_1.existsSync)(storyBoardPath)) {
|
|
1488
|
+
return createErrorResponse(`The storyboard file ${storyBoardFile} is missing or not in materials path. Please create it first.`, 'generate-image');
|
|
1587
1489
|
}
|
|
1588
1490
|
}
|
|
1589
1491
|
catch (error) {
|
|
1590
|
-
console.error('Failed to validate prompt with
|
|
1492
|
+
console.error('Failed to validate prompt with storyboard:', error);
|
|
1591
1493
|
// 如果读取或解析 storyboard.json 失败,继续执行但记录警告
|
|
1592
1494
|
}
|
|
1593
1495
|
}
|
|
@@ -1918,7 +1820,7 @@ server.registerTool('audio-video-sync', {
|
|
|
1918
1820
|
audioVolume: zod_1.z
|
|
1919
1821
|
.number()
|
|
1920
1822
|
.optional()
|
|
1921
|
-
.default(0.
|
|
1823
|
+
.default(0.316) // -10db
|
|
1922
1824
|
.describe('0.0 to 2.0.'),
|
|
1923
1825
|
videoAudioVolume: zod_1.z
|
|
1924
1826
|
.number()
|