@remixmate/cli 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +166 -0
- package/dist/argv.d.ts +15 -0
- package/dist/argv.js +51 -0
- package/dist/cli.d.ts +13 -0
- package/dist/cli.js +83 -0
- package/dist/handlers/gen-image.d.ts +13 -0
- package/dist/handlers/gen-image.js +161 -0
- package/dist/handlers/gen-voice.d.ts +16 -0
- package/dist/handlers/gen-voice.js +110 -0
- package/dist/handlers/index.d.ts +13 -0
- package/dist/handlers/index.js +14 -0
- package/dist/handlers/template-bind.d.ts +15 -0
- package/dist/handlers/template-bind.js +149 -0
- package/dist/http.d.ts +52 -0
- package/dist/http.js +110 -0
- package/dist/index.d.ts +9 -0
- package/dist/index.js +8 -0
- package/dist/manifest.json +260 -0
- package/dist/progress.d.ts +23 -0
- package/dist/progress.js +15 -0
- package/dist/registry.d.ts +33 -0
- package/dist/registry.js +72 -0
- package/dist/runner.d.ts +17 -0
- package/dist/runner.js +63 -0
- package/package.json +54 -0
- package/skills/export-jianying/SKILL.md +251 -0
- package/skills/export-jianying/scripts/gen_jianying_draft.py +1099 -0
- package/skills/export-jianying/skill.json +24 -0
- package/skills/export-jianying/version.json +7 -0
- package/skills/gen-digital-human/SKILL.md +211 -0
- package/skills/gen-digital-human/scripts/gen_digital_human_video.py +574 -0
- package/skills/gen-digital-human/skill.json +23 -0
- package/skills/gen-digital-human/version.json +7 -0
- package/skills/gen-image/SKILL.md +130 -0
- package/skills/gen-image/skill.json +22 -0
- package/skills/gen-image/version.json +7 -0
- package/skills/gen-script/SKILL.md +283 -0
- package/skills/gen-script/scripts/gen_script.py +1208 -0
- package/skills/gen-script/skill.json +41 -0
- package/skills/gen-script/version.json +7 -0
- package/skills/gen-video/SKILL.md +170 -0
- package/skills/gen-video/scripts/gen_video.py +591 -0
- package/skills/gen-video/skill.json +24 -0
- package/skills/gen-video/version.json +7 -0
- package/skills/gen-voice/SKILL.md +104 -0
- package/skills/gen-voice/skill.json +21 -0
- package/skills/gen-voice/version.json +7 -0
- package/skills/prepare-video-assets/SKILL.md +192 -0
- package/skills/prepare-video-assets/scripts/prepare_video_assets.py +57 -0
- package/skills/prepare-video-assets/skill.json +23 -0
- package/skills/prepare-video-assets/version.json +7 -0
- package/skills/render-video/SKILL.md +187 -0
- package/skills/render-video/scripts/_chrome_vendor.py +305 -0
- package/skills/render-video/scripts/_video_probe.py +174 -0
- package/skills/render-video/scripts/_vod_polling.py +86 -0
- package/skills/render-video/scripts/remote_renderer_client.py +236 -0
- package/skills/render-video/scripts/render_video.py +2288 -0
- package/skills/render-video/scripts/upload_video.py +193 -0
- package/skills/render-video/skill.json +21 -0
- package/skills/render-video/version.json +7 -0
- package/skills/template-bind/README.md +63 -0
- package/skills/template-bind/SKILL.md +234 -0
- package/skills/template-bind/scripts/check_contracts.py +196 -0
- package/skills/template-bind/scripts/match_template.py +215 -0
- package/skills/template-bind/scripts/registry_loader.py +379 -0
- package/skills/template-bind/scripts/render_job_client.py +162 -0
- package/skills/template-bind/scripts/sync_registry.py +44 -0
- package/skills/template-bind/scripts/template_paths.py +72 -0
- package/skills/template-bind/skill.json +20 -0
- package/skills/template-bind/version.json +7 -0
- package/skills/template-bind/video_dsl/README.md +97 -0
- package/skills/template-bind/video_dsl/runtime/__init__.py +11 -0
- package/skills/template-bind/video_dsl/runtime/dsl_validator.py +495 -0
- package/skills/template-bind/video_dsl/runtime/prompt_enhancer.py +274 -0
- package/skills/template-bind/video_dsl/runtime/template_binder.py +87 -0
- package/skills/template-bind/video_dsl/runtime/timeline_compiler.py +312 -0
- package/skills/template-bind/video_dsl/schema/render-plan-v1alpha1.json +193 -0
- package/skills/template-bind/video_dsl/schema/template-binding-v1alpha1.json +122 -0
- package/skills/template-bind/video_dsl/schema/template-definition-v1alpha1.json +247 -0
- package/skills/template-bind/video_dsl/schema/video-dsl-v1alpha1.json +378 -0
- package/skills/video-parser/SKILL.md +238 -0
- package/skills/video-parser/scripts/analyze_video.py +1038 -0
- package/skills/video-parser/scripts/deconstruct_video.py +602 -0
- package/skills/video-parser/scripts/parse_via_render.py +148 -0
- package/skills/video-parser/skill.json +21 -0
- package/skills/video-parser/version.json +7 -0
- package/skills/web-capture/SKILL.md +183 -0
- package/skills/web-capture/scripts/web_capture.py +461 -0
- package/skills/web-capture/skill.json +29 -0
- package/skills/web-capture/version.json +7 -0
|
@@ -0,0 +1,193 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""
|
|
3
|
+
视频上传工具 - 将本地 MP4 文件通过 multipart/form-data 上传到 ab-api,
|
|
4
|
+
获取可用 URL(直传阿里云 OSS,同步返回)。
|
|
5
|
+
|
|
6
|
+
默认行为:
|
|
7
|
+
读取本地文件,以 multipart/form-data 形式 POST 到
|
|
8
|
+
`<MM_API_BASE_URL>/file/uploadMediaFile`,解析响应 JSON,输出 fileUrl。
|
|
9
|
+
|
|
10
|
+
用法:
|
|
11
|
+
python3 upload_video.py --file output/video.mp4
|
|
12
|
+
python3 upload_video.py --file output/video.mp4 --title "我的视频"
|
|
13
|
+
python3 upload_video.py --file output/video.mp4 --json-output
|
|
14
|
+
|
|
15
|
+
环境变量:
|
|
16
|
+
MM_API_BASE_URL - API 根地址(默认 MM_API_BASE_URL 环境变量或 http://localhost:3001/api)
|
|
17
|
+
|
|
18
|
+
注意:
|
|
19
|
+
- `uploadMediaFile` 接口视频大小限制 ≤ 50MB,音频 ≤ 15MB
|
|
20
|
+
- 进度信息输出到 stderr,最终 URL 输出到 stdout(最后一行)
|
|
21
|
+
"""
|
|
22
|
+
|
|
23
|
+
import argparse
|
|
24
|
+
import http.client
|
|
25
|
+
import json
|
|
26
|
+
import mimetypes
|
|
27
|
+
import os
|
|
28
|
+
import sys
|
|
29
|
+
import urllib.error
|
|
30
|
+
import urllib.parse
|
|
31
|
+
import urllib.request
|
|
32
|
+
import uuid
|
|
33
|
+
|
|
34
|
+
# DEFAULT_API_BASE_URL = "http://localhost:3001/api"
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
def _guess_content_type(file_path: str) -> str:
|
|
38
|
+
ctype, _ = mimetypes.guess_type(file_path)
|
|
39
|
+
return ctype or "application/octet-stream"
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
def _build_multipart_body(file_path: str, title: str = None) -> (bytes, str):
|
|
43
|
+
"""
|
|
44
|
+
手动构建 multipart/form-data body。
|
|
45
|
+
|
|
46
|
+
返回 (body_bytes, content_type_header)。
|
|
47
|
+
"""
|
|
48
|
+
boundary = "----MediaMakerBoundary" + uuid.uuid4().hex
|
|
49
|
+
crlf = b"\r\n"
|
|
50
|
+
parts = []
|
|
51
|
+
|
|
52
|
+
# title 字段(可选)
|
|
53
|
+
if title:
|
|
54
|
+
parts.append(("--" + boundary).encode("utf-8"))
|
|
55
|
+
parts.append(b'Content-Disposition: form-data; name="title"')
|
|
56
|
+
parts.append(b"")
|
|
57
|
+
parts.append(title.encode("utf-8"))
|
|
58
|
+
|
|
59
|
+
# file 字段
|
|
60
|
+
file_name = os.path.basename(file_path)
|
|
61
|
+
content_type = _guess_content_type(file_path)
|
|
62
|
+
with open(file_path, "rb") as f:
|
|
63
|
+
file_data = f.read()
|
|
64
|
+
|
|
65
|
+
parts.append(("--" + boundary).encode("utf-8"))
|
|
66
|
+
parts.append(
|
|
67
|
+
(
|
|
68
|
+
'Content-Disposition: form-data; name="file"; filename="%s"'
|
|
69
|
+
% file_name
|
|
70
|
+
).encode("utf-8")
|
|
71
|
+
)
|
|
72
|
+
parts.append(("Content-Type: %s" % content_type).encode("utf-8"))
|
|
73
|
+
parts.append(b"")
|
|
74
|
+
parts.append(file_data)
|
|
75
|
+
|
|
76
|
+
# 结尾 boundary
|
|
77
|
+
parts.append(("--" + boundary + "--").encode("utf-8"))
|
|
78
|
+
parts.append(b"")
|
|
79
|
+
|
|
80
|
+
body = crlf.join(parts)
|
|
81
|
+
ct_header = "multipart/form-data; boundary=%s" % boundary
|
|
82
|
+
return body, ct_header
|
|
83
|
+
|
|
84
|
+
|
|
85
|
+
def upload_media_file(
|
|
86
|
+
file_path: str,
|
|
87
|
+
api_base_url: str = None,
|
|
88
|
+
title: str = None,
|
|
89
|
+
timeout: int = 600,
|
|
90
|
+
) -> dict:
|
|
91
|
+
"""
|
|
92
|
+
将本地媒体文件以 multipart/form-data 形式上传到 ab-api。
|
|
93
|
+
|
|
94
|
+
返回服务端 data 字段,通常形如 {"fileUrl": "https://..."}。
|
|
95
|
+
上传失败会抛出异常。
|
|
96
|
+
"""
|
|
97
|
+
if not os.path.exists(file_path):
|
|
98
|
+
raise FileNotFoundError(f"file not found: {file_path}")
|
|
99
|
+
|
|
100
|
+
api_base_url = (api_base_url or os.environ.get("MM_API_BASE_URL") or DEFAULT_API_BASE_URL).rstrip("/")
|
|
101
|
+
url = f"{api_base_url}/file/uploadMediaFile"
|
|
102
|
+
|
|
103
|
+
file_size = os.path.getsize(file_path)
|
|
104
|
+
print(f"📤 Uploading: {file_path} ({file_size / 1024 / 1024:.2f} MB)", file=sys.stderr)
|
|
105
|
+
print(f" target: {url}", file=sys.stderr)
|
|
106
|
+
|
|
107
|
+
body, content_type = _build_multipart_body(file_path, title=title)
|
|
108
|
+
headers = {
|
|
109
|
+
"Content-Type": content_type,
|
|
110
|
+
"Content-Length": str(len(body)),
|
|
111
|
+
}
|
|
112
|
+
|
|
113
|
+
# Use http.client directly to avoid urllib RemoteDisconnected issues
|
|
114
|
+
# with servers that return 'connection: close'
|
|
115
|
+
parsed = urllib.parse.urlparse(url)
|
|
116
|
+
host = parsed.hostname
|
|
117
|
+
port = parsed.port
|
|
118
|
+
path = parsed.path
|
|
119
|
+
|
|
120
|
+
if parsed.scheme == "https":
|
|
121
|
+
import ssl
|
|
122
|
+
conn = http.client.HTTPSConnection(host, port, timeout=timeout,
|
|
123
|
+
context=ssl.create_default_context())
|
|
124
|
+
else:
|
|
125
|
+
conn = http.client.HTTPConnection(host, port, timeout=timeout)
|
|
126
|
+
|
|
127
|
+
try:
|
|
128
|
+
conn.request("POST", path, body=body, headers=headers)
|
|
129
|
+
resp = conn.getresponse()
|
|
130
|
+
resp_body = resp.read().decode("utf-8")
|
|
131
|
+
|
|
132
|
+
if resp.status >= 400:
|
|
133
|
+
raise RuntimeError(f"upload failed (HTTP {resp.status}): {resp_body}")
|
|
134
|
+
except http.client.RemoteDisconnected as e:
|
|
135
|
+
raise RuntimeError(f"server disconnected: {e}") from e
|
|
136
|
+
except (OSError, Exception) as e:
|
|
137
|
+
if isinstance(e, RuntimeError):
|
|
138
|
+
raise
|
|
139
|
+
raise RuntimeError(f"network error: {e}") from e
|
|
140
|
+
finally:
|
|
141
|
+
conn.close()
|
|
142
|
+
|
|
143
|
+
try:
|
|
144
|
+
result = json.loads(resp_body)
|
|
145
|
+
except json.JSONDecodeError as e:
|
|
146
|
+
raise RuntimeError(f"failed to parse response: {resp_body}") from e
|
|
147
|
+
|
|
148
|
+
# ab-api shared response: { code: 0, data: {...}, msg: "..." }
|
|
149
|
+
if isinstance(result, dict) and result.get("code") not in (0, 200):
|
|
150
|
+
raise RuntimeError(f"server returned an error: code={result.get('code')}, msg={result.get('msg')}")
|
|
151
|
+
|
|
152
|
+
data = result.get("data") if isinstance(result, dict) else None
|
|
153
|
+
if not isinstance(data, dict) or not data.get("fileUrl"):
|
|
154
|
+
raise RuntimeError(f"response is missing fileUrl: {resp_body}")
|
|
155
|
+
|
|
156
|
+
print(f"✅ uploaded: {data['fileUrl']}", file=sys.stderr)
|
|
157
|
+
return data
|
|
158
|
+
|
|
159
|
+
|
|
160
|
+
def main():
|
|
161
|
+
parser = argparse.ArgumentParser(description="Upload a media file to ab-api / Alibaba Cloud OSS")
|
|
162
|
+
parser.add_argument("--file", required=True, help="Local file path")
|
|
163
|
+
parser.add_argument("--title", default=None, help="File title (optional, defaults to filename)")
|
|
164
|
+
parser.add_argument(
|
|
165
|
+
"--api-base-url",
|
|
166
|
+
default=None,
|
|
167
|
+
help="API root URL (defaults to MM_API_BASE_URL env var or http://localhost:3001/api)",
|
|
168
|
+
)
|
|
169
|
+
parser.add_argument("--timeout", type=int, default=600, help="Upload timeout in seconds (default 600)")
|
|
170
|
+
parser.add_argument("--json-output", action="store_true", help="Emit the full response as JSON on stdout")
|
|
171
|
+
|
|
172
|
+
args = parser.parse_args()
|
|
173
|
+
|
|
174
|
+
try:
|
|
175
|
+
data = upload_media_file(
|
|
176
|
+
file_path=args.file,
|
|
177
|
+
api_base_url=args.api_base_url,
|
|
178
|
+
title=args.title,
|
|
179
|
+
timeout=args.timeout,
|
|
180
|
+
)
|
|
181
|
+
except Exception as e:
|
|
182
|
+
print(f"❌ {e}", file=sys.stderr)
|
|
183
|
+
sys.exit(1)
|
|
184
|
+
|
|
185
|
+
if args.json_output:
|
|
186
|
+
print(json.dumps(data, ensure_ascii=False))
|
|
187
|
+
else:
|
|
188
|
+
# 最终 URL 输出到 stdout(最后一行),方便其他脚本捕获
|
|
189
|
+
print(data["fileUrl"])
|
|
190
|
+
|
|
191
|
+
|
|
192
|
+
if __name__ == "__main__":
|
|
193
|
+
main()
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "render-video",
|
|
3
|
+
"toolName": "render_video",
|
|
4
|
+
"tier": "orchestration",
|
|
5
|
+
"title": "Remotion Video Renderer",
|
|
6
|
+
"description": "Loads a persisted RenderPlan by job_id and drives the Remotion engine to produce the final video. Assets must already be generated via prepare_video_assets — this skill never resolves or regenerates assets.",
|
|
7
|
+
"envVars": ["PRIV_TOKEN", "MM_API_BASE_URL", "MM_BACKEND_API_URL", "AGENT_NAME", "REMOTION_RENDER_API_URL", "REMOTION_RENDER_MODE", "REMOTION_OUTPUT_DIR", "ASSET_CACHE_DIR"],
|
|
8
|
+
"scriptPath": "scripts/render_video.py",
|
|
9
|
+
"parameters": {
|
|
10
|
+
"type": "object",
|
|
11
|
+
"properties": {
|
|
12
|
+
"job_id": { "type": "number", "minimum": 1, "description": "Render job id (positive integer, e.g. 8). This is the integer N parsed from prepare_video_assets's stdout line '📦 render job jobId: N' (also tolerated: legacy zh '📦 渲染任务 jobId: N'). Never pass 0, a placeholder string, or descriptive text." },
|
|
13
|
+
"save_job": { "type": "boolean", "description": "Persist the render Manifest back to the database under the same jobId (default true). Auto-degrades to false when PRIV_TOKEN is missing. Pass false to opt out explicitly." },
|
|
14
|
+
"upload_title": { "type": "string", "description": "Upload title for the resulting video file. Defaults to the local output filename." },
|
|
15
|
+
"no_upload": { "type": "boolean", "description": "Skip the upload step. Only effective for local rendering; remote rendering uploads server-side." },
|
|
16
|
+
"renderer": { "type": "string", "enum": ["local", "remote"], "description": "Render mode: local runs the Remotion CLI on this host; remote calls the standalone remotion-renderer service. Defaults to REMOTION_RENDER_MODE." },
|
|
17
|
+
"render_plan": { "type": "string", "description": "Existing RenderPlan file path. Local / single-user fallback used only when job_id is unavailable; prefer job_id." }
|
|
18
|
+
},
|
|
19
|
+
"required": ["job_id"]
|
|
20
|
+
}
|
|
21
|
+
}
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"skillName": "render-video",
|
|
3
|
+
"repoName": "agent-skill-media-maker",
|
|
4
|
+
"skillId": "473",
|
|
5
|
+
"version": "V15",
|
|
6
|
+
"skillDescription": "Final-render skill (Phase 3 of the two-phase video pipeline). Loads a persisted RenderPlan by job_id and drives Remotion to produce the final video. Assets must already be generated via prepare_video_assets."
|
|
7
|
+
}
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# template-bind
|
|
2
|
+
|
|
3
|
+
CLI 暴露面是「列出可用模板」(`--list-templates`)。把 Video DSL 场景绑到具体模板 / 生成 TemplateBinding 的 Python 逻辑由本 skill 提供为**库**(`scripts/match_template.py`),被 `render-video` / `prepare-video-assets` 通过 `sys.path` 内联 import,**没有独立 CLI**。
|
|
4
|
+
|
|
5
|
+
## 模板数据来源
|
|
6
|
+
|
|
7
|
+
模板元数据维护在独立仓库 **template-library**。本 skill 通过 `scripts/registry_loader.py` 按以下顺序解析:
|
|
8
|
+
|
|
9
|
+
| 优先级 | 来源 | 触发条件 | 适用场景 |
|
|
10
|
+
|---|---|---|---|
|
|
11
|
+
| 1 | `VIDEO_TEMPLATE_REGISTRY` 指向的本地 JSON 文件 | 环境变量非空 | CI / 离线 / pinned 调试 |
|
|
12
|
+
| 2 | `VIDEO_TEMPLATE_REGISTRY_URL`(ab-api 接口) | URL 设置 + `PRIV_TOKEN` 有效 | **生产** —— 多租户隔离由 ab-api 做 |
|
|
13
|
+
| 3 | monorepo `template-library/packages/metadata/registry.json` | 本地源码可见 | 本地 dev / 单仓部署 |
|
|
14
|
+
|
|
15
|
+
`VIDEO_TEMPLATE_REGISTRY_PREFER_LOCAL=1` 开启后,#3 抢在 #2 之前——本地源码改动立即生效,不被 HTTP 缓存覆盖。
|
|
16
|
+
|
|
17
|
+
ab-api 响应包装格式 `{code, msg, data}` 由 `registry_loader._fetch_http` 透明拆封;静态 JSON endpoint 也支持(直接返回 registry 文档)。
|
|
18
|
+
|
|
19
|
+
> 早期文档曾描述本 skill 通过 `@ab-templates/metadata` npm 包消费 registry —— 该路径已被 ab-api HTTP 取代,但保留 monorepo 文件路径作为兜底。
|
|
20
|
+
|
|
21
|
+
## 新增模板工作流
|
|
22
|
+
|
|
23
|
+
新增模板**不需要改 ab-skill / template-bind 代码**:
|
|
24
|
+
|
|
25
|
+
1. 在 template-library 仓库定义新模板(`template.json` + 组件代码)。
|
|
26
|
+
2. template-library CI 校验 schema + 契约。
|
|
27
|
+
3. 合并后 template-library 的发布流水线将新 registry 推到 ab-api。
|
|
28
|
+
4. ab-skill 端无需更新——下一次 `--list-templates` 即可看到新模板。
|
|
29
|
+
|
|
30
|
+
本地调试时可通过 `VIDEO_TEMPLATE_REGISTRY_PREFER_LOCAL=1` 让 template-library 的源码改动立即生效,不依赖发布周期。
|
|
31
|
+
|
|
32
|
+
## 契约一致性
|
|
33
|
+
|
|
34
|
+
template-library 仓库内置契约验证器,确保:
|
|
35
|
+
|
|
36
|
+
- 每个 `template.json` 中引用的 `compositionId` 都存在于 remotion-renderer 的 `manifest.ts`
|
|
37
|
+
- manifest 中的每个 compositionId 都被某个 `template.json` 引用
|
|
38
|
+
- 所有 `template.json` 通过 JSON Schema 校验
|
|
39
|
+
|
|
40
|
+
本地可运行 `check_contracts.py` 做跨仓库一致性检查(从 monorepo `<repo>/remotion-renderer/src/core/compositions/manifest.ts` 读 manifest):
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
python3 <SkillDir>/scripts/check_contracts.py
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
## 共享 Python 模块
|
|
47
|
+
|
|
48
|
+
`scripts/` 是 ab-skill 内**跨 skill 共享 Python 代码**的约定位置。当前住户:
|
|
49
|
+
|
|
50
|
+
| 模块 | 谁在用 | 作用 |
|
|
51
|
+
|---|---|---|
|
|
52
|
+
| `template_paths.py` | registry_loader / template_binder / gen-script / check_contracts | 统一计算 monorepo / template-library 路径,代替 4 个 caller 各自数 `..` |
|
|
53
|
+
| `registry_loader.py` | template-bind handler / match_template / render-video / gen-script / dsl_validator | 加载 registry(HTTP + cache + file) |
|
|
54
|
+
| `match_template.py` | render-video(`auto_bind_template`) | DSL → TemplateBinding 算法 |
|
|
55
|
+
| `render_job_client.py` | render-video / export-jianying | ab-api renderJob HTTP 客户端 |
|
|
56
|
+
| `check_contracts.py` | 维护脚本 | template-library ↔ remotion-renderer 契约校验 |
|
|
57
|
+
| `sync_registry.py` | 维护脚本 | 把 registry 摘要落盘供 LLM prompt 使用 |
|
|
58
|
+
|
|
59
|
+
sibling skill 通过 `sys.path.insert(0, "<…>/template-bind/scripts")` 然后直接 `import` 模块名,模式见 `render_video.py` 顶部的"Single sys.path setup"段。
|
|
60
|
+
|
|
61
|
+
## 详细使用说明
|
|
62
|
+
|
|
63
|
+
完整的 CLI 参数、Props 提取规则、多项卡片配音写法等操作细节请参阅 [SKILL.md](./SKILL.md)。
|
|
@@ -0,0 +1,234 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: template-bind
|
|
3
|
+
description: |
|
|
4
|
+
Video-template registry and binding skill. Stores every video-template definition and produces a TemplateBinding for a given template id.
|
|
5
|
+
|
|
6
|
+
Use this skill as soon as the user mentions any of these intents:
|
|
7
|
+
- View available templates / list every template
|
|
8
|
+
- Bind a template to a DSL (a template-id must be supplied)
|
|
9
|
+
|
|
10
|
+
Also use this skill whenever the agent has a Video DSL and needs to produce a TemplateBinding.
|
|
11
|
+
triggers:
|
|
12
|
+
- View available templates / list every template
|
|
13
|
+
- Bind a template to a DSL (a template-id must be supplied)
|
|
14
|
+
- Agent has a Video DSL and needs a TemplateBinding
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
# Video Template Registry & Binding Skill
|
|
18
|
+
|
|
19
|
+
Stores every video-template definition, loads a template by **template-id**, and produces a **TemplateBinding** for a **Video DSL**, mapping each scene to a template slot.
|
|
20
|
+
|
|
21
|
+
## Core concepts
|
|
22
|
+
|
|
23
|
+
- **Template Manifest**: template metadata describing supported aspect ratios, durations, styles, and scene patterns.
|
|
24
|
+
- **TemplateBinding**: the binding result — maps each DSL scene to a template slot + Remotion Composition.
|
|
25
|
+
- **Template registry**: `registry.json` shipped by the `@ab-templates/metadata` package (housed in the template-library repo).
|
|
26
|
+
|
|
27
|
+
## Template registry
|
|
28
|
+
|
|
29
|
+
Template metadata lives in the standalone **template-library** monorepo and ships via the `@ab-templates/metadata` package.
|
|
30
|
+
|
|
31
|
+
template-bind reads `template-library/packages/metadata/registry.json` to load every template definition (with the full slotMapping, compositions, etc.).
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
template-library/packages/
|
|
35
|
+
├── templates/src/
|
|
36
|
+
│ ├── html-slide/ # Knowledge-board template
|
|
37
|
+
│ │ ├── Composition.tsx
|
|
38
|
+
│ │ └── template.json
|
|
39
|
+
│ ├── image-slide/ # Basic image + narration template (includes the tech variant)
|
|
40
|
+
│ │ └── ...
|
|
41
|
+
│ ├── picture-book-en/ # English picture-book video template
|
|
42
|
+
│ │ └── ...
|
|
43
|
+
│ └── screen-walkthrough/ # Screen-recording walkthrough template
|
|
44
|
+
│ └── ...
|
|
45
|
+
└── metadata/
|
|
46
|
+
└── registry.json # Aggregated registry (every template.json merged)
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## Agent behavior: image URL handling
|
|
50
|
+
|
|
51
|
+
**DSL AssetRef entries must use HTTPS URLs. Local file paths are forbidden.**
|
|
52
|
+
|
|
53
|
+
HTTPS URLs serve two phases:
|
|
54
|
+
1. **Resolve phase** (`--resolve-only`): the agent previews already-generated images via the HTTPS URL and asks the user to confirm.
|
|
55
|
+
2. **Render phase** (`--render-plan`): `render_video.py` automatically downloads each image from its HTTPS URL into `output/public-assets/` before launching Remotion, then exposes it through `--public-dir` so Remotion loads everything locally with no external network.
|
|
56
|
+
|
|
57
|
+
- **Forbidden**: writing local paths like `./image_1.png` or `/tmp/image_1.png` into the DSL (the resolve phase cannot preview them and the script cannot reuse them).
|
|
58
|
+
- **Forbidden**: using WebFetch against internal image domains (blocked by security policy).
|
|
59
|
+
- **Allowed**: writing raw HTTPS URLs directly into the DSL AssetRef.
|
|
60
|
+
- **Allowed**: downloading an image locally with `curl` when you need to read the content, but the DSL still carries the original HTTPS URL.
|
|
61
|
+
|
|
62
|
+
## Authentication & environment
|
|
63
|
+
|
|
64
|
+
The skill itself only reads the registry; whether a token is needed depends on the configured source.
|
|
65
|
+
|
|
66
|
+
| Env var | Description | Default |
|
|
67
|
+
|---------|-------------|---------|
|
|
68
|
+
| `VIDEO_TEMPLATE_REGISTRY_URL` | ab-api endpoint returning the registry (production source of truth). | unset |
|
|
69
|
+
| `PRIV_TOKEN` | Sent as `X-Priv-Token` when hitting `VIDEO_TEMPLATE_REGISTRY_URL`. | unset |
|
|
70
|
+
| `VIDEO_TEMPLATE_REGISTRY_HTTP_METHOD` | `POST` (default) or `GET`. POST shape matches ab-api `{code,msg,data}`. | `POST` |
|
|
71
|
+
| `VIDEO_TEMPLATE_REGISTRY` | Explicit local file path — bypasses HTTP/fallback entirely (CI / pinned debug). | unset |
|
|
72
|
+
| `VIDEO_TEMPLATE_REGISTRY_PREFER_LOCAL` | When `1`, monorepo `template-library/packages/metadata/registry.json` wins over HTTP. Defaults to HTTP-first. | unset |
|
|
73
|
+
|
|
74
|
+
Resolution order (see `scripts/registry_loader.py` for the canonical implementation):
|
|
75
|
+
`VIDEO_TEMPLATE_REGISTRY` → (optional PREFER_LOCAL fast-path to monorepo registry) → `VIDEO_TEMPLATE_REGISTRY_URL` → monorepo `template-library/packages/metadata/registry.json` fallback.
|
|
76
|
+
|
|
77
|
+
## Steps
|
|
78
|
+
|
|
79
|
+
> This skill is now a built-in handler of the ab-skill CLI (`entry.type: builtin`). The agent tool name `template_bind` is still the only entry; locally reproduce via `ab-skill template-bind --list-templates`.
|
|
80
|
+
>
|
|
81
|
+
> The Python binding logic that maps DSL → TemplateBinding lives in `scripts/match_template.py` but is **no longer exposed as a CLI** — it is only consumed as a Python library by `render-video`'s `render_video.py` via `import match_template`. The `template-bind` skill itself no longer has a Python entry script.
|
|
82
|
+
|
|
83
|
+
1. **List available templates**: run `ab-skill template-bind --list-templates` to view the templates in the registry along with their supported aspect ratios / style tags, and decide which `templateId` to pick.
|
|
84
|
+
2. **Write the DSL**: when generating the Video DSL, put the chosen `templateId` into `meta.templateId` (the canonical location). Use `meta.templateVariant` / `renderHints.templateVariant` to explicitly select a variant. The legacy `renderHints.templatePreference[0]` is still tolerated by `match_template.py` and `dsl_validator._pick_template_id` during transition, but new authors should write `meta.templateId`.
|
|
85
|
+
3. **Produce the TemplateBinding**: there is no standalone CLI for DSL → TemplateBinding; `prepare_video_assets` calls `match_template.build_binding(template, dsl)` inline during the asset-resolution pipeline and embeds the binding into the RenderPlan it hands to the renderer — no separate `*.binding.json` file is written.
|
|
86
|
+
|
|
87
|
+
> Design trade-off: collapsing the binding step into the asset-prep pipeline (no CLI, no on-disk artifact) avoids binding files drifting between the agent, the database, and the file system; any hand-edited `.binding.json` would never be consumed by the renderer anyway. For local debugging you can still `import match_template.build_binding` from Python.
|
|
88
|
+
|
|
89
|
+
### List available templates
|
|
90
|
+
|
|
91
|
+
```bash
|
|
92
|
+
ab-skill template-bind --list-templates
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
### Sync the registry cache (optional, used for offline / LLM prompt)
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
python3 <SkillDir>/scripts/sync_registry.py
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
`sync_registry.py` and `check_contracts.py` are **maintenance helpers** for template-library, not skill entry points. Keep invoking them directly via `python3`.
|
|
102
|
+
|
|
103
|
+
### Verify template / Remotion contract consistency
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
python3 <SkillDir>/scripts/check_contracts.py
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
### Build a binding locally (debug only, not part of the render pipeline)
|
|
110
|
+
|
|
111
|
+
```python
|
|
112
|
+
import json, sys
|
|
113
|
+
sys.path.insert(0, "<SkillDir>/scripts")
|
|
114
|
+
from match_template import build_binding, load_full_template, load_registry
|
|
115
|
+
|
|
116
|
+
dsl = json.load(open("my-video.dsl.json"))
|
|
117
|
+
template = next(t for t in load_registry() if t["templateId"] == "image-slide")
|
|
118
|
+
binding = build_binding(load_full_template(template), dsl)
|
|
119
|
+
print(json.dumps(binding, ensure_ascii=False, indent=2))
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
## Common CLI flags
|
|
123
|
+
|
|
124
|
+
| Flag | Description | Default |
|
|
125
|
+
|------|-------------|---------|
|
|
126
|
+
| `--list-templates` | List every available template and exit (currently the only CLI verb). | — |
|
|
127
|
+
|
|
128
|
+
## Props extraction rules
|
|
129
|
+
|
|
130
|
+
`match_template.py` converts each DSL scene into a `props` dict for the renderer. Extraction runs through **two paths** (highest priority first). Earlier versions also had two fallback layers that did "implicit pass-through of fields named in `requiredProps ∪ optionalProps`"; those were removed in P1.3 because the priority conflict between the allowlist and propExtractors confused authors. Only the two most explicit paths remain:
|
|
131
|
+
|
|
132
|
+
### Extraction paths
|
|
133
|
+
|
|
134
|
+
1. **`slotMapping[purpose].propExtractors` (explicit extraction)**
|
|
135
|
+
- Looks like `"titleText": { "from": "textLayers", "role": "headline" }`.
|
|
136
|
+
- Two forms are supported: `{ from: "textLayers", role: "..." }` matches a textLayer by role; `{ from: "a.b.c" }` reads a value from the scene by dotted path.
|
|
137
|
+
2. **`customPayload` full-field pass-through (no allowlist check)**
|
|
138
|
+
- First `scene.customPayload.templateData.*` is injected; then `scene.customPayload.*` (except `templateData` itself). The latter overrides same-named values from the former, but neither can override fields already produced by propExtractors.
|
|
139
|
+
- This is the channel for templates such as html-slide that define their own sub-schema and need to pass `slideId` / `items` / `highlightMap`, etc.
|
|
140
|
+
- `scene.templateData` (without the `customPayload` wrapper) is also recognized as a `templateData` source — equivalent shorthand.
|
|
141
|
+
- **Note**: `requiredProps` / `optionalProps` are currently **just template-schema documentation**; they trigger no implicit pass-through. A field reaches `props` either through a propExtractor or through `customPayload`.
|
|
142
|
+
|
|
143
|
+
### Which form to use
|
|
144
|
+
|
|
145
|
+
| Field type | Form | Example |
|
|
146
|
+
|---|---|---|
|
|
147
|
+
| Comes from the standard DSL shape (textLayers, audio.narration.assetRef) and needs **renaming / re-routing** | declare in `propExtractors` | `titleText ← textLayers[role=headline]` |
|
|
148
|
+
| Comes from a scene-level custom field whose name matches a prop directly | put it in `customPayload.<propName>` | `customPayload.slideId`, `customPayload.background`, `customPayload.bullets` |
|
|
149
|
+
| Structured data internal to the template (arrays, sub-objects, etc.) | use `customPayload.templateData.*` | `templateData.concepts`, `templateData.items` (shared by feature-grid / timeline / column-compare) |
|
|
150
|
+
|
|
151
|
+
**Common pitfall**: writing the same field in both `propExtractors` and `customPayload` — when they collide, `propExtractors` wins. Keep each field in exactly one place.
|
|
152
|
+
|
|
153
|
+
> Historical issue: an early html-slide `point` slot declared `slideId` as `propExtractors.slideId={from:"templateData.slideId"}` and also marked it as `requiredProps`. But every existing DSL wrote `slideId` at `customPayload.slideId` (top-level, not inside templateData), so the propExtractor returned nothing and rendering fell back to the generic layout. The convention now is unified: **slideId flows through customPayload top-level pass-through**, propExtractors no longer declares it. New templates must follow the same convention.
|
|
154
|
+
|
|
155
|
+
## Narration for multi-card scenes (html-slide template)
|
|
156
|
+
|
|
157
|
+
When `html-slide` selects `demo-concept-overview` / `feature-grid` / `timeline` / `column-compare` (all "many-card sequential highlight" slides), you **must** use structured narration to avoid highlight/audio drift:
|
|
158
|
+
|
|
159
|
+
```jsonc
|
|
160
|
+
"audio": {
|
|
161
|
+
"narration": {
|
|
162
|
+
"intro": "Hermes Agent has five core capabilities.", // optional: while this line plays, no card is highlighted
|
|
163
|
+
"items": [ // one line per card
|
|
164
|
+
"Persistent memory remembers everything across sessions.",
|
|
165
|
+
"More than forty built-in tools cover search, files, image generation, and more.",
|
|
166
|
+
"Plugs into Telegram, WeChat, and other channels.",
|
|
167
|
+
"Built-in task scheduling.",
|
|
168
|
+
"Extends to unlimited capabilities via the MCP protocol."
|
|
169
|
+
],
|
|
170
|
+
"outro": "", // optional
|
|
171
|
+
"assetRef": "narration-overview"
|
|
172
|
+
}
|
|
173
|
+
}
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
Rules:
|
|
177
|
+
- `items.length` must equal `customPayload.templateData.<concepts|items>.length` (demo-concept-overview uses `concepts`; the other three slides use `items`), otherwise `dsl_validator` rejects it.
|
|
178
|
+
- Do not write `templateData.highlightMap`; `render_video.py` builds it automatically from the per-line timestamps that TTS returns.
|
|
179
|
+
- If the author explicitly provides a `highlightMap`, the system keeps it as-is.
|
|
180
|
+
- The legacy flat `narration.text` form still works but the author then has to keep `highlightMap` in sync with the subtitle segmentation (error-prone, not recommended).
|
|
181
|
+
|
|
182
|
+
## Image sources (picture-book-en and every other template)
|
|
183
|
+
|
|
184
|
+
Image assets support two source forms:
|
|
185
|
+
|
|
186
|
+
### AI-generated image
|
|
187
|
+
|
|
188
|
+
```jsonc
|
|
189
|
+
{
|
|
190
|
+
"assetId": "img-page1",
|
|
191
|
+
"type": "image",
|
|
192
|
+
"source": "gen-image",
|
|
193
|
+
"status": "planned",
|
|
194
|
+
"payload": {
|
|
195
|
+
"prompt": "Children's picture book illustration...",
|
|
196
|
+
"ratio": "1:1"
|
|
197
|
+
}
|
|
198
|
+
}
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
### User-supplied link (use an existing image directly)
|
|
202
|
+
|
|
203
|
+
```jsonc
|
|
204
|
+
{
|
|
205
|
+
"assetId": "img-page1",
|
|
206
|
+
"type": "image",
|
|
207
|
+
"source": "existing",
|
|
208
|
+
"status": "ready",
|
|
209
|
+
"url": "https://cdn.example.com/my-image.jpg"
|
|
210
|
+
}
|
|
211
|
+
```
|
|
212
|
+
|
|
213
|
+
**Rules:**
|
|
214
|
+
- `source: "existing"` + `status: "ready"` + top-level `url` → resolver uses the URL as-is, no generation API call.
|
|
215
|
+
- `source: "gen-image"` + `status: "planned"` + `payload.prompt` → resolver calls the gen-image skill.
|
|
216
|
+
- Local paths (`./image.png`, `/tmp/...`) are forbidden; HTTPS URLs only.
|
|
217
|
+
- Both forms can be mixed inside the same DSL — each asset decides independently.
|
|
218
|
+
|
|
219
|
+
## Picture-book narration rule (picture-book-en template)
|
|
220
|
+
|
|
221
|
+
With `picture-book-en`, **narration reads English only, never Chinese**:
|
|
222
|
+
|
|
223
|
+
- `audio.narration.text` and the corresponding `gen-voice` asset's `payload.text` **must contain only the English source** (ASCII letters + standard English punctuation); any CJK character or Chinese punctuation is forbidden.
|
|
224
|
+
- The Chinese translation appears only on screen via `textLayers[role=subheadline].content` (mapped to `chineseTitle` / `chineseText`); it never enters TTS.
|
|
225
|
+
- Layout is fixed:
|
|
226
|
+
- **Cover**: top half is the title area (English `englishTitle` on top in a large round-bold font; Chinese `chineseTitle` below in a smaller font); bottom half is the cover image / video.
|
|
227
|
+
- **Content page**: top half is the text area (English `englishText` on top in a large round-bold font; Chinese `chineseText` below); bottom half is the illustration.
|
|
228
|
+
- `textLayers` role mapping is mandatory: `role=headline` carries the English text, `role=subheadline` carries the Chinese text.
|
|
229
|
+
|
|
230
|
+
## Error handling
|
|
231
|
+
|
|
232
|
+
- **No template specified**: the DSL must set `meta.templateId` (or, for legacy DSLs only, `renderHints.templatePreference[0]`); otherwise `prepare_video_assets` refuses to build a binding and prints the list of available templates.
|
|
233
|
+
- **Malformed DSL**: validate first with `gen-script --validate` (delegates to `video_dsl.runtime.dsl_validator.validate_structural`).
|
|
234
|
+
- **Empty template registry**: confirm `VIDEO_TEMPLATE_REGISTRY_URL` is reachable, or that the monorepo `template-library/packages/metadata/registry.json` fallback exists.
|