@remixmate/cli 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +166 -0
- package/dist/argv.d.ts +15 -0
- package/dist/argv.js +51 -0
- package/dist/cli.d.ts +13 -0
- package/dist/cli.js +83 -0
- package/dist/handlers/gen-image.d.ts +13 -0
- package/dist/handlers/gen-image.js +161 -0
- package/dist/handlers/gen-voice.d.ts +16 -0
- package/dist/handlers/gen-voice.js +110 -0
- package/dist/handlers/index.d.ts +13 -0
- package/dist/handlers/index.js +14 -0
- package/dist/handlers/template-bind.d.ts +15 -0
- package/dist/handlers/template-bind.js +149 -0
- package/dist/http.d.ts +52 -0
- package/dist/http.js +110 -0
- package/dist/index.d.ts +9 -0
- package/dist/index.js +8 -0
- package/dist/manifest.json +260 -0
- package/dist/progress.d.ts +23 -0
- package/dist/progress.js +15 -0
- package/dist/registry.d.ts +33 -0
- package/dist/registry.js +72 -0
- package/dist/runner.d.ts +17 -0
- package/dist/runner.js +63 -0
- package/package.json +54 -0
- package/skills/export-jianying/SKILL.md +251 -0
- package/skills/export-jianying/scripts/gen_jianying_draft.py +1099 -0
- package/skills/export-jianying/skill.json +24 -0
- package/skills/export-jianying/version.json +7 -0
- package/skills/gen-digital-human/SKILL.md +211 -0
- package/skills/gen-digital-human/scripts/gen_digital_human_video.py +574 -0
- package/skills/gen-digital-human/skill.json +23 -0
- package/skills/gen-digital-human/version.json +7 -0
- package/skills/gen-image/SKILL.md +130 -0
- package/skills/gen-image/skill.json +22 -0
- package/skills/gen-image/version.json +7 -0
- package/skills/gen-script/SKILL.md +283 -0
- package/skills/gen-script/scripts/gen_script.py +1208 -0
- package/skills/gen-script/skill.json +41 -0
- package/skills/gen-script/version.json +7 -0
- package/skills/gen-video/SKILL.md +170 -0
- package/skills/gen-video/scripts/gen_video.py +591 -0
- package/skills/gen-video/skill.json +24 -0
- package/skills/gen-video/version.json +7 -0
- package/skills/gen-voice/SKILL.md +104 -0
- package/skills/gen-voice/skill.json +21 -0
- package/skills/gen-voice/version.json +7 -0
- package/skills/prepare-video-assets/SKILL.md +192 -0
- package/skills/prepare-video-assets/scripts/prepare_video_assets.py +57 -0
- package/skills/prepare-video-assets/skill.json +23 -0
- package/skills/prepare-video-assets/version.json +7 -0
- package/skills/render-video/SKILL.md +187 -0
- package/skills/render-video/scripts/_chrome_vendor.py +305 -0
- package/skills/render-video/scripts/_video_probe.py +174 -0
- package/skills/render-video/scripts/_vod_polling.py +86 -0
- package/skills/render-video/scripts/remote_renderer_client.py +236 -0
- package/skills/render-video/scripts/render_video.py +2288 -0
- package/skills/render-video/scripts/upload_video.py +193 -0
- package/skills/render-video/skill.json +21 -0
- package/skills/render-video/version.json +7 -0
- package/skills/template-bind/README.md +63 -0
- package/skills/template-bind/SKILL.md +234 -0
- package/skills/template-bind/scripts/check_contracts.py +196 -0
- package/skills/template-bind/scripts/match_template.py +215 -0
- package/skills/template-bind/scripts/registry_loader.py +379 -0
- package/skills/template-bind/scripts/render_job_client.py +162 -0
- package/skills/template-bind/scripts/sync_registry.py +44 -0
- package/skills/template-bind/scripts/template_paths.py +72 -0
- package/skills/template-bind/skill.json +20 -0
- package/skills/template-bind/version.json +7 -0
- package/skills/template-bind/video_dsl/README.md +97 -0
- package/skills/template-bind/video_dsl/runtime/__init__.py +11 -0
- package/skills/template-bind/video_dsl/runtime/dsl_validator.py +495 -0
- package/skills/template-bind/video_dsl/runtime/prompt_enhancer.py +274 -0
- package/skills/template-bind/video_dsl/runtime/template_binder.py +87 -0
- package/skills/template-bind/video_dsl/runtime/timeline_compiler.py +312 -0
- package/skills/template-bind/video_dsl/schema/render-plan-v1alpha1.json +193 -0
- package/skills/template-bind/video_dsl/schema/template-binding-v1alpha1.json +122 -0
- package/skills/template-bind/video_dsl/schema/template-definition-v1alpha1.json +247 -0
- package/skills/template-bind/video_dsl/schema/video-dsl-v1alpha1.json +378 -0
- package/skills/video-parser/SKILL.md +238 -0
- package/skills/video-parser/scripts/analyze_video.py +1038 -0
- package/skills/video-parser/scripts/deconstruct_video.py +602 -0
- package/skills/video-parser/scripts/parse_via_render.py +148 -0
- package/skills/video-parser/skill.json +21 -0
- package/skills/video-parser/version.json +7 -0
- package/skills/web-capture/SKILL.md +183 -0
- package/skills/web-capture/scripts/web_capture.py +461 -0
- package/skills/web-capture/skill.json +29 -0
- package/skills/web-capture/version.json +7 -0
|
@@ -0,0 +1,104 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: gen-voice
|
|
3
|
+
description: |
|
|
4
|
+
Text-to-speech (TTS) skill: synthesize narration audio from text via ab-api's `/tool/minimaxT2aV2` endpoint (Minimax TTS).
|
|
5
|
+
|
|
6
|
+
Use this skill immediately whenever the user asks for any of:
|
|
7
|
+
- AI voice-over, TTS, text-to-speech, generate narration audio
|
|
8
|
+
- Use Minimax for speech synthesis
|
|
9
|
+
- Read a piece of text aloud / produce an audio file from text
|
|
10
|
+
|
|
11
|
+
Even when the user does not explicitly say "AI", any request that turns text into speech should route here.
|
|
12
|
+
triggers:
|
|
13
|
+
- AI voice-over, TTS, text-to-speech, generate narration audio
|
|
14
|
+
- Use Minimax for speech synthesis
|
|
15
|
+
- Read a piece of text aloud / produce an audio file from text
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
# Text-to-Speech (TTS) Skill
|
|
19
|
+
|
|
20
|
+
Wraps ab-api's `POST /tool/minimaxT2aV2` (the same endpoint the web studio uses), authenticated with the **Tianyan privateToken**, backed by the **Minimax TTS** model.
|
|
21
|
+
|
|
22
|
+
## Auth & environment
|
|
23
|
+
|
|
24
|
+
There is no skill-local env file — the executing process inherits the system environment. Examples below say `python`; on macOS you may need `python3`.
|
|
25
|
+
|
|
26
|
+
- **Enterprise OpenClaw**: auth is already injected, **no need** to set `PRIV_TOKEN` / `--priv-token`.
|
|
27
|
+
- **Other environments**: configure the token. See the Tianyan privateToken doc internally. Without a token, non-interactive runs fail; interactive runs prompt.
|
|
28
|
+
|
|
29
|
+
| Env var | Description | Default |
|
|
30
|
+
|---------|-------------|---------|
|
|
31
|
+
| `PRIV_TOKEN` | Tianyan token; `--priv-token` overrides | none |
|
|
32
|
+
| `MM_API_BASE_URL` | API root | see `gen_voice.py` default |
|
|
33
|
+
| `AGENT_NAME` | Optional `x-invoke-agent` header | none |
|
|
34
|
+
|
|
35
|
+
## Operations
|
|
36
|
+
|
|
37
|
+
> This skill was migrated from a Python script to an ab-skill CLI HTTP handler (`entry.type: http`). The agent invocation is unchanged (same tool name `gen_voice`, same params as in `skill.json`); local repro goes through `ab-skill gen-voice ...`. The legacy `--download` flag has been removed — audio URLs are persisted in the cloud and play directly.
|
|
38
|
+
|
|
39
|
+
1. **Text**: confirm what to synthesize. Punctuation drives pauses (commas short, periods long).
|
|
40
|
+
2. **Voice**: if the user prefers a specific voice, run `--list-voices` first and pick a matching id.
|
|
41
|
+
3. **Output**: by default the audio URL is printed; add `--json-output` for a structured result that includes per-line subtitle timestamps.
|
|
42
|
+
|
|
43
|
+
### List available voices
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
ab-skill gen-voice --list-voices
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
To inspect the local language-tagged fallback catalog used by the voice resolver (no remote API call), add `--local`:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
ab-skill gen-voice --list-voices --local
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
The local catalog prints one voice per line as `<voice-id>\t<lang>\t<display-name>`.
|
|
56
|
+
|
|
57
|
+
### Default synthesis (URL output)
|
|
58
|
+
|
|
59
|
+
```bash
|
|
60
|
+
ab-skill gen-voice --text "<text-to-synthesize>"
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
### With voice + speed
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
ab-skill gen-voice \
|
|
67
|
+
--text "<text-to-synthesize>" \
|
|
68
|
+
--voice-id "female-shaonv" \
|
|
69
|
+
--speed 1.2
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
### JSON output (with subtitle timestamps)
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
ab-skill gen-voice --text "<text-to-synthesize>" --json-output
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
## Common CLI flags
|
|
79
|
+
|
|
80
|
+
| Flag | Description | Default |
|
|
81
|
+
|------|-------------|---------|
|
|
82
|
+
| `--text` | Text to synthesize (required, except with `--list-voices`) | — |
|
|
83
|
+
| `--voice-id` | Voice id (use `--list-voices` to discover) | `Chinese (Mandarin)_Male_Announcer` |
|
|
84
|
+
| `--speed` | Speech rate, 0.5–2.0 | `1.0` |
|
|
85
|
+
| `--list-voices` | List available voices and exit | — |
|
|
86
|
+
| `--local` | Used with `--list-voices`: print the local fallback catalog (offline) | off |
|
|
87
|
+
| `--json-output` | Emit JSON (url, audio_length_ms, subtitles) | off |
|
|
88
|
+
| `--priv-token` | Override token | env var |
|
|
89
|
+
|
|
90
|
+
## Speed guidance
|
|
91
|
+
|
|
92
|
+
| Use case | Recommended speed |
|
|
93
|
+
|----------|-------------------|
|
|
94
|
+
| Reading / audiobook | 0.8–1.0 |
|
|
95
|
+
| Voice-over / narration | 1.0–1.2 |
|
|
96
|
+
| Fast announcement | 1.2–1.5 |
|
|
97
|
+
|
|
98
|
+
## Error handling
|
|
99
|
+
|
|
100
|
+
- **401** / **token missing** (non-OpenClaw): set `PRIV_TOKEN`.
|
|
101
|
+
- **Business `code != 0`**: read `msg` on stderr.
|
|
102
|
+
- **429**: rate-limited; retry later.
|
|
103
|
+
- **Invalid voice id**: re-check via `--list-voices`.
|
|
104
|
+
- **Network**: verify connectivity and `MM_API_BASE_URL`.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "gen-voice",
|
|
3
|
+
"toolName": "gen_voice",
|
|
4
|
+
"tier": "atomic",
|
|
5
|
+
"title": "Text-to-Speech (Minimax)",
|
|
6
|
+
"description": "Text-to-speech (TTS): synthesize narration audio from text via the Minimax TTS model. Returns the persisted audio URL — no download needed.",
|
|
7
|
+
"envVars": ["PRIV_TOKEN", "MM_API_BASE_URL", "AGENT_NAME"],
|
|
8
|
+
"entry": { "type": "http", "handler": "gen-voice" },
|
|
9
|
+
"parameters": {
|
|
10
|
+
"type": "object",
|
|
11
|
+
"properties": {
|
|
12
|
+
"text": { "type": "string", "description": "Text to synthesize (required)" },
|
|
13
|
+
"voice_id": { "type": "string", "description": "Voice id. Default 'Chinese (Mandarin)_Male_Announcer'. When unsure, call with list_voices=true first to see what's available — do not invent ids." },
|
|
14
|
+
"speed": { "type": "number", "description": "Speech rate, 0.5-2.0" },
|
|
15
|
+
"list_voices": { "type": "boolean", "description": "List available voices and exit" },
|
|
16
|
+
"local": { "type": "boolean", "description": "Used together with list_voices=true: print the voice-resolver fallback catalog with language tags (no remote /voice/page call). Output is one '<id>\\t<lang>\\t<name>' line per voice." },
|
|
17
|
+
"json_output": { "type": "boolean", "description": "Emit a JSON result (url, audio_length_ms, subtitles)" }
|
|
18
|
+
},
|
|
19
|
+
"required": []
|
|
20
|
+
}
|
|
21
|
+
}
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"skillName": "gen-voice",
|
|
3
|
+
"repoName": "agent-skill-media-maker",
|
|
4
|
+
"skillId": "338",
|
|
5
|
+
"version": "V5",
|
|
6
|
+
"skillDescription": "语音合成(TTS)技能,将文字转为语音音频(调用 ab-api /tool/minimaxT2aV2,基于 Minimax TTS)。\n\n当用户提到以下任何需求时,立即使用本 skill:\n- AI 配音、语音合成、文字转语音、TTS、文本转音频、生成语音\n- 使用 minimax 语音合成\n- 用户想要将一段文字朗读出来、生成音频\n\n即使用户没有明确说「使用 AI」,只要他们想要将文字转为语音,也要使用本 skill。"
|
|
7
|
+
}
|
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: prepare-video-assets
|
|
3
|
+
description: |
|
|
4
|
+
Asset-preparation skill: resolves and generates every asset (image / audio / video) referenced by a Video DSL, persists a RenderPlan to the database, and returns a `job_id` for the subsequent `render_video` call.
|
|
5
|
+
|
|
6
|
+
Use this skill as soon as the user mentions any of these intents:
|
|
7
|
+
- Generate / prepare video assets, resolve assets, render-ready
|
|
8
|
+
- "Make me a video about X" (the agent calls gen_script → prepare_video_assets → render_video)
|
|
9
|
+
- Regenerate one asset (image / audio) for a specific scene
|
|
10
|
+
|
|
11
|
+
Next step: after the user confirms the resolved assets, call `render_video` with the `job_id` returned by this skill.
|
|
12
|
+
triggers:
|
|
13
|
+
- Generate / prepare video assets
|
|
14
|
+
- Resolve missing assets in a DSL
|
|
15
|
+
- User confirmed the script and the agent needs to prepare assets for review
|
|
16
|
+
- User asks to regenerate a specific scene's image or narration
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
# Prepare Video Assets Skill
|
|
20
|
+
|
|
21
|
+
Phase 1 of the two-phase video pipeline. Takes a Video DSL plus a template binding, walks every `AssetRef` declared in the DSL, calls the matching atomic skills to fill in missing assets (`gen-image`, `gen-voice`, `gen-video`, `gen-digital-human`), persists the resulting RenderPlan to ab-api, and prints a `job_id` for the user-confirmation step.
|
|
22
|
+
|
|
23
|
+
> **Next step**: after the user reviews the assets and confirms, call **`render_video`** with the `job_id` returned by this skill.
|
|
24
|
+
|
|
25
|
+
## Pipeline (this skill's part only)
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
DSL + template_id (or binding)
|
|
29
|
+
↓
|
|
30
|
+
[1] DSL Validator → schema check + normalize
|
|
31
|
+
↓
|
|
32
|
+
[2] Template Binder → produce RenderPlan
|
|
33
|
+
↓
|
|
34
|
+
[3] Asset Resolver → for every AssetRef, call atomic skills as needed
|
|
35
|
+
↓
|
|
36
|
+
[3b] Persist RenderPlan to DB → stdout "📦 render job jobId: N"
|
|
37
|
+
↓
|
|
38
|
+
(user reviews → confirms → render_video --job-id N)
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## Atomic-skill dependencies
|
|
42
|
+
|
|
43
|
+
The asset resolver invokes the following skills based on `AssetRef` declarations:
|
|
44
|
+
|
|
45
|
+
| Asset type | Skill called | Notes |
|
|
46
|
+
|------------|--------------|-------|
|
|
47
|
+
| `image` + `source: gen-image` | `gen-image` | Text-to-image / image-to-image. |
|
|
48
|
+
| `audio` + `source: gen-voice` | `gen-voice` | Narration TTS. |
|
|
49
|
+
| `video` + `source: gen-video` | `gen-video` | AI-generated video clips. |
|
|
50
|
+
| `avatar` + `source: gen-digital-human` | `gen-digital-human` | Talking-head digital-human segments. |
|
|
51
|
+
|
|
52
|
+
## Authentication & environment
|
|
53
|
+
|
|
54
|
+
| Env var | Description | Default |
|
|
55
|
+
|---------|-------------|---------|
|
|
56
|
+
| `PRIV_TOKEN` | Tianyan token (needed for asset generation + DB persist). | (none) |
|
|
57
|
+
| `MM_API_BASE_URL` | Asset-generation API root (ab-agent proxy). | `http://localhost:3001/api` |
|
|
58
|
+
| `MM_BACKEND_API_URL` | Backend API root (RenderPlan persistence). | `http://localhost:2999/api` |
|
|
59
|
+
| `ASSET_CACHE_DIR` | Asset cache directory. | `./.asset-cache/` |
|
|
60
|
+
|
|
61
|
+
## Canonical usage (inline DSL JSON, no temp files)
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
python3 <SkillDir>/scripts/prepare_video_assets.py \
|
|
65
|
+
--dsl-json '<DSL_JSON_STRING>' \
|
|
66
|
+
--template-id html-slide
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
- `<DSL_JSON_STRING>` is the JSON the agent received from `gen_script` — pass inline; **do not** write it to disk.
|
|
70
|
+
- `--save-job` defaults to on, persisting the RenderPlan and printing `📦 render job jobId: N`.
|
|
71
|
+
- Image and audio URLs are printed to stdout under `🔊 TTS audio:` and `🖼 Image assets:` sections.
|
|
72
|
+
- When `PRIV_TOKEN` / `MM_BACKEND_API_URL` are missing the wrapper auto-degrades to file mode and prints a notice.
|
|
73
|
+
|
|
74
|
+
## File-mode fallback (single-user / local debug)
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
python3 <SkillDir>/scripts/prepare_video_assets.py \
|
|
78
|
+
--dsl my-video.dsl.json \
|
|
79
|
+
--template-id html-slide
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
`--template-id` automatically calls template-bind internally; or pass `--binding my-video.binding.json` if you have a pre-generated binding.
|
|
83
|
+
|
|
84
|
+
## Image-URL handling (agent rules)
|
|
85
|
+
|
|
86
|
+
**DSL image assets must use HTTPS URLs.** During Remotion rendering, headless Chrome fetches the images directly over the network — no local download is needed.
|
|
87
|
+
|
|
88
|
+
| Situation | Correct action |
|
|
89
|
+
|-----------|----------------|
|
|
90
|
+
| Image in a DSL AssetRef | Use the raw HTTPS URL. |
|
|
91
|
+
| Showing an asset preview to the user | Use Markdown inline image syntax: ``. **Do not** Read an HTTPS URL. |
|
|
92
|
+
| Need to understand the image (to draft a title / bullets) | `curl` it down locally and read it; the DSL still carries the HTTPS URL. |
|
|
93
|
+
| Avoid WebFetch on images | Internal-domain images are blocked by the WebFetch security policy. |
|
|
94
|
+
|
|
95
|
+
## Pre-render user confirmation (Phase 2)
|
|
96
|
+
|
|
97
|
+
This skill is the **first half** of a two-phase user-confirmation flow. After it returns, the agent must show the user the resolved assets and wait for confirmation before calling `render_video`. The recommended summary format:
|
|
98
|
+
|
|
99
|
+
```markdown
|
|
100
|
+
## Render confirmation
|
|
101
|
+
|
|
102
|
+
### Video overview
|
|
103
|
+
| Field | Value |
|
|
104
|
+
|------|-----|
|
|
105
|
+
| Title | ... |
|
|
106
|
+
| Actual duration | xxs (after audio adjustment) |
|
|
107
|
+
| Aspect ratio | 9:16 |
|
|
108
|
+
| Scenes | x |
|
|
109
|
+
|
|
110
|
+
### Scenes & assets
|
|
111
|
+
| # | Purpose | Duration | Narration excerpt | Image | Audio |
|
|
112
|
+
|---|---------|----------|--------------------|-------|-------|
|
|
113
|
+
| 1 | opening | 6.2s | ... |  | [🔊 listen](http://cdn.../narration-scene-01.mp3) |
|
|
114
|
+
| 2 | point | 11.5s | ... |  | [🔊 listen](http://cdn.../narration-scene-02.mp3) |
|
|
115
|
+
|
|
116
|
+
### Asset status
|
|
117
|
+
- ✅ Images: x/x generated
|
|
118
|
+
- ✅ TTS: x/x generated
|
|
119
|
+
- ❌ Failed: list any failures
|
|
120
|
+
|
|
121
|
+
> Reply "continue" to start rendering. To regenerate a specific asset, tell me which one.
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
## Regeneration pattern
|
|
125
|
+
|
|
126
|
+
To regenerate a specific asset (e.g. the user is unhappy with scene 2's image):
|
|
127
|
+
|
|
128
|
+
1. Update that scene's prompt / params in the DSL.
|
|
129
|
+
2. Call this skill again with the updated `dsl_json` (or the minimal narration-override JSON when the gen_script skeleton is cached in the session).
|
|
130
|
+
3. A new `job_id` is issued. Show the new assets and ask for confirmation again.
|
|
131
|
+
4. After confirmation, call `render_video` with the **new** `job_id`.
|
|
132
|
+
|
|
133
|
+
## Test mode: skip asset generation (`--stub-image-url` / `--stub-video-url`)
|
|
134
|
+
|
|
135
|
+
During dev / debug the user may want to exercise the pipeline without burning gen-image / gen-video quota. With a stub URL set, every `image+source=gen-image` AssetRef (or `video+source=gen-video`) is short-circuited at the resolver to that URL — no generation request is sent. TTS is unaffected and always runs for real.
|
|
136
|
+
|
|
137
|
+
### Usage rules
|
|
138
|
+
|
|
139
|
+
- The URL is passed **verbatim**; do not rewrite it.
|
|
140
|
+
- If the user expressed the intent but did not supply a URL, the agent must ask which fallback URL to use — never invent one.
|
|
141
|
+
- The upstream `gen_script.py --stub-image-url / --stub-video-url` is the source-of-truth solution; this skill's flag is the render-layer safety net. Both layers can be enabled at the same time.
|
|
142
|
+
- If the user does not re-state test mode in a later turn, **do not** carry the previous stub URL forward.
|
|
143
|
+
|
|
144
|
+
### Command examples
|
|
145
|
+
|
|
146
|
+
```bash
|
|
147
|
+
python3 <SkillDir>/scripts/prepare_video_assets.py \
|
|
148
|
+
--dsl-json '<DSL_JSON_STRING>' \
|
|
149
|
+
--template-id picture-book-en \
|
|
150
|
+
--stub-image-url "https://cdn.example.com/placeholder.jpg"
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Env vars `STUB_IMAGE_URL` / `STUB_VIDEO_URL` also work — their priority is lower than the CLI flag.
|
|
154
|
+
|
|
155
|
+
## Common CLI flags (inherited from render_video.py via the wrapper)
|
|
156
|
+
|
|
157
|
+
| Flag | Description | Default |
|
|
158
|
+
|------|-------------|---------|
|
|
159
|
+
| `--dsl` | Input DSL file path. | — |
|
|
160
|
+
| `--dsl-json` | DSL JSON as an inline string (preferred). | — |
|
|
161
|
+
| `--template-id` | Template id (auto-bind). | — |
|
|
162
|
+
| `--binding` | TemplateBinding file path. | — |
|
|
163
|
+
| `--binding-json` | TemplateBinding JSON as an inline string. | — |
|
|
164
|
+
| `--save-job` | Persist RenderPlan to the database; pass `--no-save-job` to disable. | **on by default** |
|
|
165
|
+
| `--asset-cache-dir` | Asset cache directory. | `.asset-cache/` |
|
|
166
|
+
| `--max-asset-retries` | Max retries per asset. | `3` |
|
|
167
|
+
| `--asset-timeout` | Per-asset generation timeout (seconds). | `300` |
|
|
168
|
+
| `--private-token` | Tianyan token. | env var |
|
|
169
|
+
| `--stub-image-url` | Test mode: short-circuit image+source=gen-image assets to this URL. | — |
|
|
170
|
+
| `--stub-video-url` | Test mode: short-circuit video+source=gen-video assets to this URL. | — |
|
|
171
|
+
|
|
172
|
+
## Asset-resolution strategy
|
|
173
|
+
|
|
174
|
+
The Asset Resolver handles each `AssetRef` in this order:
|
|
175
|
+
|
|
176
|
+
1. **status = generated / approved**: asset is ready — use the URL as-is.
|
|
177
|
+
2. **status = planned / missing**: call the matching atomic skill to generate it.
|
|
178
|
+
3. **Generation failed**: retry up to `maxRetries`; final failures are recorded in the RenderPlan errors.
|
|
179
|
+
4. **Parallel generation**: assets of the same type are generated in parallel; different types are sequenced by dependency.
|
|
180
|
+
5. **Cache reuse**: assets with the same payload are checked against `asset-cache-dir` to avoid duplicate generation.
|
|
181
|
+
|
|
182
|
+
## Error handling
|
|
183
|
+
|
|
184
|
+
- **DSL validation failed**: pre-validate with `gen-script --validate`.
|
|
185
|
+
- **Template not found**: confirm the templateId is in the registry.
|
|
186
|
+
- **Asset generation failed**: inspect the RenderPlan `errors` field and the atomic skill logs.
|
|
187
|
+
- **Stale skeleton**: if the agent submits gen_script's raw skeleton verbatim as `dsl_json`, the agent-layer precall rejects with a skeleton-placeholder error. Fill in real narration for every scene.
|
|
188
|
+
|
|
189
|
+
## See also
|
|
190
|
+
|
|
191
|
+
- **`render_video`** — the Phase 3 skill that consumes this skill's `job_id` and produces the final video.
|
|
192
|
+
- **`template-bind`** — Lists available templates. Called internally by this skill when `--template-id` is provided.
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Thin wrapper that delegates to the canonical render_video.py with --resolve-only forced on.
|
|
3
|
+
|
|
4
|
+
The two-phase video pipeline shares one Python implementation:
|
|
5
|
+
|
|
6
|
+
prepare_video_assets → render_video.py --resolve-only ... (this file)
|
|
7
|
+
render_video → render_video.py ... (sibling skill)
|
|
8
|
+
|
|
9
|
+
By exec'ing the canonical script we get verbatim stdout / stderr / exit-code / signal forwarding
|
|
10
|
+
without any subprocess wrapper layer. The agent layer's stdout parsing for the
|
|
11
|
+
"📦 render job jobId: N" line therefore continues to work unchanged.
|
|
12
|
+
|
|
13
|
+
The wrapper unconditionally injects --resolve-only as the first argument. The MCP tool schema for
|
|
14
|
+
this skill excludes job_id / render_plan / resolve_only / upload_title / no_upload / renderer, so
|
|
15
|
+
the LLM cannot push this entry point into Phase 3 territory; even if a shell caller passes those
|
|
16
|
+
flags directly to this wrapper, --resolve-only still wins because asset-resolve-and-exit always
|
|
17
|
+
short-circuits before the render step in render_video.py.
|
|
18
|
+
"""
|
|
19
|
+
from __future__ import annotations
|
|
20
|
+
|
|
21
|
+
import os
|
|
22
|
+
import sys
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
def _canonical_script_path() -> str:
|
|
26
|
+
"""Resolve the absolute path to the render-video skill's render_video.py.
|
|
27
|
+
|
|
28
|
+
Layout (relative to this file):
|
|
29
|
+
apps/ab-skill/skills/prepare-video-assets/scripts/prepare_video_assets.py <-- HERE
|
|
30
|
+
apps/ab-skill/skills/render-video/scripts/render_video.py <-- target
|
|
31
|
+
|
|
32
|
+
The relative jump is ../../render-video/scripts/render_video.py.
|
|
33
|
+
"""
|
|
34
|
+
here = os.path.dirname(os.path.abspath(__file__))
|
|
35
|
+
return os.path.normpath(
|
|
36
|
+
os.path.join(here, "..", "..", "render-video", "scripts", "render_video.py")
|
|
37
|
+
)
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
def main() -> None:
|
|
41
|
+
script = _canonical_script_path()
|
|
42
|
+
if not os.path.isfile(script):
|
|
43
|
+
# Fail fast with a human-readable message rather than letting execv raise OSError.
|
|
44
|
+
print(
|
|
45
|
+
f"❌ prepare_video_assets wrapper could not locate the canonical render script.\n"
|
|
46
|
+
f" Expected at: {script}\n"
|
|
47
|
+
f" This indicates the render-video skill is missing from the skill tree.",
|
|
48
|
+
file=sys.stderr,
|
|
49
|
+
)
|
|
50
|
+
sys.exit(2)
|
|
51
|
+
# Force --resolve-only into argv before any user-supplied flag.
|
|
52
|
+
forwarded = ["--resolve-only", *sys.argv[1:]]
|
|
53
|
+
os.execv(sys.executable, [sys.executable, script, *forwarded])
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
if __name__ == "__main__":
|
|
57
|
+
main()
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "prepare-video-assets",
|
|
3
|
+
"toolName": "prepare_video_assets",
|
|
4
|
+
"tier": "orchestration",
|
|
5
|
+
"title": "Video Asset Preparation",
|
|
6
|
+
"description": "Resolves and generates every asset (image / audio / video) referenced by a Video DSL, persists a RenderPlan to the database, and returns a job_id for the subsequent render_video call. This is Phase 1 of the two-phase video pipeline; Phase 3 (Remotion render) lives in render_video.",
|
|
7
|
+
"envVars": ["PRIV_TOKEN", "MM_API_BASE_URL", "MM_BACKEND_API_URL", "AGENT_NAME", "REMOTION_RENDER_API_URL", "REMOTION_RENDER_MODE", "REMOTION_OUTPUT_DIR", "ASSET_CACHE_DIR"],
|
|
8
|
+
"scriptPath": "scripts/prepare_video_assets.py",
|
|
9
|
+
"parameters": {
|
|
10
|
+
"type": "object",
|
|
11
|
+
"properties": {
|
|
12
|
+
"dsl_json": { "type": "string", "description": "DSL JSON as an inline string (preferred — no disk write needed; required for multi-user concurrent flows). Pass the full DSL JSON, or — when a gen_script skeleton is cached in the session — a minimal JSON with only scenes[].audio.narration.text overrides (the agent layer merges narration onto the cached skeleton)." },
|
|
13
|
+
"dsl": { "type": "string", "description": "DSL file path (legacy fallback; prefer dsl_json inline)." },
|
|
14
|
+
"template_id": { "type": "string", "description": "Template id. When provided, the script invokes template-bind internally; no separate binding step is needed." },
|
|
15
|
+
"binding_json": { "type": "string", "description": "TemplateBinding JSON as an inline string (alternative to template_id; no file needed)." },
|
|
16
|
+
"binding": { "type": "string", "description": "TemplateBinding file path (optional; auto-generated when template_id is provided)." },
|
|
17
|
+
"save_job": { "type": "boolean", "description": "Persist the RenderPlan to the database (default true). On success the stdout contains '📦 render job jobId: N'. Auto-degrades to false when PRIV_TOKEN is missing, falling back to file mode. Pass false to opt out explicitly." },
|
|
18
|
+
"stub_image_url": { "type": "string", "description": "Test-mode image stub URL (renderer-side short-circuit). Only pass when the user explicitly says things like 'just testing / don't actually generate / use a placeholder image / save credits' AND provides a concrete URL. With this set, every image+source=gen-image AssetRef is short-circuited to that URL with no gen-image call. Do not pass otherwise; if the user expressed the intent without a URL, ask for one — do not invent one." },
|
|
19
|
+
"stub_video_url": { "type": "string", "description": "Test-mode video stub URL (renderer-side short-circuit). Only pass when the user explicitly says things like 'just testing / don't actually generate the video / use a placeholder clip / save credits' AND provides a concrete URL. With this set, every video+source=gen-video AssetRef is short-circuited to that URL with no gen-video call. Do not pass otherwise; if the user expressed the intent without a URL, ask for one — do not invent one." }
|
|
20
|
+
},
|
|
21
|
+
"required": []
|
|
22
|
+
}
|
|
23
|
+
}
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
{
|
|
2
|
+
"skillName": "prepare-video-assets",
|
|
3
|
+
"repoName": "agent-skill-media-maker",
|
|
4
|
+
"skillId": "476",
|
|
5
|
+
"version": "V1",
|
|
6
|
+
"skillDescription": "Asset-preparation skill (Phase 1 of the two-phase video pipeline). Resolves and generates every asset (image / audio / video) referenced by a Video DSL, persists a RenderPlan to the database, and returns a job_id for the subsequent render_video call."
|
|
7
|
+
}
|
|
@@ -0,0 +1,187 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: render-video
|
|
3
|
+
description: |
|
|
4
|
+
Final-render skill: loads a persisted RenderPlan by `job_id` and drives the Remotion engine to produce the final video.
|
|
5
|
+
|
|
6
|
+
Use this skill as soon as the user mentions any of these intents (after assets are already prepared):
|
|
7
|
+
- Render the video, composite the video, export the video
|
|
8
|
+
- Turn the prepared assets into the final clip
|
|
9
|
+
- Render with Remotion
|
|
10
|
+
|
|
11
|
+
Prerequisite: assets must already be generated via `prepare_video_assets`. This skill never resolves or regenerates assets — pass it a `job_id` from a previous `prepare_video_assets` call.
|
|
12
|
+
triggers:
|
|
13
|
+
- Render the video, composite the video, export the video
|
|
14
|
+
- Turn the prepared assets into the final clip
|
|
15
|
+
- Render with Remotion
|
|
16
|
+
- Agent has a job_id from prepare_video_assets and needs the final video
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
# Render Video Skill
|
|
20
|
+
|
|
21
|
+
Loads a persisted RenderPlan from the database (or a local file fallback), compiles the timeline, drives the **Remotion** engine, and emits the final playable video file.
|
|
22
|
+
|
|
23
|
+
> **Prerequisite**: assets must already be generated via `prepare_video_assets`. That skill returns a `job_id` integer; pass it here.
|
|
24
|
+
|
|
25
|
+
## Render pipeline (this skill's part only)
|
|
26
|
+
|
|
27
|
+
```
|
|
28
|
+
job_id ─► RenderPlan loaded from DB
|
|
29
|
+
│
|
|
30
|
+
▼
|
|
31
|
+
[4] Timeline Compiler ─► Remotion props + frame numbers
|
|
32
|
+
│
|
|
33
|
+
▼
|
|
34
|
+
[5] Renderer ─► Remotion ─► MP4
|
|
35
|
+
│
|
|
36
|
+
▼ (auto-upload by default; --no-upload skips)
|
|
37
|
+
[6] Uploader ─► Alibaba OSS ─► playback URL
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Phases 1–3 (DSL validation, template binding, asset resolution) are owned by **`prepare_video_assets`**. This skill picks up at phase 4.
|
|
41
|
+
|
|
42
|
+
## Authentication & environment
|
|
43
|
+
|
|
44
|
+
Rendering supports two modes:
|
|
45
|
+
|
|
46
|
+
- **local** (default if `REMOTION_RENDER_MODE` unset): runs Remotion CLI locally; requires Node.js 18+ and a headless Chrome.
|
|
47
|
+
- **remote**: calls the standalone **remotion-renderer** service (at `<monorepo-root>/remotion-renderer/`) — best for environments without Node.js, long videos, or to avoid consuming local resources.
|
|
48
|
+
|
|
49
|
+
| Env var | Description | Default |
|
|
50
|
+
|---------|-------------|---------|
|
|
51
|
+
| `PRIV_TOKEN` | Tianyan token (needed to load the RenderPlan from the DB and to upload). | (none) |
|
|
52
|
+
| `MM_BACKEND_API_URL` | Backend API root URL (RenderPlan / VOD lookups). | `http://localhost:2999/api` |
|
|
53
|
+
| `REMOTION_RENDER_API_URL` | Remote renderer base URL (ab-render). | `http://localhost:3000` |
|
|
54
|
+
| `REMOTION_RENDER_MODE` | Default render mode (`remote` / `local`); CLI `--renderer` overrides it. | `remote` |
|
|
55
|
+
| `REMOTION_OUTPUT_DIR` | Render output directory. | `./output/` |
|
|
56
|
+
| `REMOTION_CONCURRENCY` | Local Remotion render concurrency. | `2` |
|
|
57
|
+
| `ASSET_CACHE_DIR` | Asset cache directory (read-only for this skill — assets are already resolved). | `./.asset-cache/` |
|
|
58
|
+
| `REMOTION_REMOTE_POLL_TIMEOUT` | Remote-render polling timeout (seconds). | `1800` |
|
|
59
|
+
| `REMOTION_REMOTE_POLL_INTERVAL` | Remote-render polling interval (seconds). | `5` |
|
|
60
|
+
|
|
61
|
+
## Canonical usage (database job mode)
|
|
62
|
+
|
|
63
|
+
> ⚠️ **`job_id` must be a positive integer.** It is the value N from `prepare_video_assets`'s stdout line `📦 render job jobId: N`. Never pass `0`, a placeholder string, or descriptive text.
|
|
64
|
+
|
|
65
|
+
```bash
|
|
66
|
+
python3 <SkillDir>/scripts/render_video.py --job-id <jobId>
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
`--save-job` is on by default. After the render, the Manifest is written back into the database under the same `jobId`.
|
|
70
|
+
|
|
71
|
+
### Fallback: file mode (single-user / local debug)
|
|
72
|
+
|
|
73
|
+
When the DB is unavailable, the script accepts a pre-generated RenderPlan file:
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
python3 <SkillDir>/scripts/render_video.py --render-plan output/render-plan.json
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
### Render and upload to Alibaba Cloud OSS (default behavior)
|
|
80
|
+
|
|
81
|
+
```bash
|
|
82
|
+
python3 <SkillDir>/scripts/render_video.py --job-id 42 --upload-title "My video"
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
After rendering, the script auto-uploads by default; `render-manifest.json` then contains an `upload.fileUrl` field. **Prefer `upload.playbackUrl`** (transcoded) over `upload.fileUrl` (raw VOD) when surfacing the URL to a user.
|
|
86
|
+
|
|
87
|
+
### Render but skip the upload
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
python3 <SkillDir>/scripts/render_video.py --job-id 42 --no-upload
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
### Upload an existing video standalone
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
python3 <SkillDir>/scripts/upload_video.py --file output/video.mp4 --title "My video"
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
`upload_video.py` is a standalone upload tool that uploads any local video file to Alibaba Cloud OSS and returns the usable URL. Use cases:
|
|
100
|
+
|
|
101
|
+
- Manual upload after a `--no-upload` render.
|
|
102
|
+
- Uploading video files not produced by Remotion.
|
|
103
|
+
|
|
104
|
+
| Flag | Description | Default |
|
|
105
|
+
|------|-------------|---------|
|
|
106
|
+
| `--file` | Local video file path. | required |
|
|
107
|
+
| `--title` | Upload title. | filename |
|
|
108
|
+
| `--priv-token` | Override the token. | env var |
|
|
109
|
+
|
|
110
|
+
### Specify the output path
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
python3 <SkillDir>/scripts/render_video.py --job-id 42 --output final-video.mp4
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
## Common CLI flags
|
|
117
|
+
|
|
118
|
+
| Flag | Description | Default |
|
|
119
|
+
|------|-------------|---------|
|
|
120
|
+
| `--job-id` | RenderPlan job id loaded from the database (primary input). | — |
|
|
121
|
+
| `--render-plan` | Existing RenderPlan file path (single-user / local debug fallback). | — |
|
|
122
|
+
| `--save-job` | Persist Manifest to the database; pass `--no-save-job` to disable. | on (auto-degrades when token missing) |
|
|
123
|
+
| `-o` / `--output` | Output video file path. | `output/video.mp4` |
|
|
124
|
+
| `--asset-cache-dir` | Asset cache directory. | `.asset-cache/` |
|
|
125
|
+
| `--private-token` | Tianyan token. | env var |
|
|
126
|
+
| `--upload` | Auto-upload to Alibaba Cloud OSS after rendering (local mode only). | **on** |
|
|
127
|
+
| `--no-upload` | Skip the upload (local mode only). | off |
|
|
128
|
+
| `--upload-title` | Upload title (defaults to filename). | — |
|
|
129
|
+
| `--renderer` | Render mode: `remote` (default) / `local`. | `remote` |
|
|
130
|
+
| `--remote-poll-timeout` | Remote-render polling timeout (seconds). | `1800` |
|
|
131
|
+
| `--remote-poll-interval` | Remote-render polling interval (seconds). | `5` |
|
|
132
|
+
|
|
133
|
+
The script also accepts `--resolve-only`, `--dsl`, `--dsl-json`, `--template-id`, `--binding`, `--binding-json`, `--stub-image-url`, `--stub-video-url` for shell-level convenience and back-compat, but **the MCP tool surface of this skill exposes only the rendering parameters**. To resolve / regenerate assets, call `prepare_video_assets` instead.
|
|
134
|
+
|
|
135
|
+
## Render outputs
|
|
136
|
+
|
|
137
|
+
After rendering, the output directory contains:
|
|
138
|
+
|
|
139
|
+
```
|
|
140
|
+
output/
|
|
141
|
+
├── video.mp4 # Final video
|
|
142
|
+
├── render-plan.json # RenderPlan snapshot
|
|
143
|
+
├── render-manifest.json # Render metadata (duration, resolution, frame count; includes upload.fileUrl by default)
|
|
144
|
+
└── remotion-props.json # The full props passed to Remotion (debug)
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
## Remote-render mode
|
|
148
|
+
|
|
149
|
+
When to use:
|
|
150
|
+
|
|
151
|
+
- The local machine does not have Node.js 18+ / headless Chrome.
|
|
152
|
+
- Long or high-resolution videos where you do not want to tie up the local machine.
|
|
153
|
+
- CI / batch rendering.
|
|
154
|
+
|
|
155
|
+
### Example invocation
|
|
156
|
+
|
|
157
|
+
```bash
|
|
158
|
+
python3 <SkillDir>/scripts/render_video.py --job-id 42 --renderer remote --upload-title "Remote render test"
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
In remote mode:
|
|
162
|
+
|
|
163
|
+
- `POST /render` returns a `taskId` immediately; the script polls `POST /renderStatus`.
|
|
164
|
+
- **No** local `output/video.mp4` is produced (the MP4 is rendered on the server and uploaded directly to OSS).
|
|
165
|
+
- `render-manifest.json.upload.fileUrl` comes from the remote response.
|
|
166
|
+
- `render-plan.json` gains a `renderMode: "remote"` field and a `remoteTaskId`.
|
|
167
|
+
- `--upload` / `--no-upload` are ineffective in remote mode (the server handles the upload).
|
|
168
|
+
|
|
169
|
+
### Fallback on failure
|
|
170
|
+
|
|
171
|
+
If the remote service is unreachable or auth fails, the script exits with a detailed message. Re-run the same `--job-id` with `--renderer local` as a fallback.
|
|
172
|
+
|
|
173
|
+
### Service deployment
|
|
174
|
+
|
|
175
|
+
The Remotion render project lives at the monorepo root under `<monorepo-root>/remotion-renderer/` and is deployed as a **standalone HTTP service**. The client script in this skill only sends POST requests and does not participate in server-side code sync.
|
|
176
|
+
|
|
177
|
+
- Server code + deployment docs: `<monorepo-root>/remotion-renderer/README.md`.
|
|
178
|
+
- Local debugging: in the renderer directory run `npm install && npm run server:dev`; the client defaults to `http://localhost:3000`.
|
|
179
|
+
- Production: set `REMOTION_RENDER_API_URL` to the production base URL.
|
|
180
|
+
|
|
181
|
+
## Error handling
|
|
182
|
+
|
|
183
|
+
- **`job_id` missing or not assets-ready**: the script exits non-zero with a message like "render job N is not in assets-ready state". The agent should call `prepare_video_assets` again with the updated DSL to regenerate, then re-call this skill with the new `job_id`.
|
|
184
|
+
- **DSL validation failed** (only reachable via `--render-plan` file mode if the plan is malformed): pre-validate with `gen-script --validate`.
|
|
185
|
+
- **Remotion render failed**: check the Node.js version (18+ required). When deps are missing the script auto-runs `npm install`; if that fails, run it manually inside `<monorepo-root>/remotion-renderer/` (overridable via `REMOTION_RENDERER_DIR`).
|
|
186
|
+
- **Remote render failed**: inspect `render-plan.json.errors` and the server log; fall back to `--renderer local` if needed.
|
|
187
|
+
- **Timeouts**: remote rendering accepts `--remote-poll-timeout`.
|