@remixmate/cli 0.9.50 → 0.9.52
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -26
- package/dist/manifest.json +27 -14
- package/package.json +10 -13
- package/skills/doc-parse/SKILL.md +6 -6
- package/skills/doc-parse/scripts/extract_via_render.py +3 -2
- package/skills/doc-parse/skill.json +1 -1
- package/skills/gen-script/SKILL.md +19 -18
- package/skills/gen-script/scripts/gen_script.py +170 -38
- package/skills/gen-script/skill.json +14 -4
- package/skills/gen-script/version.json +1 -1
- package/skills/prepare-video-assets/SKILL.md +3 -3
- package/skills/prepare-video-assets/skill.json +1 -1
- package/skills/prepare-video-assets/version.json +1 -1
- package/skills/render-video/scripts/render_video.py +53 -42
- package/skills/render-video/version.json +1 -1
- package/skills/template-registry/README.md +4 -15
- package/skills/template-registry/SKILL.md +20 -52
- package/skills/template-registry/scripts/countdown_spec.py +56 -6
- package/skills/template-registry/scripts/list_templates.py +133 -8
- package/skills/template-registry/scripts/registry_loader.py +3 -4
- package/skills/template-registry/skill.json +2 -2
- package/skills/template-registry/version.json +1 -1
- package/skills/template-registry/video_dsl/README.md +10 -53
- package/skills/template-registry/video_dsl/runtime/__init__.py +0 -1
- package/skills/template-registry/video_dsl/runtime/dsl_validator.py +108 -35
- package/skills/template-registry/video_dsl/runtime/narration_budget.py +129 -0
- package/skills/template-registry/video_dsl/schema/video-dsl-v1alpha1.json +1 -1
- package/skills/template-registry/scripts/check_contracts.py +0 -179
- package/skills/template-registry/scripts/sync_registry.py +0 -44
- package/skills/template-registry/scripts/template_paths.py +0 -72
- package/skills/template-registry/scripts/verify_props_contract.py +0 -273
- package/skills/template-registry/video_dsl/runtime/template_binder.py +0 -87
package/README.md
CHANGED
|
@@ -259,32 +259,20 @@ npm run smoke
|
|
|
259
259
|
# CLI unit tests (argv parser + skill schema)
|
|
260
260
|
npm run test:cli
|
|
261
261
|
|
|
262
|
-
# ──
|
|
263
|
-
#
|
|
264
|
-
#
|
|
265
|
-
# template
|
|
266
|
-
npm run
|
|
267
|
-
|
|
268
|
-
#
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
#
|
|
275
|
-
|
|
276
|
-
npm run test:narration-speed
|
|
277
|
-
|
|
278
|
-
# L3 — selective, real Remotion render of chosen template(s). NOT full-library
|
|
279
|
-
# by design (rendering is slow + needs asset generation). Catches crashes /
|
|
280
|
-
# black frames the static layers can't. Needs PRIV_TOKEN for narration TTS;
|
|
281
|
-
# pass --stub-image-url to keep image-heavy renders cheap.
|
|
282
|
-
npm run check:render -- --template spotlight-card --frame
|
|
283
|
-
|
|
284
|
-
# Template defs come from the registry (ab-api HTTP + /tmp cache), exactly like
|
|
285
|
-
# render_video in production. A warm cache or reachable ab-api is required.
|
|
286
|
-
# In a multi-repo checkout the scripts auto-detect template-library; override
|
|
287
|
-
# with AB_TEMPLATES_SRC=/path/to/template-library/packages/templates/src.
|
|
262
|
+
# ── Unit tests for the Python pipeline ──
|
|
263
|
+
# Fully offline. They exercise the CLI's generic mechanisms against the synthetic
|
|
264
|
+
# templates in scripts/fixtures/ — never against a real template library, so
|
|
265
|
+
# template changes cannot turn them red.
|
|
266
|
+
npm run test:unit
|
|
267
|
+
|
|
268
|
+
# ── Integration: run the CLI against a real template library ──
|
|
269
|
+
# Validation, binding, RenderPlan and props contracts over every template (and
|
|
270
|
+
# dsl-example) in AB_TEMPLATES_SRC. Names no template; needs no ab-api or login.
|
|
271
|
+
AB_TEMPLATES_SRC=/path/to/template-library/packages/templates/src npm run integration
|
|
272
|
+
|
|
273
|
+
# Selective real Remotion render of chosen template(s). Needs PRIV_TOKEN for
|
|
274
|
+
# narration TTS; pass --stub-image-url to keep image-heavy renders cheap.
|
|
275
|
+
AB_TEMPLATES_SRC=... npm run integration:render -- --template <id> --frame
|
|
288
276
|
|
|
289
277
|
# Python skills forward --help to the underlying script:
|
|
290
278
|
remixmate gen-script --help
|
package/dist/manifest.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"schemaVersion": 1,
|
|
3
|
-
"version": "0.9.
|
|
4
|
-
"generatedAt": "2026-09-
|
|
3
|
+
"version": "0.9.52",
|
|
4
|
+
"generatedAt": "2026-09-27T14:43:11.552Z",
|
|
5
5
|
"skills": [
|
|
6
6
|
{
|
|
7
7
|
"id": "doc-parse",
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
"tier": "tool",
|
|
10
10
|
"category": "consuming",
|
|
11
11
|
"title": "Document Extraction",
|
|
12
|
-
"summary": "Extract a document (.pptx) into structured data: per-page title and body text, the speaker notes, and — when asked for the layout layer — the exact shapes, coordinates, colors and animation beats of every slide. Runs server-side (ab-render /extract); nothing is installed locally and no model is involved, so the result is deterministic. Two layers via `fields`: `content` is lightweight text (title / text / rawNotes) and is what you want for narration or for summarizing a deck; `layout` additionally returns the slide payload that
|
|
12
|
+
"summary": "Extract a document (.pptx) into structured data: per-page title and body text, the speaker notes, and — when asked for the layout layer — the exact shapes, coordinates, colors and animation beats of every slide. Runs server-side (ab-render /extract); nothing is installed locally and no model is involved, so the result is deterministic. Two layers via `fields`: `content` is lightweight text (title / text / rawNotes) and is what you want for narration or for summarizing a deck; `layout` additionally returns the per-page slide payload that templates declaring capabilities.payloadHydration (docKind pptx) render one-to-one from the original file — it embeds every icon as a data URI and is large, so it is written to a file rather than printed.",
|
|
13
13
|
"triggers": [
|
|
14
14
|
"Turn this PPT / slide deck into a video",
|
|
15
15
|
"Extract text or speaker notes from a .pptx",
|
|
@@ -437,7 +437,7 @@
|
|
|
437
437
|
},
|
|
438
438
|
"scenes": {
|
|
439
439
|
"type": "number",
|
|
440
|
-
"description": "Scene count. Only applies to templates with capabilities.sceneStrategy=arc (the default); single / fixed / countdown templates own their own count. Bounded below by 3 and above by the template's capabilities.maxScenes (default 20) — a clamp is reported on stderr and is NOT an error, so always read the real count back from the returned DSL (scenes[].length) before describing the script to the user. For a template that maps one scene per source page (
|
|
440
|
+
"description": "Scene count. Only applies to templates with capabilities.sceneStrategy=arc (the default); single / fixed / countdown templates own their own count. Bounded below by 3 and above by the template's capabilities.maxScenes (default 20) — a clamp is reported on stderr and is NOT an error, so always read the real count back from the returned DSL (scenes[].length) before describing the script to the user. For a template that maps one scene per source page (capabilities.payloadHydration), pass the page count you saw in doc_parse's summary."
|
|
441
441
|
},
|
|
442
442
|
"voice_id": {
|
|
443
443
|
"type": "string",
|
|
@@ -447,13 +447,21 @@
|
|
|
447
447
|
"type": "number",
|
|
448
448
|
"description": "Narration speech rate, 0.5-2.0 (default 1.0). Stored at global.narration.speed and applied when prepare_video_assets synthesizes the voice-over. **Must** pass when the user picked a speech rate — a template's 语速 / speed parameter arrives as a multiplier like 1.1, and without this flag it is silently dropped and the narration comes back at 1.0. Above ~1.3 subtitle alignment drifts and the delivery sounds mechanical; suggest shortening the script instead of raising it further."
|
|
449
449
|
},
|
|
450
|
+
"variant": {
|
|
451
|
+
"type": "string",
|
|
452
|
+
"description": "Template visual variant id (a key of the template's variants; template_registry lists them as variantIds). Written to meta.templateVariant, which beats the fuzzy `style` match. **Must** pass when the user picked a style: the replicate card shows it as `名字(id)` — e.g. `视觉风格: 紫色科技(tech-purple)` or `板面颜色: 黑板(black-board)` → pass the id in the parentheses. Without it the choice is silently dropped. Requires template_id; an id the template does not declare is rejected."
|
|
453
|
+
},
|
|
454
|
+
"background_preset": {
|
|
455
|
+
"type": "string",
|
|
456
|
+
"description": "Scene background preset, one of the template's customPayloadSchema.background.preset values. Written to every scene's customPayload.background.preset. **Must** pass when the user picked a background: the replicate card shows `背景样式: 纯色(solid)` → pass `solid`. Without it the scene keeps the template default. Requires template_id; unsupported values are rejected. `image` / `video` also need customPayload.background.assetRef on each scene."
|
|
457
|
+
},
|
|
450
458
|
"template_id": {
|
|
451
459
|
"type": "string",
|
|
452
|
-
"description": "Template id (
|
|
460
|
+
"description": "Template id (from template_registry). The template owns outputLanguage and may also declare a defaultVoiceId; both flow into the produced DSL."
|
|
453
461
|
},
|
|
454
462
|
"source_document_url": {
|
|
455
463
|
"type": "string",
|
|
456
|
-
"description": "Source document URL (.pptx) for a template that builds its per-scene picture from that document (capabilities.payloadHydration
|
|
464
|
+
"description": "Source document URL (.pptx) for a template that builds its per-scene picture from that document (capabilities.payloadHydration). **Must** pass it for such a template: it is recorded at meta.sourceDocument, and prepare_video_assets then fetches the layout itself and fills the customPayload field the template declares (payloadHydration.payloadField) on every scene. Pass the same URL you gave doc_parse. You are NOT expected to author or even read that layout data — one page is hundreds of KB, so it never travels through the conversation; without this reference the DSL is rejected downstream after you have already written all the narration."
|
|
457
465
|
},
|
|
458
466
|
"headline": {
|
|
459
467
|
"type": "string",
|
|
@@ -475,7 +483,7 @@
|
|
|
475
483
|
"items": {
|
|
476
484
|
"type": "string"
|
|
477
485
|
},
|
|
478
|
-
"description": "Per-scene visuals for a visual-overlay template (
|
|
486
|
+
"description": "Per-scene visuals for a visual-overlay template (capabilities.payloadStyle=visual-overlay) — the counterpart of carousel_items for templates that have real scenes. Each element is EITHER a full URL (use that existing asset for the scene, no gen-image call) OR the literal string 'ai' / 'ai:<prompt>' (leave that scene to gen-image; the text after the colon, when present, is the user's own image prompt for it). Mapping is POSITIONAL: element i is scene i, so an AI scene can sit anywhere in the order — pass ['url1','ai','url2'] verbatim when the user arranged it that way. Scenes past the last element still get an AI-generated image, so 2 images still produce a full video. When `scenes` is not set the scene count is raised (never lowered) to fit the elements, so nothing the user arranged is dropped and handing over fewer images does not shrink the video. **Must** pass when the user supplies images for such a template — otherwise their images are ignored entirely and every scene is AI-generated. Do NOT pass for carousel-caption templates; those take carousel_items."
|
|
479
487
|
},
|
|
480
488
|
"caption_lines": {
|
|
481
489
|
"type": "array",
|
|
@@ -520,6 +528,8 @@
|
|
|
520
528
|
"scenes",
|
|
521
529
|
"voice_id",
|
|
522
530
|
"speed",
|
|
531
|
+
"variant",
|
|
532
|
+
"background_preset",
|
|
523
533
|
"template_id",
|
|
524
534
|
"source_document_url",
|
|
525
535
|
"headline",
|
|
@@ -784,7 +794,7 @@
|
|
|
784
794
|
},
|
|
785
795
|
"source_document_url": {
|
|
786
796
|
"type": "string",
|
|
787
|
-
"description": "Source document URL (.pptx) for a template that builds its per-scene picture from that document (capabilities.payloadHydration
|
|
797
|
+
"description": "Source document URL (.pptx) for a template that builds its per-scene picture from that document (capabilities.payloadHydration). Normally unnecessary: gen_script already recorded it at meta.sourceDocument when you passed source_document_url there, and the cached skeleton carries it through. Pass it here only to recover when the skeleton lacks the reference — it saves a full gen_script round trip. The layout data is fetched here and written into the customPayload field the template declares (payloadHydration.payloadField) on every scene; you neither author nor see it."
|
|
788
798
|
},
|
|
789
799
|
"binding_json": {
|
|
790
800
|
"type": "string",
|
|
@@ -934,16 +944,19 @@
|
|
|
934
944
|
},
|
|
935
945
|
"template_id": {
|
|
936
946
|
"type": "string",
|
|
937
|
-
"description": "Return this template's
|
|
947
|
+
"description": "Return this template's full authoring contract instead of the summary list: llmHint in full, customPayloadSchema (every template-specific field incl. the legal slideId values), slotMapping, compositions, variants. Multi-layout templates return slideSchemas as a MENU (per layout: description, field names, card-list key / item fields / maxItems, and whether narration must be split into items) — enough to pick layouts and draft the script. Read it once per conversation; it does not change between turns."
|
|
948
|
+
},
|
|
949
|
+
"slide_id": {
|
|
950
|
+
"type": "array",
|
|
951
|
+
"items": {
|
|
952
|
+
"type": "string"
|
|
953
|
+
},
|
|
954
|
+
"description": "With template_id: return the FULL templateData schema for these layouts (the slideIds you chose from the menu). Call this once, right before writing templateData, with every layout the script uses. 'all' returns every layout (large — avoid)."
|
|
938
955
|
},
|
|
939
956
|
"full": {
|
|
940
957
|
"type": "boolean",
|
|
941
958
|
"description": "List mode only: emit full definitions for every listed template instead of summaries. Rejected when the result would be too large for one tool result — prefer template_id, or narrow with the filter_* parameters."
|
|
942
959
|
},
|
|
943
|
-
"list_examples": {
|
|
944
|
-
"type": "boolean",
|
|
945
|
-
"description": "List the *.dsl.json / *.binding.json reference examples under template-registry/video_dsl/schema/examples/, grouped by templateId. That directory is optional and is not part of the published package, so this is normally empty — to read a template's reference shape, use template_id instead (the registry's own contract)."
|
|
946
|
-
},
|
|
947
960
|
"filter_tag": {
|
|
948
961
|
"type": "string",
|
|
949
962
|
"description": "Show only templates whose styleTags contain (or are contained in) this string. Case-insensitive. E.g. 'tech' / '科普' / 'walkthrough'."
|
|
@@ -972,8 +985,8 @@
|
|
|
972
985
|
"advanced": [
|
|
973
986
|
"list_templates",
|
|
974
987
|
"template_id",
|
|
988
|
+
"slide_id",
|
|
975
989
|
"full",
|
|
976
|
-
"list_examples",
|
|
977
990
|
"filter_tag",
|
|
978
991
|
"filter_aspect",
|
|
979
992
|
"filter_language",
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@remixmate/cli",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.52",
|
|
4
4
|
"description": "AI media generation skills for Claude Code / Codex — 12 skills covering image, video, voice, digital human, web screenshot, web recording, script, template registry, rendering, Jianying export, and video deconstruction.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -20,29 +20,26 @@
|
|
|
20
20
|
"list": "node dist/cli.js --list",
|
|
21
21
|
"smoke": "node scripts/smoke.mjs",
|
|
22
22
|
"test:cli": "npm run build && node --test test/*.test.mjs",
|
|
23
|
-
"test:validators": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-validators.py",
|
|
24
23
|
"test:web-read": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-web-read.py",
|
|
25
24
|
"test:web-read:browser": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-web-read-browser.py",
|
|
26
|
-
"test:template-pipeline": "PYTHONDONTWRITEBYTECODE=1 node dist/cli.js exec -- python3 scripts/test-template-pipeline.py",
|
|
27
|
-
"test:props-contract": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-props-contract.py",
|
|
28
|
-
"test:card-array-keys": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-card-array-keys.py",
|
|
29
25
|
"test:narration-speed": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-narration-speed.py",
|
|
26
|
+
"test:template-look": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-template-look.py",
|
|
30
27
|
"test:narration-tts": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-narration-tts.py",
|
|
31
28
|
"test:render-resume": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-render-resume.py",
|
|
32
29
|
"test:required-props": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-required-props.py",
|
|
33
|
-
"test:render-plan": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-render-plan-snapshot.py",
|
|
34
|
-
"test:render-plan:update": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-render-plan-snapshot.py --update",
|
|
35
|
-
"test:contracts": "PYTHONDONTWRITEBYTECODE=1 python3 skills/template-registry/scripts/check_contracts.py",
|
|
36
|
-
"regress": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/regress.py",
|
|
37
|
-
"regress:update": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/regress.py --update",
|
|
38
|
-
"check:render": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/render-template-check.py",
|
|
39
30
|
"clean": "node scripts/clean.mjs",
|
|
40
31
|
"test:countdown": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-countdown-strategy.py",
|
|
41
32
|
"test:prepare-contracts": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-prepare-contracts.py",
|
|
42
33
|
"test:image-sequence": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-image-sequence.py",
|
|
43
|
-
"test:ranking-count": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-ranking-count.py",
|
|
44
34
|
"test:image-prompt-compose": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-image-prompt-compose.py",
|
|
45
|
-
"test:
|
|
35
|
+
"test:page-contracts": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-page-contracts.py",
|
|
36
|
+
"test:declared-keys": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-declared-keys.py",
|
|
37
|
+
"test:payload-hydration": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-payload-hydration.py",
|
|
38
|
+
"test:sentence-alignment": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-sentence-alignment.py",
|
|
39
|
+
"test:authoring-budget": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/test-authoring-budget.py",
|
|
40
|
+
"test:unit": "npm run -s test:narration-speed && npm run -s test:template-look && npm run -s test:narration-tts && npm run -s test:render-resume && npm run -s test:required-props && npm run -s test:countdown && npm run -s test:prepare-contracts && npm run -s test:image-sequence && npm run -s test:image-prompt-compose && npm run -s test:page-contracts && npm run -s test:declared-keys && npm run -s test:payload-hydration && npm run -s test:sentence-alignment && npm run -s test:authoring-budget",
|
|
41
|
+
"integration": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/integration/regress.py",
|
|
42
|
+
"integration:render": "PYTHONDONTWRITEBYTECODE=1 python3 scripts/integration/render-template-check.py"
|
|
46
43
|
},
|
|
47
44
|
"keywords": [
|
|
48
45
|
"claude-code",
|
|
@@ -5,7 +5,7 @@ description: |
|
|
|
5
5
|
|
|
6
6
|
Two layers, picked with `--fields`:
|
|
7
7
|
- **content** — per-page `title`, `text[]`, and `rawNotes` (the speaker notes). Small, no base64. This is the narration source material.
|
|
8
|
-
- **layout** — the full `slide` payload
|
|
8
|
+
- **layout** — the full per-page `slide` payload: every shape, coordinate, color, clip path and the animation `beats`, plus the `mode` verdict. This is what templates declaring `capabilities.payloadHydration` with `docKind: pptx` render from. Large (icons are embedded as data URIs).
|
|
9
9
|
|
|
10
10
|
Deterministic: the endpoint parses OOXML, it does **not** call a model. Notes come back verbatim — rewriting them into narration is your job, not the endpoint's.
|
|
11
11
|
|
|
@@ -14,7 +14,7 @@ description: |
|
|
|
14
14
|
- Read / extract / summarize a .pptx, pull the text or the speaker notes out of a deck
|
|
15
15
|
- Use a PowerPoint file as source material for a script
|
|
16
16
|
|
|
17
|
-
You call this skill for the **words** (notes, body text) and for the **page inventory** (how many pages, each page's animation tier and beat count). You do not call it to obtain
|
|
17
|
+
You call this skill for the **words** (notes, body text) and for the **page inventory** (how many pages, each page's animation tier and beat count). You do not call it to obtain the layout payload for a template — that payload never passes through you; see "Using the result".
|
|
18
18
|
triggers:
|
|
19
19
|
- Turn this PPT / slide deck into a video
|
|
20
20
|
- Extract text or speaker notes from a .pptx
|
|
@@ -108,12 +108,12 @@ So the picture travels **by reference**:
|
|
|
108
108
|
doc_parse --url "https://…/deck.pptx"
|
|
109
109
|
|
|
110
110
|
# 2. pass the SAME url to gen_script, plus one scene per page
|
|
111
|
-
gen_script --template-id
|
|
111
|
+
gen_script --template-id <templateId> \
|
|
112
112
|
--source-document-url "https://…/deck.pptx" \
|
|
113
113
|
--scenes 23
|
|
114
114
|
```
|
|
115
115
|
|
|
116
|
-
`gen_script` records
|
|
116
|
+
`<templateId>` is a template that declares `capabilities.payloadHydration` with `docKind: pptx`. `gen_script` records the URL at `meta.sourceDocument`, and `prepare_video_assets` then fetches the layout itself and fills every scene's `customPayload.<payloadHydration.payloadField>` before validation. You never see that data, and you never hand-write it.
|
|
117
117
|
|
|
118
118
|
What you *do* own from this result:
|
|
119
119
|
|
|
@@ -124,9 +124,9 @@ What you *do* own from this result:
|
|
|
124
124
|
| `mode` + beat count per page | cut the narration into that many sentences |
|
|
125
125
|
| `flags` | tell the user which elements the extractor could not reconstruct |
|
|
126
126
|
|
|
127
|
-
Do not
|
|
127
|
+
Do not second-guess `mode`: the verdict comes from the extractor, which can see reconstruction problems you cannot (`modeReason` says why). Whether and how a template lets you override it is in that template's `llmHint`.
|
|
128
128
|
|
|
129
|
-
> If `prepare_video_assets` ever
|
|
129
|
+
> If `prepare_video_assets` ever reports that hydrated field as `(required)`, the missing thing is not the payload — it is the **reference**. Re-run `gen_script` with `--source-document-url`, or pass `source_document_url` to `prepare_video_assets` to recover without regenerating the script.
|
|
130
130
|
|
|
131
131
|
**For the words** — `rawNotes` and `text` are two different kinds of raw material, and they need different treatment:
|
|
132
132
|
|
|
@@ -16,8 +16,9 @@
|
|
|
16
16
|
|
|
17
17
|
⚠️ 关于 layout 层的用法,看清楚这一条:**它不是给你读的**。在 ab-agent 里模型没有
|
|
18
18
|
任何文件读取工具(这是有意的安全收窄),所以落盘的那个文件你打不开。需要 layout 的
|
|
19
|
-
|
|
20
|
-
|
|
19
|
+
模板(声明了 capabilities.payloadHydration)按引用取数:把同一个文档 URL 传给
|
|
20
|
+
``gen_script --source-document-url``,渲染管线会自己把每一页填进模板声明的 customPayload
|
|
21
|
+
字段。见 SKILL.md「Using the result」。
|
|
21
22
|
|
|
22
23
|
必要环境变量(由 ab-agent 自动注入):
|
|
23
24
|
RENDER_API_URL ab-render 服务地址(默认 https://api-render.remixmate.ai)
|
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
"tier": "tool",
|
|
5
5
|
"category": "consuming",
|
|
6
6
|
"title": "Document Extraction",
|
|
7
|
-
"description": "Extract a document (.pptx) into structured data: per-page title and body text, the speaker notes, and — when asked for the layout layer — the exact shapes, coordinates, colors and animation beats of every slide. Runs server-side (ab-render /extract); nothing is installed locally and no model is involved, so the result is deterministic. Two layers via `fields`: `content` is lightweight text (title / text / rawNotes) and is what you want for narration or for summarizing a deck; `layout` additionally returns the slide payload that
|
|
7
|
+
"description": "Extract a document (.pptx) into structured data: per-page title and body text, the speaker notes, and — when asked for the layout layer — the exact shapes, coordinates, colors and animation beats of every slide. Runs server-side (ab-render /extract); nothing is installed locally and no model is involved, so the result is deterministic. Two layers via `fields`: `content` is lightweight text (title / text / rawNotes) and is what you want for narration or for summarizing a deck; `layout` additionally returns the per-page slide payload that templates declaring capabilities.payloadHydration (docKind pptx) render one-to-one from the original file — it embeds every icon as a data URI and is large, so it is written to a file rather than printed.",
|
|
8
8
|
"auth": "required",
|
|
9
9
|
"envVars": [
|
|
10
10
|
"RENDER_API_URL",
|
|
@@ -39,7 +39,7 @@ Turns a user-supplied **topic** into a **Video DSL v1alpha1** JSON describing wh
|
|
|
39
39
|
|
|
40
40
|
The full schema is `template-registry` skill's `video_dsl/schema/video-dsl-v1alpha1.json`.
|
|
41
41
|
|
|
42
|
-
**Per-template reference DSLs do not ship with this package.** `video_dsl/schema/examples/` is a local-only directory: it is absent from the published npm package (and has never existed in the repo)
|
|
42
|
+
**Per-template reference DSLs do not ship with this package.** `video_dsl/schema/examples/` is a local-only directory: it is absent from the published npm package (and has never existed in the repo). Do not go looking for examples — the authoritative per-template contract ships inside the registry's `template.json`, see below.
|
|
43
43
|
|
|
44
44
|
## Agent behavior: DSL generation when a template is selected
|
|
45
45
|
|
|
@@ -52,24 +52,23 @@ The intent is to avoid the failure mode where "the generic DSL looks compatible
|
|
|
52
52
|
|
|
53
53
|
### Mandatory steps
|
|
54
54
|
|
|
55
|
-
1. Read the requested template's full definition from the registry — `template_registry` with `template_id=<the id>`
|
|
55
|
+
1. Read the requested template's full definition from the registry — `template_registry` with `template_id=<the id>` emits that one template's contract. This is the shipping source of truth. Read it **once per conversation**: it is still in context on the next turn, so do not re-fetch it after the user confirms the script. Do **not** ask for every template's full definition (`list_templates=true` + `full=true`): the whole registry is far more JSON than one tool result can carry, and the call fails instead of returning the contract.
|
|
56
56
|
2. Read its `llmHint` end to end — that is where the template states how its on-screen text and its layouts must be authored.
|
|
57
57
|
3. Read its `customPayloadSchema`: every template-specific field lives there, including the enum of legal `customPayload.slideId` values for multi-layout templates.
|
|
58
|
-
3b. If the template ships `slideSchemas` (multi-layout templates do), **author `templateData` straight from `slideSchemas[slideId]`** — it is the only place the per-layout field names exist. Do not infer them: the same concept is named `name` in one layout, `label` in another and `era` in a third, and a wrong name is dropped in silence (see below).
|
|
58
|
+
3b. If the template ships `slideSchemas` (multi-layout templates do), step 1 returns them as a **menu** — per layout: description, field names, card-list key / item fields / `maxItems`, and a `narration` note. Pick layouts and draft the script from the menu. Then, before writing `templateData`, call `template_registry` once with `template_id` and `slide_id=[every chosen layout]`, and **author `templateData` straight from those `slideSchemas[slideId]`** — it is the only place the per-layout field names exist. Do not infer them: the same concept is named `name` in one layout, `label` in another and `era` in a third, and a wrong name is dropped in silence (see below).
|
|
59
59
|
4. Combine that with the template's `slotMapping`, `requiredProps`, `optionalProps`, `propExtractors`, `assetRequirements`, `supportedAspectRatios`, `supportedDurations`, `constraints`, and `scenePatterns`.
|
|
60
60
|
5. Generate the DSL in the template's native shape — not the generic DSL shape.
|
|
61
61
|
6. Make sure the DSL explicitly contains every template-specific field, e.g. `templateData.words`, `customPayload.slideId`, `visuals.avatar.assetRef`.
|
|
62
|
-
7.
|
|
63
|
-
8. After generation, run the schema check and show the script summary to the user for confirmation.
|
|
62
|
+
7. After generation, run the schema check and show the script summary to the user for confirmation.
|
|
64
63
|
|
|
65
64
|
### Document-driven templates: pass the source URL, don't author the payload
|
|
66
65
|
|
|
67
|
-
Some templates build each scene's picture from a source file rather than from JSON you write. They declare it as `capabilities.payloadHydration`
|
|
66
|
+
Some templates build each scene's picture from a source file rather than from JSON you write. They declare it as `capabilities.payloadHydration` — `payloadField` names the `customPayload` field the pipeline fills, `docKind` the kind of source file (e.g. `.pptx`).
|
|
68
67
|
|
|
69
68
|
For those, pass **`--source-document-url`** — the same URL you gave `doc_parse`:
|
|
70
69
|
|
|
71
70
|
```bash
|
|
72
|
-
gen_script --template-id
|
|
71
|
+
gen_script --template-id <templateId> \
|
|
73
72
|
--source-document-url "https://…/deck.pptx" \
|
|
74
73
|
--scenes 23 --duration 600
|
|
75
74
|
```
|
|
@@ -93,7 +92,7 @@ If the count you need is above the template's maximum, say so and stop — do no
|
|
|
93
92
|
|
|
94
93
|
### Multi-layout templates: pick a layout per scene
|
|
95
94
|
|
|
96
|
-
When a template's `customPayloadSchema.slideId.enum` holds more than one value
|
|
95
|
+
When a template's `customPayloadSchema.slideId.enum` holds more than one value, **every scene must name one**, chosen to fit that scene's information shape — a comparison, a timeline, a code walkthrough and a set of numbers are four different layouts.
|
|
97
96
|
|
|
98
97
|
`gen_script.py` cannot make that choice: it emits the placeholder `"slideId": "__CHOOSE_SLIDE__"`, and the DSL validator rejects any DSL that still carries it, listing the legal values in the error. Replace every placeholder, and do not reuse one layout for the whole video.
|
|
99
98
|
|
|
@@ -103,7 +102,7 @@ A missing `slideId` is the quietest failure in the pipeline: the renderer drops
|
|
|
103
102
|
|
|
104
103
|
Picking the layout is only half of it. Each layout reads its own field names out of `templateData`, and a field the layout does not read is **dropped without a word** — that slot renders empty, or shows the component's built-in placeholder text, and the render still reports success.
|
|
105
104
|
|
|
106
|
-
This has shipped broken videos: a 7-scene deck
|
|
105
|
+
This has shipped broken videos: a 7-scene deck guessed generic names (`title`, `description`, `ctaText`) where the chosen layouts read different ones. Five of seven scenes rendered as empty cards and placeholder copy — after paying for TTS and the render. Which names each layout reads is the template's fact; the template's `llmHint` calls out the usual traps.
|
|
107
106
|
|
|
108
107
|
So: **copy the field names out of `slideSchemas[slideId]`.** The DSL validator now rejects unknown field names before any asset is generated and tells you the declared ones, but that check only fires for templates that ship `slideSchemas` — reading the contract is still the primary move, not the fallback.
|
|
109
108
|
|
|
@@ -257,7 +256,7 @@ Every URL the user gave becomes one carousel item, in order. How long each item
|
|
|
257
256
|
## Agent behavior: user-supplied images for scene-based templates (`--scene-images`)
|
|
258
257
|
|
|
259
258
|
The section above covers `carousel-caption` templates. Templates whose `payloadStyle` is
|
|
260
|
-
`visual-overlay`
|
|
259
|
+
`visual-overlay` have **real scenes**, each with its own background
|
|
261
260
|
image and its own narration — for those, user-supplied images go through `--scene-images`,
|
|
262
261
|
not `--carousel-items`.
|
|
263
262
|
|
|
@@ -342,13 +341,13 @@ describes an `ai:` prompt as if it were a picture you have seen.
|
|
|
342
341
|
|
|
343
342
|
### Command example
|
|
344
343
|
|
|
345
|
-
User: "
|
|
344
|
+
User: "用这个模板做一条讲 RAG 的视频,配图用这两张"(a `visual-overlay` template)
|
|
346
345
|
+ two asset lines with `desc:`
|
|
347
346
|
|
|
348
347
|
```bash
|
|
349
348
|
python3 <SkillDir>/scripts/gen_script.py \
|
|
350
349
|
--topic "三分钟看懂 RAG" \
|
|
351
|
-
--template-id
|
|
350
|
+
--template-id <templateId> \
|
|
352
351
|
--scene-images "https://cdn.example.com/a.png" \
|
|
353
352
|
--scene-images "ai: 赛博朋克风格的服务器机房" \
|
|
354
353
|
--scene-images "https://cdn.example.com/b.png"
|
|
@@ -388,14 +387,14 @@ Typical phrasings (non-exhaustive):
|
|
|
388
387
|
```bash
|
|
389
388
|
python3 <SkillDir>/scripts/gen_script.py \
|
|
390
389
|
--topic "English picture-book story" \
|
|
391
|
-
--template-id
|
|
390
|
+
--template-id <templateId> \
|
|
392
391
|
--stub-image-url "https://cdn.example.com/placeholder.jpg"
|
|
393
392
|
```
|
|
394
393
|
|
|
395
394
|
```bash
|
|
396
395
|
python3 <SkillDir>/scripts/gen_script.py \
|
|
397
396
|
--topic "GitHub project tour" \
|
|
398
|
-
--template-id
|
|
397
|
+
--template-id <templateId> \
|
|
399
398
|
--stub-image-url "https://cdn.example.com/img.jpg" \
|
|
400
399
|
--stub-video-url "https://cdn.example.com/demo.mp4"
|
|
401
400
|
```
|
|
@@ -407,7 +406,7 @@ Env vars `STUB_IMAGE_URL` / `STUB_VIDEO_URL` also work — their priority is low
|
|
|
407
406
|
1. **Script path**: read the system-injected `Base directory for this skill: <path>` as `<SkillDir>`. Every command below uses `<SkillDir>/scripts/gen_script.py`; never hard-code paths.
|
|
408
407
|
2. **Understand the request**: extract topic, duration, style, audience, etc.
|
|
409
408
|
3. **If the user specifies a template, read it first**: load the matching `template.json` via `template-registry` and confirm its template-specific fields and asset requirements. Do not skip this step.
|
|
410
|
-
4. **Generate the DSL**: pick the
|
|
409
|
+
4. **Generate the DSL**: pick the flags the target template's `capabilities` call for (see *Common CLI flags*); the template's `llmHint` says what to fill in afterwards.
|
|
411
410
|
5. **Schema validation**: make sure the output DSL conforms to v1alpha1.
|
|
412
411
|
6. **User confirmation**: show the script summary and wait for confirmation, per the rules above.
|
|
413
412
|
|
|
@@ -454,6 +453,8 @@ python3 <SkillDir>/scripts/gen_script.py \
|
|
|
454
453
|
| `--scenes` | Scene count. | auto-planned |
|
|
455
454
|
| `--voice-id` | Narration voice id. | resolved from template + language fallback |
|
|
456
455
|
| `--speed` | Narration speech rate (0.5–2.0), written to `global.narration.speed` and applied when `prepare_video_assets` calls gen-voice. Pass it whenever the user picked a speed (the replicate form's 语速 field sends one). Above ~1.3 subtitle alignment drifts and the delivery turns mechanical — shorten the script instead. | `1.0` |
|
|
456
|
+
| `--variant` | Template visual variant id (a key of the template's `variants`; `template-registry --list-templates` prints them as `variantIds`). Written to `meta.templateVariant`, which takes precedence over the fuzzy `--style` match. Pass it whenever the user picked a style — the replicate card shows it as `名字(id)`, e.g. `视觉风格: 紫色科技(tech-purple)`; pass the id in the parentheses. Requires `--template-id`; unknown ids are rejected. | template default |
|
|
457
|
+
| `--background-preset` | Scene background preset, one of the template's `customPayloadSchema.background.preset` values. Written to every scene's `customPayload.background.preset` (other keys of the template's default background are kept). Pass it whenever the user picked a background, e.g. `背景样式: 纯色(solid)` → `solid`. `image` / `video` also need `customPayload.background.assetRef`. Requires `--template-id`; unsupported values are rejected. | template default |
|
|
457
458
|
| `--allow-digital-human` | Whether to allow digital-human assets. | off |
|
|
458
459
|
| `--allow-ai-video` | Whether to allow AI-generated video assets. | off |
|
|
459
460
|
| `--validate` | Validate-only mode: only validate the input DSL. | — |
|
|
@@ -462,7 +463,7 @@ python3 <SkillDir>/scripts/gen_script.py \
|
|
|
462
463
|
| `--stub-image-url` | Test mode: every image AssetRef is written as existing + generated + this URL, no prompt (env: `STUB_IMAGE_URL`). | — |
|
|
463
464
|
| `--stub-video-url` | Test mode: every video AssetRef is written as existing + generated + this URL, no prompt (env: `STUB_VIDEO_URL`). | — |
|
|
464
465
|
| `--carousel-items` | Repeatable. Media URL placed directly into `customPayload.carousel.items` for `carousel-caption` templates. Bypasses gen-image. **Required** for `carousel-caption` templates when the user supplies images. | — |
|
|
465
|
-
| `--scene-images` | Repeatable. One scene's visual for a **`visual-overlay`** template
|
|
466
|
+
| `--scene-images` | Repeatable. One scene's visual for a **`visual-overlay`** template, mapped **positionally**. Each entry is a URL (that existing asset) or `ai` / `ai:<prompt>` (that scene goes to gen-image, optionally with the user's own prompt) — so an AI scene can sit anywhere in the order. Scenes past the last entry still get AI images. Scene count is raised (never lowered) to fit the entries when `--scenes` is absent. **Required** whenever the user supplies images for such a template — omitting it silently ignores every image they gave. | — |
|
|
466
467
|
| `--caption-lines` | Repeatable. On-screen caption line for `carousel-caption` templates → `customPayload.caption.lines`. Line limits and `**emphasis**` support are declared per template (see its `llmHint`). **Required** for `durationStrategy: fit-caption` templates — write the lines yourself if the user did not supply them. Optional only for `fit-images` templates. | — |
|
|
467
468
|
|
|
468
469
|
## DSL generation principles
|
|
@@ -471,7 +472,7 @@ Apply the following principles when producing the DSL:
|
|
|
471
472
|
|
|
472
473
|
1. **Narrative-first**: opening hook → point expansion → CTA close.
|
|
473
474
|
2. **Assets are declared, not executed**: every asset starts as `AssetRef` with `status: planned`; no generation API call from this skill.
|
|
474
|
-
3. **Narration drives duration**:
|
|
475
|
+
3. **Narration drives duration**: the rendered length of a scene is its narration's reading time plus a short tail, not `scenes[].duration`. Each skeleton placeholder states the scene's budget (e.g. `(8s,整段≤32 字;分行旁白每多一行约少 4 字)`) — write inside it. The budget comes from `video_dsl/runtime/narration_budget.py` (≈4.5 zh chars/s at 1.0× speed, ~0.9s pause per extra narration line, 1.5s tail per scene unless the template declares `capabilities.defaultTailPadSec`). If you go over, report the longer estimated length at script confirmation; `prepare_video_assets` also warns when the estimate exceeds `meta.targetDuration` by more than 30%.
|
|
475
476
|
4. **One complete thought per line — do NOT pre-split at subtitle width.** Separate lines inside `audio.narration.text` with `\n`, and put **one full clause or sentence** on each line (roughly 15–30 Chinese characters, or an equivalent number of English words). A `\n` is a hard cut in the *audio*: TTS treats each line as its own segment and returns a timestamp for it, so lines should fall where a human would actually pause for breath.
|
|
476
477
|
|
|
477
478
|
**Do not cut mid-phrase to make lines "subtitle-sized".** The renderer splits each line into on-screen subtitles itself, and it does that better than you can from the text alone: it measures the real glyph widths for the actual font size and frame (`HNSW` is not as wide as four Chinese characters), knows how many characters fit on one line of *this* template and aspect ratio, and picks cut points by punctuation strength (sentence end → colon → comma → enumeration comma → space). It also guarantees no wrapping. Pre-splitting takes all of that away and just makes the captions choppy.
|
|
@@ -524,4 +525,4 @@ prefix and version — not a short alias:
|
|
|
524
525
|
|------|---------|
|
|
525
526
|
| `gen_script.py` | Core script — produces the Video DSL JSON from a topic. |
|
|
526
527
|
|
|
527
|
-
The authoritative per-template contract is the registry's `template.json` (`llmHint` + `customPayloadSchema` + `slideSchemas` + `slotMapping`) — everything you need to author against is in there, so read it rather than inferring field names from the template's name or from another template.
|
|
528
|
+
The authoritative per-template contract is the registry's `template.json` (`llmHint` + `customPayloadSchema` + `slideSchemas` + `slotMapping`) — everything you need to author against is in there, so read it rather than inferring field names from the template's name or from another template.
|