@mevdragon/vidfarm-devcli 0.2.7 → 0.2.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.example +9 -0
- package/GETTING_STARTED.developers.md +1 -30
- package/README.md +22 -5
- package/SKILL.developer.md +464 -12
- package/auto-create-templates/AUTO_CREATE_TEMPLATES.md +475 -0
- package/auto-create-templates/extractor-system-prompt.md +113 -0
- package/auto-create-templates/production-graph.schema.json +276 -0
- package/auto-create-templates/runbook.md +140 -0
- package/auto-create-templates/template-plan.schema.json +230 -0
- package/auto-create-templates/template-planner-prompt.md +75 -0
- package/dist/src/account-pages-legacy.js +9064 -0
- package/dist/src/account-pages.js +41 -620
- package/dist/src/app.js +7173 -494
- package/dist/src/cli.js +525 -29
- package/dist/src/config.js +14 -7
- package/dist/src/context.js +51 -35
- package/dist/src/db.js +1049 -55
- package/dist/src/dev-app-legacy.js +693 -0
- package/dist/src/dev-app.js +4 -904
- package/dist/src/domain.js +1 -1
- package/dist/src/editor-chat-history.js +72 -0
- package/dist/src/editor-chat.js +202 -0
- package/dist/src/frontend/flockposter-cache-store.js +116 -0
- package/dist/src/frontend/homepage-client.js +114 -0
- package/dist/src/frontend/homepage-shared.js +3 -0
- package/dist/src/frontend/homepage-store.js +27 -0
- package/dist/src/frontend/homepage-view.js +147 -0
- package/dist/src/frontend/page-runtime-client.js +100 -0
- package/dist/src/frontend/page-runtime-store.js +8 -0
- package/dist/src/frontend/template-editor-chat.js +2261 -0
- package/dist/src/homepage.js +695 -339
- package/dist/src/lib/template-paths.js +10 -4
- package/dist/src/lib/template-style-options.js +95 -15
- package/dist/src/page-runtime.js +1 -0
- package/dist/src/page-shell.js +941 -0
- package/dist/src/primitive-context.js +163 -0
- package/dist/src/primitive-registry.js +340 -0
- package/dist/src/primitive-sdk.js +3 -0
- package/dist/src/primitives/remotion/html-image.js +28 -0
- package/dist/src/react-page-shell.js +34 -0
- package/dist/src/ready-post-schedule-component.js +1514 -0
- package/dist/src/registry.js +44 -18
- package/dist/src/runtime.js +6 -1
- package/dist/src/services/api-call-history.js +245 -0
- package/dist/src/services/auth.js +27 -9
- package/dist/src/services/billing.js +248 -9
- package/dist/src/services/chat-threads.js +88 -0
- package/dist/src/services/job-logs.js +10 -3
- package/dist/src/services/jobs.js +13 -2
- package/dist/src/services/providers.js +944 -55
- package/dist/src/services/rate-limits.js +236 -0
- package/dist/src/services/remotion.js +143 -16
- package/dist/src/services/storage.js +23 -8
- package/dist/src/services/template-certification.js +26 -3
- package/dist/src/services/template-loader.js +19 -1
- package/dist/src/services/template-sources.js +316 -7
- package/dist/src/template-editor-pages.js +2309 -0
- package/dist/src/template-editor-shell.js +1507 -0
- package/dist/src/worker.js +120 -22
- package/package.json +30 -6
- package/public/assets/homepage-app.js +54 -0
- package/public/assets/homepage-client-app.js +54 -0
- package/public/assets/page-runtime-client-app.js +68 -0
- package/dist/templates/template_0000/src/lib/images.js +0 -202
- package/dist/templates/template_0000/src/remotion/Root.js +0 -33
- package/dist/templates/template_0000/src/remotion/index.js +0 -3
- package/dist/templates/template_0000/src/sdk.js +0 -3
- package/dist/templates/template_0000/src/style-options.js +0 -51
- package/dist/templates/template_0000/src/template-dna.js +0 -9
- package/dist/templates/template_0000/src/template.js +0 -1441
- package/templates/template_0000/README.md +0 -77
- package/templates/template_0000/SKILL.md +0 -225
- package/templates/template_0000/assets/Abel-Regular.ttf +0 -0
- package/templates/template_0000/assets/DMSerifDisplay-Regular.ttf +0 -0
- package/templates/template_0000/assets/Montserrat[wght].ttf +0 -0
- package/templates/template_0000/assets/SourceCodePro[wght].ttf +0 -0
- package/templates/template_0000/assets/TikTokSans-SemiBold.ttf +0 -0
- package/templates/template_0000/assets/Yesteryear-Regular.ttf +0 -0
- package/templates/template_0000/composition.json +0 -11
- package/templates/template_0000/package.json +0 -28
- package/templates/template_0000/research/preview/.gitkeep +0 -1
- package/templates/template_0000/research/source_notes.md +0 -7
- package/templates/template_0000/src/lib/images.js +0 -202
- package/templates/template_0000/src/lib/images.ts +0 -241
- package/templates/template_0000/src/remotion/Root.js +0 -33
- package/templates/template_0000/src/remotion/Root.tsx +0 -75
- package/templates/template_0000/src/remotion/index.js +0 -3
- package/templates/template_0000/src/remotion/index.tsx +0 -4
- package/templates/template_0000/src/sdk.js +0 -3
- package/templates/template_0000/src/sdk.ts +0 -122
- package/templates/template_0000/src/style-options.js +0 -51
- package/templates/template_0000/src/style-options.ts +0 -60
- package/templates/template_0000/src/template-dna.js +0 -9
- package/templates/template_0000/src/template-dna.ts +0 -15
- package/templates/template_0000/src/template.js +0 -1441
- package/templates/template_0000/src/template.ts +0 -2042
- package/templates/template_0000/template.config.json +0 -21
- package/templates/template_0000/tsconfig.json +0 -19
|
@@ -0,0 +1,475 @@
|
|
|
1
|
+
# Auto-Create Templates Harness
|
|
2
|
+
|
|
3
|
+
This is the harness for turning an arbitrary TikTok or short-form video into a reusable Vidfarm template that can generate the same format in bulk.
|
|
4
|
+
|
|
5
|
+
## Start Here
|
|
6
|
+
|
|
7
|
+
Give a Codex agent these two things:
|
|
8
|
+
|
|
9
|
+
1. this entry file: `auto-create-templates/AUTO_CREATE_TEMPLATES.md`
|
|
10
|
+
2. one media reference folder, preferably shaped like:
|
|
11
|
+
|
|
12
|
+
```txt
|
|
13
|
+
drafts/<format_slug>/
|
|
14
|
+
source.md
|
|
15
|
+
preview/
|
|
16
|
+
source.mp4
|
|
17
|
+
screenshot-01.png
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
Then ask:
|
|
21
|
+
|
|
22
|
+
```txt
|
|
23
|
+
Use auto-create-templates/AUTO_CREATE_TEMPLATES.md as the harness. Analyze drafts/<format_slug>/ as the source format, create the run artifacts, generate the production graph and template plan, then scaffold and implement a runnable Vidfarm template.
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
The media folder can contain a video, screenshots, reference images, or all of them. If `source.md` is missing, the agent should create one with the original URL, creator, adaptation goal, and any constraints it can infer.
|
|
27
|
+
|
|
28
|
+
The core idea is to avoid asking an agent to "clone this video" in one prompt. The harness splits the job into staged extraction:
|
|
29
|
+
|
|
30
|
+
1. inspect source media and source notes
|
|
31
|
+
2. run Vidfarm viral and visual DNA analysis
|
|
32
|
+
3. decide which visual traits must be preserved because they support the viral mechanism
|
|
33
|
+
4. extract a production graph from the actual video
|
|
34
|
+
5. classify which parts are fixed format rules vs replaceable inputs
|
|
35
|
+
6. choose a generation strategy: pure prompt, Remotion assembly, AI image storyboard, AI video, source clip swapping, or a hybrid
|
|
36
|
+
7. generate a Vidfarm template plan with schemas, operations, jobs, prompts, and validation cases
|
|
37
|
+
|
|
38
|
+
Supporting harness files live beside this entry file in [auto-create-templates](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/vidfarm/auto-create-templates).
|
|
39
|
+
|
|
40
|
+
## Sample Read
|
|
41
|
+
|
|
42
|
+
I sampled the local drafts across the spectrum:
|
|
43
|
+
|
|
44
|
+
- Easy:
|
|
45
|
+
- `cute_diagram_grid`: one static generated infographic with repeated layout slots, title, illustrations, and short explanatory labels.
|
|
46
|
+
- `history_swap_news`: one meme/news image where the reusable mechanic is text replacement inside a familiar historical/news surface.
|
|
47
|
+
- `before_after_slides` and `video_in_chat`: short screen-native videos where most of the format is slot timing, overlay text, and a small number of replaceable media regions.
|
|
48
|
+
- Medium:
|
|
49
|
+
- `app_demo_captioned`: app interface recording with caption track and visible UI state changes across time.
|
|
50
|
+
- `face_upclose_yapper`: close-face UGC speech with top hook, word-level captions, and performance pacing driven by audio.
|
|
51
|
+
- `unique_sketch_character`: recurring illustrated character/vibe, scene beats, and animation continuity rather than simple text swaps.
|
|
52
|
+
- Hard:
|
|
53
|
+
- `movie_montage`: external clip montage plus reaction/meter overlays and app interstitials.
|
|
54
|
+
- `claymation_ads`: full AI-video storyboard with character continuity, product/problem narrative, captions, and scene-to-scene action.
|
|
55
|
+
- `full_animated_science_explainer`: horizontal drawn explainer with abstract diagrams, narration, transitions, and educational sequencing.
|
|
56
|
+
|
|
57
|
+
This implies three production tiers:
|
|
58
|
+
|
|
59
|
+
- `easy`: one image or a few fixed visual slots. Usually pure image generation, reference image editing, HTML/CSS compositing, or Remotion slideshow.
|
|
60
|
+
- `medium`: timed media replacement. Usually Remotion plus STT/TTS/captions, exact overlay timing, screen recordings, UI mocks, and optional AI image inserts.
|
|
61
|
+
- `hard`: multi-stage production. Sometimes script -> storyboard -> assets -> AI image/video generation -> Remotion edit -> captions/audio/master. Other times the source is too visually fluid for reliable Remotion reconstruction and should be treated as an AI-native prompt/reference template.
|
|
62
|
+
- `full_ai_character_video`: recurring AI-generated characters across scenes. Usually character sprite/reference cards -> image storyboard -> scene video generation -> Remotion stitch/caption/master.
|
|
63
|
+
|
|
64
|
+
## What To Extract
|
|
65
|
+
|
|
66
|
+
The obvious primitives are captions, TTS, STT, scene snipping/replacement, pointing/circling/magnifying. The harness should also extract these:
|
|
67
|
+
|
|
68
|
+
- hook mechanic: first-frame promise, curiosity gap, shock, status signal, objection, or payoff setup
|
|
69
|
+
- viral steering contract: the non-negotiable distribution psychology that every generated output must preserve
|
|
70
|
+
- visual preservation decision: which visual traits are essential, supportive, optional, or irrelevant based on the viral DNA
|
|
71
|
+
- format container: chat, app screen, map, notification, yearbook, bodycam, news, diagram, ranking, montage, list, interview, vlog, or cinematic scene
|
|
72
|
+
- shot list: ordered scene beats with start/end times, source asset type, camera framing, and purpose
|
|
73
|
+
- edit rhythm: cut frequency, beat drops, zooms, pauses, jump cuts, freeze frames, scroll speed, and loop point
|
|
74
|
+
- audio plan: music bed, narrator, on-camera voice, dialogue, sound effects, silence, and sync points
|
|
75
|
+
- caption style: font, case, color, outline, background chip, placement, max words per line, word-level highlighting, safe areas
|
|
76
|
+
- timing controls: total duration, per-scene duration, per-slide duration, caption group duration, hold time, transition duration, zoom duration, loop point, and minimum/maximum slot length
|
|
77
|
+
- typography controls: TikTok-native font choice, fallback font stack, font size, weight, line height, outline/shadow, background chip, word highlight style, and whether the caller can override them
|
|
78
|
+
- overlay inventory: arrows, circles, magnifiers, stickers, screenshots, reaction meters, emoji, app chrome, progress bars, counters, comments, and lower thirds
|
|
79
|
+
- layout anchors: bounding boxes for faces, products, captions, buttons, UI slots, and safe/unsafe zones
|
|
80
|
+
- replaceable variables: product, person, pain point, website, app screen, object, character, text, location, statistic, clip, or CTA
|
|
81
|
+
- invariant rules: what must stay consistent for the format to remain recognizable
|
|
82
|
+
- adaptive rules: what can change per brand without breaking the format
|
|
83
|
+
- asset requirements: exact user-uploaded references vs AI-generated assets vs external clips vs rendered UI
|
|
84
|
+
- continuity requirements: character identity, wardrobe, room, lighting, prop placement, visual style, and voice
|
|
85
|
+
- character bible: character names, roles, facial/body traits, wardrobe, sprite/reference cards, expression range, poses, props, negative prompts, and consistency constraints
|
|
86
|
+
- storyboard requirements: start/end frame prompts, keyframe images, scene-to-scene continuity notes, camera/action notes, and which frames feed video generation
|
|
87
|
+
- production dependencies: Remotion only, image model, video model, layout analysis, TTS, STT, OCR, background removal, object tracking, or media search
|
|
88
|
+
- architecture fit: whether the format is better as deterministic Remotion assembly, AI-native prompt/reference generation, or a hybrid
|
|
89
|
+
- compliance risks: copyrighted media dependency, celebrity likeness, medical/financial claims, unsafe camera/bodycam contexts, and platform UI imitation
|
|
90
|
+
- validation signals: first-frame legibility, caption readability, subject safe framing, slot replacement correctness, timing fidelity, audio sync, and renderability
|
|
91
|
+
|
|
92
|
+
## Pipeline
|
|
93
|
+
|
|
94
|
+
### 1. Intake
|
|
95
|
+
|
|
96
|
+
Put the source into a clean draft folder:
|
|
97
|
+
|
|
98
|
+
```txt
|
|
99
|
+
drafts/<format_slug>/
|
|
100
|
+
source.md
|
|
101
|
+
preview/
|
|
102
|
+
source.mp4
|
|
103
|
+
screenshot-01.png
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
`source.md` should include:
|
|
107
|
+
|
|
108
|
+
- original URL
|
|
109
|
+
- creator/account if known
|
|
110
|
+
- product/category being adapted, if any
|
|
111
|
+
- what the human believes is repeatable
|
|
112
|
+
- any constraints such as "must use product photo" or "no AI video"
|
|
113
|
+
|
|
114
|
+
### 2. Prepare Analyzer Artifacts
|
|
115
|
+
|
|
116
|
+
For every video, create analyzer artifacts before asking AI for a template plan:
|
|
117
|
+
|
|
118
|
+
```txt
|
|
119
|
+
auto-create-templates/runs/<run_id>/
|
|
120
|
+
source/
|
|
121
|
+
frames/contact-sheet.jpg
|
|
122
|
+
frames/frame-0001.jpg
|
|
123
|
+
audio/source.wav
|
|
124
|
+
transcripts/stt.json
|
|
125
|
+
ocr/frame_text.json
|
|
126
|
+
motion/scene-cuts.json
|
|
127
|
+
dna/viral_dna.extraction.json
|
|
128
|
+
dna/visual_dna.extraction.json
|
|
129
|
+
extraction/production_graph.json
|
|
130
|
+
plan/template_plan.json
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Minimum useful extraction:
|
|
134
|
+
|
|
135
|
+
- `ffprobe` metadata: duration, resolution, fps, audio presence
|
|
136
|
+
- contact sheet: 6-12 frames across the video
|
|
137
|
+
- scene cuts: start/end/duration and representative frame
|
|
138
|
+
- STT transcript with word timestamps when speech exists
|
|
139
|
+
- OCR pass on representative frames
|
|
140
|
+
- object/layout pass for faces, products, UI panes, captions, and overlay regions
|
|
141
|
+
- audio summary: narrator/music/sfx and approximate sync events
|
|
142
|
+
- timing table: every meaningful scene, caption, overlay, zoom, highlight, hold, and transition with exact or inferred start/end times
|
|
143
|
+
- font inventory: visible creator-authored caption/overlay fonts, approximate TikTok font match, styling, and fallback stack
|
|
144
|
+
|
|
145
|
+
### 3. Run Viral And Visual DNA
|
|
146
|
+
|
|
147
|
+
Use the existing Vidfarm analyzer before the production graph step:
|
|
148
|
+
|
|
149
|
+
```bash
|
|
150
|
+
npx @mevdragon/vidfarm-devcli analyze-viral-dna --template-dir ./templates/<template_dir>
|
|
151
|
+
npx @mevdragon/vidfarm-devcli analyze-visual-dna --template-dir ./templates/<template_dir>
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
The underlying pattern is based on:
|
|
155
|
+
|
|
156
|
+
- [1-analyze-viral-dna.mjs](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/tiktok-winning-formats/src/scripts/1-analyze-viral-dna.mjs)
|
|
157
|
+
- [1b-analyze-visual-dns.mjs](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/tiktok-winning-formats/src/scripts/1b-analyze-visual-dns.mjs)
|
|
158
|
+
|
|
159
|
+
Those scripts upload preview media to Gemini, ask for strict JSON, and store extraction results. For auto-creation, treat viral DNA and visual DNA as upstream context, not the final template plan.
|
|
160
|
+
|
|
161
|
+
Use viral DNA for why the format works. Use visual DNA for what the viewer recognizes. Use the production graph for how to recreate it.
|
|
162
|
+
|
|
163
|
+
### 3b. Decide Viral Steering And Visual Preservation
|
|
164
|
+
|
|
165
|
+
Before planning the template, create a short preservation decision:
|
|
166
|
+
|
|
167
|
+
```json
|
|
168
|
+
{
|
|
169
|
+
"viral_steering": {
|
|
170
|
+
"must_preserve": [
|
|
171
|
+
"specific hook psychology",
|
|
172
|
+
"curiosity or tension pattern",
|
|
173
|
+
"payoff structure",
|
|
174
|
+
"pacing behavior that creates retention"
|
|
175
|
+
],
|
|
176
|
+
"must_not_dilute": [
|
|
177
|
+
"things that would make the output generic or less clickable"
|
|
178
|
+
]
|
|
179
|
+
},
|
|
180
|
+
"visual_preservation": {
|
|
181
|
+
"importance": "critical | supportive | optional | low",
|
|
182
|
+
"preserve_because": "how the visual trait reinforces the viral mechanism",
|
|
183
|
+
"preserve": [
|
|
184
|
+
"only visual traits that materially support the viral DNA"
|
|
185
|
+
],
|
|
186
|
+
"can_change": [
|
|
187
|
+
"visual traits that are not part of the viral engine"
|
|
188
|
+
]
|
|
189
|
+
}
|
|
190
|
+
}
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
This decision is viral-first. A visual trait is `critical` only when changing it would damage the hook, recognition, curiosity, credibility, emotional tension, or retention loop. Some formats are presentation-driven and must preserve visual layout tightly. Other formats win because of premise, pacing, or voice, and their visuals can vary more freely.
|
|
194
|
+
|
|
195
|
+
The generated template must persist this decision in:
|
|
196
|
+
|
|
197
|
+
- `about.viral_dna`
|
|
198
|
+
- `about.visual_dna`
|
|
199
|
+
- `about.proposal_generator`
|
|
200
|
+
- `about.skeleton_prompt`
|
|
201
|
+
- `sample_prompts`
|
|
202
|
+
- template-local `SKILL.md`
|
|
203
|
+
- prompt constants or comments near the top of `src/template.ts`
|
|
204
|
+
- runtime plan/checklist output where the template has a `create_plan` operation
|
|
205
|
+
|
|
206
|
+
### 4. Extract Production Graph
|
|
207
|
+
|
|
208
|
+
Run the prompt in [extractor-system-prompt.md](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/vidfarm/auto-create-templates/extractor-system-prompt.md) against:
|
|
209
|
+
|
|
210
|
+
- source notes
|
|
211
|
+
- source media or contact sheet
|
|
212
|
+
- frame OCR
|
|
213
|
+
- transcript
|
|
214
|
+
- scene cuts
|
|
215
|
+
- viral DNA
|
|
216
|
+
- visual DNA
|
|
217
|
+
- viral steering and visual preservation decision
|
|
218
|
+
|
|
219
|
+
The output should match [production-graph.schema.json](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/vidfarm/auto-create-templates/production-graph.schema.json).
|
|
220
|
+
|
|
221
|
+
The graph is the critical artifact. It should answer:
|
|
222
|
+
|
|
223
|
+
- what is the timeline?
|
|
224
|
+
- which clips/images/text/audio are fixed?
|
|
225
|
+
- which viral mechanics must be steered in every generated output?
|
|
226
|
+
- which visual traits are actually preservation-worthy because of the viral mechanics?
|
|
227
|
+
- which elements are replaceable?
|
|
228
|
+
- which elements are generated?
|
|
229
|
+
- which elements need exact timing?
|
|
230
|
+
- which parts are best implemented in Remotion?
|
|
231
|
+
- which parts would become brittle if implemented in Remotion?
|
|
232
|
+
- whether the format should instead be generated as a prompt/reference with AI-native image or video generation
|
|
233
|
+
- which parts require AI image/video generation?
|
|
234
|
+
|
|
235
|
+
### 5. Choose Template Strategy
|
|
236
|
+
|
|
237
|
+
Use this decision table, but make an explicit architecture-fit decision instead of defaulting to Remotion:
|
|
238
|
+
|
|
239
|
+
| Source pattern | Recommended strategy |
|
|
240
|
+
| --- | --- |
|
|
241
|
+
| Single static infographic, meme, diagram, notification, poster, fake UI with simple geometry | AI image or HTML/CSS/Canvas composition, optional image reference attachment |
|
|
242
|
+
| Dense illustrated poster, painterly infographic, organic collage, or high-detail aesthetic image | AI-native prompt/reference image generation; avoid rebuilding every region in Remotion |
|
|
243
|
+
| Static multi-slide carousel | structured slide manifest -> image generation/compositing -> optional Remotion video |
|
|
244
|
+
| Screenshot or app interface with simple motion | Remotion-rendered UI shell plus replaceable media/text slots |
|
|
245
|
+
| App demo / screen recording | user-provided clip or generated UI recording -> Remotion captions/zooms/highlights |
|
|
246
|
+
| Face talking-head with captions | STT/TTS or user audio -> caption renderer -> optional B-roll slots |
|
|
247
|
+
| UGC reaction/demo | shot list with user-provided or generated clips, captions, overlays, jump cuts |
|
|
248
|
+
| Character animation / clay / anime / Pixar-like with complex motion, acting, camera, or texture | character sprite cards -> image storyboard -> generated video scenes -> Remotion stitch/captions/audio |
|
|
249
|
+
| Movie montage / celebrity montage | avoid copyrighted source dependency by making it a generic montage slot template with user-supplied clips or generated parody-safe scenes |
|
|
250
|
+
| Explainer animation with simple diagrams | script -> visual metaphor list -> Remotion/Canvas diagrams -> narration -> edit |
|
|
251
|
+
| Explainer animation with hand-drawn morphs, organic transitions, or cinematic continuity | script -> storyboard -> AI-native image/video generation -> Remotion only as final edit |
|
|
252
|
+
|
|
253
|
+
#### Remotion Ceiling Heuristics
|
|
254
|
+
|
|
255
|
+
Choose AI-native prompt/reference generation when the format has several of these traits:
|
|
256
|
+
|
|
257
|
+
- organic or painterly visual details where every frame would require many hand-positioned layers
|
|
258
|
+
- complex character acting, facial expression, body motion, or physics
|
|
259
|
+
- camera moves that rely on generated perspective, parallax, handheld realism, or cinematic continuity
|
|
260
|
+
- style consistency matters more than exact bounding boxes
|
|
261
|
+
- the source wins because the whole generated image/video feels native, not because it uses a deterministic UI layout
|
|
262
|
+
- recreating it in Remotion would require dozens of brittle scene-specific layers, masks, paths, or keyframes
|
|
263
|
+
- small visual deviations are acceptable as long as the viral hook, premise, pacing, and style family survive
|
|
264
|
+
|
|
265
|
+
Choose Remotion when the format has several of these traits:
|
|
266
|
+
|
|
267
|
+
- UI containers, screenshots, chat bubbles, scorecards, diagrams, slides, labels, or other geometric layout
|
|
268
|
+
- exact caption timing, word highlighting, arrows, circles, magnifiers, or overlays are the main reusable mechanic
|
|
269
|
+
- source clips/images are user-supplied and only need deterministic trimming, cropping, zooming, or captioning
|
|
270
|
+
- brand/product slots need reliable positioning across hundreds of outputs
|
|
271
|
+
- validation depends on exact safe areas, text legibility, and repeatable timing
|
|
272
|
+
|
|
273
|
+
Use hybrid when AI should generate complex source assets but Remotion should assemble, caption, crop, time, and package them.
|
|
274
|
+
|
|
275
|
+
#### Full AI Character Video Detection
|
|
276
|
+
|
|
277
|
+
Detect `full_ai_character_video` when the source has several of these traits:
|
|
278
|
+
|
|
279
|
+
- recurring generated character or mascot across multiple scenes
|
|
280
|
+
- consistent face, body, wardrobe, material style, or prop relationship
|
|
281
|
+
- multiple story beats where the same character performs actions, reacts, or changes location
|
|
282
|
+
- animation style is part of recognition, such as claymation, anime, Pixar-like, game cutscene, illustrated skit, or puppet/doll style
|
|
283
|
+
- video cannot be reduced to one static image or deterministic UI shell
|
|
284
|
+
- quality depends on maintaining identity through a sequence, not just generating one good shot
|
|
285
|
+
|
|
286
|
+
For this case, the template should usually expose this operation family:
|
|
287
|
+
|
|
288
|
+
- `create_character_cards`: generate or normalize character sprite/reference cards, including front/side/close-up/expression/pose references.
|
|
289
|
+
- `create_storyboard`: generate a scene-by-scene image storyboard from the viral steering contract, character cards, and user brief.
|
|
290
|
+
- `generate_scene_videos`: turn storyboard frames/keyframes into short AI video clips with scene-level action and camera prompts.
|
|
291
|
+
- `stitch_video`: assemble clips, narration, captions, audio, transitions, CTA, and final safe-area formatting.
|
|
292
|
+
|
|
293
|
+
Use Remotion for `stitch_video`, captions, audio, timing, crop/fit, and light overlays. Do not use Remotion to fake character acting frame by frame.
|
|
294
|
+
|
|
295
|
+
### 6. Plan Vidfarm Template
|
|
296
|
+
|
|
297
|
+
Run [template-planner-prompt.md](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/vidfarm/auto-create-templates/template-planner-prompt.md) with the production graph. It should output a template plan containing:
|
|
298
|
+
|
|
299
|
+
- slug and title
|
|
300
|
+
- `about.viral_dna`
|
|
301
|
+
- `about.visual_dna`
|
|
302
|
+
- `about.proposal_generator`
|
|
303
|
+
- `about.skeleton_prompt`
|
|
304
|
+
- `about.sample_prompts`
|
|
305
|
+
- viral steering rules that every prompt/output must satisfy
|
|
306
|
+
- visual preservation rules, with an explicit importance level
|
|
307
|
+
- config schema
|
|
308
|
+
- operation list
|
|
309
|
+
- operation input schemas
|
|
310
|
+
- workflow/job plan
|
|
311
|
+
- Remotion composition plan if needed
|
|
312
|
+
- architecture fit decision, including why Remotion is or is not appropriate
|
|
313
|
+
- AI-native prompt/reference plan when deterministic assembly would be brittle
|
|
314
|
+
- character-consistency workflow when the source is a full AI character video
|
|
315
|
+
- provider requirements
|
|
316
|
+
- validation checklist
|
|
317
|
+
- risk notes
|
|
318
|
+
|
|
319
|
+
The output should match [template-plan.schema.json](/Users/localghost/Projects/OfficeX/OfficeX/ZoomGTM/vidfarm/auto-create-templates/template-plan.schema.json).
|
|
320
|
+
|
|
321
|
+
### 7. Generate And Validate
|
|
322
|
+
|
|
323
|
+
Scaffold normally:
|
|
324
|
+
|
|
325
|
+
```bash
|
|
326
|
+
npx @mevdragon/vidfarm-devcli generate-template \
|
|
327
|
+
--slug-id <slug_id> \
|
|
328
|
+
--template-dir ./templates/vidfarm_template_<slug> \
|
|
329
|
+
--link-to-original "<source_url>" \
|
|
330
|
+
--source-notes-path ./drafts/<format_slug>/source.md \
|
|
331
|
+
--source-preview-dir ./drafts/<format_slug>/preview
|
|
332
|
+
```
|
|
333
|
+
|
|
334
|
+
Then implement from the plan:
|
|
335
|
+
|
|
336
|
+
- public operations stay coarse-grained and async
|
|
337
|
+
- viral DNA is persisted as a steering contract, not just descriptive metadata; generation prompts should actively enforce the hook, tension, payoff, pacing, and audience psychology that make the format work
|
|
338
|
+
- visual DNA is persisted according to its preservation decision; preserve tightly only when visuals support the viral mechanism, otherwise allow controlled variation
|
|
339
|
+
- timing parameters are exposed when useful instead of being buried as constants; at minimum support `duration_ms` per scene/slide/media slot and sensible min/max clamping
|
|
340
|
+
- caption/font parameters prefer TikTok-native defaults and can be configured when the format's recognizability depends on typography
|
|
341
|
+
- generated media goes through `ctx.storage`
|
|
342
|
+
- video render props reference artifact URLs, not base64
|
|
343
|
+
- Remotion is used only where it is the right tool: deterministic layout, captions, overlays, timing, simple motion, and final edit
|
|
344
|
+
- if the format exceeds Remotion's practical complexity ceiling, preserve it as an AI-native prompt/reference template and use Remotion only for optional wrapping, captions, or stitching
|
|
345
|
+
- image/video providers are used for assets that cannot be deterministically composed
|
|
346
|
+
- template-local `SKILL.md` must describe operations, payloads, provider caveats, and examples
|
|
347
|
+
|
|
348
|
+
Validate:
|
|
349
|
+
|
|
350
|
+
```bash
|
|
351
|
+
npx @mevdragon/vidfarm-devcli validate-template --template-id <slug_id>
|
|
352
|
+
npx @mevdragon/vidfarm-devcli session
|
|
353
|
+
```
|
|
354
|
+
|
|
355
|
+
Then run at least one real local operation and inspect artifacts.
|
|
356
|
+
|
|
357
|
+
## Operation Patterns
|
|
358
|
+
|
|
359
|
+
Most auto-created templates should expose two paths:
|
|
360
|
+
|
|
361
|
+
- `create_plan`: turn a prompt/product brief into a concrete script, shot list, asset list, and render plan.
|
|
362
|
+
- `render_video` or `create_video`: generate/assemble the final output from explicit assets or from the plan.
|
|
363
|
+
|
|
364
|
+
`create_plan` must include a viral compliance section in its output:
|
|
365
|
+
|
|
366
|
+
- which viral DNA rules the plan is using
|
|
367
|
+
- how the first frame/hook preserves the source format's click driver
|
|
368
|
+
- how the middle preserves tension, curiosity, status, proof, or emotional movement
|
|
369
|
+
- how the ending preserves payoff, loop, share/save behavior, or CTA
|
|
370
|
+
- whether visual DNA is critical, supportive, optional, or low-importance for this format
|
|
371
|
+
- which visual rules are enforced vs allowed to vary
|
|
372
|
+
|
|
373
|
+
Every video-producing template should make timing explicit. The default input shape should allow:
|
|
374
|
+
|
|
375
|
+
- `duration_ms` for each scene, slide, or media slot
|
|
376
|
+
- `caption_timing_mode`: transcript-derived, fixed beat groups, or manual timestamps
|
|
377
|
+
- `transition_ms`
|
|
378
|
+
- `hold_ms`
|
|
379
|
+
- optional `total_duration_ms` target with proportional scaling
|
|
380
|
+
- min/max duration clamps so batch generation cannot create broken pacing
|
|
381
|
+
|
|
382
|
+
Every captioned template should strongly prefer TikTok-native typography. Use checked-in fonts when available, especially `TikTokSans-SemiBold.ttf` from the reference template assets, with practical fallbacks such as Montserrat or system sans. Expose caption styling in config only where it is useful; otherwise default to the closest source-matching TikTok style.
|
|
383
|
+
|
|
384
|
+
For simple image formats:
|
|
385
|
+
|
|
386
|
+
- `create_image`: exact/director-controlled path
|
|
387
|
+
- `create_image_auto`: AI-brief path that writes exact text and image prompts first
|
|
388
|
+
|
|
389
|
+
For slideshow formats:
|
|
390
|
+
|
|
391
|
+
- `create_slides`: returns frames, manifest, and render input
|
|
392
|
+
- `render_video`: turns frames or mixed media slots into MP4
|
|
393
|
+
|
|
394
|
+
For complex video formats:
|
|
395
|
+
|
|
396
|
+
- `create_script`: hook, narration/dialogue, beat list
|
|
397
|
+
- `create_storyboard`: frame prompts and continuity notes
|
|
398
|
+
- `generate_assets`: AI images/video clips/audio
|
|
399
|
+
- `render_video`: deterministic edit and overlays
|
|
400
|
+
|
|
401
|
+
For full AI character videos:
|
|
402
|
+
|
|
403
|
+
- `create_character_cards`: produces character reference images and a character bible manifest
|
|
404
|
+
- `create_storyboard`: produces ordered storyboard frames/keyframes with scene prompts, camera notes, action notes, and continuity notes
|
|
405
|
+
- `generate_scene_videos`: produces one video clip per storyboard scene, using character cards and storyboard frames as references where the provider supports it
|
|
406
|
+
- `stitch_video`: combines generated clips, voice/music/sfx, captions, transitions, and CTA into the final video
|
|
407
|
+
|
|
408
|
+
The output manifest should keep character cards, storyboard frames, scene clips, audio, and final render as separate artifacts so the workflow can be resumed or revised at any stage.
|
|
409
|
+
|
|
410
|
+
For AI-native reference formats:
|
|
411
|
+
|
|
412
|
+
- `create_prompt_plan`: convert a product/audience brief into a source-faithful prompt/reference contract
|
|
413
|
+
- `generate_reference_asset`: produce the final image or video using source DNA, attachments, and style constraints
|
|
414
|
+
- `wrap_video` or `render_video`: optional Remotion wrapper for platform-safe duration, captions, music, CTA, or end card
|
|
415
|
+
|
|
416
|
+
## Production Graph Principles
|
|
417
|
+
|
|
418
|
+
The graph should be precise enough that an agent can implement a template without rewatching the source.
|
|
419
|
+
|
|
420
|
+
Every timeline item should include:
|
|
421
|
+
|
|
422
|
+
- `start_ms` and `end_ms`
|
|
423
|
+
- `duration_ms`
|
|
424
|
+
- `role`: hook, setup, proof, gag, reveal, CTA, transition, b-roll, UI, caption, overlay, audio
|
|
425
|
+
- `asset_kind`: source_video, generated_video, generated_image, uploaded_image, uploaded_video, rendered_ui, text, audio, shape, effect
|
|
426
|
+
- `replaceability`: fixed, variable, optional, generated
|
|
427
|
+
- `implementation`: remotion, ai_image, ai_video, tts, stt, user_upload, external_clip, html_canvas
|
|
428
|
+
- `architecture_fit`: remotion, ai_native_reference, hybrid, or manual_review
|
|
429
|
+
- `character_consistency`: none, light, strict, or identity_locked
|
|
430
|
+
- `safe_area_notes`
|
|
431
|
+
- `validation_notes`
|
|
432
|
+
|
|
433
|
+
The graph should separate four layers:
|
|
434
|
+
|
|
435
|
+
- viral layer: hook, audience psychology, tension, payoff, retention, share/save reason
|
|
436
|
+
- narrative layer: what story or argument happens
|
|
437
|
+
- visual layer: frames, shots, UI, layout, overlays
|
|
438
|
+
- audio layer: speech, captions, music, sfx
|
|
439
|
+
- interaction layer: pointing, circling, zooming, highlighting, magnifying, camera movement
|
|
440
|
+
|
|
441
|
+
## Guardrails
|
|
442
|
+
|
|
443
|
+
- Do not turn copyrighted movie/TV clips into a template dependency. Model those formats as user-supplied clip slots or generate parody-safe replacements.
|
|
444
|
+
- Do not require AWS, S3, or cloud Remotion setup for local template creation.
|
|
445
|
+
- Do not write binary media into JSON outputs. Store with `ctx.storage` and return URLs.
|
|
446
|
+
- Do not invent REST endpoints. Use standard Vidfarm operations and jobs.
|
|
447
|
+
- Do not treat DNA analysis as sufficient. It explains why and what; the production graph explains how.
|
|
448
|
+
- Do not bury viral DNA as static `about` text only. It must steer prompts, plans, operation outputs, validation checks, and template usage docs.
|
|
449
|
+
- Do not preserve visual traits blindly. First decide whether each trait materially supports the viral DNA; then preserve only the traits that matter.
|
|
450
|
+
- Do not overuse AI video when Remotion or HTML composition can reproduce the format deterministically.
|
|
451
|
+
- Do not force Remotion onto formats whose value comes from organic generated style, character acting, cinematic movement, or dense visual texture. Use AI-native prompt/reference generation when it will be more robust.
|
|
452
|
+
- Do not ignore audio. Many medium/hard formats are paced by voice, captions, and beat timing.
|
|
453
|
+
|
|
454
|
+
## Suggested Automation Command
|
|
455
|
+
|
|
456
|
+
A future CLI command could wrap the harness like this:
|
|
457
|
+
|
|
458
|
+
```bash
|
|
459
|
+
npx @mevdragon/vidfarm-devcli auto-create-template \
|
|
460
|
+
--source ./drafts/<format_slug>/preview/source.mp4 \
|
|
461
|
+
--source-notes ./drafts/<format_slug>/source.md \
|
|
462
|
+
--slug-id <slug_id> \
|
|
463
|
+
--template-dir ./templates/vidfarm_template_<slug> \
|
|
464
|
+
--run-dir ./auto-create-templates/runs/<run_id>
|
|
465
|
+
```
|
|
466
|
+
|
|
467
|
+
Internally it should:
|
|
468
|
+
|
|
469
|
+
1. stage source media
|
|
470
|
+
2. extract media artifacts
|
|
471
|
+
3. scaffold template with DNA analysis
|
|
472
|
+
4. generate production graph
|
|
473
|
+
5. generate template plan
|
|
474
|
+
6. optionally open a code-generation pass to implement the template
|
|
475
|
+
7. validate template metadata and run smoke jobs
|
|
@@ -0,0 +1,113 @@
|
|
|
1
|
+
# Production Graph Extractor Prompt
|
|
2
|
+
|
|
3
|
+
You are a Vidfarm template production analyst.
|
|
4
|
+
|
|
5
|
+
Your job is to inspect one TikTok or short-form source format and extract the repeatable production process as structured JSON. Do not merely describe the video. Convert it into a reusable template recipe that can be implemented with Vidfarm operations, Remotion, AI image/video generation, TTS, STT, OCR, and user-supplied media slots.
|
|
6
|
+
|
|
7
|
+
The viral DNA is the primary steering contract. The visual DNA is only preservation-critical when it materially supports the viral mechanism. Decide this explicitly before extracting implementation details.
|
|
8
|
+
|
|
9
|
+
Return only JSON matching `production-graph.schema.json`.
|
|
10
|
+
|
|
11
|
+
## Inputs You May Receive
|
|
12
|
+
|
|
13
|
+
- source notes
|
|
14
|
+
- original URL and creator
|
|
15
|
+
- full source video or preview images
|
|
16
|
+
- contact sheet
|
|
17
|
+
- representative frames
|
|
18
|
+
- ffprobe metadata
|
|
19
|
+
- scene cut list
|
|
20
|
+
- STT transcript with word timestamps
|
|
21
|
+
- OCR text from frames
|
|
22
|
+
- viral DNA JSON
|
|
23
|
+
- visual DNA JSON
|
|
24
|
+
- viral steering and visual preservation decision, if already produced
|
|
25
|
+
|
|
26
|
+
## Extraction Rules
|
|
27
|
+
|
|
28
|
+
- Treat all supplied media as one format.
|
|
29
|
+
- Separate viral mechanism, visual system, and production implementation.
|
|
30
|
+
- Use viral DNA to decide what the template must optimize for: hook, curiosity, emotional tension, status, proof, payoff, retention, share/save behavior, or native credibility.
|
|
31
|
+
- Evaluate visual DNA through that viral lens. Mark each visual trait as critical, supportive, optional, or low-importance. Do not preserve visual details merely because they are visible.
|
|
32
|
+
- Ignore TikTok platform chrome unless the creator-authored format intentionally imitates an app surface.
|
|
33
|
+
- Identify which elements are fixed, variable, optional, or generated.
|
|
34
|
+
- Extract exact or inferred durations for scenes, captions, overlays, transitions, zooms, holds, and loop points.
|
|
35
|
+
- Treat timing as a reusable control surface. If a source slot has a duration, decide whether future users should be able to override it.
|
|
36
|
+
- Strongly identify TikTok-native typography: font family match, weight, case, outline, shadow, background chip, word highlight, line wrapping, and fallback font stack.
|
|
37
|
+
- Prefer deterministic Remotion/HTML composition for layout, captions, overlays, app UI, and final edit.
|
|
38
|
+
- Do not force Remotion when it would become brittle. If the format depends on organic visual detail, character acting, cinematic continuity, generated texture, painterly style, or complex physical motion, mark it as AI-native prompt/reference or hybrid.
|
|
39
|
+
- Use AI image/video only for assets that require generative content.
|
|
40
|
+
- Use TTS/STT when audio drives timing, narration, or captions.
|
|
41
|
+
- Identify compliance risks and avoid making copyrighted media or celebrity likenesses a required dependency.
|
|
42
|
+
|
|
43
|
+
## Required Analysis Dimensions
|
|
44
|
+
|
|
45
|
+
Extract:
|
|
46
|
+
|
|
47
|
+
- first-frame hook
|
|
48
|
+
- viewer promise and payoff
|
|
49
|
+
- viral steering rules that must survive every adaptation
|
|
50
|
+
- visual preservation importance and rationale
|
|
51
|
+
- timeline beats
|
|
52
|
+
- scene/shot list
|
|
53
|
+
- text overlays and captions
|
|
54
|
+
- audio/narration/music/sfx
|
|
55
|
+
- timing controls and duration ranges
|
|
56
|
+
- font and caption styling controls
|
|
57
|
+
- camera moves and edit rhythm
|
|
58
|
+
- UI/screenshot/app surfaces
|
|
59
|
+
- generated-image or generated-video needs
|
|
60
|
+
- media replacement slots
|
|
61
|
+
- Remotion ceiling risk: what would become too brittle if hand-assembled
|
|
62
|
+
- architecture fit: `remotion`, `ai_native_reference`, `hybrid`, or `manual_review`
|
|
63
|
+
- character bible and identity consistency requirements when the same AI-generated character appears across scenes
|
|
64
|
+
- storyboard/keyframe workflow requirements for full AI character videos
|
|
65
|
+
- pointing/circling/magnifying/highlight effects
|
|
66
|
+
- transition rules
|
|
67
|
+
- safe areas and layout constraints
|
|
68
|
+
- provider/tool dependencies
|
|
69
|
+
- validation checklist
|
|
70
|
+
|
|
71
|
+
## Implementation Classification
|
|
72
|
+
|
|
73
|
+
For every meaningful element choose one implementation:
|
|
74
|
+
|
|
75
|
+
- `remotion`: deterministic layout, captions, overlays, cuts, zooms, UI, final render
|
|
76
|
+
- `html_canvas`: static or frame-based graphic composition
|
|
77
|
+
- `ai_image`: generated still image or edited reference
|
|
78
|
+
- `ai_video`: generated motion clip
|
|
79
|
+
- `ai_native_reference`: whole asset or scene generated from source DNA and reference attachments instead of deterministic layer reconstruction
|
|
80
|
+
- `tts`: generated speech
|
|
81
|
+
- `stt`: transcription and word timing
|
|
82
|
+
- `ocr`: source text extraction
|
|
83
|
+
- `user_upload`: caller must provide media
|
|
84
|
+
- `external_clip`: caller-provided clip; do not fetch copyrighted media automatically
|
|
85
|
+
- `media_search`: optional search/reference gathering
|
|
86
|
+
- `manual_review`: human must approve or adjust
|
|
87
|
+
|
|
88
|
+
## Full AI Character Video Detection
|
|
89
|
+
|
|
90
|
+
If the source appears to be a full AI-generated video with recurring characters, extract it as a character-consistency workflow. Look for consistent faces, wardrobe, body shape, art material, recurring props, repeated locations, or the same mascot/person acting across multiple scenes.
|
|
91
|
+
|
|
92
|
+
When detected, the production graph should require:
|
|
93
|
+
|
|
94
|
+
- `create_character_cards`: character sprite/reference cards and a character bible
|
|
95
|
+
- `create_storyboard`: image storyboard or keyframes for the full video
|
|
96
|
+
- `generate_scene_videos`: one generated video clip per scene
|
|
97
|
+
- `stitch_video`: final Remotion assembly for clips, captions, audio, timing, and overlays
|
|
98
|
+
|
|
99
|
+
Do not recommend frame-by-frame character acting in Remotion. Remotion is the editor/stitcher for this case; AI image/video generation owns the character motion and scene content.
|
|
100
|
+
|
|
101
|
+
## Output Bias
|
|
102
|
+
|
|
103
|
+
Be concrete. A production graph should be implementable.
|
|
104
|
+
|
|
105
|
+
Bad: "Use engaging captions and quick cuts."
|
|
106
|
+
|
|
107
|
+
Good: "Render 2-line white uppercase captions centered at y=178 with black 2px shadow, max 22 characters per line, timed to word groups of 700-1400 ms. Insert 1.08x punch-in zoom on every sentence start."
|
|
108
|
+
|
|
109
|
+
Typography bias: if the format uses creator-authored captions or overlays, default to TikTok-native typography. Prefer TikTok Sans Semibold when available; otherwise choose the closest checked-in or system fallback and record the fallback explicitly.
|
|
110
|
+
|
|
111
|
+
Viral preservation bias: if a choice conflicts between visual sameness and viral effectiveness, preserve the viral mechanism first. Example: if the source wins because of a shocking first-frame claim, preserve the claim structure and pacing even if the background scene changes. If it wins because of a recognizable fake-chat, notification, or game UI container, then visual preservation is likely critical.
|
|
112
|
+
|
|
113
|
+
Architecture bias: Remotion is excellent for deterministic layout and edit assembly, but brittle for highly organic visuals. Prefer AI-native reference prompting when exact layer reconstruction would be more fragile than asking a capable image/video model to regenerate the style and scene directly.
|