@amg555/frameforge-mcp 2.3.1 → 2.3.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. package/.mcp.json +2 -4
  2. package/CHANGELOG.md +20 -1
  3. package/CLAUDE.md +154 -18
  4. package/README.md +16 -12
  5. package/SECURITY.md +1 -3
  6. package/install.sh +1 -1
  7. package/package.json +7 -3
  8. package/plugins/video-pipeline/.claude-plugin/plugin.json +19 -0
  9. package/plugins/video-pipeline/skills/html-renderer/SKILL.md +292 -0
  10. package/plugins/video-pipeline/skills/video-pipeline/SKILL.md +308 -0
  11. package/plugins/video-pipeline/skills/video-pipeline/assets/project/ANIMATION_GUIDE.md +257 -0
  12. package/plugins/video-pipeline/skills/video-pipeline/assets/project/STORYBOARD.md +77 -0
  13. package/plugins/video-pipeline/skills/video-pipeline/assets/project/index.html +80 -0
  14. package/plugins/video-pipeline/skills/video-pipeline/assets/project/pipeline/briefings/diagram-author.md +43 -0
  15. package/plugins/video-pipeline/skills/video-pipeline/assets/project/pipeline/briefings/orchestrator.md +56 -0
  16. package/plugins/video-pipeline/skills/video-pipeline/assets/project/pipeline/briefings/scene-author.md +42 -0
  17. package/plugins/video-pipeline/skills/video-pipeline/assets/project/reel/engine.css +140 -0
  18. package/plugins/video-pipeline/skills/video-pipeline/assets/project/reel/engine.js +870 -0
  19. package/plugins/video-pipeline/skills/video-pipeline/assets/project/reel/virtual-clock.js +138 -0
  20. package/plugins/video-pipeline/skills/video-pipeline/references/mcp-tools.md +167 -0
  21. package/plugins/video-pipeline/skills/video-pipeline/references/providers.md +163 -0
  22. package/scripts/db-migrate.js +112 -0
  23. package/scripts/deep-validate.js +326 -0
  24. package/scripts/generate-initial-assets.js +342 -0
  25. package/scripts/html-renderer/new-project.mjs +408 -0
  26. package/scripts/html-renderer/transcribe.py +294 -0
  27. package/src/asset-library/README.md +2 -2
  28. package/src/asset-library/asset-library.js +1 -1
  29. package/src/cli/doctor.js +31 -1
  30. package/src/cli/make.js +52 -71
  31. package/src/cli/setup-keys.js +24 -19
  32. package/src/fact-checker/review-gate.js +5 -1
  33. package/src/harness/bin/harness.mjs +324 -0
  34. package/src/harness/lib/browser.mjs +137 -0
  35. package/src/harness/lib/capture.mjs +161 -0
  36. package/src/harness/lib/encode.mjs +89 -0
  37. package/src/harness/lib/jobs.mjs +135 -0
  38. package/src/harness/lib/pixels.mjs +137 -0
  39. package/src/harness/lib/sheets.mjs +164 -0
  40. package/src/harness/lib/verify-purity.mjs +119 -0
  41. package/src/harness/test/fixtures/broken-scene.html +27 -0
  42. package/src/harness/test/fixtures/reference-scene.html +23 -0
  43. package/src/harness/test/fixtures/sample-script.json +18 -0
  44. package/src/mcp-server/server.js +318 -66
  45. package/src/pipeline/external-adapter.js +55 -15
  46. package/src/pipeline/providers.js +46 -0
  47. package/src/pipeline/telemetry.js +1 -0
  48. package/src/utils/log-redaction.js +5 -3
  49. package/src/video-renderer/renderer.js +12 -4
  50. package/src/worker/server-render-free.js +2 -3
  51. package/src/workflows/assets.js +20 -43
  52. package/src/workflows/ingest.js +5 -5
@@ -0,0 +1,292 @@
1
+ ---
2
+ name: html-renderer
3
+ description: |
4
+ HTML-as-compositor renderer for FrameForge. Every video frame is a pure
5
+ function of time rendered by a headless browser (Playwright + Chromium)
6
+ and encoded by FFmpeg. Produces exact, pixel-perfect animation — React
7
+ component quality without a React runtime. Use this as an alternative to
8
+ FFmpeg-only zoompan when richer motion graphics are required: kinetic text,
9
+ morphing layouts, spring-physics, counters, progress rings, captions
10
+ synced to audio, and any CSS/SVG animation. Pairs with the FrameForge
11
+ pipeline: script JSON → HTML compositor → frame PNGs → FFmpeg encode.
12
+ Compatible with IBM Bob, Kiro, Cursor, Windsurf, Claude Code, and any
13
+ agentic IDE connected via MCP. Activate when the user asks for richer
14
+ animation, Remotion-style rendering, HTML-based video, or "make it look
15
+ like the reference repo".
16
+ ---
17
+
18
+ # HTML Renderer — FrameForge Extension
19
+
20
+ Renders video frames as HTML pages (one page = one frame clock) using
21
+ Playwright for screenshot capture and FFmpeg for final encoding.
22
+
23
+ Built on deterministic `seek(t)` — every style is a pure function of time `t`.
24
+ No CSS transitions, no `setTimeout`, no frame state that depends on the
25
+ previous frame (Playwright renders frames out of order for parallelism).
26
+
27
+ ---
28
+
29
+ ## When to use this renderer
30
+
31
+ | Use FFmpeg zoompan (default) | Use HTML renderer (this skill) |
32
+ |------------------------------------|--------------------------------------|
33
+ | Batch overnight — speed matters | Animation quality is the priority |
34
+ | Simple Ken-Burns + captions | Kinetic text, counters, data charts |
35
+ | CI/server with no browser | Morphing layouts, spring physics |
36
+ | < 5 min render time required | Creator wants Remotion-style output |
37
+ | Raspberry Pi / low-RAM machine | macOS/Linux with 4GB+ free RAM |
38
+
39
+ ---
40
+
41
+ ## Requirements
42
+
43
+ - Node.js 18+, FFmpeg, `npx playwright install chromium`
44
+ - macOS or Linux recommended (Windows: WSL2)
45
+ - 4 GB+ free RAM (Playwright renders up to 6 subframes per output frame in
46
+ parallel for motion blur)
47
+ - `uv` (Python) for audio processing scripts: `pip install uv` or
48
+ `curl -LsSf https://astral.sh/uv/install.sh | sh`
49
+
50
+ ---
51
+
52
+ ## Setup
53
+
54
+ ```bash
55
+ # Install Playwright chromium (one-time)
56
+ npx playwright install chromium
57
+
58
+ # Scaffold a new HTML-renderer project from a FrameForge script JSON
59
+ node <frameforge-root>/scripts/html-renderer/new-project.mjs \
60
+ --script output/<productionId>-script.json \
61
+ --out output/<productionId>-html/
62
+ ```
63
+
64
+ The scaffold copies `index.html` (compositor), starter `edit.js`, the
65
+ `reel/engine.js` runtime, and all render scripts into the project folder.
66
+
67
+ ---
68
+
69
+ ## Core concept: `seek(t)`
70
+
71
+ ```js
72
+ // edit.js — every element style is computed from t (seconds)
73
+ import { reel, faceAt, layer, text, at, captions } from './reel/engine.js';
74
+
75
+ reel({ width: 1920, height: 1080, fps: 30, duration: 65 });
76
+
77
+ // Title card: fades in at t=0, stays for 3s
78
+ layer('hook').on(t => ({
79
+ opacity: t < 0.3 ? t / 0.3 : t > 2.7 ? (3 - t) / 0.3 : 1,
80
+ }));
81
+
82
+ at(0, text('Why Ancient Humans Started Cooking', {
83
+ font: '72px/1.2 "Archivo"', color: '#fff',
84
+ x: 'center', y: 200,
85
+ }));
86
+
87
+ // Caption synced to FrameForge TTS audio
88
+ captions('audio/outline.txt'); // generated by transcribe.py
89
+ ```
90
+
91
+ **Iron rule:** Every style must be a pure function of `t`. If a layer is
92
+ active during frames 5–15 s and you `return` early for t > 15, it must
93
+ explicitly reset every property it drove — otherwise the last frame's value
94
+ sticks on all later frames.
95
+
96
+ ---
97
+
98
+ ## Workflow (integrates with FrameForge pipeline)
99
+
100
+ ### Step 1: Run FrameForge dry-run to get script JSON
101
+
102
+ ```bash
103
+ npm run make -- --topic "Your Topic" --dry-run --out-dir output/my-topic
104
+ # Produces: output/my-topic/<productionId>-script.json
105
+ ```
106
+
107
+ ### Step 2: Scaffold the HTML compositor project
108
+
109
+ ```bash
110
+ node scripts/html-renderer/new-project.mjs \
111
+ --script output/my-topic/<productionId>-script.json \
112
+ --out output/my-topic/html-render/
113
+ ```
114
+
115
+ The scaffolder reads each scene's `narration`, `caption`,
116
+ `visual_description`, `camera_hint`, and `music_mood` from the script JSON
117
+ and generates a starter `edit.js` with one `layer()` per scene, timed to
118
+ the expected narration duration (estimated at 2.5 words/second).
119
+
120
+ ### Step 3: Gather assets (optional)
121
+
122
+ ```bash
123
+ # Stock footage
124
+ uv run scripts/stock.py video "ancient cooking fire" --portrait
125
+
126
+ # Website screenshot as b-roll
127
+ node scripts/capture.mjs site https://example.com broll-site
128
+
129
+ # X post card
130
+ node scripts/capture.mjs tweet https://x.com/... post-card
131
+ ```
132
+
133
+ ### Step 4: Check visually before full render
134
+
135
+ ```bash
136
+ # Contact sheet — one frame every 0.5s
137
+ node scripts/render.mjs sheet
138
+
139
+ # Specific timestamps
140
+ node scripts/render.mjs stills 0 5.5 12 28 45 60
141
+ ```
142
+
143
+ ### Step 5: Mix audio
144
+
145
+ ```bash
146
+ uv run scripts/mix_audio.py
147
+ # Reads FrameForge TTS outputs, applies -14 LUFS, writes out/audio_mix.m4a
148
+ ```
149
+
150
+ ### Step 6: Full render + verify
151
+
152
+ ```bash
153
+ node scripts/render.mjs full
154
+ uv run scripts/verify.py
155
+ ```
156
+
157
+ Output: `out/<name>.mp4` (1920x1080 or 1080x1920 for Shorts), verified
158
+ against the same FrameForge quality gates (resolution, codec, LUFS, A/V sync).
159
+
160
+ ### Step 7: Re-inject into FrameForge pipeline
161
+
162
+ The rendered MP4 can be injected back into FrameForge via `video_path`:
163
+
164
+ ```json
165
+ {
166
+ "scene_id": 1,
167
+ "video_path": "output/my-topic/html-render/out/video.mp4"
168
+ }
169
+ ```
170
+
171
+ Or pass it directly to `publish_video` via MCP:
172
+
173
+ ```
174
+ publish_video(productionId, masterPath="output/my-topic/html-render/out/video.mp4", channels=["file","webhook"])
175
+ ```
176
+
177
+ ---
178
+
179
+ ## Engine API quick reference
180
+
181
+ | Function | What it does |
182
+ |---|---|
183
+ | `reel({ width, height, fps, duration })` | Configure the compositor |
184
+ | `layer('name').on(t => styles)` | Declare a layer; styles recomputed every frame |
185
+ | `at(t, element)` | Place element at time t |
186
+ | `stagger(t, dt, elements[])` | Place elements with dt spacing |
187
+ | `text(str, opts)` | Animated text with font/color/position |
188
+ | `chip(str, opts)` | Pill-shaped label (tags, captions) |
189
+ | `wallCard(lines[], opts)` | Frosted-glass card over background |
190
+ | `imageCard(src, opts)` | Image with optional caption overlay |
191
+ | `browserCard(url, opts)` | Screenshot-as-card |
192
+ | `terminal(lines[], opts)` | Animated terminal output |
193
+ | `stamp(str, opts)` | Bold diagonal stamp (VERIFIED, SOURCE, etc.) |
194
+ | `codeCard(code, lang, opts)` | Syntax-highlighted code block |
195
+ | `captions(outlinePath)` | Auto-sync captions from whisper outline |
196
+ | `cuePhrase(phrase, offset)` | Word-level trigger with fail-fast transcript check |
197
+ | `cue(sfxName, t, vol)` | Trigger sound effect at time t |
198
+ | `faceAt(layout, t)` | Position talking-head face (FULL, SPLIT, PIP) |
199
+ | `hash(i)` | Deterministic PRNG float [0, 1) for index i |
200
+ | `boilSeed(key, t)` | Deterministic stepped float [0, 1) in 1/12s window |
201
+ | `kf(t, keys, ease)` | Keyframe interpolation with easing curves |
202
+ | `onTwos(t)` | Quantize time to 12 fps (traditional animation on twos) |
203
+ | `seg(t, a, b)` | Clamp t into [0, 1] interval progress |
204
+ | `pulse(t, k)` | Exponential impulse peak at t=0, decaying with rate k |
205
+ | `shots(timeline)` | Time-ordered shot registry fn(t, lt, dur) for parallel render |
206
+ | `on(t => ...)` | Raw per-frame compute (for custom animations) |
207
+
208
+ ---
209
+
210
+ ## Storyboard Gates, Purity Verification & Review Modes
211
+
212
+ ```bash
213
+ # 1. Diagnostic & Setup checks
214
+ node scripts/harness.mjs doctor # Verify Chrome, GPU flags (D3D11/Metal), WebGL context
215
+ node scripts/look.mjs # Pre-render linter (fonts, clipping, NaN transforms)
216
+
217
+ # 2. Purity verification gate (3-point repeat, wander, cold-jump test)
218
+ node scripts/harness.mjs verify # Must pass a===b and a===c across 9 sampled timestamps
219
+
220
+ # 3. Gate stills & review commands
221
+ node scripts/harness.mjs storyboard # 4 keyframes (0%, 33%, 66%, 100%) or validate STORYBOARD.md
222
+ node scripts/harness.mjs sheet # Contact sheet (0.5s interval)
223
+ node scripts/harness.mjs strip 1.0 1.5 # Horizontal frame strip (motion check)
224
+ node scripts/harness.mjs crop 9:16 # Focal-point mobile crop preview
225
+ node scripts/harness.mjs pixels # Blind-spot frame analysis (blank, blown, corner-confined)
226
+
227
+ # 4. Master render & encode
228
+ node scripts/harness.mjs frames # Resumable atomic frame render (.tmp + renameSync)
229
+ node scripts/harness.mjs encode # BT.709 color-tagged master encode + audio mux
230
+ node scripts/harness.mjs all # Complete gate-checked end-to-end production
231
+ node scripts/render.mjs mux # Re-mux audio only (no re-render)
232
+ node scripts/render.mjs kit # Editor kit (Premiere/Resolve XML)
233
+ ```
234
+
235
+ Full render uses 6-page parallel Playwright pool (motion blur subframes).
236
+ Long videos render in 60s resumable chunks — re-run to continue after crash.
237
+
238
+ ---
239
+
240
+ ## Gotchas
241
+
242
+ - **Pure functions of t.** No CSS transitions, `setTimeout`, or state that
243
+ depends on previous frames. Playwright seeks out of order.
244
+ - **Every style must reset.** If a layer returns early after t > 15s, it
245
+ must explicitly set opacity/display/visibility to 0/none/hidden — or the
246
+ last rendered value bleeds onto all later frames.
247
+ - **HDR iPhone footage** needs tonemapping before compositing
248
+ (`uv run scripts/assemble.py` does this automatically).
249
+ - **Motion blur needs 6 subframes.** Fewer subframes show fast morphs as
250
+ ghost copies. Default is 6 (180° shutter), a 60s video takes ~4 min.
251
+ - **Measure text after fonts load.** Components that need widths (strike-
252
+ through, underline) measure lazily on first `seek()`.
253
+ - **Captions over the mouth.** Use `REEL.captionY` per layout to move them
254
+ to the seam in split-screen mode.
255
+
256
+ ---
257
+
258
+ ## IBM Bob integration
259
+
260
+ This skill works inside IBM Bob as a `~/.bob/skills/html-renderer/` skill.
261
+
262
+ ```bash
263
+ # Install
264
+ cp -r plugins/video-pipeline/skills/html-renderer/ ~/.bob/skills/html-renderer/
265
+
266
+ # Bob will pick it up automatically — ask:
267
+ # "use the html-renderer skill to animate my FrameForge script"
268
+ ```
269
+
270
+ The skill pairs with the existing `video-pipeline` skill: use
271
+ `video-pipeline` for research/script/TTS/publish and `html-renderer` for
272
+ the animation/render stage.
273
+
274
+ ---
275
+
276
+ ## Files this skill adds
277
+
278
+ ```
279
+ plugins/video-pipeline/skills/html-renderer/
280
+ SKILL.md ← this file
281
+ references/engine-api.md ← full engine function reference
282
+ references/design.md ← layouts, safe zones, type scale, pacing
283
+ assets/project/ ← template copied into each render project
284
+ index.html ← compositor page (loads engine.js)
285
+ edit.js ← starter edit (one layer per FrameForge scene)
286
+ reel/engine.js ← springs, layouts, components, captions, seek API
287
+ reel/engine.css ← default fonts + base reset
288
+ scripts/ ← render.mjs, mix_audio.py, verify.py, stock.py,
289
+ capture.mjs, transcribe.py, new-project.mjs
290
+ scripts/html-renderer/
291
+ new-project.mjs ← scaffolds a render project from script JSON
292
+ ```
@@ -0,0 +1,308 @@
1
+ ---
2
+ name: video-pipeline
3
+ description: >
4
+ Autonomous AI video production pipeline. Given a topic, this skill
5
+ researches, fact-checks, writes the script, synthesises voice, renders
6
+ with FFmpeg, runs quality gates, and saves the output locally or POSTs
7
+ to any webhook — all without human intervention.
8
+ Works in IBM Bob, Codex, opencode, Cursor, Windsurf, Claude Code,
9
+ or any IDE that loads skills from ~/.claude/skills/ or .claude/skills/.
10
+ Trigger phrases: "make a video about", "produce a reel on", "overnight
11
+ batch", "queue videos", "generate video", "run the pipeline", "publish
12
+ video". Also activates when a user pastes a topic list and says
13
+ "make these" or "do the batch".
14
+ ---
15
+
16
+ # Video Pipeline Skill
17
+
18
+ End-to-end autonomous video production. The AI proposes every creative
19
+ decision; deterministic systems (FFmpeg, TTS, image gen, MCP) execute them;
20
+ a quality gate blocks bad output before delivery.
21
+
22
+ ## Architecture
23
+
24
+ ```
25
+ IDE Agent (IBM Bob / Codex / opencode / Cursor / Windsurf / Claude Code)
26
+ │
27
+ ▼ Cloudflare Remote MCP (recommended) ─────────────────────────────
28
+ │ https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp
29
+ │ 17 tools: create_video_from_script · render_from_scenes
30
+ │ ingest_media · edit_scenes · render_tts · multi_voice_tts
31
+ │ generate_scene_image · assemble_video · publish_video
32
+ │ research_topic · write_script · external_platform_call
33
+ │ queue_full_pipeline · get_job_status
34
+ │ scaffold_animation_project · validate_storyboard · render_animation
35
+ │
36
+ ▼ OR — Local stdio / HTTP MCP ────────────────────────────────────
37
+ │ node src/mcp-server/stdio.js (stdio)
38
+ │ node src/mcp-server/server.js (HTTP :3210)
39
+ │
40
+ ▼ Execution layer
41
+ │ LLM provider: PROVIDER_LLM=groq|openrouter|anthropic|gemini
42
+ │ |mistral|together|cerebras|openai
43
+ │ |ollama|webhook
44
+ │ TTS provider: PROVIDER_TTS=edge-tts|elevenlabs|azure|openai|webhook
45
+ │ Visuals: Code-driven animations (default) + user media (video_path / image_path)
46
+ │
47
+ ▼ Optional: queue_full_pipeline
48
+ queue_full_pipeline MCP tool → POST job to worker API (WORKER_API_URL)
49
+ ```
50
+
51
+ All secrets come from environment variables — **never** hardcoded.
52
+ See `docs/env.template` for the full reference.
53
+
54
+ ---
55
+
56
+ ## Setup
57
+
58
+ ### Option A — Cloudflare Remote MCP (Recommended)
59
+
60
+ Deploy once; works in every IDE with no local server process:
61
+
62
+ ```bash
63
+ cd cloudflare-mcp
64
+ npm install
65
+ wrangler secret put WORKER_API_URL # https://your-render-app.onrender.com
66
+ wrangler secret put MCP_AUTH_TOKEN # random 32-byte hex
67
+ wrangler secret put INTERNAL_API_TOKEN # shared secret with render node
68
+ wrangler deploy
69
+ ```
70
+
71
+ Add to any IDE's MCP config:
72
+
73
+ ```json
74
+ {
75
+ "mcpServers": {
76
+ "frameforge": {
77
+ "url": "https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp",
78
+ "headers": { "Authorization": "Bearer YOUR_MCP_AUTH_TOKEN" }
79
+ }
80
+ }
81
+ }
82
+ ```
83
+
84
+ ### Option B — Local stdio MCP
85
+
86
+ **IBM Bob** (`~/.bob/mcp.json`):
87
+ ```json
88
+ {
89
+ "mcpServers": {
90
+ "frameforge": {
91
+ "command": "node",
92
+ "args": ["<absolute-path>/src/mcp-server/stdio.js"],
93
+ "env": { "PROVIDER_LLM": "groq", "GROQ_API_KEY": "gsk_..." }
94
+ }
95
+ }
96
+ }
97
+ ```
98
+
99
+ **Codex / opencode / Cursor / Windsurf / Claude Code** (`~/.claude/mcp.json` or `.mcp.json`):
100
+ ```json
101
+ {
102
+ "mcpServers": {
103
+ "frameforge": {
104
+ "command": "node",
105
+ "args": ["<absolute-path>/src/mcp-server/stdio.js"],
106
+ "env": { "PROVIDER_LLM": "groq", "GROQ_API_KEY": "${GROQ_API_KEY}" }
107
+ }
108
+ }
109
+ }
110
+ ```
111
+
112
+ ### 2. Set environment variables
113
+
114
+ Copy `docs/env.template` → `.env` and fill in. Minimum:
115
+
116
+ ```bash
117
+ # LLM (pick one — Groq is free with no credit card)
118
+ PROVIDER_LLM=groq
119
+ GROQ_API_KEY=gsk_...
120
+
121
+ # TTS (edge-tts is free offline)
122
+ PROVIDER_TTS=edge-tts
123
+
124
+ # Visuals are 100% code-driven (kinetic typography/motion graphics) + user media. No image key needed.
125
+
126
+ # Delivery (optional — default is 'file' which copies to N8N_FILES_DIR/published/)
127
+ # PUBLISH_WEBHOOK_URL=https://hooks.example.com/frameforge
128
+ # REVIEW_NOTIFY_URL=https://hooks.example.com/review-gate
129
+ ```
130
+
131
+ Run the interactive wizard for guided setup:
132
+ ```bash
133
+ npm run setup
134
+ ```
135
+
136
+ ---
137
+
138
+ ## Workflow (step-by-step)
139
+
140
+ ### Path 1 — Agent-First (FFmpeg + AI images)
141
+
142
+ **You are already running an LLM — use it.** Write the scenes yourself and hand
143
+ them to FrameForge for rendering. No `GROQ_API_KEY` or any other LLM key needed.
144
+ FrameForge only runs FFmpeg, TTS, and image generation:
145
+
146
+ ```
147
+ create_video_from_script({
148
+ scenes: [
149
+ { narration: "WiFi sends data as radio waves...", caption: "RADIO WAVES",
150
+ visual_description: "router emitting waves, flat illustration" },
151
+ { narration: "Your device picks up the signal...", caption: "YOUR DEVICE",
152
+ visual_description: "laptop receiving signal, clean diagram" }
153
+ ],
154
+ title: "How WiFi Works"
155
+ })
156
+ ```
157
+
158
+ Then deliver it:
159
+ ```
160
+ publish_video({ productionId, masterPath, channels: ["file"] })
161
+ ```
162
+
163
+ > **Image generation fallback:** If the configured image provider is unavailable, FrameForge automatically falls back to a dark-background placeholder per scene — the render completes. Switch to `PROVIDER_IMAGE=falai` for a more reliable free provider.
164
+
165
+ ### Path 2 — Code-Driven Animated Video (Playwright + ffmpeg, no AI images)
166
+
167
+ Use when: the user wants Remotion-style motion graphics, image gen is unavailable, or they ask for "animated video" or "make it look good". Every pixel is a pure function of time `t`.
168
+
169
+ > **Beat sheet gate:** Reply with a beat sheet and 4 keyframe descriptions first. Stop. Wait for approval before writing any scene code.
170
+
171
+ ```
172
+ // 1. Synthesise voice first — audio timing is ground truth
173
+ multi_voice_tts({ text: "Narration.", outputPath: ".frameforge-files/voice_0.m4a" })
174
+
175
+ // 2. Scaffold the compositor project (copies engine + all render scripts + briefings)
176
+ scaffold_animation_project({
177
+ outDir: ".frameforge-files/my-anim/",
178
+ scenes: [ { narration: "...", caption: "...", visual_description: "..." } ],
179
+ audioPath: ".frameforge-files/voice_0.m4a"
180
+ })
181
+ // → returns nextSteps[] — read ANIMATION_GUIDE.md and edit STORYBOARD.md next
182
+
183
+ // 3. Validate storyboard (four hard gates — must pass before scene code)
184
+ validate_storyboard({ storyboardPath: ".frameforge-files/my-anim/STORYBOARD.md" })
185
+
186
+ // 4. Write real animation in edit.js (pure functions of t — read ANIMATION_GUIDE.md)
187
+
188
+ // 5. Pre-flight linter & purity verification (3-point repeat, wander, cold-jump)
189
+ render_animation({ projectDir: ".frameforge-files/my-anim/", mode: "look" })
190
+ render_animation({ projectDir: ".frameforge-files/my-anim/", mode: "verify" })
191
+
192
+ // 6. Review stills (at least one sheet per shot, one strip per key motion)
193
+ render_animation({ projectDir: ".frameforge-files/my-anim/", mode: "sheet" })
194
+ render_animation({ projectDir: ".frameforge-files/my-anim/", mode: "strip",
195
+ timestamps: [2.1, 2.6] }) // every frame of one motion
196
+ render_animation({ projectDir: ".frameforge-files/my-anim/", mode: "crop",
197
+ timestamps: ["9:16", 3.0] }) // mobile crop check
198
+
199
+ // 7. Full render (only after stills pass)
200
+ render_animation({ projectDir: ".frameforge-files/my-anim/", mode: "full",
201
+ audioPath: ".frameforge-files/voice_0.m4a" })
202
+ ```
203
+
204
+ Requires one-time: `cd <outDir> && npm install playwright && npx playwright install chromium`
205
+
206
+ ### Path 3 — Auto-Research (for unattended / batch runs)
207
+
208
+ Use this when FrameForge runs **without** an IDE in the loop — overnight batch,
209
+ `queue_full_pipeline`, or a standalone worker. The pipeline calls its own LLM
210
+ for research and scripting, so `PROVIDER_LLM` + the matching API key are required:
211
+
212
+ ```
213
+ 1. research_topic({ topic: "...", style: "educational", format: "short-form reel" })
214
+ 2. write_script({ productionId, claims, topic, persona: "tech reviewer" })
215
+ 3. render_from_scenes({ productionId })
216
+ 4. publish_video({ productionId, masterPath, channels: ["file"] })
217
+ ```
218
+
219
+ ### Voice Synthesis (per call)
220
+
221
+ ```
222
+ multi_voice_tts({
223
+ text: "Narration text here",
224
+ outputPath: "/path/in/files-dir/voice_0.m4a",
225
+ platform: "elevenlabs",
226
+ voice: "your_cloned_voice_id",
227
+ apiKey: "sk_..."
228
+ })
229
+ ```
230
+
231
+ ### Overnight batch
232
+
233
+ ```
234
+ I'm going to sleep. Make as many videos as you can before 9 AM.
235
+ Topics (queue them all):
236
+ 1. How does GPS actually work?
237
+ 2. Why planes don't fly over the Pacific in a straight line
238
+ 3. The engineering inside a modern CPU
239
+
240
+ Rules:
241
+ - Strong hook, double-verify every fact.
242
+ - After each video: run the quality gate, then publish to the output directory.
243
+ - Leave me a summary when done.
244
+ ```
245
+
246
+ The skill calls MCP tools sequentially using the StateMachine to track progress.
247
+ If a stage fails it retries with exponential backoff; if it fails terminally it
248
+ marks that production FAILED and moves to the next topic.
249
+
250
+ ---
251
+
252
+ ## Provider reference
253
+
254
+ | Role | Provider | Key env var | Notes |
255
+ |-------|-------------|--------------------------|-------------------------------|
256
+ | LLM | groq | `GROQ_API_KEY` | Default. Fast, free tier. |
257
+ | LLM | openrouter | `OPENROUTER_API_KEY` | Any model via one key |
258
+ | LLM | anthropic | `ANTHROPIC_API_KEY` | Claude Sonnet/Haiku/Opus |
259
+ | LLM | gemini | `GEMINI_API_KEY` | Gemini 2.0 Flash / 2.5 Pro |
260
+ | LLM | cerebras | `CEREBRAS_API_KEY` | 2000+ tok/s Llama |
261
+ | LLM | openai | `OPENAI_API_KEY` | GPT-4o / o1 |
262
+ | LLM | ollama | _(none)_ | Local, OLLAMA_BASE_URL |
263
+ | TTS | edge-tts | _(none)_ | Free, offline, 300+ voices |
264
+ | TTS | elevenlabs | `ELEVENLABS_API_KEY` | Voice cloning |
265
+ | TTS | azure | `AZURE_SPEECH_KEY` | Enterprise |
266
+ | TTS | openai | `OPENAI_API_KEY` | High quality |
267
+ | Visual| code-driven | _(none)_ | Default — kinetic cards & typography |
268
+ | Visual| user-media | _(none)_ | Caller `image_path` / `video_path` |
269
+
270
+ ---
271
+
272
+ ## Quality gates (hard — will block delivery)
273
+
274
+ | Gate | Threshold |
275
+ |---------------------|----------------------|
276
+ | File size | > 1 MB |
277
+ | Video stream | present |
278
+ | Resolution (master) | 1920×1080 |
279
+ | Resolution (short) | 1080×1920 |
280
+ | Codec | H.264 |
281
+ | Pixel format | yuv420p |
282
+ | Frame rate | 24–30 fps |
283
+ | Integrated loudness | −14 ±1.5 LUFS |
284
+ | True peak | ≤ −0.5 dBTP |
285
+ | Black frames | none > 0.5 s |
286
+ | Frozen frames | none > 0.5 s |
287
+
288
+ ---
289
+
290
+ ## Key files
291
+
292
+ ```
293
+ src/mcp-server/server.js MCP HTTP server (17 tools, port 3210)
294
+ src/mcp-server/stdio.js MCP stdio server (17 tools)
295
+ cloudflare-mcp/src/index.js Cloudflare Workers Remote MCP proxy
296
+ src/pipeline/providers.js LLM / TTS / Image abstraction (env-var-switchable providers)
297
+ src/pipeline/state-machine.js File + PostgreSQL production state
298
+ src/pipeline/retry-policy.js Exponential backoff, stage timeouts
299
+ src/pipeline/telemetry.js NDJSON cost tracking
300
+ src/quality-gate/gate.js 9 hard gates
301
+ src/audio-mixer/mixer.js Sidechain ducking, LUFS normalisation
302
+ src/shorts-extractor/extractor.js Silence-based 9:16 crop for Shorts
303
+ src/fact-checker/checker.js Claim risk classification + evidence gates
304
+ src/video-renderer/renderer.js Scene render + master assembly
305
+ src/worker/server-render-free.js Render-optimised worker API (Render free)
306
+ src/worker/server.js Full worker API
307
+ docs/env.template Environment variable reference
308
+ ```