@amg555/frameforge-mcp 0.0.0-stage → 2.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (51) hide show
  1. package/.bob/artifacts/enterprise-ai-video-agent-platform-agnostic-implementation-guide.html +464 -0
  2. package/.claude-plugin/marketplace.json +45 -0
  3. package/.gitattributes +19 -0
  4. package/.mcp.json +45 -0
  5. package/CHANGELOG.md +447 -0
  6. package/CLAUDE.md +320 -0
  7. package/CONTRIBUTING.md +112 -0
  8. package/LICENSE +21 -0
  9. package/Procfile +1 -0
  10. package/README.md +377 -2
  11. package/SECURITY.md +71 -0
  12. package/START_HERE.md +200 -0
  13. package/assets/metadata/assets.json +5 -0
  14. package/assets/metadata/embeddings.json +5 -0
  15. package/install.sh +64 -0
  16. package/package.json +58 -4
  17. package/src/animation/animation-system.js +229 -0
  18. package/src/asset-library/README.md +188 -0
  19. package/src/asset-library/asset-library.js +639 -0
  20. package/src/audio-mixer/mixer.js +187 -0
  21. package/src/cli/doctor.js +173 -0
  22. package/src/cli/make.js +655 -0
  23. package/src/cli/setup-keys.js +392 -0
  24. package/src/config/load-config.js +72 -0
  25. package/src/fact-checker/checker.js +384 -0
  26. package/src/fact-checker/review-gate.js +306 -0
  27. package/src/fact-checker/validation-set.js +1078 -0
  28. package/src/mcp-server/server.js +1299 -0
  29. package/src/mcp-server/stdio.js +87 -0
  30. package/src/pipeline/artifact-manifest.js +323 -0
  31. package/src/pipeline/artifact-retention.js +189 -0
  32. package/src/pipeline/external-adapter.js +366 -0
  33. package/src/pipeline/providers.js +853 -0
  34. package/src/pipeline/retry-policy.js +134 -0
  35. package/src/pipeline/state-machine.js +297 -0
  36. package/src/pipeline/telemetry.js +162 -0
  37. package/src/quality-gate/gate.js +229 -0
  38. package/src/schema/scene.schema.json +101 -0
  39. package/src/shorts-extractor/extractor.js +343 -0
  40. package/src/utils/log-redaction.js +274 -0
  41. package/src/video-renderer/renderer.js +364 -0
  42. package/src/worker/server-render-free.js +420 -0
  43. package/src/worker/server.js +328 -0
  44. package/src/workflows/assets.js +131 -0
  45. package/src/workflows/ingest.js +299 -0
  46. package/src/workflows/packaging.js +156 -0
  47. package/src/workflows/publishing.js +206 -0
  48. package/src/workflows/render.js +102 -0
  49. package/src/workflows/research.js +151 -0
  50. package/src/workflows/script.js +210 -0
  51. package/src/workflows/shorts.js +113 -0
package/README.md CHANGED
@@ -1,3 +1,378 @@
1
- # Temporary Holding Version
1
+ <div align="center">
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ # ⚡ FrameForge
4
+
5
+ ### AI Video Production Engine for Any Agentic IDE
6
+
7
+ **Any IDE. Any topic. Any style. 1080p videos from a single prompt.**
8
+
9
+ [![npm](https://img.shields.io/npm/v/@amg555/frameforge-mcp.svg?label=npm)](https://www.npmjs.com/package/@amg555/frameforge-mcp)
10
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
11
+ [![Node.js](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg)](https://nodejs.org/)
12
+ [![Version](https://img.shields.io/badge/version-2.3.1-blue.svg)](CHANGELOG.md)
13
+ [![MCP Spec](https://img.shields.io/badge/MCP-2024--11--05-blue.svg)](https://modelcontextprotocol.io/)
14
+ [![Tests](https://img.shields.io/badge/tests-512%20passing-brightgreen.svg)](#testing)
15
+
16
+ </div>
17
+
18
+ ---
19
+
20
+ FrameForge is an MCP server your IDE agent calls to make videos. You (or your agent) write the scenes — FrameForge handles voice, images, FFmpeg compositing, and quality checking. Works in Bob, Cursor, Windsurf, Codex, opencode, Claude Code, and any MCP-capable IDE.
21
+
22
+ ---
23
+
24
+ ## Zero-install quickstart (npx)
25
+
26
+ No cloning. No `npm install`. FFmpeg and edge-tts run on your own machine.
27
+
28
+ ```bash
29
+ # 1. Install prerequisites (one-time, system-level)
30
+ # FFmpeg: https://ffmpeg.org/download.html → add bin/ to PATH
31
+ # edge-tts: pip install edge-tts
32
+
33
+ # 2. Configure your API keys (one-time, saved to ~/.frameforge/config.json)
34
+ npx --package=@amg555/frameforge-mcp frameforge-setup
35
+
36
+ # 3. Check everything is ready
37
+ npx --package=@amg555/frameforge-mcp frameforge-doctor
38
+
39
+ # 4. Add to your IDE — then ask your agent to make a video
40
+ ```
41
+
42
+ Add to your IDE's MCP config (works in Bob, Cursor, Windsurf, Codex, Claude Code, opencode):
43
+
44
+ ```json
45
+ {
46
+ "mcpServers": {
47
+ "frameforge": {
48
+ "command": "npx",
49
+ "args": ["--package=@amg555/frameforge-mcp", "-y", "frameforge-mcp"]
50
+ }
51
+ }
52
+ }
53
+ ```
54
+
55
+ That's it. Your agent now has 17 video production tools. FFmpeg and TTS run on your machine — no cloud render costs.
56
+
57
+ ---
58
+
59
+ ## Other connection methods
60
+
61
+ ### Cloned repo (contributors / local dev)
62
+
63
+ ```bash
64
+ git clone https://github.com/AMG555/FrameForge.git && cd FrameForge && npm run doctor && npm run setup
65
+ ```
66
+
67
+ ```json
68
+ {
69
+ "mcpServers": {
70
+ "frameforge": {
71
+ "command": "node",
72
+ "args": ["/absolute/path/to/FrameForge/src/mcp-server/stdio.js"]
73
+ }
74
+ }
75
+ }
76
+ ```
77
+
78
+ ### Remote URL (Cloudflare Workers — FFmpeg must run separately)
79
+
80
+ Deploy once to Cloudflare (free):
81
+
82
+ ```bash
83
+ cd cloudflare-mcp && npm install
84
+ wrangler secret put WORKER_API_URL # your FrameForge backend URL
85
+ wrangler secret put MCP_AUTH_TOKEN # any random 32-char string
86
+ wrangler deploy
87
+ ```
88
+
89
+ ```json
90
+ {
91
+ "mcpServers": {
92
+ "frameforge": {
93
+ "url": "https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp",
94
+ "headers": { "Authorization": "Bearer YOUR_MCP_AUTH_TOKEN" }
95
+ }
96
+ }
97
+ }
98
+ ```
99
+
100
+ ---
101
+
102
+ ## Make your first video
103
+
104
+ **New to FrameForge?** Read [`START_HERE.md`](START_HERE.md) — it's a single file you paste into any IDE chat that starts a guided conversation: the agent asks what you want to make, produces a beat sheet for your approval, then drives the full pipeline.
105
+
106
+ Once connected, tell your IDE:
107
+
108
+ > *"Make a 60-second explainer video about how GPS works"*
109
+
110
+ Or call the tool directly:
111
+
112
+ ```
113
+ create_video_from_script({
114
+ scenes: [
115
+ {
116
+ narration: "GPS works by triangulating signals from at least four satellites.",
117
+ caption: "GPS TRIANGULATION",
118
+ visual_description: "satellite orbiting earth, flat illustration, bold colors"
119
+ },
120
+ {
121
+ narration: "Your phone measures tiny timing differences to pinpoint your location.",
122
+ caption: "YOUR LOCATION",
123
+ visual_description: "phone with pin on map, white background, minimal"
124
+ }
125
+ ],
126
+ title: "How GPS Works"
127
+ })
128
+ ```
129
+
130
+ Output lands in `.frameforge-files/`:
131
+
132
+ ```
133
+ final_video_<id>.mp4 → 1080p landscape master
134
+ short_1_<id>.mp4 → auto-extracted 9:16 vertical Short
135
+ video_meta.json → title, description, tags, chapters
136
+ ```
137
+
138
+ **No API keys required for this path.** `edge-tts` (voice) and `pollinations` (images) are both free with no account. If Pollinations is slow, add `FALAI_API_KEY` and set `PROVIDER_IMAGE=falai`.
139
+
140
+ ---
141
+
142
+ ## Two ways to use it
143
+
144
+ ### Your IDE writes the script — no LLM key needed
145
+
146
+ Your IDE's model writes the scenes. FrameForge only runs FFmpeg, TTS, and image generation. Use `create_video_from_script`. This is the fastest path and what most people want.
147
+
148
+ ### FrameForge researches and scripts itself
149
+
150
+ For batch runs or when running without an IDE. Requires a free LLM key (Groq or Gemini both have free tiers).
151
+
152
+ ```bash
153
+ npm run make -- --topic "How does GPS actually work?"
154
+ ```
155
+
156
+ ---
157
+
158
+ ## Bring your own media
159
+
160
+ Skip AI generation for any scene by supplying your own files:
161
+
162
+ ```json
163
+ {
164
+ "narration": "Here is the sorting algorithm running step by step.",
165
+ "caption": "ALGORITHM",
166
+ "video_path": "/recordings/screen_capture.mp4",
167
+ "audio_path": "/recordings/my_voice.mp3"
168
+ }
169
+ ```
170
+
171
+ Any combination of `image_path`, `video_path`, `audio_path` works — FrameForge only generates what you don't supply.
172
+
173
+ ---
174
+
175
+ ## Tools
176
+
177
+ 17 tools, grouped by what you're doing:
178
+
179
+ **Make a video from your own script (no LLM key)**
180
+
181
+ | Tool | What it does |
182
+ |------|-------------|
183
+ | `create_video_from_script` | Full render from scenes you provide — voice, images, FFmpeg, quality gate |
184
+ | `ingest_media` | Import your own clips, images, and audio into a scenes manifest |
185
+ | `edit_scenes` | Update captions, swap media files, or append scenes to a manifest |
186
+ | `render_from_scenes` | Re-render from an existing manifest, skipping research and scripting |
187
+ | `assemble_video` | FFmpeg render + quality gate — produces the master MP4 |
188
+ | `render_tts` | Generate voice audio for one scene |
189
+ | `multi_voice_tts` | Override TTS provider per call without touching `.env` |
190
+ | `generate_scene_image` | Generate one scene image via the configured image provider |
191
+ | `publish_video` | Deliver the finished video via file copy or HTTP webhook |
192
+ | `external_platform_call` | HTTP bridge to ElevenLabs, Runway, Replicate, or any custom platform |
193
+
194
+ **Auto-research pipeline (needs a free LLM key)**
195
+
196
+ | Tool | What it does |
197
+ |------|-------------|
198
+ | `research_topic` | Research any topic and extract fact-checked claims |
199
+ | `write_script` | Generate a structured scene script from research claims |
200
+ | `queue_full_pipeline` | Queue a full research → render → publish run; returns `{ jobId }` immediately |
201
+ | `get_job_status` | Poll a queued job for `{ status, stage, result }` |
202
+
203
+ **Code-driven animated video (Playwright + FFmpeg)**
204
+
205
+ | Tool | What it does |
206
+ |------|-------------|
207
+ | `scaffold_animation_project` | Create a project where every frame is a pure function of time `t` |
208
+ | `validate_storyboard` | Check `STORYBOARD.md` against the 4-gate beat-sheet contract |
209
+ | `render_animation` | Run the Playwright renderer: sheet / strip / storyboard / crop / full |
210
+
211
+ ---
212
+
213
+ ## Providers
214
+
215
+ Switch any provider with one `.env` change — no code changes.
216
+
217
+ **Voice (TTS)**
218
+
219
+ | Provider | Free? | Key needed |
220
+ |----------|-------|------------|
221
+ | `edge-tts` | ✅ free, offline | none — **default** |
222
+ | `elevenlabs` | paid | `ELEVENLABS_API_KEY` |
223
+ | `azure` | paid | `AZURE_SPEECH_KEY` + `AZURE_SPEECH_REGION` |
224
+ | `openai` | paid | `OPENAI_API_KEY` |
225
+ | `webhook` | — | `WEBHOOK_TTS_URL` |
226
+
227
+ Set `PROVIDER_TTS=<name>`.
228
+
229
+ **Images**
230
+
231
+ | Provider | Free? | Key needed |
232
+ |----------|-------|------------|
233
+ | `pollinations` | ✅ free, no account | none — **default** |
234
+ | `falai` | ✅ free tier | `FALAI_API_KEY` at fal.ai |
235
+ | `replicate` | paid | `REPLICATE_API_KEY` |
236
+ | `dalle` | paid | `OPENAI_API_KEY` |
237
+ | `webhook` | — | `WEBHOOK_IMAGE_URL` |
238
+
239
+ Set `PROVIDER_IMAGE=<name>`.
240
+
241
+ **LLM — only for the auto-research path**
242
+
243
+ > Skip entirely when using `create_video_from_script`.
244
+
245
+ | Provider | Free? | Key needed |
246
+ |----------|-------|------------|
247
+ | `groq` | ✅ free tier | `GROQ_API_KEY` at console.groq.com |
248
+ | `gemini` | ✅ free tier | `GEMINI_API_KEY` at aistudio.google.com |
249
+ | `ollama` | ✅ fully local | none — set `OLLAMA_MODEL` |
250
+ | `openrouter` | pay per use | `OPENROUTER_API_KEY` |
251
+ | `anthropic` | paid | `ANTHROPIC_API_KEY` |
252
+ | `openai` | paid | `OPENAI_API_KEY` |
253
+
254
+ Set `PROVIDER_LLM=<name>`.
255
+
256
+ ---
257
+
258
+ ## Quality gates
259
+
260
+ Every video passes 7 checks before delivery. Any failure blocks the output.
261
+
262
+ | Check | Requirement |
263
+ |-------|------------|
264
+ | Resolution | 1920×1080 landscape or 1080×1920 portrait |
265
+ | Video codec | H.264, high profile, yuv420p |
266
+ | Loudness | −14 LUFS ± 1.5 LU (broadcast standard) |
267
+ | True peak | ≤ −0.5 dBTP |
268
+ | Black frames | None longer than 0.5 s |
269
+ | Frozen frames | None longer than 0.5 s |
270
+ | File integrity | > 1 MB, valid video + audio streams |
271
+
272
+ ---
273
+
274
+ ## CLI commands
275
+
276
+ **Via npx (no clone needed):**
277
+
278
+ | Command | What it does |
279
+ |---------|-------------|
280
+ | `npx --package=@amg555/frameforge-mcp frameforge-setup` | Interactive key wizard — saves to `~/.frameforge/config.json` |
281
+ | `npx --package=@amg555/frameforge-mcp frameforge-doctor` | Check Node, FFmpeg, edge-tts, and env vars |
282
+ | `npx --package=@amg555/frameforge-mcp frameforge --topic "..."` | Full pipeline: research → render → deliver |
283
+
284
+ **Via cloned repo:**
285
+
286
+ | Command | What it does |
287
+ |---------|-------------|
288
+ | `npm run doctor` | Check Node, FFmpeg, edge-tts, and env vars |
289
+ | `npm run setup` | Interactive key wizard — writes `.env` |
290
+ | `npm run mcp` | Start MCP server on port 3210 |
291
+ | `npm run mcp:stdio` | stdio MCP server for direct IDE connection |
292
+ | `npm run make -- --topic "..."` | Full pipeline: research → render → deliver |
293
+ | `npm run validate` | Deep structural validation (113 checks) |
294
+ | `npm test` | Test suite (512 tests) |
295
+
296
+ ---
297
+
298
+ ## Architecture
299
+
300
+ ```
301
+ YOUR IDE (Bob / Codex / opencode / Cursor / Windsurf / Claude Code)
302
+ │
303
+ │ HTTPS + SSE (Cloudflare Remote MCP — optional)
304
+ ▼
305
+ [Cloudflare Worker] proxies tool calls, keeps IDE context lean
306
+ │
307
+ │ HTTP + INTERNAL_API_TOKEN
308
+ ▼
309
+ [FrameForge Node] runs on VPS / Pi / your machine / Render.com
310
+ │
311
+ ├─ src/mcp-server/ 17 MCP tools
312
+ ├─ src/workflows/ research → script → assets → render → publish
313
+ ├─ src/pipeline/ state machine, retry, telemetry, manifest
314
+ ├─ src/video-renderer/ FFmpeg scene render + concat
315
+ ├─ src/audio-mixer/ LUFS normalise, sidechain ducking
316
+ └─ src/quality-gate/ 7-point output validation
317
+ ```
318
+
319
+ ---
320
+
321
+ ## Security
322
+
323
+ - **Timing-safe auth** — all token comparisons use `crypto.timingSafeEqual`
324
+ - **SSRF protection** — webhook and pipeline URLs validated against RFC-1918 / loopback / link-local blocklist
325
+ - **Path traversal guards** — `productionId`, `audioPath`, `localPath` validated before any file I/O
326
+ - **CLI injection guards** — animation timestamps allowlisted before passing to `render.mjs`
327
+ - **Worker API auth** — `INTERNAL_API_TOKEN` Bearer check on all internal routes
328
+ - **Body size limits** — all HTTP servers reject requests > 4 MB with HTTP 413
329
+ - **Credential redaction** — 14+ regex patterns strip API keys from all logs
330
+
331
+ Report vulnerabilities: [SECURITY.md](SECURITY.md)
332
+
333
+ ---
334
+
335
+ ## Deployment
336
+
337
+ **npx (no install — recommended for users):**
338
+
339
+ ```bash
340
+ npx --package=@amg555/frameforge-mcp frameforge-setup
341
+ # then add the IDE config from the quickstart above
342
+ ```
343
+
344
+ **VPS / Pi / self-hosted (cloned repo):**
345
+
346
+ ```bash
347
+ git clone https://github.com/AMG555/FrameForge.git && cd FrameForge
348
+ cp docs/env.template .env # edit with your keys
349
+ npm run mcp
350
+ ```
351
+
352
+ See [`docs/deployment.md`](docs/deployment.md) for Docker, systemd, and Render.com setup.
353
+
354
+ ---
355
+
356
+ ## Testing
357
+
358
+ ```bash
359
+ npm test # 512 tests, 0 failures
360
+ npm run validate # 113 structural checks
361
+ ```
362
+
363
+ ---
364
+
365
+ ## Contributing
366
+
367
+ See [CONTRIBUTING.md](CONTRIBUTING.md).
368
+
369
+ 1. `npm run doctor` passes on your OS
370
+ 2. `npm test` passes (512/512)
371
+ 3. `npm run validate` passes (113/113)
372
+ 4. FFmpeg commands keep `-nostdin` and cross-platform font paths
373
+
374
+ ---
375
+
376
+ ## License
377
+
378
+ MIT — see [LICENSE](LICENSE).
package/SECURITY.md ADDED
@@ -0,0 +1,71 @@
1
+ # Security Policy
2
+
3
+ ## Supported Versions
4
+
5
+ | Version | Supported |
6
+ |:--- |:---: |
7
+ | 2.x (current) | ✅ |
8
+ | 1.x | ✅ security fixes only |
9
+ | < 1.0 | ❌ |
10
+
11
+ ---
12
+
13
+ ## Reporting a Vulnerability
14
+
15
+ **Please do not open a public GitHub Issue for security vulnerabilities.**
16
+
17
+ Report security issues by emailing **amgt503@gmail.com** with:
18
+
19
+ 1. A description of the vulnerability and its potential impact.
20
+ 2. Steps to reproduce (minimal proof of concept if possible).
21
+ 3. Affected versions.
22
+ 4. Any suggested mitigation.
23
+
24
+ You will receive an acknowledgement within **48 hours** and a resolution
25
+ timeline within **7 days**. We will credit you in the release notes unless you
26
+ prefer to remain anonymous.
27
+
28
+ ---
29
+
30
+ ## Security Model
31
+
32
+ ### Secrets handling
33
+ - All credentials are loaded exclusively from environment variables (`.env`).
34
+ - No secrets are committed to git — the `.gitignore` excludes `.env` and all
35
+ `*secrets*` files.
36
+ - `src/utils/log-redaction.js` masks known credential patterns (API keys,
37
+ tokens, connection strings, Bearer tokens) from all log output.
38
+
39
+ ### Authentication
40
+ - The Cloudflare Workers MCP endpoint requires a `Bearer` token
41
+ (`MCP_AUTH_TOKEN`) on every request.
42
+ - The internal render API (`server-render-free.js`) requires
43
+ `INTERNAL_API_TOKEN` on all non-health routes.
44
+ - Tokens are compared with `crypto.timingSafeEqual` to prevent timing attacks.
45
+
46
+ ### Shell injection prevention
47
+ - All external process calls use `spawnSync(cmd, [args, as, array])` — never
48
+ template string interpolation into shell commands.
49
+
50
+ ### Network
51
+ - FFmpeg calls use `-nostdin` to prevent interactive prompts from blocking.
52
+ - All outbound HTTP calls (Pollinations, search APIs, webhook endpoints) use
53
+ Node.js `https` with explicit timeouts and no silent redirects beyond 5 hops.
54
+
55
+ ### Cloudflare Workers
56
+ - The CF Workers MCP proxy does not store any credentials. `WORKER_API_URL`,
57
+ `MCP_AUTH_TOKEN`, and `INTERNAL_API_TOKEN` are set as Wrangler secrets
58
+ (encrypted at rest by Cloudflare).
59
+
60
+ ---
61
+
62
+ ## Known Limitations
63
+
64
+ - The LLM fallback in `FactChecker._querySources()` produces synthesized
65
+ sources when no `SEARCH_API_KEY` is set. These are not real citations and
66
+ should not be presented as authoritative without a real search provider.
67
+ - Pollinations.ai image requests are unauthenticated — do not include
68
+ private or sensitive content in image prompts.
69
+ - `PUBLISH_WEBHOOK_URL` and `REVIEW_NOTIFY_URL` receivers are trusted
70
+ endpoints; ensure they are authenticated on your side before deploying
71
+ to production.
package/START_HERE.md ADDED
@@ -0,0 +1,200 @@
1
+ # FrameForge — Start Here
2
+
3
+ > **Paste this file's path into your agent IDE chat to start a guided video session.**
4
+ > The agent will read this file, ask you what you want to make, and drive the full pipeline.
5
+
6
+ ---
7
+
8
+ ## For the Agent
9
+
10
+ You are a video producer powered by FrameForge. Your job is to have a short conversation
11
+ with the user to understand what video they want, then use the FrameForge MCP tools to
12
+ produce it — from research through render to final MP4.
13
+
14
+ **Do not assume anything. Ask first. Produce second.**
15
+
16
+ ---
17
+
18
+ ## Opening Message (say this verbatim to the user)
19
+
20
+ > 👋 I'm your FrameForge video studio. I can make:
21
+ >
22
+ > - **Short-form reels** (Instagram, TikTok, YouTube Shorts — 15–60s, 9:16 portrait)
23
+ > - **YouTube videos** (explainers, tutorials, long-form — 3–15 min, 16:9 landscape)
24
+ > - **Code-driven animated explainers** (motion graphics, no stock images needed — any length)
25
+ >
26
+ > Let's start with a few quick questions:
27
+ >
28
+ > 1. **What's the topic or idea?** (e.g. "How black holes form", "5 tips for sourdough bread", "A travel reel about Tokyo")
29
+ > 2. **Format?** `Reel / Short / TikTok (9:16)` · `YouTube video (16:9)` · `Animated explainer (any)`
30
+ > 3. **Style?** `educational` · `comedic` · `documentary` · `cinematic` · `tutorial` · or describe your own
31
+ > 4. **How long?** `Short (15–60s)` · `Medium (1–3 min)` · `Long (5–10 min)`
32
+ > 5. **Voice?** `Auto-select` · or name a style: `professional narrator`, `friendly teacher`, `energetic host`
33
+ > 6. **Any footage, images, or audio to include?** (paste file paths, or say "generate everything")
34
+ >
35
+ > Answer as many or as few as you like. I'll fill in the rest.
36
+
37
+ ---
38
+
39
+ ## After the User Answers — Agent Instructions
40
+
41
+ Once you have the user's answers, choose the right render path and follow it precisely.
42
+
43
+ ---
44
+
45
+ ## Render Paths
46
+
47
+ ### PATH A — IDE-written scenes (fastest, most common)
48
+
49
+ Use for: any video where you (the IDE agent) write the narration and visual descriptions.
50
+ No LLM key needed — FrameForge generates TTS + AI images and renders with FFmpeg.
51
+
52
+ **Step A1 — Beat Sheet Gate (MANDATORY)**
53
+
54
+ Before calling any render tool, produce a beat sheet and **stop** — wait for approval:
55
+
56
+ ```
57
+ BEAT SHEET — "<title>"
58
+ Duration: Xs | Style: <style> | Format: <landscape/portrait>
59
+
60
+ Scene 1 [0:00–0:08] <what happens + what the viewer learns> → TRANSITION
61
+ Scene 2 [0:08–0:18] <what happens + what the viewer learns> → TRANSITION
62
+ ...
63
+ Scene N [X:XX–X:XX] <what happens + what the viewer learns> → FADE OUT
64
+
65
+ Voice: <voice description>
66
+ Visual style: <1-sentence description>
67
+ ```
68
+
69
+ Gate rules — revise if ANY are true:
70
+ - A scene has no event (action that changes something)
71
+ - Two scenes share the same timestamp range
72
+ - Any seam is missing a transition label
73
+ - Total duration doesn't match requested length ±10%
74
+
75
+ **Step A2 — Render**
76
+
77
+ ```
78
+ create_video_from_script({
79
+ scenes: [ { narration: "...", caption: "...", visual_description: "...", camera_hint: "zoom_in" } ],
80
+ title: "My Video",
81
+ orientation: "landscape", // "portrait" for Reels/Shorts/TikTok
82
+ voice: "en-US-BrianNeural", // or use language: "en"
83
+ withMusic: true
84
+ })
85
+ ```
86
+
87
+ **Step A3 — Publish**
88
+
89
+ Ask: "Save locally, or deliver somewhere?" Then call `publish_video`.
90
+
91
+ ---
92
+
93
+ ### PATH B — Code-driven animated video (motion graphics, explainers, reels)
94
+
95
+ Use for: motion graphics, animated explainers, reels that must look high-quality without stock images.
96
+ Requires one-time: `npm install playwright && npx playwright install chromium` (run in the project dir).
97
+
98
+ **Step B1 — Beat sheet first (same gate as Path A — MANDATORY)**
99
+
100
+ **Step B2 — TTS and transcription**
101
+
102
+ ```
103
+ multi_voice_tts({ text: "Full narration.", outputPath: ".frameforge-files/voice_0.m4a" })
104
+ // After scaffolding: python scripts/transcribe.py .frameforge-files/voice_0.m4a
105
+ // → voice_0.captions.json — word timestamps for cuePhrase() sync
106
+ ```
107
+
108
+ **Step B3 — Scaffold**
109
+
110
+ ```
111
+ scaffold_animation_project({
112
+ outDir: ".frameforge-files/my-animation/",
113
+ title: "My Video",
114
+ width: 1920, height: 1080, // 1080×1920 for portrait reels
115
+ scenes: [...],
116
+ audioPath: ".frameforge-files/voice_0.m4a"
117
+ })
118
+ ```
119
+
120
+ **Step B4 — Validate STORYBOARD.md**
121
+
122
+ ```
123
+ validate_storyboard({ storyboardPath: ".frameforge-files/my-animation/STORYBOARD.md" })
124
+ ```
125
+
126
+ **Step B5 — Write `edit.js`, then stills, then full render**
127
+
128
+ ```
129
+ render_animation({ projectDir: "...", mode: "look" }) // pre-flight
130
+ render_animation({ projectDir: "...", mode: "sheet" }) // one sheet per shot
131
+ render_animation({ projectDir: "...", mode: "strip", timestamps: [2.1, 2.6] }) // motion check
132
+ render_animation({ projectDir: "...", mode: "full", audioPath: ".frameforge-files/voice_0.m4a" })
133
+ ```
134
+
135
+ **For reels: make 2–3 hook variants** (change only the first 3s of `edit.js`, full-render each):
136
+ ```
137
+ render_animation({ projectDir: "...", mode: "stills", timestamps: [0.0, 1.5, 3.0] })
138
+ // deliver hook_a.mp4, hook_b.mp4 so the user can A/B test
139
+ ```
140
+
141
+ ---
142
+
143
+ ### PATH C — Auto-research (needs `PROVIDER_LLM` + key in `.env`)
144
+
145
+ Use for: overnight batch, topics where you want FrameForge to research and script everything.
146
+
147
+ ```
148
+ queue_full_pipeline({ topic: "How GPS works", style: "educational" })
149
+ → { jobId }
150
+ get_job_status({ jobId }) // poll every 30s
151
+ ```
152
+
153
+ ---
154
+
155
+ ## Quick Reference — MCP Tools
156
+
157
+ | Tool | What it does |
158
+ |------|-------------|
159
+ | `research_topic` | Research any topic, extract verified claims |
160
+ | `write_script` | Turn claims + beat sheet into a full narrated script |
161
+ | `generate_scene_image` | AI-generated scene image from a prompt |
162
+ | `render_tts` | Text-to-speech narration for a scene |
163
+ | `create_video_from_script` | **IDE path** — you write the scenes, FrameForge renders |
164
+ | `assemble_video` | Assemble rendered scenes into a master MP4 |
165
+ | `render_animation` | Render HTML animation scenes (code-driven videos) |
166
+ | `scaffold_animation_project` | Create a new code-driven animation project |
167
+ | `ingest_media` | Import user-supplied images, clips, or audio |
168
+ | `edit_scenes` | Re-render specific scenes with new prompts/captions |
169
+ | `get_job_status` | Check async job progress |
170
+ | `publish_video` | Upload to YouTube, push to webhook, or copy to folder |
171
+ | `get_pipeline_status` | Health check and current job queue |
172
+ | `external_platform_call` | Call any external API during production |
173
+
174
+ Full tool docs: [`src/mcp-server/server.js`](src/mcp-server/server.js)
175
+
176
+ ---
177
+
178
+ ## Setup Check
179
+
180
+ Before starting, confirm the server is running:
181
+
182
+ ```bash
183
+ npm run doctor # checks ffmpeg, fonts, TTS, LLM key
184
+ npm run mcp # starts the MCP server on port 3210
185
+ ```
186
+
187
+ If `npm run doctor` reports missing dependencies, fix them before proceeding.
188
+ See [`docs/env.template`](docs/env.template) for all environment variables.
189
+
190
+ ---
191
+
192
+ ## Determinism Note (for code-driven animations)
193
+
194
+ If the user wants a **code-driven animation** (like the viral "Claude explains X" style),
195
+ use `scaffold_animation_project` to create a project, then use `render_animation` to
196
+ render it. Every frame is a pure function of time `t` — same input always produces the
197
+ same pixels, enabling parallel render, resume, and diff-based review.
198
+
199
+ See [`SKILL.md`](plugins/video-pipeline/skills/video-pipeline/SKILL.md) for the full
200
+ code-driven animation workflow.
@@ -0,0 +1,5 @@
1
+ {
2
+ "schema_version": "1.0.0",
3
+ "last_updated": "2026-09-09T19:45:00Z",
4
+ "assets": []
5
+ }
@@ -0,0 +1,5 @@
1
+ {
2
+ "schema_version": "1.0.0",
3
+ "last_updated": "2026-09-09T19:45:00Z",
4
+ "embeddings": {}
5
+ }