@amg555/frameforge-mcp 2.7.1 → 2.7.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,426 +1,426 @@
1
- <div align="center">
2
-
3
- # ⚡ FrameForge
4
-
5
- ### AI Video Production Engine for Any Agentic IDE
6
-
7
- **Any IDE. Any topic. Any style. Broadcast-quality videos from a single prompt.**
8
-
9
- [![npm](https://img.shields.io/npm/v/@amg555/frameforge-mcp.svg?label=npm)](https://www.npmjs.com/package/@amg555/frameforge-mcp)
10
- [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
11
- [![Node.js](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg)](https://nodejs.org/)
12
- [![Version](https://img.shields.io/badge/version-2.7.0-blue.svg)](CHANGELOG.md)
13
- [![MCP Spec](https://img.shields.io/badge/MCP-2024--11--05-blue.svg)](https://modelcontextprotocol.io/)
14
- [![Tests](https://img.shields.io/badge/tests-636%20passing-brightgreen.svg)](#testing)
15
-
16
- </div>
17
-
18
- ---
19
-
20
- FrameForge is an MCP server your IDE agent calls to make videos. You (or your agent) write the scenes — FrameForge handles voice, images, FFmpeg compositing, and quality checking. Works in Bob, Cursor, Windsurf, Codex, opencode, Claude Code, and any MCP-capable IDE.
21
-
22
- **28 MCP tools.** Includes Whisper transcription, yt-dlp media ingestion, broadcast-grade audio post-production (EQ + de-esser + SFX synthesis + room acoustics), and auto-updating motion craft from 9 frontier video repositories.
23
-
24
- ---
25
-
26
- ## Zero-install quickstart (npx)
27
-
28
- No cloning. No `npm install`. FFmpeg and edge-tts run on your own machine.
29
-
30
- ```bash
31
- # 1. Install prerequisites (one-time, system-level)
32
- # FFmpeg: https://ffmpeg.org/download.html → add bin/ to PATH
33
- # edge-tts: pip install edge-tts
34
-
35
- # 2. Configure your API keys (one-time, saved to ~/.frameforge/config.json)
36
- npx --package=@amg555/frameforge-mcp frameforge-setup
37
-
38
- # 3. Check everything is ready
39
- npx --package=@amg555/frameforge-mcp frameforge-doctor
40
-
41
- # 4. Add to your IDE — then ask your agent to make a video
42
- ```
43
-
44
- Add to your IDE's MCP config (works in Bob, Cursor, Windsurf, Codex, Claude Code, opencode):
45
-
46
- ```json
47
- {
48
- "mcpServers": {
49
- "frameforge": {
50
- "command": "npx",
51
- "args": ["--package=@amg555/frameforge-mcp", "-y", "frameforge-mcp"]
52
- }
53
- }
54
- }
55
- ```
56
-
57
- That's it. Your agent now has 28 video production & autonomous editing tools. FFmpeg and TTS run on your machine — no cloud render costs.
58
-
59
- ### Standalone CLI Commands (no IDE required)
60
-
61
- ```bash
62
- # Ingest any web clip or audio via yt-dlp
63
- npx --package=@amg555/frameforge-mcp frameforge download "https://youtube.com/watch?v=..." --subtitles
64
-
65
- # Transcribe speech with word-level timestamps & SRT via Whisper
66
- npx --package=@amg555/frameforge-mcp frameforge transcribe ./narration.mp3 --format all
67
-
68
- # Sync upstream techniques from 9 frontier video repositories
69
- npx --package=@amg555/frameforge-mcp frameforge sync
70
-
71
- # Diagnostics and system check
72
- npx --package=@amg555/frameforge-mcp frameforge doctor
73
- ```
74
-
75
- ---
76
-
77
- ## Other connection methods
78
-
79
- ### Cloned repo (contributors / local dev)
80
-
81
- ```bash
82
- git clone https://github.com/AMG555/FrameForge.git && cd FrameForge && npm run doctor && npm run setup
83
- ```
84
-
85
- ```json
86
- {
87
- "mcpServers": {
88
- "frameforge": {
89
- "command": "node",
90
- "args": ["/absolute/path/to/FrameForge/src/mcp-server/stdio.js"]
91
- }
92
- }
93
- }
94
- ```
95
-
96
- ### Remote URL (Cloudflare Workers — FFmpeg must run separately)
97
-
98
- Deploy once to Cloudflare (free):
99
-
100
- ```bash
101
- cd cloudflare-mcp && npm install
102
- wrangler secret put WORKER_API_URL # your FrameForge backend URL
103
- wrangler secret put MCP_AUTH_TOKEN # any random 32-char string
104
- wrangler deploy
105
- ```
106
-
107
- ```json
108
- {
109
- "mcpServers": {
110
- "frameforge": {
111
- "url": "https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp",
112
- "headers": { "Authorization": "Bearer YOUR_MCP_AUTH_TOKEN" }
113
- }
114
- }
115
- }
116
- ```
117
-
118
- ---
119
-
120
- ## Make your first video
121
-
122
- **New to FrameForge?** Read [`START_HERE.md`](START_HERE.md) — it's a single file you paste into any IDE chat that starts a guided conversation: the agent asks what you want to make, produces a beat sheet for your approval, then drives the full pipeline.
123
-
124
- Once connected, tell your IDE:
125
-
126
- > *"Make a 60-second explainer video about how GPS works"*
127
-
128
- Or call the tool directly:
129
-
130
- ```
131
- create_video_from_script({
132
- scenes: [
133
- {
134
- narration: "GPS works by triangulating signals from at least four satellites.",
135
- caption: "GPS TRIANGULATION",
136
- visual_description: "satellite orbiting earth, flat illustration, bold colors"
137
- },
138
- {
139
- narration: "Your phone measures tiny timing differences to pinpoint your location.",
140
- caption: "YOUR LOCATION",
141
- visual_description: "phone with pin on map, white background, minimal"
142
- }
143
- ],
144
- title: "How GPS Works"
145
- })
146
- ```
147
-
148
- Output lands in `.frameforge-files/`:
149
-
150
- ```
151
- final_video_<id>.mp4 → 1080p landscape master
152
- short_1_<id>.mp4 → auto-extracted 9:16 vertical Short
153
- video_meta.json → title, description, tags, chapters
154
- ```
155
-
156
- **No API keys required for this path.** `edge-tts` (voice) runs offline and visual cards render procedurally via FFmpeg/Playwright. 100% deterministic, zero diffusion flaws.
157
-
158
- ---
159
-
160
- ## Two ways to use it
161
-
162
- ### Your IDE writes the script — no LLM key needed
163
-
164
- Your IDE's model writes the scenes. FrameForge runs FFmpeg, TTS, and code-driven kinetic animations or user media. Use `create_video_from_script`. This is the fastest path and what most people want.
165
-
166
- ### FrameForge researches and scripts itself
167
-
168
- For batch runs or when running without an IDE. Requires a free LLM key (Groq or Gemini both have free tiers).
169
-
170
- ```bash
171
- npm run make -- --topic "How does GPS actually work?"
172
- ```
173
-
174
- ---
175
-
176
- ## Bring your own media
177
-
178
- Skip AI generation for any scene by supplying your own files:
179
-
180
- ```json
181
- {
182
- "narration": "Here is the sorting algorithm running step by step.",
183
- "caption": "ALGORITHM",
184
- "video_path": "/recordings/screen_capture.mp4",
185
- "audio_path": "/recordings/my_voice.mp3"
186
- }
187
- ```
188
-
189
- Any combination of `image_path`, `video_path`, `audio_path` works — FrameForge only generates what you don't supply.
190
-
191
- ---
192
-
193
- ## Tools
194
-
195
- 28 tools, grouped by what you're doing:
196
-
197
- **Make a video from your own script (no LLM key)**
198
-
199
- | Tool | What it does |
200
- |------|-------------|
201
- | `create_video_from_script` | Full render from scenes you provide — voice, procedural motion graphics / custom code, FFmpeg, quality gate |
202
- | `edit_video` | Autonomous local video editor — cut, trim, speed-ramp, color-grade, reframe (16:9 / 9:16 / 1:1), add captions, apply xfade transitions, and mix audio |
203
- | `ingest_media` | Import your own clips, images, and audio into a scenes manifest |
204
- | `edit_scenes` | Update captions, swap media files, or append scenes to a manifest |
205
- | `render_from_scenes` | Re-render from an existing manifest, skipping research and scripting |
206
- | `assemble_video` | FFmpeg render + quality gate — produces the master MP4 |
207
- | `render_tts` | Generate voice audio for one scene |
208
- | `multi_voice_tts` | Override TTS provider per call without touching `.env` |
209
- | `generate_scene_image` | Generate one scene image via the configured image provider |
210
- | `publish_video` | Deliver the finished video via file copy or HTTP webhook |
211
- | `external_platform_call` | Bridge to ElevenLabs, Runway, VoxForge, or local FFmpeg editor fallback |
212
- | `export_to_editor` | Bridge/import into DaVinci Resolve, Premiere Pro, After Effects, Remotion, or CapCut |
213
- | `plan_video_edit` | Plan/Approve/Render: analyze footage, generate `edit-plan.md` + `edit.json` with rationale per cut |
214
-
215
- **Media ingestion (yt-dlp & Whisper)**
216
-
217
- | Tool | What it does |
218
- |------|-------------|
219
- | `ingest_url` | Download video/audio from YouTube, Vimeo, Twitter/X, TikTok, and 1000+ sites via yt-dlp |
220
- | `transcribe_media` | Whisper transcription with word-level timestamps → NDJSON (for `cuePhrase()`), SRT, JSON |
221
-
222
- **Professional audio post-production**
223
-
224
- | Tool | What it does |
225
- |------|-------------|
226
- | `mix_audio_pro` | Broadcast-grade chain: HPF → EQ → de-esser → compressor → reverb → duck → stereo widen → SFX synthesis → two-pass EBU R128 |
227
-
228
- **Auto-research pipeline (needs a free LLM key)**
229
-
230
- | Tool | What it does |
231
- |------|-------------|
232
- | `research_topic` | Research any topic and extract fact-checked claims |
233
- | `write_script` | Generate a structured scene script from research claims |
234
- | `queue_full_pipeline` | Queue a full research → render → publish run; returns `{ jobId }` immediately |
235
- | `get_job_status` | Poll a queued job for `{ status, stage, result }` |
236
-
237
- **Code-driven animated video (Playwright + FFmpeg)**
238
-
239
- | Tool | What it does |
240
- |------|-------------|
241
- | `scaffold_animation_project` | Create a project where every frame is a pure function of time `t` |
242
- | `validate_storyboard` | Check `STORYBOARD.md` against the 4-gate beat-sheet contract |
243
- | `render_animation` | Run the Playwright renderer: sheet / strip / storyboard / crop / full / quality / golden |
244
- | `preview_scene` | Render still frames of a scene HTML/code for visual inspection before full render |
245
- | `verify_scene_quality` | Closed-loop visual QA gate: 1–10 score on content, exposure, layout, and motion delta |
246
-
247
- **Upstream skills**
248
-
249
- | Tool | What it does |
250
- |------|-------------|
251
- | `sync_upstream_skills` | Fetch latest commits from 9 reference repos and update local manifest. Returns per-category update summary with `userImpact` and affected tools. |
252
- | `check_upstream_updates` | Fast offline read of local manifest — shows last sync time, per-repo SHA, and sync status without any network requests. |
253
-
254
- ---
255
-
256
- ## Providers
257
-
258
- Switch any provider with one `.env` change — no code changes.
259
-
260
- **Voice (TTS)**
261
-
262
- | Provider | Free? | Key needed |
263
- |----------|-------|------------|
264
- | `edge-tts` | ✅ free, offline | none — **default** |
265
- | `voxforge` | ✅ free, local/private | none — `VOXFORGE_URL` ([VoxForge](https://github.com/AMG555/VoxForge)) |
266
- | `elevenlabs` | paid | `ELEVENLABS_API_KEY` |
267
- | `azure` | paid | `AZURE_SPEECH_KEY` + `AZURE_SPEECH_REGION` |
268
- | `openai` | paid | `OPENAI_API_KEY` |
269
- | `webhook` | — | `WEBHOOK_TTS_URL` |
270
-
271
- Set `PROVIDER_TTS=<name>`.
272
-
273
- **Visuals & Media**
274
-
275
- FrameForge is **code-driven first**: scenes render via crisp vector animations, kinetic typography, and user-provided media (clips, b-roll, photos) — zero diffusion artifacts, zero AI slop, 100% deterministic.
276
-
277
- | Visual Mode / Provider | Type | Key needed |
278
- |------------------------|------|------------|
279
- | `code-driven` | **Default** — kinetic cards, typography, spring motion | none — **recommended default** |
280
- | `user-media` | User-supplied `image_path` or `video_path` in scene | none |
281
- | `pollinations` | Optional AI diffusion (opt-in via `--ai-images`) | none |
282
- | `falai` | Optional AI diffusion (fast FLUX) | `FALAI_API_KEY` at fal.ai |
283
- | `replicate` | Optional AI diffusion (FLUX-schnell) | `REPLICATE_API_KEY` |
284
- | `dalle` | Optional AI diffusion (DALL-E 3) | `OPENAI_API_KEY` |
285
- | `webhook` | Custom image/video generation endpoint | `WEBHOOK_IMAGE_URL` |
286
-
287
- Set `ENABLE_AI_IMAGES=true` or pass `--ai-images` only if you explicitly want AI diffusion images. Default is clean code-driven animation.
288
-
289
- **LLM — only for the auto-research path**
290
-
291
- > Skip entirely when using `create_video_from_script`.
292
-
293
- | Provider | Free? | Key needed |
294
- |----------|-------|------------|
295
- | `groq` | ✅ free tier | `GROQ_API_KEY` at console.groq.com |
296
- | `gemini` | ✅ free tier | `GEMINI_API_KEY` at aistudio.google.com |
297
- | `ollama` | ✅ fully local | none — set `OLLAMA_MODEL` |
298
- | `openrouter` | pay per use | `OPENROUTER_API_KEY` |
299
- | `anthropic` | paid | `ANTHROPIC_API_KEY` |
300
- | `openai` | paid | `OPENAI_API_KEY` |
301
-
302
- Set `PROVIDER_LLM=<name>`.
303
-
304
- ---
305
-
306
- ## Quality gates
307
-
308
- Every video passes 7 checks before delivery. Any failure blocks the output.
309
-
310
- | Check | Requirement |
311
- |-------|------------|
312
- | Resolution | 1920×1080 landscape or 1080×1920 portrait |
313
- | Video codec | H.264, high profile, yuv420p |
314
- | Loudness | −14 LUFS ± 1.5 LU (broadcast standard) |
315
- | True peak | ≤ −0.5 dBTP |
316
- | Black frames | None longer than 0.5 s |
317
- | Frozen frames | None longer than 0.5 s |
318
- | File integrity | > 1 MB, valid video + audio streams |
319
-
320
- ---
321
-
322
- ## CLI commands
323
-
324
- **Via npx (no clone needed):**
325
-
326
- | Command | What it does |
327
- |---------|-------------|
328
- | `npx --package=@amg555/frameforge-mcp frameforge-setup` | Interactive key wizard — saves to `~/.frameforge/config.json` |
329
- | `npx --package=@amg555/frameforge-mcp frameforge-doctor` | Check Node, FFmpeg, edge-tts, and env vars |
330
- | `npx --package=@amg555/frameforge-mcp frameforge --topic "..."` | Full pipeline: research → render → deliver |
331
-
332
- **Via cloned repo:**
333
-
334
- | Command | What it does |
335
- |---------|-------------|
336
- | `npm run doctor` | Check Node, FFmpeg, edge-tts, and env vars |
337
- | `npm run setup` | Interactive key wizard — writes `.env` |
338
- | `npm run mcp` | Start MCP server on port 3210 |
339
- | `npm run mcp:stdio` | stdio MCP server for direct IDE connection |
340
- | `npm run make -- --topic "..."` | Full pipeline: research → render → deliver |
341
- | `npm run validate` | Deep structural validation (113 checks) |
342
- | `npm test` | Test suite (512 tests) |
343
-
344
- ---
345
-
346
- ## Architecture
347
-
348
- ```
349
- YOUR IDE (Bob / Codex / opencode / Cursor / Windsurf / Claude Code)
350
- │
351
- │ HTTPS + SSE (Cloudflare Remote MCP — optional)
352
- ▼
353
- [Cloudflare Worker] proxies tool calls, keeps IDE context lean
354
- │
355
- │ HTTP + INTERNAL_API_TOKEN
356
- ▼
357
- [FrameForge Node] runs on VPS / Pi / your machine / Render.com
358
- │
359
- ├─ src/mcp-server/ 17 MCP tools
360
- ├─ src/workflows/ research → script → assets → render → publish
361
- ├─ src/pipeline/ state machine, retry, telemetry, manifest
362
- ├─ src/video-renderer/ FFmpeg scene render + concat
363
- ├─ src/audio-mixer/ LUFS normalise, sidechain ducking
364
- └─ src/quality-gate/ 7-point output validation
365
- ```
366
-
367
- ---
368
-
369
- ## Security
370
-
371
- - **Timing-safe auth** — all token comparisons use `crypto.timingSafeEqual`
372
- - **SSRF protection** — webhook and pipeline URLs validated against RFC-1918 / loopback / link-local blocklist
373
- - **Path traversal guards** — `productionId`, `audioPath`, `localPath` validated before any file I/O
374
- - **CLI injection guards** — animation timestamps allowlisted before passing to `render.mjs`
375
- - **Worker API auth** — `INTERNAL_API_TOKEN` Bearer check on all internal routes
376
- - **Body size limits** — all HTTP servers reject requests > 4 MB with HTTP 413
377
- - **Credential redaction** — 14+ regex patterns strip API keys from all logs
378
-
379
- Report vulnerabilities: [SECURITY.md](SECURITY.md)
380
-
381
- ---
382
-
383
- ## Deployment
384
-
385
- **npx (no install — recommended for users):**
386
-
387
- ```bash
388
- npx --package=@amg555/frameforge-mcp frameforge-setup
389
- # then add the IDE config from the quickstart above
390
- ```
391
-
392
- **VPS / Pi / self-hosted (cloned repo):**
393
-
394
- ```bash
395
- git clone https://github.com/AMG555/FrameForge.git && cd FrameForge
396
- cp docs/env.template .env # edit with your keys
397
- npm run mcp
398
- ```
399
-
400
- See [`docs/deployment.md`](docs/deployment.md) for Docker, systemd, and Render.com setup.
401
-
402
- ---
403
-
404
- ## Testing
405
-
406
- ```bash
407
- npm test # 512 tests, 0 failures
408
- npm run validate # 113 structural checks
409
- ```
410
-
411
- ---
412
-
413
- ## Contributing
414
-
415
- See [CONTRIBUTING.md](CONTRIBUTING.md).
416
-
417
- 1. `npm run doctor` passes on your OS
418
- 2. `npm test` passes (512/512)
419
- 3. `npm run validate` passes (113/113)
420
- 4. FFmpeg commands keep `-nostdin` and cross-platform font paths
421
-
422
- ---
423
-
424
- ## License
425
-
426
- MIT — see [LICENSE](LICENSE).
1
+ <div align="center">
2
+
3
+ # ⚡ FrameForge
4
+
5
+ ### AI Video Production Engine for Any Agentic IDE
6
+
7
+ **Any IDE. Any topic. Any style. Broadcast-quality videos from a single prompt.**
8
+
9
+ [![npm](https://img.shields.io/npm/v/@amg555/frameforge-mcp.svg?label=npm)](https://www.npmjs.com/package/@amg555/frameforge-mcp)
10
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
11
+ [![Node.js](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg)](https://nodejs.org/)
12
+ [![Version](https://img.shields.io/badge/version-2.7.2-blue.svg)](CHANGELOG.md)
13
+ [![MCP Spec](https://img.shields.io/badge/MCP-2024--11--05-blue.svg)](https://modelcontextprotocol.io/)
14
+ [![Tests](https://img.shields.io/badge/tests-658%20passing-brightgreen.svg)](#testing)
15
+
16
+ </div>
17
+
18
+ ---
19
+
20
+ FrameForge is an MCP server your IDE agent calls to make videos. You (or your agent) write the scenes — FrameForge handles voice, images, FFmpeg compositing, and quality checking. Works in Bob, Cursor, Windsurf, Codex, opencode, Claude Code, and any MCP-capable IDE.
21
+
22
+ **28 MCP tools.** Includes Whisper transcription, yt-dlp media ingestion, broadcast-grade audio post-production (EQ + de-esser + SFX synthesis + room acoustics), and auto-updating motion craft from 9 frontier video repositories.
23
+
24
+ ---
25
+
26
+ ## Zero-install quickstart (npx)
27
+
28
+ No cloning. No `npm install`. FFmpeg and edge-tts run on your own machine.
29
+
30
+ ```bash
31
+ # 1. Install prerequisites (one-time, system-level)
32
+ # FFmpeg: https://ffmpeg.org/download.html → add bin/ to PATH
33
+ # edge-tts: pip install edge-tts
34
+
35
+ # 2. Configure your API keys (one-time, saved to ~/.frameforge/config.json)
36
+ npx --package=@amg555/frameforge-mcp frameforge-setup
37
+
38
+ # 3. Check everything is ready
39
+ npx --package=@amg555/frameforge-mcp frameforge-doctor
40
+
41
+ # 4. Add to your IDE — then ask your agent to make a video
42
+ ```
43
+
44
+ Add to your IDE's MCP config (works in Bob, Cursor, Windsurf, Codex, Claude Code, opencode):
45
+
46
+ ```json
47
+ {
48
+ "mcpServers": {
49
+ "frameforge": {
50
+ "command": "npx",
51
+ "args": ["--package=@amg555/frameforge-mcp", "-y", "frameforge-mcp"]
52
+ }
53
+ }
54
+ }
55
+ ```
56
+
57
+ That's it. Your agent now has 28 video production & autonomous editing tools. FFmpeg and TTS run on your machine — no cloud render costs.
58
+
59
+ ### Standalone CLI Commands (no IDE required)
60
+
61
+ ```bash
62
+ # Ingest any web clip or audio via yt-dlp
63
+ npx --package=@amg555/frameforge-mcp frameforge download "https://youtube.com/watch?v=..." --subtitles
64
+
65
+ # Transcribe speech with word-level timestamps & SRT via Whisper
66
+ npx --package=@amg555/frameforge-mcp frameforge transcribe ./narration.mp3 --format all
67
+
68
+ # Sync upstream techniques from 9 frontier video repositories
69
+ npx --package=@amg555/frameforge-mcp frameforge sync
70
+
71
+ # Diagnostics and system check
72
+ npx --package=@amg555/frameforge-mcp frameforge doctor
73
+ ```
74
+
75
+ ---
76
+
77
+ ## Other connection methods
78
+
79
+ ### Cloned repo (contributors / local dev)
80
+
81
+ ```bash
82
+ git clone https://github.com/AMG555/FrameForge.git && cd FrameForge && npm run doctor && npm run setup
83
+ ```
84
+
85
+ ```json
86
+ {
87
+ "mcpServers": {
88
+ "frameforge": {
89
+ "command": "node",
90
+ "args": ["/absolute/path/to/FrameForge/src/mcp-server/stdio.js"]
91
+ }
92
+ }
93
+ }
94
+ ```
95
+
96
+ ### Remote URL (Cloudflare Workers — FFmpeg must run separately)
97
+
98
+ Deploy once to Cloudflare (free):
99
+
100
+ ```bash
101
+ cd cloudflare-mcp && npm install
102
+ wrangler secret put WORKER_API_URL # your FrameForge backend URL
103
+ wrangler secret put MCP_AUTH_TOKEN # any random 32-char string
104
+ wrangler deploy
105
+ ```
106
+
107
+ ```json
108
+ {
109
+ "mcpServers": {
110
+ "frameforge": {
111
+ "url": "https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp",
112
+ "headers": { "Authorization": "Bearer YOUR_MCP_AUTH_TOKEN" }
113
+ }
114
+ }
115
+ }
116
+ ```
117
+
118
+ ---
119
+
120
+ ## Make your first video
121
+
122
+ **New to FrameForge?** Read [`START_HERE.md`](START_HERE.md) — it's a single file you paste into any IDE chat that starts a guided conversation: the agent asks what you want to make, produces a beat sheet for your approval, then drives the full pipeline.
123
+
124
+ Once connected, tell your IDE:
125
+
126
+ > *"Make a 60-second explainer video about how GPS works"*
127
+
128
+ Or call the tool directly:
129
+
130
+ ```
131
+ create_video_from_script({
132
+ scenes: [
133
+ {
134
+ narration: "GPS works by triangulating signals from at least four satellites.",
135
+ caption: "GPS TRIANGULATION",
136
+ visual_description: "satellite orbiting earth, flat illustration, bold colors"
137
+ },
138
+ {
139
+ narration: "Your phone measures tiny timing differences to pinpoint your location.",
140
+ caption: "YOUR LOCATION",
141
+ visual_description: "phone with pin on map, white background, minimal"
142
+ }
143
+ ],
144
+ title: "How GPS Works"
145
+ })
146
+ ```
147
+
148
+ Output lands in `.frameforge-files/`:
149
+
150
+ ```
151
+ final_video_<id>.mp4 → 1080p landscape master
152
+ short_1_<id>.mp4 → auto-extracted 9:16 vertical Short
153
+ video_meta.json → title, description, tags, chapters
154
+ ```
155
+
156
+ **No API keys required for this path.** `edge-tts` (voice) runs offline and visual cards render procedurally via FFmpeg/Playwright. 100% deterministic, zero diffusion flaws.
157
+
158
+ ---
159
+
160
+ ## Two ways to use it
161
+
162
+ ### Your IDE writes the script — no LLM key needed
163
+
164
+ Your IDE's model writes the scenes. FrameForge runs FFmpeg, TTS, and code-driven kinetic animations or user media. Use `create_video_from_script`. This is the fastest path and what most people want.
165
+
166
+ ### FrameForge researches and scripts itself
167
+
168
+ For batch runs or when running without an IDE. Requires a free LLM key (Groq or Gemini both have free tiers).
169
+
170
+ ```bash
171
+ npm run make -- --topic "How does GPS actually work?"
172
+ ```
173
+
174
+ ---
175
+
176
+ ## Bring your own media
177
+
178
+ Skip AI generation for any scene by supplying your own files:
179
+
180
+ ```json
181
+ {
182
+ "narration": "Here is the sorting algorithm running step by step.",
183
+ "caption": "ALGORITHM",
184
+ "video_path": "/recordings/screen_capture.mp4",
185
+ "audio_path": "/recordings/my_voice.mp3"
186
+ }
187
+ ```
188
+
189
+ Any combination of `image_path`, `video_path`, `audio_path` works — FrameForge only generates what you don't supply.
190
+
191
+ ---
192
+
193
+ ## Tools
194
+
195
+ 28 tools, grouped by what you're doing:
196
+
197
+ **Make a video from your own script (no LLM key)**
198
+
199
+ | Tool | What it does |
200
+ |------|-------------|
201
+ | `create_video_from_script` | Full render from scenes you provide — voice, procedural motion graphics / custom code, FFmpeg, quality gate |
202
+ | `edit_video` | Autonomous local video editor — cut, trim, speed-ramp, color-grade, reframe (16:9 / 9:16 / 1:1), add captions, apply xfade transitions, and mix audio |
203
+ | `ingest_media` | Import your own clips, images, and audio into a scenes manifest |
204
+ | `edit_scenes` | Update captions, swap media files, or append scenes to a manifest |
205
+ | `render_from_scenes` | Re-render from an existing manifest, skipping research and scripting |
206
+ | `assemble_video` | FFmpeg render + quality gate — produces the master MP4 |
207
+ | `render_tts` | Generate voice audio for one scene |
208
+ | `multi_voice_tts` | Override TTS provider per call without touching `.env` |
209
+ | `generate_scene_image` | Generate one scene image via the configured image provider |
210
+ | `publish_video` | Deliver the finished video via file copy or HTTP webhook |
211
+ | `external_platform_call` | Bridge to ElevenLabs, Runway, VoxForge, or local FFmpeg editor fallback |
212
+ | `export_to_editor` | Bridge/import into DaVinci Resolve, Premiere Pro, After Effects, Remotion, or CapCut |
213
+ | `plan_video_edit` | Plan/Approve/Render: analyze footage, generate `edit-plan.md` + `edit.json` with rationale per cut |
214
+
215
+ **Media ingestion (yt-dlp & Whisper)**
216
+
217
+ | Tool | What it does |
218
+ |------|-------------|
219
+ | `ingest_url` | Download video/audio from YouTube, Vimeo, Twitter/X, TikTok, and 1000+ sites via yt-dlp |
220
+ | `transcribe_media` | Whisper transcription with word-level timestamps → NDJSON (for `cuePhrase()`), SRT, JSON |
221
+
222
+ **Professional audio post-production**
223
+
224
+ | Tool | What it does |
225
+ |------|-------------|
226
+ | `mix_audio_pro` | Broadcast-grade chain: HPF → EQ → de-esser → compressor → reverb → duck → stereo widen → SFX synthesis → two-pass EBU R128 |
227
+
228
+ **Auto-research pipeline (needs a free LLM key)**
229
+
230
+ | Tool | What it does |
231
+ |------|-------------|
232
+ | `research_topic` | Research any topic and extract fact-checked claims |
233
+ | `write_script` | Generate a structured scene script from research claims |
234
+ | `queue_full_pipeline` | Queue a full research → render → publish run; returns `{ jobId }` immediately |
235
+ | `get_job_status` | Poll a queued job for `{ status, stage, result }` |
236
+
237
+ **Code-driven animated video (Playwright + FFmpeg)**
238
+
239
+ | Tool | What it does |
240
+ |------|-------------|
241
+ | `scaffold_animation_project` | Create a project where every frame is a pure function of time `t` |
242
+ | `validate_storyboard` | Check `STORYBOARD.md` against the 4-gate beat-sheet contract |
243
+ | `render_animation` | Run the Playwright renderer: sheet / strip / storyboard / crop / full / quality / golden |
244
+ | `preview_scene` | Render still frames of a scene HTML/code for visual inspection before full render |
245
+ | `verify_scene_quality` | Closed-loop visual QA gate: 1–10 score on content, exposure, layout, and motion delta |
246
+
247
+ **Upstream skills**
248
+
249
+ | Tool | What it does |
250
+ |------|-------------|
251
+ | `sync_upstream_skills` | Fetch latest commits from 9 reference repos and update local manifest. Returns per-category update summary with `userImpact` and affected tools. |
252
+ | `check_upstream_updates` | Fast offline read of local manifest — shows last sync time, per-repo SHA, and sync status without any network requests. |
253
+
254
+ ---
255
+
256
+ ## Providers
257
+
258
+ Switch any provider with one `.env` change — no code changes.
259
+
260
+ **Voice (TTS)**
261
+
262
+ | Provider | Free? | Key needed |
263
+ |----------|-------|------------|
264
+ | `edge-tts` | ✅ free, offline | none — **default** |
265
+ | `voxforge` | ✅ free, local/private | none — `VOXFORGE_URL` ([VoxForge](https://github.com/AMG555/VoxForge)) |
266
+ | `elevenlabs` | paid | `ELEVENLABS_API_KEY` |
267
+ | `azure` | paid | `AZURE_SPEECH_KEY` + `AZURE_SPEECH_REGION` |
268
+ | `openai` | paid | `OPENAI_API_KEY` |
269
+ | `webhook` | — | `WEBHOOK_TTS_URL` |
270
+
271
+ Set `PROVIDER_TTS=<name>`.
272
+
273
+ **Visuals & Media**
274
+
275
+ FrameForge is **code-driven first**: scenes render via crisp vector animations, kinetic typography, and user-provided media (clips, b-roll, photos) — zero diffusion artifacts, zero AI slop, 100% deterministic.
276
+
277
+ | Visual Mode / Provider | Type | Key needed |
278
+ |------------------------|------|------------|
279
+ | `code-driven` | **Default** — kinetic cards, typography, spring motion | none — **recommended default** |
280
+ | `user-media` | User-supplied `image_path` or `video_path` in scene | none |
281
+ | `pollinations` | Optional AI diffusion (opt-in via `--ai-images`) | none |
282
+ | `falai` | Optional AI diffusion (fast FLUX) | `FALAI_API_KEY` at fal.ai |
283
+ | `replicate` | Optional AI diffusion (FLUX-schnell) | `REPLICATE_API_KEY` |
284
+ | `dalle` | Optional AI diffusion (DALL-E 3) | `OPENAI_API_KEY` |
285
+ | `webhook` | Custom image/video generation endpoint | `WEBHOOK_IMAGE_URL` |
286
+
287
+ Set `ENABLE_AI_IMAGES=true` or pass `--ai-images` only if you explicitly want AI diffusion images. Default is clean code-driven animation.
288
+
289
+ **LLM — only for the auto-research path**
290
+
291
+ > Skip entirely when using `create_video_from_script`.
292
+
293
+ | Provider | Free? | Key needed |
294
+ |----------|-------|------------|
295
+ | `groq` | ✅ free tier | `GROQ_API_KEY` at console.groq.com |
296
+ | `gemini` | ✅ free tier | `GEMINI_API_KEY` at aistudio.google.com |
297
+ | `ollama` | ✅ fully local | none — set `OLLAMA_MODEL` |
298
+ | `openrouter` | pay per use | `OPENROUTER_API_KEY` |
299
+ | `anthropic` | paid | `ANTHROPIC_API_KEY` |
300
+ | `openai` | paid | `OPENAI_API_KEY` |
301
+
302
+ Set `PROVIDER_LLM=<name>`.
303
+
304
+ ---
305
+
306
+ ## Quality gates
307
+
308
+ Every video passes 7 checks before delivery. Any failure blocks the output.
309
+
310
+ | Check | Requirement |
311
+ |-------|------------|
312
+ | Resolution | 1920×1080 landscape or 1080×1920 portrait |
313
+ | Video codec | H.264, high profile, yuv420p |
314
+ | Loudness | −14 LUFS ± 1.5 LU (broadcast standard) |
315
+ | True peak | ≤ −0.5 dBTP |
316
+ | Black frames | None longer than 0.5 s |
317
+ | Frozen frames | None longer than 0.5 s |
318
+ | File integrity | > 1 MB, valid video + audio streams |
319
+
320
+ ---
321
+
322
+ ## CLI commands
323
+
324
+ **Via npx (no clone needed):**
325
+
326
+ | Command | What it does |
327
+ |---------|-------------|
328
+ | `npx --package=@amg555/frameforge-mcp frameforge-setup` | Interactive key wizard — saves to `~/.frameforge/config.json` |
329
+ | `npx --package=@amg555/frameforge-mcp frameforge-doctor` | Check Node, FFmpeg, edge-tts, and env vars |
330
+ | `npx --package=@amg555/frameforge-mcp frameforge --topic "..."` | Full pipeline: research → render → deliver |
331
+
332
+ **Via cloned repo:**
333
+
334
+ | Command | What it does |
335
+ |---------|-------------|
336
+ | `npm run doctor` | Check Node, FFmpeg, edge-tts, and env vars |
337
+ | `npm run setup` | Interactive key wizard — writes `.env` |
338
+ | `npm run mcp` | Start MCP server on port 3210 |
339
+ | `npm run mcp:stdio` | stdio MCP server for direct IDE connection |
340
+ | `npm run make -- --topic "..."` | Full pipeline: research → render → deliver |
341
+ | `npm run validate` | Deep structural validation (113 checks) |
342
+ | `npm test` | Test suite (512 tests) |
343
+
344
+ ---
345
+
346
+ ## Architecture
347
+
348
+ ```
349
+ YOUR IDE (Bob / Codex / opencode / Cursor / Windsurf / Claude Code)
350
+ │
351
+ │ HTTPS + SSE (Cloudflare Remote MCP — optional)
352
+ ▼
353
+ [Cloudflare Worker] proxies tool calls, keeps IDE context lean
354
+ │
355
+ │ HTTP + INTERNAL_API_TOKEN
356
+ ▼
357
+ [FrameForge Node] runs on VPS / Pi / your machine / Render.com
358
+ │
359
+ ├─ src/mcp-server/ 17 MCP tools
360
+ ├─ src/workflows/ research → script → assets → render → publish
361
+ ├─ src/pipeline/ state machine, retry, telemetry, manifest
362
+ ├─ src/video-renderer/ FFmpeg scene render + concat
363
+ ├─ src/audio-mixer/ LUFS normalise, sidechain ducking
364
+ └─ src/quality-gate/ 7-point output validation
365
+ ```
366
+
367
+ ---
368
+
369
+ ## Security
370
+
371
+ - **Timing-safe auth** — all token comparisons use `crypto.timingSafeEqual`
372
+ - **SSRF protection** — webhook and pipeline URLs validated against RFC-1918 / loopback / link-local blocklist
373
+ - **Path traversal guards** — `productionId`, `audioPath`, `localPath` validated before any file I/O
374
+ - **CLI injection guards** — animation timestamps allowlisted before passing to `render.mjs`
375
+ - **Worker API auth** — `INTERNAL_API_TOKEN` Bearer check on all internal routes
376
+ - **Body size limits** — all HTTP servers reject requests > 4 MB with HTTP 413
377
+ - **Credential redaction** — 14+ regex patterns strip API keys from all logs
378
+
379
+ Report vulnerabilities: [SECURITY.md](SECURITY.md)
380
+
381
+ ---
382
+
383
+ ## Deployment
384
+
385
+ **npx (no install — recommended for users):**
386
+
387
+ ```bash
388
+ npx --package=@amg555/frameforge-mcp frameforge-setup
389
+ # then add the IDE config from the quickstart above
390
+ ```
391
+
392
+ **VPS / Pi / self-hosted (cloned repo):**
393
+
394
+ ```bash
395
+ git clone https://github.com/AMG555/FrameForge.git && cd FrameForge
396
+ cp docs/env.template .env # edit with your keys
397
+ npm run mcp
398
+ ```
399
+
400
+ See [`docs/deployment.md`](docs/deployment.md) for Docker, systemd, and Render.com setup.
401
+
402
+ ---
403
+
404
+ ## Testing
405
+
406
+ ```bash
407
+ npm test # 512 tests, 0 failures
408
+ npm run validate # 113 structural checks
409
+ ```
410
+
411
+ ---
412
+
413
+ ## Contributing
414
+
415
+ See [CONTRIBUTING.md](CONTRIBUTING.md).
416
+
417
+ 1. `npm run doctor` passes on your OS
418
+ 2. `npm test` passes (512/512)
419
+ 3. `npm run validate` passes (113/113)
420
+ 4. FFmpeg commands keep `-nostdin` and cross-platform font paths
421
+
422
+ ---
423
+
424
+ ## License
425
+
426
+ MIT — see [LICENSE](LICENSE).