@amg555/frameforge-mcp 0.0.0-stage → 2.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.bob/artifacts/enterprise-ai-video-agent-platform-agnostic-implementation-guide.html +464 -0
- package/.claude-plugin/marketplace.json +45 -0
- package/.gitattributes +19 -0
- package/.mcp.json +45 -0
- package/CHANGELOG.md +447 -0
- package/CLAUDE.md +320 -0
- package/CONTRIBUTING.md +112 -0
- package/LICENSE +21 -0
- package/Procfile +1 -0
- package/README.md +377 -2
- package/SECURITY.md +71 -0
- package/START_HERE.md +200 -0
- package/assets/metadata/assets.json +5 -0
- package/assets/metadata/embeddings.json +5 -0
- package/install.sh +64 -0
- package/package.json +58 -4
- package/src/animation/animation-system.js +229 -0
- package/src/asset-library/README.md +188 -0
- package/src/asset-library/asset-library.js +639 -0
- package/src/audio-mixer/mixer.js +187 -0
- package/src/cli/doctor.js +173 -0
- package/src/cli/make.js +655 -0
- package/src/cli/setup-keys.js +392 -0
- package/src/config/load-config.js +72 -0
- package/src/fact-checker/checker.js +384 -0
- package/src/fact-checker/review-gate.js +306 -0
- package/src/fact-checker/validation-set.js +1078 -0
- package/src/mcp-server/server.js +1299 -0
- package/src/mcp-server/stdio.js +87 -0
- package/src/pipeline/artifact-manifest.js +323 -0
- package/src/pipeline/artifact-retention.js +189 -0
- package/src/pipeline/external-adapter.js +366 -0
- package/src/pipeline/providers.js +853 -0
- package/src/pipeline/retry-policy.js +134 -0
- package/src/pipeline/state-machine.js +297 -0
- package/src/pipeline/telemetry.js +162 -0
- package/src/quality-gate/gate.js +229 -0
- package/src/schema/scene.schema.json +101 -0
- package/src/shorts-extractor/extractor.js +343 -0
- package/src/utils/log-redaction.js +274 -0
- package/src/video-renderer/renderer.js +364 -0
- package/src/worker/server-render-free.js +420 -0
- package/src/worker/server.js +328 -0
- package/src/workflows/assets.js +131 -0
- package/src/workflows/ingest.js +299 -0
- package/src/workflows/packaging.js +156 -0
- package/src/workflows/publishing.js +206 -0
- package/src/workflows/render.js +102 -0
- package/src/workflows/research.js +151 -0
- package/src/workflows/script.js +210 -0
- package/src/workflows/shorts.js +113 -0
package/README.md
CHANGED
|
@@ -1,3 +1,378 @@
|
|
|
1
|
-
|
|
1
|
+
<div align="center">
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
# ⚡ FrameForge
|
|
4
|
+
|
|
5
|
+
### AI Video Production Engine for Any Agentic IDE
|
|
6
|
+
|
|
7
|
+
**Any IDE. Any topic. Any style. 1080p videos from a single prompt.**
|
|
8
|
+
|
|
9
|
+
[](https://www.npmjs.com/package/@amg555/frameforge-mcp)
|
|
10
|
+
[](LICENSE)
|
|
11
|
+
[](https://nodejs.org/)
|
|
12
|
+
[](CHANGELOG.md)
|
|
13
|
+
[](https://modelcontextprotocol.io/)
|
|
14
|
+
[](#testing)
|
|
15
|
+
|
|
16
|
+
</div>
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
FrameForge is an MCP server your IDE agent calls to make videos. You (or your agent) write the scenes — FrameForge handles voice, images, FFmpeg compositing, and quality checking. Works in Bob, Cursor, Windsurf, Codex, opencode, Claude Code, and any MCP-capable IDE.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## Zero-install quickstart (npx)
|
|
25
|
+
|
|
26
|
+
No cloning. No `npm install`. FFmpeg and edge-tts run on your own machine.
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
# 1. Install prerequisites (one-time, system-level)
|
|
30
|
+
# FFmpeg: https://ffmpeg.org/download.html → add bin/ to PATH
|
|
31
|
+
# edge-tts: pip install edge-tts
|
|
32
|
+
|
|
33
|
+
# 2. Configure your API keys (one-time, saved to ~/.frameforge/config.json)
|
|
34
|
+
npx --package=@amg555/frameforge-mcp frameforge-setup
|
|
35
|
+
|
|
36
|
+
# 3. Check everything is ready
|
|
37
|
+
npx --package=@amg555/frameforge-mcp frameforge-doctor
|
|
38
|
+
|
|
39
|
+
# 4. Add to your IDE — then ask your agent to make a video
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Add to your IDE's MCP config (works in Bob, Cursor, Windsurf, Codex, Claude Code, opencode):
|
|
43
|
+
|
|
44
|
+
```json
|
|
45
|
+
{
|
|
46
|
+
"mcpServers": {
|
|
47
|
+
"frameforge": {
|
|
48
|
+
"command": "npx",
|
|
49
|
+
"args": ["--package=@amg555/frameforge-mcp", "-y", "frameforge-mcp"]
|
|
50
|
+
}
|
|
51
|
+
}
|
|
52
|
+
}
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
That's it. Your agent now has 17 video production tools. FFmpeg and TTS run on your machine — no cloud render costs.
|
|
56
|
+
|
|
57
|
+
---
|
|
58
|
+
|
|
59
|
+
## Other connection methods
|
|
60
|
+
|
|
61
|
+
### Cloned repo (contributors / local dev)
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
git clone https://github.com/AMG555/FrameForge.git && cd FrameForge && npm run doctor && npm run setup
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
```json
|
|
68
|
+
{
|
|
69
|
+
"mcpServers": {
|
|
70
|
+
"frameforge": {
|
|
71
|
+
"command": "node",
|
|
72
|
+
"args": ["/absolute/path/to/FrameForge/src/mcp-server/stdio.js"]
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
}
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
### Remote URL (Cloudflare Workers — FFmpeg must run separately)
|
|
79
|
+
|
|
80
|
+
Deploy once to Cloudflare (free):
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
cd cloudflare-mcp && npm install
|
|
84
|
+
wrangler secret put WORKER_API_URL # your FrameForge backend URL
|
|
85
|
+
wrangler secret put MCP_AUTH_TOKEN # any random 32-char string
|
|
86
|
+
wrangler deploy
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
```json
|
|
90
|
+
{
|
|
91
|
+
"mcpServers": {
|
|
92
|
+
"frameforge": {
|
|
93
|
+
"url": "https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp",
|
|
94
|
+
"headers": { "Authorization": "Bearer YOUR_MCP_AUTH_TOKEN" }
|
|
95
|
+
}
|
|
96
|
+
}
|
|
97
|
+
}
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
---
|
|
101
|
+
|
|
102
|
+
## Make your first video
|
|
103
|
+
|
|
104
|
+
**New to FrameForge?** Read [`START_HERE.md`](START_HERE.md) — it's a single file you paste into any IDE chat that starts a guided conversation: the agent asks what you want to make, produces a beat sheet for your approval, then drives the full pipeline.
|
|
105
|
+
|
|
106
|
+
Once connected, tell your IDE:
|
|
107
|
+
|
|
108
|
+
> *"Make a 60-second explainer video about how GPS works"*
|
|
109
|
+
|
|
110
|
+
Or call the tool directly:
|
|
111
|
+
|
|
112
|
+
```
|
|
113
|
+
create_video_from_script({
|
|
114
|
+
scenes: [
|
|
115
|
+
{
|
|
116
|
+
narration: "GPS works by triangulating signals from at least four satellites.",
|
|
117
|
+
caption: "GPS TRIANGULATION",
|
|
118
|
+
visual_description: "satellite orbiting earth, flat illustration, bold colors"
|
|
119
|
+
},
|
|
120
|
+
{
|
|
121
|
+
narration: "Your phone measures tiny timing differences to pinpoint your location.",
|
|
122
|
+
caption: "YOUR LOCATION",
|
|
123
|
+
visual_description: "phone with pin on map, white background, minimal"
|
|
124
|
+
}
|
|
125
|
+
],
|
|
126
|
+
title: "How GPS Works"
|
|
127
|
+
})
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
Output lands in `.frameforge-files/`:
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
final_video_<id>.mp4 → 1080p landscape master
|
|
134
|
+
short_1_<id>.mp4 → auto-extracted 9:16 vertical Short
|
|
135
|
+
video_meta.json → title, description, tags, chapters
|
|
136
|
+
```
|
|
137
|
+
|
|
138
|
+
**No API keys required for this path.** `edge-tts` (voice) and `pollinations` (images) are both free with no account. If Pollinations is slow, add `FALAI_API_KEY` and set `PROVIDER_IMAGE=falai`.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## Two ways to use it
|
|
143
|
+
|
|
144
|
+
### Your IDE writes the script — no LLM key needed
|
|
145
|
+
|
|
146
|
+
Your IDE's model writes the scenes. FrameForge only runs FFmpeg, TTS, and image generation. Use `create_video_from_script`. This is the fastest path and what most people want.
|
|
147
|
+
|
|
148
|
+
### FrameForge researches and scripts itself
|
|
149
|
+
|
|
150
|
+
For batch runs or when running without an IDE. Requires a free LLM key (Groq or Gemini both have free tiers).
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
npm run make -- --topic "How does GPS actually work?"
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
---
|
|
157
|
+
|
|
158
|
+
## Bring your own media
|
|
159
|
+
|
|
160
|
+
Skip AI generation for any scene by supplying your own files:
|
|
161
|
+
|
|
162
|
+
```json
|
|
163
|
+
{
|
|
164
|
+
"narration": "Here is the sorting algorithm running step by step.",
|
|
165
|
+
"caption": "ALGORITHM",
|
|
166
|
+
"video_path": "/recordings/screen_capture.mp4",
|
|
167
|
+
"audio_path": "/recordings/my_voice.mp3"
|
|
168
|
+
}
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
Any combination of `image_path`, `video_path`, `audio_path` works — FrameForge only generates what you don't supply.
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## Tools
|
|
176
|
+
|
|
177
|
+
17 tools, grouped by what you're doing:
|
|
178
|
+
|
|
179
|
+
**Make a video from your own script (no LLM key)**
|
|
180
|
+
|
|
181
|
+
| Tool | What it does |
|
|
182
|
+
|------|-------------|
|
|
183
|
+
| `create_video_from_script` | Full render from scenes you provide — voice, images, FFmpeg, quality gate |
|
|
184
|
+
| `ingest_media` | Import your own clips, images, and audio into a scenes manifest |
|
|
185
|
+
| `edit_scenes` | Update captions, swap media files, or append scenes to a manifest |
|
|
186
|
+
| `render_from_scenes` | Re-render from an existing manifest, skipping research and scripting |
|
|
187
|
+
| `assemble_video` | FFmpeg render + quality gate — produces the master MP4 |
|
|
188
|
+
| `render_tts` | Generate voice audio for one scene |
|
|
189
|
+
| `multi_voice_tts` | Override TTS provider per call without touching `.env` |
|
|
190
|
+
| `generate_scene_image` | Generate one scene image via the configured image provider |
|
|
191
|
+
| `publish_video` | Deliver the finished video via file copy or HTTP webhook |
|
|
192
|
+
| `external_platform_call` | HTTP bridge to ElevenLabs, Runway, Replicate, or any custom platform |
|
|
193
|
+
|
|
194
|
+
**Auto-research pipeline (needs a free LLM key)**
|
|
195
|
+
|
|
196
|
+
| Tool | What it does |
|
|
197
|
+
|------|-------------|
|
|
198
|
+
| `research_topic` | Research any topic and extract fact-checked claims |
|
|
199
|
+
| `write_script` | Generate a structured scene script from research claims |
|
|
200
|
+
| `queue_full_pipeline` | Queue a full research → render → publish run; returns `{ jobId }` immediately |
|
|
201
|
+
| `get_job_status` | Poll a queued job for `{ status, stage, result }` |
|
|
202
|
+
|
|
203
|
+
**Code-driven animated video (Playwright + FFmpeg)**
|
|
204
|
+
|
|
205
|
+
| Tool | What it does |
|
|
206
|
+
|------|-------------|
|
|
207
|
+
| `scaffold_animation_project` | Create a project where every frame is a pure function of time `t` |
|
|
208
|
+
| `validate_storyboard` | Check `STORYBOARD.md` against the 4-gate beat-sheet contract |
|
|
209
|
+
| `render_animation` | Run the Playwright renderer: sheet / strip / storyboard / crop / full |
|
|
210
|
+
|
|
211
|
+
---
|
|
212
|
+
|
|
213
|
+
## Providers
|
|
214
|
+
|
|
215
|
+
Switch any provider with one `.env` change — no code changes.
|
|
216
|
+
|
|
217
|
+
**Voice (TTS)**
|
|
218
|
+
|
|
219
|
+
| Provider | Free? | Key needed |
|
|
220
|
+
|----------|-------|------------|
|
|
221
|
+
| `edge-tts` | ✅ free, offline | none — **default** |
|
|
222
|
+
| `elevenlabs` | paid | `ELEVENLABS_API_KEY` |
|
|
223
|
+
| `azure` | paid | `AZURE_SPEECH_KEY` + `AZURE_SPEECH_REGION` |
|
|
224
|
+
| `openai` | paid | `OPENAI_API_KEY` |
|
|
225
|
+
| `webhook` | — | `WEBHOOK_TTS_URL` |
|
|
226
|
+
|
|
227
|
+
Set `PROVIDER_TTS=<name>`.
|
|
228
|
+
|
|
229
|
+
**Images**
|
|
230
|
+
|
|
231
|
+
| Provider | Free? | Key needed |
|
|
232
|
+
|----------|-------|------------|
|
|
233
|
+
| `pollinations` | ✅ free, no account | none — **default** |
|
|
234
|
+
| `falai` | ✅ free tier | `FALAI_API_KEY` at fal.ai |
|
|
235
|
+
| `replicate` | paid | `REPLICATE_API_KEY` |
|
|
236
|
+
| `dalle` | paid | `OPENAI_API_KEY` |
|
|
237
|
+
| `webhook` | — | `WEBHOOK_IMAGE_URL` |
|
|
238
|
+
|
|
239
|
+
Set `PROVIDER_IMAGE=<name>`.
|
|
240
|
+
|
|
241
|
+
**LLM — only for the auto-research path**
|
|
242
|
+
|
|
243
|
+
> Skip entirely when using `create_video_from_script`.
|
|
244
|
+
|
|
245
|
+
| Provider | Free? | Key needed |
|
|
246
|
+
|----------|-------|------------|
|
|
247
|
+
| `groq` | ✅ free tier | `GROQ_API_KEY` at console.groq.com |
|
|
248
|
+
| `gemini` | ✅ free tier | `GEMINI_API_KEY` at aistudio.google.com |
|
|
249
|
+
| `ollama` | ✅ fully local | none — set `OLLAMA_MODEL` |
|
|
250
|
+
| `openrouter` | pay per use | `OPENROUTER_API_KEY` |
|
|
251
|
+
| `anthropic` | paid | `ANTHROPIC_API_KEY` |
|
|
252
|
+
| `openai` | paid | `OPENAI_API_KEY` |
|
|
253
|
+
|
|
254
|
+
Set `PROVIDER_LLM=<name>`.
|
|
255
|
+
|
|
256
|
+
---
|
|
257
|
+
|
|
258
|
+
## Quality gates
|
|
259
|
+
|
|
260
|
+
Every video passes 7 checks before delivery. Any failure blocks the output.
|
|
261
|
+
|
|
262
|
+
| Check | Requirement |
|
|
263
|
+
|-------|------------|
|
|
264
|
+
| Resolution | 1920×1080 landscape or 1080×1920 portrait |
|
|
265
|
+
| Video codec | H.264, high profile, yuv420p |
|
|
266
|
+
| Loudness | −14 LUFS ± 1.5 LU (broadcast standard) |
|
|
267
|
+
| True peak | ≤ −0.5 dBTP |
|
|
268
|
+
| Black frames | None longer than 0.5 s |
|
|
269
|
+
| Frozen frames | None longer than 0.5 s |
|
|
270
|
+
| File integrity | > 1 MB, valid video + audio streams |
|
|
271
|
+
|
|
272
|
+
---
|
|
273
|
+
|
|
274
|
+
## CLI commands
|
|
275
|
+
|
|
276
|
+
**Via npx (no clone needed):**
|
|
277
|
+
|
|
278
|
+
| Command | What it does |
|
|
279
|
+
|---------|-------------|
|
|
280
|
+
| `npx --package=@amg555/frameforge-mcp frameforge-setup` | Interactive key wizard — saves to `~/.frameforge/config.json` |
|
|
281
|
+
| `npx --package=@amg555/frameforge-mcp frameforge-doctor` | Check Node, FFmpeg, edge-tts, and env vars |
|
|
282
|
+
| `npx --package=@amg555/frameforge-mcp frameforge --topic "..."` | Full pipeline: research → render → deliver |
|
|
283
|
+
|
|
284
|
+
**Via cloned repo:**
|
|
285
|
+
|
|
286
|
+
| Command | What it does |
|
|
287
|
+
|---------|-------------|
|
|
288
|
+
| `npm run doctor` | Check Node, FFmpeg, edge-tts, and env vars |
|
|
289
|
+
| `npm run setup` | Interactive key wizard — writes `.env` |
|
|
290
|
+
| `npm run mcp` | Start MCP server on port 3210 |
|
|
291
|
+
| `npm run mcp:stdio` | stdio MCP server for direct IDE connection |
|
|
292
|
+
| `npm run make -- --topic "..."` | Full pipeline: research → render → deliver |
|
|
293
|
+
| `npm run validate` | Deep structural validation (113 checks) |
|
|
294
|
+
| `npm test` | Test suite (512 tests) |
|
|
295
|
+
|
|
296
|
+
---
|
|
297
|
+
|
|
298
|
+
## Architecture
|
|
299
|
+
|
|
300
|
+
```
|
|
301
|
+
YOUR IDE (Bob / Codex / opencode / Cursor / Windsurf / Claude Code)
|
|
302
|
+
│
|
|
303
|
+
│ HTTPS + SSE (Cloudflare Remote MCP — optional)
|
|
304
|
+
▼
|
|
305
|
+
[Cloudflare Worker] proxies tool calls, keeps IDE context lean
|
|
306
|
+
│
|
|
307
|
+
│ HTTP + INTERNAL_API_TOKEN
|
|
308
|
+
▼
|
|
309
|
+
[FrameForge Node] runs on VPS / Pi / your machine / Render.com
|
|
310
|
+
│
|
|
311
|
+
├─ src/mcp-server/ 17 MCP tools
|
|
312
|
+
├─ src/workflows/ research → script → assets → render → publish
|
|
313
|
+
├─ src/pipeline/ state machine, retry, telemetry, manifest
|
|
314
|
+
├─ src/video-renderer/ FFmpeg scene render + concat
|
|
315
|
+
├─ src/audio-mixer/ LUFS normalise, sidechain ducking
|
|
316
|
+
└─ src/quality-gate/ 7-point output validation
|
|
317
|
+
```
|
|
318
|
+
|
|
319
|
+
---
|
|
320
|
+
|
|
321
|
+
## Security
|
|
322
|
+
|
|
323
|
+
- **Timing-safe auth** — all token comparisons use `crypto.timingSafeEqual`
|
|
324
|
+
- **SSRF protection** — webhook and pipeline URLs validated against RFC-1918 / loopback / link-local blocklist
|
|
325
|
+
- **Path traversal guards** — `productionId`, `audioPath`, `localPath` validated before any file I/O
|
|
326
|
+
- **CLI injection guards** — animation timestamps allowlisted before passing to `render.mjs`
|
|
327
|
+
- **Worker API auth** — `INTERNAL_API_TOKEN` Bearer check on all internal routes
|
|
328
|
+
- **Body size limits** — all HTTP servers reject requests > 4 MB with HTTP 413
|
|
329
|
+
- **Credential redaction** — 14+ regex patterns strip API keys from all logs
|
|
330
|
+
|
|
331
|
+
Report vulnerabilities: [SECURITY.md](SECURITY.md)
|
|
332
|
+
|
|
333
|
+
---
|
|
334
|
+
|
|
335
|
+
## Deployment
|
|
336
|
+
|
|
337
|
+
**npx (no install — recommended for users):**
|
|
338
|
+
|
|
339
|
+
```bash
|
|
340
|
+
npx --package=@amg555/frameforge-mcp frameforge-setup
|
|
341
|
+
# then add the IDE config from the quickstart above
|
|
342
|
+
```
|
|
343
|
+
|
|
344
|
+
**VPS / Pi / self-hosted (cloned repo):**
|
|
345
|
+
|
|
346
|
+
```bash
|
|
347
|
+
git clone https://github.com/AMG555/FrameForge.git && cd FrameForge
|
|
348
|
+
cp docs/env.template .env # edit with your keys
|
|
349
|
+
npm run mcp
|
|
350
|
+
```
|
|
351
|
+
|
|
352
|
+
See [`docs/deployment.md`](docs/deployment.md) for Docker, systemd, and Render.com setup.
|
|
353
|
+
|
|
354
|
+
---
|
|
355
|
+
|
|
356
|
+
## Testing
|
|
357
|
+
|
|
358
|
+
```bash
|
|
359
|
+
npm test # 512 tests, 0 failures
|
|
360
|
+
npm run validate # 113 structural checks
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
---
|
|
364
|
+
|
|
365
|
+
## Contributing
|
|
366
|
+
|
|
367
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
368
|
+
|
|
369
|
+
1. `npm run doctor` passes on your OS
|
|
370
|
+
2. `npm test` passes (512/512)
|
|
371
|
+
3. `npm run validate` passes (113/113)
|
|
372
|
+
4. FFmpeg commands keep `-nostdin` and cross-platform font paths
|
|
373
|
+
|
|
374
|
+
---
|
|
375
|
+
|
|
376
|
+
## License
|
|
377
|
+
|
|
378
|
+
MIT — see [LICENSE](LICENSE).
|
package/SECURITY.md
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
# Security Policy
|
|
2
|
+
|
|
3
|
+
## Supported Versions
|
|
4
|
+
|
|
5
|
+
| Version | Supported |
|
|
6
|
+
|:--- |:---: |
|
|
7
|
+
| 2.x (current) | ✅ |
|
|
8
|
+
| 1.x | ✅ security fixes only |
|
|
9
|
+
| < 1.0 | ❌ |
|
|
10
|
+
|
|
11
|
+
---
|
|
12
|
+
|
|
13
|
+
## Reporting a Vulnerability
|
|
14
|
+
|
|
15
|
+
**Please do not open a public GitHub Issue for security vulnerabilities.**
|
|
16
|
+
|
|
17
|
+
Report security issues by emailing **amgt503@gmail.com** with:
|
|
18
|
+
|
|
19
|
+
1. A description of the vulnerability and its potential impact.
|
|
20
|
+
2. Steps to reproduce (minimal proof of concept if possible).
|
|
21
|
+
3. Affected versions.
|
|
22
|
+
4. Any suggested mitigation.
|
|
23
|
+
|
|
24
|
+
You will receive an acknowledgement within **48 hours** and a resolution
|
|
25
|
+
timeline within **7 days**. We will credit you in the release notes unless you
|
|
26
|
+
prefer to remain anonymous.
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## Security Model
|
|
31
|
+
|
|
32
|
+
### Secrets handling
|
|
33
|
+
- All credentials are loaded exclusively from environment variables (`.env`).
|
|
34
|
+
- No secrets are committed to git — the `.gitignore` excludes `.env` and all
|
|
35
|
+
`*secrets*` files.
|
|
36
|
+
- `src/utils/log-redaction.js` masks known credential patterns (API keys,
|
|
37
|
+
tokens, connection strings, Bearer tokens) from all log output.
|
|
38
|
+
|
|
39
|
+
### Authentication
|
|
40
|
+
- The Cloudflare Workers MCP endpoint requires a `Bearer` token
|
|
41
|
+
(`MCP_AUTH_TOKEN`) on every request.
|
|
42
|
+
- The internal render API (`server-render-free.js`) requires
|
|
43
|
+
`INTERNAL_API_TOKEN` on all non-health routes.
|
|
44
|
+
- Tokens are compared with `crypto.timingSafeEqual` to prevent timing attacks.
|
|
45
|
+
|
|
46
|
+
### Shell injection prevention
|
|
47
|
+
- All external process calls use `spawnSync(cmd, [args, as, array])` — never
|
|
48
|
+
template string interpolation into shell commands.
|
|
49
|
+
|
|
50
|
+
### Network
|
|
51
|
+
- FFmpeg calls use `-nostdin` to prevent interactive prompts from blocking.
|
|
52
|
+
- All outbound HTTP calls (Pollinations, search APIs, webhook endpoints) use
|
|
53
|
+
Node.js `https` with explicit timeouts and no silent redirects beyond 5 hops.
|
|
54
|
+
|
|
55
|
+
### Cloudflare Workers
|
|
56
|
+
- The CF Workers MCP proxy does not store any credentials. `WORKER_API_URL`,
|
|
57
|
+
`MCP_AUTH_TOKEN`, and `INTERNAL_API_TOKEN` are set as Wrangler secrets
|
|
58
|
+
(encrypted at rest by Cloudflare).
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Known Limitations
|
|
63
|
+
|
|
64
|
+
- The LLM fallback in `FactChecker._querySources()` produces synthesized
|
|
65
|
+
sources when no `SEARCH_API_KEY` is set. These are not real citations and
|
|
66
|
+
should not be presented as authoritative without a real search provider.
|
|
67
|
+
- Pollinations.ai image requests are unauthenticated — do not include
|
|
68
|
+
private or sensitive content in image prompts.
|
|
69
|
+
- `PUBLISH_WEBHOOK_URL` and `REVIEW_NOTIFY_URL` receivers are trusted
|
|
70
|
+
endpoints; ensure they are authenticated on your side before deploying
|
|
71
|
+
to production.
|
package/START_HERE.md
ADDED
|
@@ -0,0 +1,200 @@
|
|
|
1
|
+
# FrameForge — Start Here
|
|
2
|
+
|
|
3
|
+
> **Paste this file's path into your agent IDE chat to start a guided video session.**
|
|
4
|
+
> The agent will read this file, ask you what you want to make, and drive the full pipeline.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## For the Agent
|
|
9
|
+
|
|
10
|
+
You are a video producer powered by FrameForge. Your job is to have a short conversation
|
|
11
|
+
with the user to understand what video they want, then use the FrameForge MCP tools to
|
|
12
|
+
produce it — from research through render to final MP4.
|
|
13
|
+
|
|
14
|
+
**Do not assume anything. Ask first. Produce second.**
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## Opening Message (say this verbatim to the user)
|
|
19
|
+
|
|
20
|
+
> 👋 I'm your FrameForge video studio. I can make:
|
|
21
|
+
>
|
|
22
|
+
> - **Short-form reels** (Instagram, TikTok, YouTube Shorts — 15–60s, 9:16 portrait)
|
|
23
|
+
> - **YouTube videos** (explainers, tutorials, long-form — 3–15 min, 16:9 landscape)
|
|
24
|
+
> - **Code-driven animated explainers** (motion graphics, no stock images needed — any length)
|
|
25
|
+
>
|
|
26
|
+
> Let's start with a few quick questions:
|
|
27
|
+
>
|
|
28
|
+
> 1. **What's the topic or idea?** (e.g. "How black holes form", "5 tips for sourdough bread", "A travel reel about Tokyo")
|
|
29
|
+
> 2. **Format?** `Reel / Short / TikTok (9:16)` · `YouTube video (16:9)` · `Animated explainer (any)`
|
|
30
|
+
> 3. **Style?** `educational` · `comedic` · `documentary` · `cinematic` · `tutorial` · or describe your own
|
|
31
|
+
> 4. **How long?** `Short (15–60s)` · `Medium (1–3 min)` · `Long (5–10 min)`
|
|
32
|
+
> 5. **Voice?** `Auto-select` · or name a style: `professional narrator`, `friendly teacher`, `energetic host`
|
|
33
|
+
> 6. **Any footage, images, or audio to include?** (paste file paths, or say "generate everything")
|
|
34
|
+
>
|
|
35
|
+
> Answer as many or as few as you like. I'll fill in the rest.
|
|
36
|
+
|
|
37
|
+
---
|
|
38
|
+
|
|
39
|
+
## After the User Answers — Agent Instructions
|
|
40
|
+
|
|
41
|
+
Once you have the user's answers, choose the right render path and follow it precisely.
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Render Paths
|
|
46
|
+
|
|
47
|
+
### PATH A — IDE-written scenes (fastest, most common)
|
|
48
|
+
|
|
49
|
+
Use for: any video where you (the IDE agent) write the narration and visual descriptions.
|
|
50
|
+
No LLM key needed — FrameForge generates TTS + AI images and renders with FFmpeg.
|
|
51
|
+
|
|
52
|
+
**Step A1 — Beat Sheet Gate (MANDATORY)**
|
|
53
|
+
|
|
54
|
+
Before calling any render tool, produce a beat sheet and **stop** — wait for approval:
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
BEAT SHEET — "<title>"
|
|
58
|
+
Duration: Xs | Style: <style> | Format: <landscape/portrait>
|
|
59
|
+
|
|
60
|
+
Scene 1 [0:00–0:08] <what happens + what the viewer learns> → TRANSITION
|
|
61
|
+
Scene 2 [0:08–0:18] <what happens + what the viewer learns> → TRANSITION
|
|
62
|
+
...
|
|
63
|
+
Scene N [X:XX–X:XX] <what happens + what the viewer learns> → FADE OUT
|
|
64
|
+
|
|
65
|
+
Voice: <voice description>
|
|
66
|
+
Visual style: <1-sentence description>
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
Gate rules — revise if ANY are true:
|
|
70
|
+
- A scene has no event (action that changes something)
|
|
71
|
+
- Two scenes share the same timestamp range
|
|
72
|
+
- Any seam is missing a transition label
|
|
73
|
+
- Total duration doesn't match requested length ±10%
|
|
74
|
+
|
|
75
|
+
**Step A2 — Render**
|
|
76
|
+
|
|
77
|
+
```
|
|
78
|
+
create_video_from_script({
|
|
79
|
+
scenes: [ { narration: "...", caption: "...", visual_description: "...", camera_hint: "zoom_in" } ],
|
|
80
|
+
title: "My Video",
|
|
81
|
+
orientation: "landscape", // "portrait" for Reels/Shorts/TikTok
|
|
82
|
+
voice: "en-US-BrianNeural", // or use language: "en"
|
|
83
|
+
withMusic: true
|
|
84
|
+
})
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
**Step A3 — Publish**
|
|
88
|
+
|
|
89
|
+
Ask: "Save locally, or deliver somewhere?" Then call `publish_video`.
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
### PATH B — Code-driven animated video (motion graphics, explainers, reels)
|
|
94
|
+
|
|
95
|
+
Use for: motion graphics, animated explainers, reels that must look high-quality without stock images.
|
|
96
|
+
Requires one-time: `npm install playwright && npx playwright install chromium` (run in the project dir).
|
|
97
|
+
|
|
98
|
+
**Step B1 — Beat sheet first (same gate as Path A — MANDATORY)**
|
|
99
|
+
|
|
100
|
+
**Step B2 — TTS and transcription**
|
|
101
|
+
|
|
102
|
+
```
|
|
103
|
+
multi_voice_tts({ text: "Full narration.", outputPath: ".frameforge-files/voice_0.m4a" })
|
|
104
|
+
// After scaffolding: python scripts/transcribe.py .frameforge-files/voice_0.m4a
|
|
105
|
+
// → voice_0.captions.json — word timestamps for cuePhrase() sync
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
**Step B3 — Scaffold**
|
|
109
|
+
|
|
110
|
+
```
|
|
111
|
+
scaffold_animation_project({
|
|
112
|
+
outDir: ".frameforge-files/my-animation/",
|
|
113
|
+
title: "My Video",
|
|
114
|
+
width: 1920, height: 1080, // 1080×1920 for portrait reels
|
|
115
|
+
scenes: [...],
|
|
116
|
+
audioPath: ".frameforge-files/voice_0.m4a"
|
|
117
|
+
})
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
**Step B4 — Validate STORYBOARD.md**
|
|
121
|
+
|
|
122
|
+
```
|
|
123
|
+
validate_storyboard({ storyboardPath: ".frameforge-files/my-animation/STORYBOARD.md" })
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
**Step B5 — Write `edit.js`, then stills, then full render**
|
|
127
|
+
|
|
128
|
+
```
|
|
129
|
+
render_animation({ projectDir: "...", mode: "look" }) // pre-flight
|
|
130
|
+
render_animation({ projectDir: "...", mode: "sheet" }) // one sheet per shot
|
|
131
|
+
render_animation({ projectDir: "...", mode: "strip", timestamps: [2.1, 2.6] }) // motion check
|
|
132
|
+
render_animation({ projectDir: "...", mode: "full", audioPath: ".frameforge-files/voice_0.m4a" })
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
**For reels: make 2–3 hook variants** (change only the first 3s of `edit.js`, full-render each):
|
|
136
|
+
```
|
|
137
|
+
render_animation({ projectDir: "...", mode: "stills", timestamps: [0.0, 1.5, 3.0] })
|
|
138
|
+
// deliver hook_a.mp4, hook_b.mp4 so the user can A/B test
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
---
|
|
142
|
+
|
|
143
|
+
### PATH C — Auto-research (needs `PROVIDER_LLM` + key in `.env`)
|
|
144
|
+
|
|
145
|
+
Use for: overnight batch, topics where you want FrameForge to research and script everything.
|
|
146
|
+
|
|
147
|
+
```
|
|
148
|
+
queue_full_pipeline({ topic: "How GPS works", style: "educational" })
|
|
149
|
+
→ { jobId }
|
|
150
|
+
get_job_status({ jobId }) // poll every 30s
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
---
|
|
154
|
+
|
|
155
|
+
## Quick Reference — MCP Tools
|
|
156
|
+
|
|
157
|
+
| Tool | What it does |
|
|
158
|
+
|------|-------------|
|
|
159
|
+
| `research_topic` | Research any topic, extract verified claims |
|
|
160
|
+
| `write_script` | Turn claims + beat sheet into a full narrated script |
|
|
161
|
+
| `generate_scene_image` | AI-generated scene image from a prompt |
|
|
162
|
+
| `render_tts` | Text-to-speech narration for a scene |
|
|
163
|
+
| `create_video_from_script` | **IDE path** — you write the scenes, FrameForge renders |
|
|
164
|
+
| `assemble_video` | Assemble rendered scenes into a master MP4 |
|
|
165
|
+
| `render_animation` | Render HTML animation scenes (code-driven videos) |
|
|
166
|
+
| `scaffold_animation_project` | Create a new code-driven animation project |
|
|
167
|
+
| `ingest_media` | Import user-supplied images, clips, or audio |
|
|
168
|
+
| `edit_scenes` | Re-render specific scenes with new prompts/captions |
|
|
169
|
+
| `get_job_status` | Check async job progress |
|
|
170
|
+
| `publish_video` | Upload to YouTube, push to webhook, or copy to folder |
|
|
171
|
+
| `get_pipeline_status` | Health check and current job queue |
|
|
172
|
+
| `external_platform_call` | Call any external API during production |
|
|
173
|
+
|
|
174
|
+
Full tool docs: [`src/mcp-server/server.js`](src/mcp-server/server.js)
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## Setup Check
|
|
179
|
+
|
|
180
|
+
Before starting, confirm the server is running:
|
|
181
|
+
|
|
182
|
+
```bash
|
|
183
|
+
npm run doctor # checks ffmpeg, fonts, TTS, LLM key
|
|
184
|
+
npm run mcp # starts the MCP server on port 3210
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
If `npm run doctor` reports missing dependencies, fix them before proceeding.
|
|
188
|
+
See [`docs/env.template`](docs/env.template) for all environment variables.
|
|
189
|
+
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
## Determinism Note (for code-driven animations)
|
|
193
|
+
|
|
194
|
+
If the user wants a **code-driven animation** (like the viral "Claude explains X" style),
|
|
195
|
+
use `scaffold_animation_project` to create a project, then use `render_animation` to
|
|
196
|
+
render it. Every frame is a pure function of time `t` — same input always produces the
|
|
197
|
+
same pixels, enabling parallel render, resume, and diff-based review.
|
|
198
|
+
|
|
199
|
+
See [`SKILL.md`](plugins/video-pipeline/skills/video-pipeline/SKILL.md) for the full
|
|
200
|
+
code-driven animation workflow.
|