@amg555/frameforge-mcp 2.3.1 → 2.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -0
- package/CLAUDE.md +67 -9
- package/README.md +1 -1
- package/install.sh +1 -1
- package/package.json +1 -1
- package/src/cli/doctor.js +31 -1
- package/src/cli/setup-keys.js +10 -3
- package/src/mcp-server/server.js +117 -6
- package/src/pipeline/external-adapter.js +55 -15
- package/src/pipeline/providers.js +46 -0
package/CHANGELOG.md
CHANGED
|
@@ -4,6 +4,25 @@ All notable changes to FrameForge are documented here.
|
|
|
4
4
|
|
|
5
5
|
---
|
|
6
6
|
|
|
7
|
+
## [2.3.2] — 2026-11-06
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- **Tool 18: `export_to_editor`** — send a finished animation MP4 + EDL to any local editing app via an HTTP bridge: Premiere Pro (`:7777`), DaVinci Resolve (`:8080`), CapCut (`:9090`), After Effects (`:7878`), or Remotion (`:3001`). Pass `editorUrl` to override any port. Bridge setup docs: `docs/editor-bridges/`.
|
|
12
|
+
- **VoxForge TTS provider** (`PROVIDER_TTS=voxforge`) — named first-class provider in `TTSProvider`. Tries `/v1/audio/speech` (OpenAI-compatible) then falls back to `/v1/tts`. Configured via `VOXFORGE_URL`, `VOXFORGE_KEY`, `VOXFORGE_VOICE`, `VOXFORGE_MODEL`. Also surfaced in `multi_voice_tts` platform enum and `setup-keys.js` wizard.
|
|
13
|
+
- **Editing app presets in `ExternalAdapter`** — `premiere`, `resolve`, `capcut`, `aftereffects` (local bridge), `kling` (cloud), plus updated `voxforge` preset pointing to `/v1/audio/speech`.
|
|
14
|
+
- **`render_animation` mode `"kit"`** — now exposed in the MCP tool enum. Generates an EDL file in `out/` for import into Premiere / Resolve.
|
|
15
|
+
|
|
16
|
+
### Changed
|
|
17
|
+
|
|
18
|
+
- **`CLAUDE.md`**: code-driven animation promoted to **primary path**; FFmpeg + AI images demoted to "quick draft"; new sections for VoxForge voice and editor bridge workflow; tool table reorganised into Primary / Secondary / Auto-research groups (18 tools).
|
|
19
|
+
- **`START_HERE.md`**: opening question set updated to surface animated explainer as a first-class format choice.
|
|
20
|
+
- **`setup-keys.js`**: VoxForge added to TTS provider menu (second option, right after edge-tts). Four new config keys persisted: `VOXFORGE_URL`, `VOXFORGE_KEY`, `VOXFORGE_VOICE`, `VOXFORGE_MODEL`.
|
|
21
|
+
- **`scripts/deep-validate.js`**: +3 checks (`export_to_editor`, `voxforge`, tool count label updated to 18).
|
|
22
|
+
- Version bump: `2.3.1` → `2.3.2` across all version strings.
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
7
26
|
## [2.3.1] — 2026-11-04
|
|
8
27
|
|
|
9
28
|
### Added
|
package/CLAUDE.md
CHANGED
|
@@ -58,7 +58,9 @@ If you're not sure what a tool does — call it with the obvious parameters. The
|
|
|
58
58
|
|
|
59
59
|
## THREE PATHS — PICK ONE
|
|
60
60
|
|
|
61
|
-
|
|
61
|
+
> **The primary path is Path 2 (code-driven animation).** Every frame is a pure function of time `t`, rendered by Playwright + FFmpeg. This is what produces the high-quality, viral-style motion graphics clips. Path 1 (FFmpeg + AI images) is the fast fallback when Playwright is unavailable or the user wants a quick draft. Path 3 (auto-research) is for unattended batch.
|
|
62
|
+
|
|
63
|
+
### Path 1 — You write the scenes + AI images (quick draft)
|
|
62
64
|
|
|
63
65
|
Your IDE model writes the scenes. FrameForge generates TTS + AI images + renders with FFmpeg.
|
|
64
66
|
|
|
@@ -207,9 +209,20 @@ get_job_status({ jobId }) // poll every 30s until status = "SUCCEEDED"
|
|
|
207
209
|
|
|
208
210
|
---
|
|
209
211
|
|
|
210
|
-
## ALL
|
|
212
|
+
## ALL 18 MCP TOOLS
|
|
213
|
+
|
|
214
|
+
**Primary — code-driven animation (no LLM key, no image gen API):**
|
|
215
|
+
|
|
216
|
+
| Tool | What it does |
|
|
217
|
+
|------|-------------|
|
|
218
|
+
| `scaffold_animation_project` | Scaffold a Playwright + FFmpeg animation project (engine, render scripts, STORYBOARD.md) |
|
|
219
|
+
| `validate_storyboard` | Enforce the 4 hard gates before any scene code |
|
|
220
|
+
| `render_animation` | Playwright renderer: `sheet` / `strip` / `storyboard` / `crop` / `full` / `mux` / `kit` / `look` |
|
|
221
|
+
| `multi_voice_tts` | TTS per-call: `edge-tts` · **`voxforge`** · `elevenlabs` · `azure` · `openai` · `webhook` |
|
|
222
|
+
| `export_to_editor` | Send finished animation to Premiere / Resolve / CapCut / After Effects / Remotion |
|
|
223
|
+
| `external_platform_call` | HTTP bridge to VoxForge, Runway, Kling, Pika, or any platform |
|
|
211
224
|
|
|
212
|
-
**
|
|
225
|
+
**Secondary — FFmpeg + AI images (fast draft, no Playwright needed):**
|
|
213
226
|
|
|
214
227
|
| Tool | What it does |
|
|
215
228
|
|------|-------------|
|
|
@@ -219,15 +232,10 @@ get_job_status({ jobId }) // poll every 30s until status = "SUCCEEDED"
|
|
|
219
232
|
| `edit_scenes` | Patch captions, swap media files, append scenes |
|
|
220
233
|
| `assemble_video` | FFmpeg render + quality gate → master MP4 |
|
|
221
234
|
| `render_tts` | Generate voice audio for one scene |
|
|
222
|
-
| `multi_voice_tts` | Override TTS provider per call without touching `.env` |
|
|
223
235
|
| `generate_scene_image` | Generate one scene image |
|
|
224
236
|
| `publish_video` | Deliver via file copy or HTTP webhook |
|
|
225
|
-
| `external_platform_call` | HTTP bridge to ElevenLabs, Runway, Replicate, etc. |
|
|
226
|
-
| `scaffold_animation_project` | Create a code-driven animation project |
|
|
227
|
-
| `validate_storyboard` | Check STORYBOARD.md against the 4-gate contract |
|
|
228
|
-
| `render_animation` | Playwright renderer: sheet / strip / storyboard / crop / full / look |
|
|
229
237
|
|
|
230
|
-
**
|
|
238
|
+
**Auto-research (needs a free LLM key):**
|
|
231
239
|
|
|
232
240
|
| Tool | What it does |
|
|
233
241
|
|------|-------------|
|
|
@@ -238,6 +246,56 @@ get_job_status({ jobId }) // poll every 30s until status = "SUCCEEDED"
|
|
|
238
246
|
|
|
239
247
|
---
|
|
240
248
|
|
|
249
|
+
## VOICE — VoxForge custom audio
|
|
250
|
+
|
|
251
|
+
`multi_voice_tts` with `platform: "voxforge"` routes to your VoxForge instance:
|
|
252
|
+
|
|
253
|
+
```
|
|
254
|
+
multi_voice_tts({
|
|
255
|
+
text: "Your narration.",
|
|
256
|
+
outputPath: ".frameforge-files/voice_0.m4a",
|
|
257
|
+
platform: "voxforge",
|
|
258
|
+
voice: "my-cloned-voice-id", // speaker ID from VoxForge
|
|
259
|
+
apiKey: "vf-..." // or set VOXFORGE_KEY in .env
|
|
260
|
+
})
|
|
261
|
+
```
|
|
262
|
+
|
|
263
|
+
Set in `.env` for all calls:
|
|
264
|
+
```
|
|
265
|
+
PROVIDER_TTS=voxforge
|
|
266
|
+
VOXFORGE_URL=https://api.voxforge.io # or your self-hosted URL
|
|
267
|
+
VOXFORGE_KEY=vf-...
|
|
268
|
+
VOXFORGE_VOICE=my-cloned-voice-id
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
---
|
|
272
|
+
|
|
273
|
+
## EDITOR BRIDGES — send animation to editing apps
|
|
274
|
+
|
|
275
|
+
After `render_animation full`, send the finished MP4 into any editing app:
|
|
276
|
+
|
|
277
|
+
```
|
|
278
|
+
// Export EDL timeline first (for apps that support it)
|
|
279
|
+
render_animation({ projectDir: "...", mode: "kit" })
|
|
280
|
+
|
|
281
|
+
// Send to DaVinci Resolve (default bridge port :8080)
|
|
282
|
+
export_to_editor({ projectDir: "...", editor: "resolve" })
|
|
283
|
+
|
|
284
|
+
// Send to Premiere Pro (bridge port :7777)
|
|
285
|
+
export_to_editor({ projectDir: "...", editor: "premiere" })
|
|
286
|
+
|
|
287
|
+
// Send to CapCut (bridge port :9090)
|
|
288
|
+
export_to_editor({ projectDir: "...", editor: "capcut" })
|
|
289
|
+
|
|
290
|
+
// Custom endpoint
|
|
291
|
+
export_to_editor({ projectDir: "...", editor: "custom", editorUrl: "http://localhost:4000" })
|
|
292
|
+
```
|
|
293
|
+
|
|
294
|
+
Bridge servers run locally alongside the app. Setup docs: `docs/editor-bridges/`.
|
|
295
|
+
Override ports: `RESOLVE_BRIDGE_URL`, `PREMIERE_BRIDGE_URL`, `CAPCUT_BRIDGE_URL`, `AE_BRIDGE_URL`.
|
|
296
|
+
|
|
297
|
+
---
|
|
298
|
+
|
|
241
299
|
## SCENE FIELDS
|
|
242
300
|
|
|
243
301
|
| Field | Required | Notes |
|
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
[](https://www.npmjs.com/package/@amg555/frameforge-mcp)
|
|
10
10
|
[](LICENSE)
|
|
11
11
|
[](https://nodejs.org/)
|
|
12
|
-
[](CHANGELOG.md)
|
|
13
13
|
[](https://modelcontextprotocol.io/)
|
|
14
14
|
[](#testing)
|
|
15
15
|
|
package/install.sh
CHANGED
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@amg555/frameforge-mcp",
|
|
3
|
-
"version": "2.3.
|
|
3
|
+
"version": "2.3.2",
|
|
4
4
|
"description": "FrameForge — AI video production engine for agentic IDEs. npx @amg555/frameforge-mcp. FFmpeg + TTS run on your machine. 17 MCP tools.",
|
|
5
5
|
"publishConfig": {
|
|
6
6
|
"access": "public"
|
package/src/cli/doctor.js
CHANGED
|
@@ -89,7 +89,37 @@ if (edgeTts.status === 0) {
|
|
|
89
89
|
);
|
|
90
90
|
}
|
|
91
91
|
|
|
92
|
-
// 5.
|
|
92
|
+
// 5. Playwright / Chromium (required for code-driven animation — Path 2)
|
|
93
|
+
const playwright = spawnSync('node', ['-e', "require('playwright')"], {
|
|
94
|
+
encoding: 'utf8', timeout: 5000,
|
|
95
|
+
// Look in the project's own node_modules first (how it ships in the npm package)
|
|
96
|
+
cwd: path.resolve(__dirname, '../..'),
|
|
97
|
+
});
|
|
98
|
+
const playwrightGlobal = spawnSync('node', ['-e', "require('playwright')"], {
|
|
99
|
+
encoding: 'utf8', timeout: 5000,
|
|
100
|
+
});
|
|
101
|
+
const hasPlaywright = playwright.status === 0 || playwrightGlobal.status === 0;
|
|
102
|
+
|
|
103
|
+
// Check for a locally installed Chromium browser
|
|
104
|
+
const chromiumPaths = [
|
|
105
|
+
path.join(require('os').homedir(), '.cache', 'ms-playwright'), // Linux / macOS default
|
|
106
|
+
path.join(process.env.LOCALAPPDATA || '', 'ms-playwright'), // Windows
|
|
107
|
+
path.join(require('os').homedir(), 'Library', 'Caches', 'ms-playwright'), // macOS alt
|
|
108
|
+
];
|
|
109
|
+
const hasChromium = chromiumPaths.some(p => { try { return require('fs').readdirSync(p).length > 0; } catch { return false; } });
|
|
110
|
+
|
|
111
|
+
if (hasPlaywright && hasChromium) {
|
|
112
|
+
pass('Playwright', 'playwright package + Chromium browser installed (code-driven animation ready)');
|
|
113
|
+
} else if (hasPlaywright && !hasChromium) {
|
|
114
|
+
warn('Playwright', 'playwright package found but Chromium browser not installed',
|
|
115
|
+
'Run in your animation project dir: npx playwright install chromium');
|
|
116
|
+
} else {
|
|
117
|
+
warn('Playwright', 'Not installed — required for code-driven animation (Path 2)',
|
|
118
|
+
'Run in your animation project dir: npm install playwright && npx playwright install chromium\n' +
|
|
119
|
+
' (Path 1 via create_video_from_script does not need Playwright)');
|
|
120
|
+
}
|
|
121
|
+
|
|
122
|
+
// 6. System Font (for video subtitles/captions)
|
|
93
123
|
const FONT_CANDIDATES = [
|
|
94
124
|
'C:\\Windows\\Fonts\\arialbd.ttf',
|
|
95
125
|
'C:\\Windows\\Fonts\\calibrib.ttf',
|
package/src/cli/setup-keys.js
CHANGED
|
@@ -240,13 +240,20 @@ Press Enter to accept a default. Ctrl+C to cancel.
|
|
|
240
240
|
|
|
241
241
|
cfg.PROVIDER_TTS = await menu('Choose TTS provider', [
|
|
242
242
|
['edge-tts', 'edge-tts — free, offline, 400+ voices. No key needed.'],
|
|
243
|
+
['voxforge', 'VoxForge — custom voice cloning (github.com/amg555/voxforge). Get key: your VoxForge instance.'],
|
|
243
244
|
['elevenlabs', 'ElevenLabs — best quality + voice cloning. elevenlabs.io'],
|
|
244
245
|
['azure', 'Azure Cognitive Speech. portal.azure.com'],
|
|
245
246
|
['openai', 'OpenAI TTS (uses OPENAI_API_KEY above)'],
|
|
246
|
-
['webhook', 'Custom/
|
|
247
|
+
['webhook', 'Custom/any OpenAI-compatible TTS endpoint'],
|
|
247
248
|
], 'edge-tts');
|
|
248
249
|
|
|
249
|
-
if (cfg.PROVIDER_TTS === '
|
|
250
|
+
if (cfg.PROVIDER_TTS === 'voxforge') {
|
|
251
|
+
cfg.VOXFORGE_URL = await ask(' VoxForge base URL', 'https://api.voxforge.io');
|
|
252
|
+
cfg.VOXFORGE_KEY = await askSecret(' VoxForge API key (leave blank if self-hosted without auth)');
|
|
253
|
+
cfg.VOXFORGE_VOICE = await ask(' Voice ID or cloned speaker ID', 'narrator');
|
|
254
|
+
cfg.VOXFORGE_MODEL = await ask(' Model ID (leave blank for default)', 'voxforge-v1');
|
|
255
|
+
console.log(' ℹ️ Self-hosted? Set VOXFORGE_URL to your instance URL (e.g. http://localhost:8080).');
|
|
256
|
+
} else if (cfg.PROVIDER_TTS === 'elevenlabs') {
|
|
250
257
|
cfg.ELEVENLABS_API_KEY = await askSecret(' ElevenLabs API key');
|
|
251
258
|
cfg.ELEVENLABS_VOICE_ID = await ask(' Voice ID (leave blank for default Adam)', 'pNInz6obpgDQGcFmaJgB');
|
|
252
259
|
console.log(' ℹ️ For voice cloning: set ELEVENLABS_VOICE_ID to your cloned voice ID.');
|
|
@@ -326,7 +333,7 @@ Press Enter to accept a default. Ctrl+C to cancel.
|
|
|
326
333
|
|
|
327
334
|
const groups = {
|
|
328
335
|
'# ── LLM ─────────────────────────────────────────────────────────': ['PROVIDER_LLM','GROQ_API_KEY','CEREBRAS_API_KEY','OPENROUTER_API_KEY','ANTHROPIC_API_KEY','GEMINI_API_KEY','MISTRAL_API_KEY','TOGETHER_API_KEY','OPENAI_API_KEY','OLLAMA_BASE_URL','OLLAMA_MODEL','WEBHOOK_LLM_URL','WEBHOOK_LLM_HEADERS','WEBHOOK_LLM_RESPONSE_PATH','LLM_MODEL'],
|
|
329
|
-
'# ── TTS ─────────────────────────────────────────────────────────': ['PROVIDER_TTS','ELEVENLABS_API_KEY','ELEVENLABS_VOICE_ID','AZURE_SPEECH_KEY','AZURE_SPEECH_REGION','WEBHOOK_TTS_URL','WEBHOOK_TTS_RESPONSE','WEBHOOK_TTS_RESPONSE_PATH','TTS_VOICE','TTS_RATE'],
|
|
336
|
+
'# ── TTS ─────────────────────────────────────────────────────────': ['PROVIDER_TTS','VOXFORGE_URL','VOXFORGE_KEY','VOXFORGE_VOICE','VOXFORGE_MODEL','ELEVENLABS_API_KEY','ELEVENLABS_VOICE_ID','AZURE_SPEECH_KEY','AZURE_SPEECH_REGION','WEBHOOK_TTS_URL','WEBHOOK_TTS_RESPONSE','WEBHOOK_TTS_RESPONSE_PATH','TTS_VOICE','TTS_RATE'],
|
|
330
337
|
'# ── Image generation ─────────────────────────────────────────────': ['PROVIDER_IMAGE','FALAI_API_KEY','REPLICATE_API_KEY','WEBHOOK_IMAGE_URL','WEBHOOK_IMAGE_RESPONSE','WEBHOOK_IMAGE_RESPONSE_PATH','POLLINATIONS_MODEL'],
|
|
331
338
|
'# ── Delivery webhooks ────────────────────────────────────────────': ['PUBLISH_WEBHOOK_URL','REVIEW_NOTIFY_URL'],
|
|
332
339
|
'# ── External platform adapter ────────────────────────────────────': ['EXTERNAL_PLATFORM_NAME','EXTERNAL_PLATFORM_URL','EXTERNAL_PLATFORM_KEY','EXTERNAL_PLATFORM_INGEST_PATH','EXTERNAL_PLATFORM_RENDER_PATH','EXTERNAL_PLATFORM_STATUS_PATH'],
|
package/src/mcp-server/server.js
CHANGED
|
@@ -592,14 +592,14 @@ Example scene:
|
|
|
592
592
|
// ── Tool 12: Multi-platform TTS ──────────────────────────────────────────
|
|
593
593
|
// Call any voice platform per-request without changing global .env
|
|
594
594
|
multi_voice_tts: {
|
|
595
|
-
description: 'Convert text to speech using any supported voice platform per-call. Overrides the global PROVIDER_TTS for this one request. Supports: edge-tts, elevenlabs, azure, openai,
|
|
595
|
+
description: 'Convert text to speech using any supported voice platform per-call. Overrides the global PROVIDER_TTS for this one request. Supports: edge-tts, voxforge, elevenlabs, azure, openai, webhook.',
|
|
596
596
|
inputSchema: {
|
|
597
597
|
type: 'object',
|
|
598
598
|
required: ['text', 'outputPath'],
|
|
599
599
|
properties: {
|
|
600
600
|
text: { type: 'string', description: 'Text to synthesise' },
|
|
601
601
|
outputPath: { type: 'string', description: 'Where to save the audio file (.m4a or .mp3)' },
|
|
602
|
-
platform: { type: 'string', enum: ['edge-tts','elevenlabs','azure','openai','webhook'], description: 'TTS platform. Overrides PROVIDER_TTS for this call.' },
|
|
602
|
+
platform: { type: 'string', enum: ['edge-tts','voxforge','elevenlabs','azure','openai','webhook'], description: 'TTS platform. Overrides PROVIDER_TTS for this call. "voxforge" = github.com/amg555/voxforge; uses VOXFORGE_URL + VOXFORGE_KEY.' },
|
|
603
603
|
voice: { type: 'string', description: 'Voice ID. edge-tts: e.g. en-US-GuyNeural. ElevenLabs: voice_id. Azure: short name. OpenAI: alloy/echo/fable/nova/onyx/shimmer.' },
|
|
604
604
|
rate: { type: 'string', description: 'Speaking rate, e.g. "-10%" for slower, "+20%" for faster (edge-tts)' },
|
|
605
605
|
apiKey: { type: 'string', description: 'API key for this call (optional — falls back to env var)' },
|
|
@@ -608,9 +608,11 @@ Example scene:
|
|
|
608
608
|
sceneIndex: { type: 'number', description: 'Optional scene index for naming the output file' },
|
|
609
609
|
},
|
|
610
610
|
},
|
|
611
|
-
handler: async ({ text, outputPath, platform, voice, rate, apiKey, webhookUrl, productionId = 'adhoc', sceneIndex }) => {
|
|
611
|
+
handler: async ({ text, outputPath, platform: _platform, voice, rate, apiKey, webhookUrl, productionId = 'adhoc', sceneIndex }) => {
|
|
612
612
|
const syncFs = require('fs');
|
|
613
613
|
const pathMod = require('path');
|
|
614
|
+
// Normalise platform name so 'voxforge' routes correctly
|
|
615
|
+
const platform = _platform ? _platform.toLowerCase() : undefined;
|
|
614
616
|
|
|
615
617
|
// Guard: outputPath must not escape FILES_DIR — prevent path traversal
|
|
616
618
|
if (outputPath) {
|
|
@@ -1003,17 +1005,18 @@ Modes:
|
|
|
1003
1005
|
"crop" — focal-point 9:16 mobile crop preview; pass timestamps as [aspect_ratio_string, t]
|
|
1004
1006
|
"full" — full render → out/<title>.mp4
|
|
1005
1007
|
"mux" — re-mux audio only (no re-render, use after changing audio)
|
|
1008
|
+
"kit" — export EDL/XML for Premiere Pro / DaVinci Resolve; run after "full"
|
|
1006
1009
|
"look" — pre-render linter (fonts, clipped text, NaN transforms, missing assets)
|
|
1007
1010
|
|
|
1008
1011
|
Review budget: at least one "sheet" per shot, one "strip" per key motion/transition, one "crop" per story-carrying face.
|
|
1009
|
-
Returns { outputPath, passed } for "full", or { outputPath } for sheet/strip/storyboard.
|
|
1012
|
+
Returns { outputPath, passed } for "full", or { outputPath } for sheet/strip/storyboard/kit.
|
|
1010
1013
|
Requires: npm install playwright && npx playwright install chromium (one-time setup in projectDir).`,
|
|
1011
1014
|
inputSchema: {
|
|
1012
1015
|
type: 'object',
|
|
1013
1016
|
required: ['projectDir', 'mode'],
|
|
1014
1017
|
properties: {
|
|
1015
1018
|
projectDir: { type: 'string', description: 'The directory created by scaffold_animation_project' },
|
|
1016
|
-
mode: { type: 'string', enum: ['sheet', 'stills', 'strip', 'storyboard', 'crop', 'full', 'mux', 'look'], description: 'Render mode' },
|
|
1019
|
+
mode: { type: 'string', enum: ['sheet', 'stills', 'strip', 'storyboard', 'crop', 'full', 'mux', 'kit', 'look'], description: 'Render mode' },
|
|
1017
1020
|
timestamps: { type: 'array', items: {}, description: 'For mode="stills": [t1, t2, ...]. For mode="strip": [startSeconds, endSeconds]. For mode="crop": ["9:16", t]. For mode="sheet": omit or pass shot times.' },
|
|
1018
1021
|
audioPath: { type: 'string', description: 'Audio file to mux in (for mode="full" or "mux")' },
|
|
1019
1022
|
workers: { type: 'number', default: 4, description: 'Parallel Playwright pages for mode="full"' },
|
|
@@ -1091,6 +1094,9 @@ Requires: npm install playwright && npx playwright install chromium (one-time se
|
|
|
1091
1094
|
} else if (mode === 'full' || mode === 'mux') {
|
|
1092
1095
|
const mp4 = files.find(f => f.endsWith('.mp4') && !f.includes('_preview'));
|
|
1093
1096
|
if (mp4) outputPath = pathMod.join(outDir, mp4);
|
|
1097
|
+
} else if (mode === 'kit') {
|
|
1098
|
+
const edl = files.find(f => f.endsWith('.edl') || f.endsWith('.xml'));
|
|
1099
|
+
if (edl) outputPath = pathMod.join(outDir, edl);
|
|
1094
1100
|
} else if (mode === 'stills' || mode === 'strip' || mode === 'crop') {
|
|
1095
1101
|
const stills = files.filter(f => f.endsWith('.png') || f.endsWith('.jpg'));
|
|
1096
1102
|
if (stills.length > 0) outputPath = stills.map(f => pathMod.join(outDir, f));
|
|
@@ -1114,13 +1120,118 @@ Requires: npm install playwright && npx playwright install chromium (one-time se
|
|
|
1114
1120
|
},
|
|
1115
1121
|
},
|
|
1116
1122
|
|
|
1123
|
+
// ── Tool 18: Export to editing app ────────────────────────────────────────
|
|
1124
|
+
// Send a finished animation project to any local editing app via MCP bridge.
|
|
1125
|
+
// Works with Premiere Pro, DaVinci Resolve, CapCut, After Effects, or Remotion.
|
|
1126
|
+
export_to_editor: {
|
|
1127
|
+
description: `Send a finished FrameForge animation to a local editing app for post-production.
|
|
1128
|
+
|
|
1129
|
+
Supported editors (configure via env or pass editorUrl directly):
|
|
1130
|
+
"premiere" — Adobe Premiere Pro via local REST bridge (PREMIERE_BRIDGE_URL, default :7777)
|
|
1131
|
+
"resolve" — DaVinci Resolve Remote API (RESOLVE_BRIDGE_URL, default :8080)
|
|
1132
|
+
"capcut" — CapCut Desktop via MCP bridge (CAPCUT_BRIDGE_URL, default :9090)
|
|
1133
|
+
"aftereffects" — Adobe After Effects via AEIO bridge (AE_BRIDGE_URL, default :7878)
|
|
1134
|
+
"remotion" — Remotion local dev server (REMOTION_BASE_URL, default :3001)
|
|
1135
|
+
"custom" — any HTTP endpoint (pass editorUrl)
|
|
1136
|
+
|
|
1137
|
+
What this does:
|
|
1138
|
+
1. Imports the rendered MP4 (and optionally the audio stem) into the editor
|
|
1139
|
+
2. Exports an EDL/XML timeline from the animation project (render_animation kit mode)
|
|
1140
|
+
3. Returns the editor's import confirmation or render job ID
|
|
1141
|
+
|
|
1142
|
+
The bridge servers are lightweight companion apps you run locally alongside the editor.
|
|
1143
|
+
They are not installed by FrameForge — see docs/editor-bridges/ for setup instructions.`,
|
|
1144
|
+
inputSchema: {
|
|
1145
|
+
type: 'object',
|
|
1146
|
+
required: ['projectDir'],
|
|
1147
|
+
properties: {
|
|
1148
|
+
projectDir: { type: 'string', description: 'The animation project directory (from scaffold_animation_project)' },
|
|
1149
|
+
editor: { type: 'string', enum: ['premiere', 'resolve', 'capcut', 'aftereffects', 'remotion', 'custom'], default: 'resolve', description: 'Target editing app' },
|
|
1150
|
+
editorUrl: { type: 'string', description: 'Override the editor bridge URL (e.g. http://localhost:8080). Useful for custom or remote bridges.' },
|
|
1151
|
+
audioPath: { type: 'string', description: 'Audio stem to import alongside the video (optional)' },
|
|
1152
|
+
importOnly: { type: 'boolean', default: false, description: 'If true, only import the clip — do not trigger an export/render in the editor.' },
|
|
1153
|
+
},
|
|
1154
|
+
},
|
|
1155
|
+
handler: async ({ projectDir, editor = 'resolve', editorUrl, audioPath, importOnly = false }) => {
|
|
1156
|
+
const syncFs = require('fs');
|
|
1157
|
+
const pathMod = require('path');
|
|
1158
|
+
const { ExternalAdapter } = require('../pipeline/external-adapter');
|
|
1159
|
+
|
|
1160
|
+
if (!projectDir) throw new Error('projectDir is required');
|
|
1161
|
+
if (!syncFs.existsSync(projectDir)) throw new Error(`Project directory not found: ${projectDir}`);
|
|
1162
|
+
|
|
1163
|
+
// Find the most recently rendered MP4 in out/
|
|
1164
|
+
const outDir = pathMod.join(projectDir, 'out');
|
|
1165
|
+
if (!syncFs.existsSync(outDir)) {
|
|
1166
|
+
throw new Error(`No out/ directory in ${projectDir}. Run render_animation with mode="full" first.`);
|
|
1167
|
+
}
|
|
1168
|
+
const mp4Files = syncFs.readdirSync(outDir)
|
|
1169
|
+
.filter(f => f.endsWith('.mp4') && !f.includes('_preview'))
|
|
1170
|
+
.map(f => pathMod.join(outDir, f))
|
|
1171
|
+
.sort((a, b) => syncFs.statSync(b).mtimeMs - syncFs.statSync(a).mtimeMs);
|
|
1172
|
+
|
|
1173
|
+
if (mp4Files.length === 0) {
|
|
1174
|
+
throw new Error(`No rendered MP4 found in ${outDir}. Run render_animation with mode="full" first.`);
|
|
1175
|
+
}
|
|
1176
|
+
const masterPath = mp4Files[0];
|
|
1177
|
+
|
|
1178
|
+
// Also find the EDL from kit mode (generate it now if missing)
|
|
1179
|
+
const edlFiles = syncFs.readdirSync(outDir).filter(f => f.endsWith('.edl') || f.endsWith('.xml'));
|
|
1180
|
+
let edlPath = edlFiles.length > 0 ? pathMod.join(outDir, edlFiles[0]) : null;
|
|
1181
|
+
|
|
1182
|
+
// Configure the adapter for the target editor
|
|
1183
|
+
// editorUrl overrides the preset base URL
|
|
1184
|
+
const adapterConfig = editorUrl
|
|
1185
|
+
? { name: editor, baseUrl: editorUrl }
|
|
1186
|
+
: { name: editor };
|
|
1187
|
+
|
|
1188
|
+
const adapter = new ExternalAdapter(adapterConfig);
|
|
1189
|
+
|
|
1190
|
+
// Build the file list to import
|
|
1191
|
+
const files = [masterPath];
|
|
1192
|
+
if (audioPath && syncFs.existsSync(audioPath)) files.push(audioPath);
|
|
1193
|
+
if (edlPath) files.push(edlPath);
|
|
1194
|
+
|
|
1195
|
+
let importResult;
|
|
1196
|
+
try {
|
|
1197
|
+
importResult = await adapter.ingestMedia({
|
|
1198
|
+
files,
|
|
1199
|
+
metadata: {
|
|
1200
|
+
source: 'frameforge',
|
|
1201
|
+
projectDir,
|
|
1202
|
+
editor,
|
|
1203
|
+
importOnly: String(importOnly),
|
|
1204
|
+
},
|
|
1205
|
+
});
|
|
1206
|
+
} catch (err) {
|
|
1207
|
+
// If the bridge is not running, give a clear message instead of a raw HTTP error
|
|
1208
|
+
throw new Error(
|
|
1209
|
+
`Could not reach ${editor} bridge at ${adapter.baseUrl}.\n` +
|
|
1210
|
+
`Make sure the bridge is running.\n` +
|
|
1211
|
+
`Error: ${err.message}\n\n` +
|
|
1212
|
+
`See docs/editor-bridges/${editor}.md for setup instructions.`
|
|
1213
|
+
);
|
|
1214
|
+
}
|
|
1215
|
+
|
|
1216
|
+
return {
|
|
1217
|
+
editor,
|
|
1218
|
+
bridgeUrl: adapter.baseUrl,
|
|
1219
|
+
importedFiles: files.map(f => pathMod.basename(f)),
|
|
1220
|
+
masterPath,
|
|
1221
|
+
edlPath,
|
|
1222
|
+
importResult,
|
|
1223
|
+
message: `Imported into ${editor} at ${adapter.baseUrl}. ${edlPath ? 'EDL/XML timeline included.' : 'Tip: run render_animation mode="kit" to also export an EDL timeline.'}`,
|
|
1224
|
+
};
|
|
1225
|
+
},
|
|
1226
|
+
},
|
|
1227
|
+
|
|
1117
1228
|
};
|
|
1118
1229
|
|
|
1119
1230
|
// ── MCP protocol handler (MCP 2024-11-05 spec) ───────────────────────────
|
|
1120
1231
|
|
|
1121
1232
|
const SERVER_INFO = {
|
|
1122
1233
|
name: 'video-pipeline',
|
|
1123
|
-
version: '2.3.
|
|
1234
|
+
version: '2.3.2',
|
|
1124
1235
|
};
|
|
1125
1236
|
|
|
1126
1237
|
const CAPABILITIES = {
|
|
@@ -150,24 +150,20 @@ class ExternalAdapter {
|
|
|
150
150
|
_applyPreset() {
|
|
151
151
|
const name = this.name.toLowerCase();
|
|
152
152
|
const presets = {
|
|
153
|
+
// ── Voice / Audio platforms ─────────────────────────────────────────────
|
|
154
|
+
voxforge: {
|
|
155
|
+
baseUrl: this.baseUrl || process.env.VOXFORGE_URL || 'https://api.voxforge.io',
|
|
156
|
+
ingestPath: '/v1/voices/add', // upload a speaker sample for cloning
|
|
157
|
+
renderPath: '/v1/audio/speech', // OpenAI-compatible TTS synthesis
|
|
158
|
+
statusPath: '/v1/voices', // list voices / health check
|
|
159
|
+
},
|
|
153
160
|
elevenlabs: {
|
|
154
161
|
baseUrl: 'https://api.elevenlabs.io',
|
|
155
|
-
ingestPath: '/v1/voices/add',
|
|
156
|
-
renderPath: '/v1/text-to-speech',
|
|
157
|
-
statusPath: '/v1/voices',
|
|
158
|
-
},
|
|
159
|
-
remotion: {
|
|
160
|
-
baseUrl: process.env.REMOTION_BASE_URL || 'http://localhost:3001',
|
|
161
|
-
ingestPath: '/compositions',
|
|
162
|
-
renderPath: '/renders',
|
|
163
|
-
statusPath: '/renders',
|
|
164
|
-
},
|
|
165
|
-
voxforge: {
|
|
166
|
-
baseUrl: this.baseUrl || 'https://api.voxforge.io',
|
|
167
|
-
ingestPath: '/v1/ingest',
|
|
168
|
-
renderPath: '/v1/render',
|
|
169
|
-
statusPath: '/v1/status',
|
|
162
|
+
ingestPath: '/v1/voices/add',
|
|
163
|
+
renderPath: '/v1/text-to-speech',
|
|
164
|
+
statusPath: '/v1/voices',
|
|
170
165
|
},
|
|
166
|
+
// ── Video generation platforms ──────────────────────────────────────────
|
|
171
167
|
runway: {
|
|
172
168
|
baseUrl: 'https://api.runwayml.com',
|
|
173
169
|
ingestPath: '/v1/assets',
|
|
@@ -180,6 +176,50 @@ class ExternalAdapter {
|
|
|
180
176
|
renderPath: '/generate',
|
|
181
177
|
statusPath: '/generations',
|
|
182
178
|
},
|
|
179
|
+
kling: {
|
|
180
|
+
baseUrl: 'https://api.klingai.com',
|
|
181
|
+
ingestPath: '/v1/videos/image2video',
|
|
182
|
+
renderPath: '/v1/videos/image2video',
|
|
183
|
+
statusPath: '/v1/videos/image2video',
|
|
184
|
+
},
|
|
185
|
+
// ── Local editing app bridges ───────────────────────────────────────────
|
|
186
|
+
// These talk to local companion servers (CEP panel, local REST shim, etc.)
|
|
187
|
+
remotion: {
|
|
188
|
+
baseUrl: process.env.REMOTION_BASE_URL || 'http://localhost:3001',
|
|
189
|
+
ingestPath: '/compositions',
|
|
190
|
+
renderPath: '/renders',
|
|
191
|
+
statusPath: '/renders',
|
|
192
|
+
},
|
|
193
|
+
premiere: {
|
|
194
|
+
// Adobe Premiere Pro — requires the FrameForge CEP panel (see docs/premiere-panel/)
|
|
195
|
+
// OR the unofficial premiere-rest-bridge: github.com/premiere-rest-bridge
|
|
196
|
+
baseUrl: process.env.PREMIERE_BRIDGE_URL || 'http://localhost:7777',
|
|
197
|
+
ingestPath: '/import', // import a clip into the current project
|
|
198
|
+
renderPath: '/export', // export timeline to file
|
|
199
|
+
statusPath: '/status',
|
|
200
|
+
},
|
|
201
|
+
resolve: {
|
|
202
|
+
// DaVinci Resolve — requires local Resolve REST API bridge (port 8080 by default)
|
|
203
|
+
// Enable via: Preferences → System → Remote Control (DaVinci Resolve 18.6+)
|
|
204
|
+
baseUrl: process.env.RESOLVE_BRIDGE_URL || 'http://localhost:8080',
|
|
205
|
+
ingestPath: '/media/import',
|
|
206
|
+
renderPath: '/render/start',
|
|
207
|
+
statusPath: '/render/status',
|
|
208
|
+
},
|
|
209
|
+
capcut: {
|
|
210
|
+
// CapCut Desktop — requires CapCut MCP bridge (https://github.com/capcut-mcp)
|
|
211
|
+
baseUrl: process.env.CAPCUT_BRIDGE_URL || 'http://localhost:9090',
|
|
212
|
+
ingestPath: '/import',
|
|
213
|
+
renderPath: '/export',
|
|
214
|
+
statusPath: '/status',
|
|
215
|
+
},
|
|
216
|
+
aftereffects: {
|
|
217
|
+
// Adobe After Effects — requires AEIO REST bridge or ExtendScript server
|
|
218
|
+
baseUrl: process.env.AE_BRIDGE_URL || 'http://localhost:7878',
|
|
219
|
+
ingestPath: '/import',
|
|
220
|
+
renderPath: '/render',
|
|
221
|
+
statusPath: '/render/status',
|
|
222
|
+
},
|
|
183
223
|
};
|
|
184
224
|
|
|
185
225
|
const preset = presets[name];
|
|
@@ -390,12 +390,58 @@ class TTSProvider {
|
|
|
390
390
|
case 'elevenlabs': return this._elevenLabs({ text, voice: resolvedVoice, outputPath });
|
|
391
391
|
case 'azure': return this._azureTTS({ text, voice: resolvedVoice, outputPath });
|
|
392
392
|
case 'openai': return this._openaiTTS({ text, voice: resolvedVoice, outputPath });
|
|
393
|
+
case 'voxforge': return this._voxforgeTTS({ text, voice: resolvedVoice, outputPath });
|
|
393
394
|
case 'custom':
|
|
394
395
|
case 'webhook': return this._customTTS({ text, voice: resolvedVoice, outputPath });
|
|
395
396
|
default: return this._edgeTTS({ text, voice: resolvedVoice, outputPath });
|
|
396
397
|
}
|
|
397
398
|
}
|
|
398
399
|
|
|
400
|
+
/**
|
|
401
|
+
* VoxForge TTS — github.com/amg555/voxforge
|
|
402
|
+
* OpenAI-compatible /v1/audio/speech endpoint with optional voice cloning.
|
|
403
|
+
* Env vars:
|
|
404
|
+
* VOXFORGE_URL — base URL, e.g. https://api.voxforge.io (or self-hosted)
|
|
405
|
+
* VOXFORGE_KEY — API key (Bearer token)
|
|
406
|
+
* VOXFORGE_VOICE — voice ID or cloned speaker ID (default: 'narrator')
|
|
407
|
+
* VOXFORGE_MODEL — model ID (default: 'voxforge-v1')
|
|
408
|
+
*/
|
|
409
|
+
async _voxforgeTTS({ text, voice, outputPath }) {
|
|
410
|
+
const baseUrl = process.env.VOXFORGE_URL || 'https://api.voxforge.io';
|
|
411
|
+
const apiKey = this._overrides.apiKey || process.env.VOXFORGE_KEY || '';
|
|
412
|
+
const model = process.env.VOXFORGE_MODEL || 'voxforge-v1';
|
|
413
|
+
const resolvedVoice = voice || process.env.VOXFORGE_VOICE || 'narrator';
|
|
414
|
+
const rate = this._overrides.rate || process.env.TTS_RATE || '+0%';
|
|
415
|
+
|
|
416
|
+
const headers = { 'Content-Type': 'application/json' };
|
|
417
|
+
if (apiKey) headers['Authorization'] = `Bearer ${apiKey}`;
|
|
418
|
+
|
|
419
|
+
// Try OpenAI-compatible /v1/audio/speech first; fall back to /v1/tts
|
|
420
|
+
const endpoints = [
|
|
421
|
+
`${baseUrl}/v1/audio/speech`,
|
|
422
|
+
`${baseUrl}/v1/tts`,
|
|
423
|
+
];
|
|
424
|
+
|
|
425
|
+
return withRetry(async () => {
|
|
426
|
+
for (const url of endpoints) {
|
|
427
|
+
try {
|
|
428
|
+
const isOpenAIPath = url.endsWith('/speech');
|
|
429
|
+
const body = isOpenAIPath
|
|
430
|
+
? { model, input: text, voice: resolvedVoice }
|
|
431
|
+
: { text, voice: resolvedVoice, rate, model };
|
|
432
|
+
const audio = await httpPost(url, body, headers);
|
|
433
|
+
await fs.writeFile(outputPath, Buffer.isBuffer(audio) ? audio : Buffer.from(audio, 'binary'));
|
|
434
|
+
const stat = await fs.stat(outputPath);
|
|
435
|
+
if (stat.size > 500) return outputPath;
|
|
436
|
+
} catch (err) {
|
|
437
|
+
if (endpoints.indexOf(url) === endpoints.length - 1) throw err;
|
|
438
|
+
// try next endpoint
|
|
439
|
+
}
|
|
440
|
+
}
|
|
441
|
+
throw new Error('VoxForge: all endpoints failed');
|
|
442
|
+
}, { maxAttempts: 3, baseDelayMs: 3000, label: 'TTS(voxforge)' });
|
|
443
|
+
}
|
|
444
|
+
|
|
399
445
|
async _customTTS({ text, voice, outputPath }) {
|
|
400
446
|
const url = this._overrides.webhookUrl || process.env.CUSTOM_TTS_URL || process.env.TTS_WEBHOOK_URL;
|
|
401
447
|
if (!url) throw new Error('CUSTOM_TTS_URL or TTS_WEBHOOK_URL not set for custom/webhook TTS provider');
|