@amg555/frameforge-mcp 2.3.6 → 2.3.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -4,6 +4,41 @@ All notable changes to FrameForge are documented here.
4
4
 
5
5
  ---
6
6
 
7
+ ## [2.3.8] — 2026-11-09
8
+
9
+ ### Fixed & Enhanced
10
+ - **Multilingual Neural TTS Voice Routing (`src/pipeline/providers.js`)**:
11
+ - Automatically routes multilingual text to native neural voices: Malayalam (`ml-IN-MidhunNeural` / `ml-IN-SobhanaNeural`), Hindi (`hi-IN-MadhurNeural` / `hi-IN-SwaraNeural`), Tamil, Telugu, Chinese, Japanese, and Arabic.
12
+ - Eliminated Windows `shell: true` path space splitting and DEP0190 warnings in `_edgeTTS` by writing text to a UTF-8 temporary file and passing `--file` with `shell: false`.
13
+ - **Harness CLI Positional Arguments (`src/harness/bin/harness.mjs`)**:
14
+ - Added support for positional `.html` scene files (e.g. `node harness.mjs verify my-scene.html`).
15
+ - Added automatic fallback to `reference-scene.html` fixture so `npm run harness:verify` runs cleanly without requiring a pre-existing root `index.html`.
16
+
17
+ ---
18
+
19
+ ## [2.3.7] — 2026-11-09
20
+
21
+ ### Added
22
+ - **Autonomous Local Video Editor (`src/video-editor/editor.js`)**:
23
+ - Full-featured local FFmpeg video editing engine for autonomous agent workflows (inspired by `video-agent-skill`).
24
+ - `probeMedia`: `ffprobe` JSON parsing for duration, dimensions, fps, streams, codecs, and channels.
25
+ - `parseTimestamp`: Flexible timestamp parsing supporting numeric seconds, `"01:23.45"`, `"83.45s"`, `"01:02:03"`.
26
+ - `editClip`: Frame-accurate trimming, speed ramping (`0.25x` to `4.0x` with chained `atempo` pitch preservation), aspect ratio conforming (`16:9` landscape, `9:16` portrait, `1:1` square), color grading, and HarfBuzz captions.
27
+ - Cinematic color grading presets: `cinematic`, `warm`, `cool`, `vintage`, `bw`, `vibrant`, and `cyberpunk`.
28
+ - `concatClipsWithTransitions`: Seamless multi-clip transitions using FFmpeg `xfade` (`fade`, `dissolve`, `wipeleft`, `wiperight`, `slideup`, `slidedown`, `circleopen`, `smoothleft`) and `acrossfade`.
29
+ - `editAndAssemble`: End-to-end autonomous assembly with ambient clip audio volume, master voiceover overlay, and ducked background music with LUFS normalisation.
30
+ - Auto-fit mode: Automatically calculates clip durations so video length perfectly matches master voiceover length.
31
+ - `applyOperations`: Imperative operation dispatcher supporting `cut`, `trim`, `speed`, `color`, `caption`, `mute`, `transition`, and `assemble`.
32
+ - **`edit_video` MCP Tool (`src/mcp-server/server.js`)**:
33
+ - Exposes dedicated autonomous video editor tool to Claude Code, Cursor, Windsurf, Bob, and Codex.
34
+ - Local fallback in `external_platform_call` for `action === 'edit'`: runs local `VideoEditor` instead of erroring out when no external platform is configured.
35
+ - Enhanced `create_video_from_script` and `VideoRenderer.renderScene()` to support `startTime`, `endTime`, `speed`, `clipVolume`, and `colorGrade` for user clips (`video_path`).
36
+ - **Worker & Remote MCP Integration**:
37
+ - Registered `edit_video` in Cloudflare Remote MCP (`cloudflare-mcp/src/index.js`).
38
+ - Added `edit_video` job execution in background worker servers (`src/worker/server.js`, `src/worker/server-render-free.js`).
39
+
40
+ ---
41
+
7
42
  ## [2.3.6] — 2026-11-09
8
43
 
9
44
  ### Added
package/README.md CHANGED
@@ -9,7 +9,7 @@
9
9
  [![npm](https://img.shields.io/npm/v/@amg555/frameforge-mcp.svg?label=npm)](https://www.npmjs.com/package/@amg555/frameforge-mcp)
10
10
  [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
11
11
  [![Node.js](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg)](https://nodejs.org/)
12
- [![Version](https://img.shields.io/badge/version-2.3.6-blue.svg)](CHANGELOG.md)
12
+ [![Version](https://img.shields.io/badge/version-2.3.8-blue.svg)](CHANGELOG.md)
13
13
  [![MCP Spec](https://img.shields.io/badge/MCP-2024--11--05-blue.svg)](https://modelcontextprotocol.io/)
14
14
  [![Tests](https://img.shields.io/badge/tests-539%20passing-brightgreen.svg)](#testing)
15
15
 
@@ -52,7 +52,7 @@ Add to your IDE's MCP config (works in Bob, Cursor, Windsurf, Codex, Claude Code
52
52
  }
53
53
  ```
54
54
 
55
- That's it. Your agent now has 17 video production tools. FFmpeg and TTS run on your machine — no cloud render costs.
55
+ That's it. Your agent now has 19 video production & autonomous editing tools. FFmpeg and TTS run on your machine — no cloud render costs.
56
56
 
57
57
  ---
58
58
 
@@ -174,13 +174,14 @@ Any combination of `image_path`, `video_path`, `audio_path` works — FrameForge
174
174
 
175
175
  ## Tools
176
176
 
177
- 17 tools, grouped by what you're doing:
177
+ 19 tools, grouped by what you're doing:
178
178
 
179
179
  **Make a video from your own script (no LLM key)**
180
180
 
181
181
  | Tool | What it does |
182
182
  |------|-------------|
183
- | `create_video_from_script` | Full render from scenes you provide — voice, images, FFmpeg, quality gate |
183
+ | `create_video_from_script` | Full render from scenes you provide — voice, procedural motion graphics / custom code, FFmpeg, quality gate |
184
+ | `edit_video` | Autonomous local video editor — cut, trim, speed-ramp, color-grade, reframe (16:9 / 9:16 / 1:1), add captions, apply xfade transitions, and mix audio |
184
185
  | `ingest_media` | Import your own clips, images, and audio into a scenes manifest |
185
186
  | `edit_scenes` | Update captions, swap media files, or append scenes to a manifest |
186
187
  | `render_from_scenes` | Re-render from an existing manifest, skipping research and scripting |
@@ -189,7 +190,8 @@ Any combination of `image_path`, `video_path`, `audio_path` works — FrameForge
189
190
  | `multi_voice_tts` | Override TTS provider per call without touching `.env` |
190
191
  | `generate_scene_image` | Generate one scene image via the configured image provider |
191
192
  | `publish_video` | Deliver the finished video via file copy or HTTP webhook |
192
- | `external_platform_call` | HTTP bridge to ElevenLabs, Runway, Replicate, or any custom platform |
193
+ | `external_platform_call` | Bridge to ElevenLabs, Runway, VoxForge, or local FFmpeg editor fallback |
194
+ | `export_to_editor` | Bridge/import into DaVinci Resolve, Premiere Pro, After Effects, Remotion, or CapCut |
193
195
 
194
196
  **Auto-research pipeline (needs a free LLM key)**
195
197
 
@@ -0,0 +1,79 @@
1
+ {
2
+ "passed": true,
3
+ "timesTested": 9,
4
+ "results": [
5
+ {
6
+ "time": 0.1,
7
+ "a": "8d21a5d5a07ed0d7",
8
+ "b": "8d21a5d5a07ed0d7",
9
+ "c": "8d21a5d5a07ed0d7",
10
+ "ok": true,
11
+ "failure": null
12
+ },
13
+ {
14
+ "time": 0.3,
15
+ "a": "8d21a5d5a07ed0d7",
16
+ "b": "8d21a5d5a07ed0d7",
17
+ "c": "8d21a5d5a07ed0d7",
18
+ "ok": true,
19
+ "failure": null
20
+ },
21
+ {
22
+ "time": 0.56,
23
+ "a": "8d21a5d5a07ed0d7",
24
+ "b": "8d21a5d5a07ed0d7",
25
+ "c": "8d21a5d5a07ed0d7",
26
+ "ok": true,
27
+ "failure": null
28
+ },
29
+ {
30
+ "time": 0.8,
31
+ "a": "8d21a5d5a07ed0d7",
32
+ "b": "8d21a5d5a07ed0d7",
33
+ "c": "8d21a5d5a07ed0d7",
34
+ "ok": true,
35
+ "failure": null
36
+ },
37
+ {
38
+ "time": 1,
39
+ "a": "8d21a5d5a07ed0d7",
40
+ "b": "8d21a5d5a07ed0d7",
41
+ "c": "8d21a5d5a07ed0d7",
42
+ "ok": true,
43
+ "failure": null
44
+ },
45
+ {
46
+ "time": 1.24,
47
+ "a": "8d21a5d5a07ed0d7",
48
+ "b": "8d21a5d5a07ed0d7",
49
+ "c": "8d21a5d5a07ed0d7",
50
+ "ok": true,
51
+ "failure": null
52
+ },
53
+ {
54
+ "time": 1.5,
55
+ "a": "8d21a5d5a07ed0d7",
56
+ "b": "8d21a5d5a07ed0d7",
57
+ "c": "8d21a5d5a07ed0d7",
58
+ "ok": true,
59
+ "failure": null
60
+ },
61
+ {
62
+ "time": 1.76,
63
+ "a": "8d21a5d5a07ed0d7",
64
+ "b": "8d21a5d5a07ed0d7",
65
+ "c": "8d21a5d5a07ed0d7",
66
+ "ok": true,
67
+ "failure": null
68
+ },
69
+ {
70
+ "time": 1.9,
71
+ "a": "8d21a5d5a07ed0d7",
72
+ "b": "8d21a5d5a07ed0d7",
73
+ "c": "8d21a5d5a07ed0d7",
74
+ "ok": true,
75
+ "failure": null
76
+ }
77
+ ],
78
+ "framesSha256": "82dff4b5de7e0d9d"
79
+ }
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@amg555/frameforge-mcp",
3
- "version": "2.3.6",
4
- "description": "FrameForge — AI video production engine for agentic IDEs. npx @amg555/frameforge-mcp. FFmpeg + TTS run on your machine. 17 MCP tools.",
3
+ "version": "2.3.8",
4
+ "description": "FrameForge — AI video production engine for agentic IDEs. npx @amg555/frameforge-mcp. FFmpeg + TTS run on your machine. 19 MCP tools.",
5
5
  "publishConfig": {
6
6
  "access": "public"
7
7
  },
@@ -30,7 +30,7 @@
30
30
  "start:local-render": "cross-env RENDER_MODE=full WORKER_PORT=3211 node src/worker/server-render-free.js",
31
31
  "db:migrate": "node scripts/db-migrate.js",
32
32
  "validate": "node scripts/deep-validate.js",
33
- "test": "node --test src/pipeline/retry-policy.test.js src/pipeline/telemetry.test.js src/pipeline/artifact-manifest.test.js src/pipeline/artifact-retention.test.js src/pipeline/external-adapter.test.js src/workflows/publishing.test.js src/workflows/script.test.js src/animation/animation-system.test.js src/fact-checker/review-gate.test.js src/fact-checker/checker.test.js src/utils/log-redaction.test.js src/worker/server-render-free.test.js src/worker/server.test.js src/mcp-server/server.test.js src/quality-gate/gate.test.js src/shorts-extractor/extractor.test.js src/audio-mixer/mixer.test.js src/video-renderer/scene-animator.test.js src/video-renderer/renderer.test.js plugins/video-pipeline/skills/video-pipeline/assets/project/reel/engine.test.js src/harness/lib/pixels.test.js src/harness/lib/verify-purity.test.js src/harness/harness.test.js src/integration.test.js",
33
+ "test": "node --test src/pipeline/retry-policy.test.js src/pipeline/telemetry.test.js src/pipeline/artifact-manifest.test.js src/pipeline/artifact-retention.test.js src/pipeline/external-adapter.test.js src/workflows/publishing.test.js src/workflows/script.test.js src/animation/animation-system.test.js src/fact-checker/review-gate.test.js src/fact-checker/checker.test.js src/utils/log-redaction.test.js src/worker/server-render-free.test.js src/worker/server.test.js src/mcp-server/server.test.js src/quality-gate/gate.test.js src/shorts-extractor/extractor.test.js src/audio-mixer/mixer.test.js src/video-renderer/scene-animator.test.js src/video-renderer/renderer.test.js src/video-editor/editor.test.js plugins/video-pipeline/skills/video-pipeline/assets/project/reel/engine.test.js src/harness/lib/pixels.test.js src/harness/lib/verify-purity.test.js src/harness/harness.test.js src/integration.test.js",
34
34
  "lint": "echo 'No linter configured — add eslint for production'",
35
35
  "check-credentials": "node src/utils/log-redaction.js",
36
36
  "generate-assets": "node scripts/generate-initial-assets.js",
@@ -26,11 +26,11 @@ IDE Agent (IBM Bob / Codex / opencode / Cursor / Windsurf / Claude Code)
26
26
  │
27
27
  ▼ Cloudflare Remote MCP (recommended) ─────────────────────────────
28
28
  │ https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp
29
- │ 17 tools: create_video_from_script · render_from_scenes
29
+ │ 19 tools: create_video_from_script · edit_video · render_from_scenes
30
30
  │ ingest_media · edit_scenes · render_tts · multi_voice_tts
31
31
  │ generate_scene_image · assemble_video · publish_video
32
32
  │ research_topic · write_script · external_platform_call
33
- │ queue_full_pipeline · get_job_status
33
+ │ queue_full_pipeline · get_job_status · export_to_editor
34
34
  │ scaffold_animation_project · validate_storyboard · render_animation
35
35
  │
36
36
  ▼ OR — Local stdio / HTTP MCP ────────────────────────────────────
@@ -229,6 +229,49 @@ for research and scripting, so `PROVIDER_LLM` + the matching API key are require
229
229
  4. publish_video({ productionId, masterPath, channels: ["file"] })
230
230
  ```
231
231
 
232
+ ### Path 4 — Autonomous Local Video Editing (`edit_video`)
233
+
234
+ When the user provides video clips (B-roll, screen recordings, camera footage) and audio/narration, FrameForge autonomously cuts, trims, color-grades, reframes, captions, and stitches them with seamless transitions locally:
235
+
236
+ ```js
237
+ edit_video({
238
+ clips: [
239
+ {
240
+ video_path: "/path/to/clip1.mp4",
241
+ start_time: "00:05.5",
242
+ end_time: "00:15.0",
243
+ speed: 1.0,
244
+ color_grade: "cinematic",
245
+ caption: "AUTONOMOUS EDITING",
246
+ transition: "dissolve",
247
+ transition_duration: 0.5
248
+ },
249
+ {
250
+ video_path: "/path/to/clip2.mp4",
251
+ start_time: 0,
252
+ duration: 6.0,
253
+ speed: 0.8, // subtle slow-mo
254
+ color_grade: "cinematic",
255
+ caption: "SEAMLESS TRANSITIONS",
256
+ clip_volume: 0.3 // duck ambient sound
257
+ }
258
+ ],
259
+ audio_path: "/path/to/master_voiceover.m4a",
260
+ music_path: "/path/to/background_track.mp3",
261
+ music_volume: 0.07,
262
+ orientation: "landscape", // or "portrait" for Reels / TikTok / Shorts
263
+ auto_fit: true // automatically slice clips to fit total voiceover duration
264
+ })
265
+ ```
266
+
267
+ Supported features:
268
+ - **Trimming & Cutting:** Accurate timestamp parsing (`"01:23.45"`, `"83.45s"`, numeric seconds).
269
+ - **Speed Ramping:** `0.25x` to `4.0x` with chained `atempo` filters preserving pitch.
270
+ - **Cinematic Color Grading:** `cinematic`, `warm`, `cool`, `vintage`, `bw`, `vibrant`, `cyberpunk`.
271
+ - **Canvas Reframe:** `landscape` (1920×1080), `portrait` (1080×1920), `square` (1080×1080).
272
+ - **Transitions:** `fade`, `dissolve`, `wipeleft`, `wiperight`, `slideup`, `slidedown`, `circleopen`, `smoothleft`.
273
+ - **Audio Mixing & Ducking:** Original ambient audio level control, master voice track, background music with sidechain ducking and LUFS normalisation.
274
+
232
275
  ### Voice Synthesis (per call)
233
276
 
234
277
  ```
@@ -20,9 +20,11 @@ check('batch handling', code.includes('Array.isArray(body)'));
20
20
  check('ping handler', code.includes("method === 'ping'"));
21
21
  check('notifications/init', code.includes("notifications/initialized"));
22
22
  check('body size limit', code.includes('MAX_BODY_BYTES'));
23
- check('SERVER_INFO version 2.3.6', code.includes("version: '2.3.6'"));
23
+ check('SERVER_INFO version 2.3.8', code.includes("version: '2.3.8'"));
24
24
  check('18 tools defined (incl get_job_status)', code.includes('get_job_status'));
25
25
  check('export_to_editor tool defined', code.includes('export_to_editor'));
26
+ check('edit_video tool defined', code.includes('edit_video'));
27
+ check('video-editor exists', fs.existsSync('./src/video-editor/editor.js'));
26
28
  check('voxforge TTS provider', code.includes('voxforge'));
27
29
 
28
30
  // ── No execSync with shell strings in renderer or extractor ───────────────
@@ -205,7 +207,7 @@ check('render-free: default channel is file', renderFreeJs.includes("
205
207
  // ── server.js — queue_full_pipeline + tool completeness ──────────────────
206
208
  console.log('\n=== server.js tool completeness ===');
207
209
  const serverjs = fs.readFileSync('./src/mcp-server/server.js', 'utf8');
208
- check('server.js exposes 14 tools (header)', serverjs.includes('14 video production tools'));
210
+ check('server.js exposes tools (header)', serverjs.includes('video production'));
209
211
  check('server.js publish_video calls runPublishing', serverjs.includes("require('../workflows/publishing')"));
210
212
  check('server.js has queue_full_pipeline tool', serverjs.includes('queue_full_pipeline'));
211
213
  check('server.js has get_job_status tool', serverjs.includes('get_job_status'));
@@ -293,7 +295,7 @@ check('cf-worker has checkRateLimit', cfWorker.includes('checkRateLimit'));
293
295
  check('cf-worker hashes token (SHA-256)', cfWorker.includes('SHA-256'));
294
296
  check('cf-worker uses RATE_LIMITS KV', cfWorker.includes('env.RATE_LIMITS'));
295
297
  check('cf-worker returns 429 on limit', cfWorker.includes('429'));
296
- check('cf-worker version 2.3.6', cfWorker.includes("'2.3.6'"));
298
+ check('cf-worker version 2.3.8', cfWorker.includes("'2.3.8'"));
297
299
  check('wrangler.toml has kv_namespaces',fs.readFileSync('./cloudflare-mcp/wrangler.toml','utf8').includes('kv_namespaces'));
298
300
 
299
301
  // ── plugins/skills present ────────────────────────────────────────────────
@@ -64,7 +64,9 @@ function resolveProjectFile(filename, fallback) {
64
64
  return inCwd;
65
65
  }
66
66
 
67
- const HTML_FILE = flags.html ? path.resolve(CWD, flags.html) : resolveProjectFile('index.html', resolveProjectFile('studio.html'));
67
+ const positionalHtml = extras.find(e => typeof e === 'string' && e.endsWith('.html'));
68
+ const fixtureFallback = path.resolve(__dirname, '../test/fixtures/reference-scene.html');
69
+ const HTML_FILE = flags.html ? path.resolve(CWD, flags.html) : (positionalHtml ? path.resolve(CWD, positionalHtml) : resolveProjectFile('index.html', resolveProjectFile('studio.html', fixtureFallback)));
68
70
  const OUT_DIR = flags.outDir ? path.resolve(CWD, flags.outDir) : path.join(CWD, 'out');
69
71
  const CHECK_DIR = path.join(OUT_DIR, 'check');
70
72
  const FRAMES_DIR = flags.frames ? path.resolve(CWD, flags.frames) : flags.out ? path.resolve(CWD, flags.out) : path.join(OUT_DIR, 'frames');
@@ -1,13 +1,14 @@
1
1
  /**
2
2
  * FrameForge MCP Server — MCP 2024-11-05 spec compliant
3
3
  *
4
- * Exposes 14 video production tools via MCP.
5
- * Works with IBM Bob, Cursor, Windsurf, Codex, opencode, or any MCP-capable IDE.
4
+ * Exposes 19 video production & autonomous editing tools via MCP.
5
+ * Works with IBM Bob, Cursor, Windsurf, Codex, opencode, Claude Code, or any MCP-capable IDE.
6
6
  * Primary connection: Cloudflare Remote MCP (cloudflare-mcp/).
7
7
  * Alternative: stdio (src/mcp-server/stdio.js) or HTTP (this file).
8
8
  *
9
9
  * Primary path for agentic IDEs:
10
10
  * create_video_from_script — IDE writes scenes, FrameForge renders (no internal LLM)
11
+ * edit_video — Autonomous local video editor (cut, trim, speed, grade, transition, mix)
11
12
  * multi_voice_tts — per-call TTS platform override
12
13
  *
13
14
  * Full pipeline path (FrameForge does the research + scripting):
@@ -15,7 +16,7 @@
15
16
  * assemble_video, publish_video
16
17
  *
17
18
  * Edit path:
18
- * ingest_media, edit_scenes, render_from_scenes, external_platform_call
19
+ * edit_video, ingest_media, edit_scenes, render_from_scenes, external_platform_call, export_to_editor
19
20
  *
20
21
  * Start: node src/mcp-server/server.js (or: npm start)
21
22
  * Port: MCP_PORT env (default 3210)
@@ -482,8 +483,8 @@ Named presets (set EXTERNAL_PLATFORM_NAME to one of these and omit platformUrl):
482
483
 
483
484
  const adapter = new ExternalAdapter(adapterConfig);
484
485
 
485
- // If no URL was provided via param or env, give a clear setup message
486
- if (!adapter.isConfigured()) {
486
+ // If no URL was provided via param or env and action !== 'edit', give clear setup message
487
+ if (!adapter.isConfigured() && action !== 'edit') {
487
488
  throw new Error(
488
489
  'No platform URL configured. Either:\n' +
489
490
  ' • Pass platformUrl: "http://localhost:8080" (connects any custom app immediately)\n' +
@@ -500,12 +501,28 @@ Named presets (set EXTERNAL_PLATFORM_NAME to one of these and omit platformUrl):
500
501
  // Validate operations
501
502
  const ops = operations || [];
502
503
  if (ops.length === 0) throw new Error('action=edit requires at least one operation in the operations array');
503
- const validOps = ['cut','trim','transition','color','caption','mute','speed','swap'];
504
+ const validOps = ['cut','trim','transition','color','caption','mute','speed','swap','assemble'];
504
505
  for (const [i, o] of ops.entries()) {
505
506
  if (!o.op) throw new Error(`operations[${i}] missing "op" field`);
506
507
  if (!validOps.includes(o.op)) throw new Error(`operations[${i}].op "${o.op}" not valid. Use: ${validOps.join(', ')}`);
507
508
  }
508
- return adapter.renderScenes({ scenes: [], productionId: 'edit', options: { operations: ops, ...(metadata || {}) } });
509
+
510
+ if (adapter.isConfigured()) {
511
+ return adapter.renderScenes({ scenes: [], productionId: 'edit', options: { operations: ops, ...(metadata || {}) } });
512
+ }
513
+
514
+ // Fallback: Local autonomous VideoEditor execution
515
+ const { VideoEditor } = require('../video-editor/editor');
516
+ const localEditor = new VideoEditor({ filesDir: FILES_DIR });
517
+ return localEditor.applyOperations({
518
+ operations: ops,
519
+ files: files || [],
520
+ outputPath,
521
+ orientation: metadata?.orientation || 'landscape',
522
+ musicPath: metadata?.musicPath || metadata?.music_path,
523
+ musicVolume: metadata?.musicVolume != null ? metadata?.musicVolume : 0.08,
524
+ audioPath: metadata?.audioPath || metadata?.audio_path,
525
+ });
509
526
  }
510
527
 
511
528
  case 'render':
@@ -1416,13 +1433,98 @@ Operations (used with action="edit"):
1416
1433
  },
1417
1434
  },
1418
1435
 
1436
+ // ── Tool 19: Autonomous Local Video Editor ──────────────────────────────
1437
+ // Edit, cut, trim, speed-ramp, color-grade, reframe, add captions, and transition
1438
+ // multiple video clips, and mix with voiceover & ducked background music locally.
1439
+ edit_video: {
1440
+ description: `Autonomous local video editor. Edit, cut, trim, speed-ramp, color-grade, reframe (16:9 landscape / 9:16 portrait / 1:1 square), add captions, apply xfade transitions, and mix clip audio with voiceover & ducked background music locally using FFmpeg. Fully autonomous — zero external APIs or subscriptions needed.
1441
+
1442
+ Features:
1443
+ - Fast & frame-accurate trimming (start_time, end_time, duration).
1444
+ - Speed ramping (0.25x to 4.0x) with audio pitch preserving (atempo).
1445
+ - Cinematic color grading (cinematic, warm, cool, vintage, bw, vibrant, cyberpunk).
1446
+ - Broadcast-quality captions with HarfBuzz text shaping & multilingual Indic/Malayalam support.
1447
+ - Transitions: fade, dissolve, wipeleft, wiperight, slideup, slidedown, circleopen, smoothleft, etc.
1448
+ - Multi-track audio: clip ambient audio volume control, master voiceover, and background music with sidechain ducking.
1449
+ - Auto-fit mode: automatically calculates clip durations so video perfectly matches master audio length.`,
1450
+ inputSchema: {
1451
+ type: 'object',
1452
+ required: ['clips'],
1453
+ properties: {
1454
+ clips: {
1455
+ type: 'array',
1456
+ description: 'Array of clip objects. Each clip: { video_path, start_time?, end_time?, duration?, speed?, clip_volume?, color_grade?, caption?, transition?, transition_duration?, audio_path? }',
1457
+ items: {
1458
+ type: 'object',
1459
+ required: ['video_path'],
1460
+ properties: {
1461
+ video_path: { type: 'string', description: 'Path to source video clip' },
1462
+ start_time: { type: ['number', 'string'], description: 'Start time offset (e.g. 5, "00:05", "01:23.45")' },
1463
+ end_time: { type: ['number', 'string'], description: 'End time offset (e.g. 15, "00:15")' },
1464
+ duration: { type: ['number', 'string'], description: 'Desired duration in seconds' },
1465
+ speed: { type: 'number', default: 1.0, description: 'Speed multiplier (e.g. 0.5 for 2x slow-mo, 1.5 for fast)' },
1466
+ clip_volume: { type: 'number', default: 1.0, description: 'Volume of original clip audio (0 to mute, 1.0 normal)' },
1467
+ color_grade: { type: 'string', enum: ['none', 'cinematic', 'warm', 'cool', 'vintage', 'bw', 'vibrant', 'cyberpunk'], description: 'Color grading preset' },
1468
+ caption: { type: 'string', description: 'Caption text to burn into the lower third of this clip' },
1469
+ transition: { type: 'string', default: 'fade', description: 'Transition into next clip (fade, dissolve, wipeleft, wiperight, slideup, slidedown, circleopen, none)' },
1470
+ transition_duration: { type: 'number', default: 0.5, description: 'Duration of transition in seconds (default 0.5)' },
1471
+ audio_path: { type: 'string', description: 'Optional scene-specific audio/voice file' },
1472
+ },
1473
+ },
1474
+ },
1475
+ audio_path: { type: 'string', description: 'Master voiceover / narration audio track across whole video' },
1476
+ music_path: { type: 'string', description: 'Background music track' },
1477
+ music_volume: { type: 'number', default: 0.08, description: 'Background music volume level (default 0.08)' },
1478
+ orientation: { type: 'string', enum: ['landscape', 'portrait', 'square'], default: 'landscape', description: 'Target canvas orientation' },
1479
+ width: { type: 'number', description: 'Custom canvas width (default 1920 for landscape, 1080 for portrait/square)' },
1480
+ height: { type: 'number', description: 'Custom canvas height (default 1080 for landscape, 1920 for portrait, 1080 for square)' },
1481
+ color_grade: { type: 'string', enum: ['none', 'cinematic', 'warm', 'cool', 'vintage', 'bw', 'vibrant', 'cyberpunk'], description: 'Global color grade fallback for all clips' },
1482
+ transition: { type: 'string', default: 'fade', description: 'Global transition fallback between clips' },
1483
+ transition_duration: { type: 'number', default: 0.5, description: 'Global transition duration in seconds' },
1484
+ auto_fit: { type: 'boolean', default: false, description: 'Auto-adjust clip durations to match audio_path duration' },
1485
+ output_path: { type: 'string', description: 'Destination file path for the assembled MP4' },
1486
+ },
1487
+ },
1488
+ handler: async ({
1489
+ clips = [],
1490
+ audio_path = null,
1491
+ music_path = null,
1492
+ music_volume = 0.08,
1493
+ orientation = 'landscape',
1494
+ width,
1495
+ height,
1496
+ color_grade = null,
1497
+ transition = 'fade',
1498
+ transition_duration = 0.5,
1499
+ auto_fit = false,
1500
+ output_path = null,
1501
+ }) => {
1502
+ const { VideoEditor } = require('../video-editor/editor');
1503
+ const editor = new VideoEditor({ filesDir: FILES_DIR });
1504
+ return editor.editAndAssemble({
1505
+ clips,
1506
+ audioPath: audio_path,
1507
+ musicPath: music_path,
1508
+ musicVolume: music_volume,
1509
+ orientation,
1510
+ width,
1511
+ height,
1512
+ colorGrade: color_grade,
1513
+ transition,
1514
+ transitionDuration: transition_duration,
1515
+ autoFit: auto_fit,
1516
+ outputPath: output_path,
1517
+ });
1518
+ },
1519
+ },
1520
+
1419
1521
  };
1420
1522
 
1421
1523
  // ── MCP protocol handler (MCP 2024-11-05 spec) ───────────────────────────
1422
1524
 
1423
1525
  const SERVER_INFO = {
1424
1526
  name: 'video-pipeline',
1425
- version: '2.3.6',
1527
+ version: '2.3.8',
1426
1528
  };
1427
1529
 
1428
1530
  const CAPABILITIES = {
@@ -368,10 +368,38 @@ class TTSProvider {
368
368
  * picks by gender preference (TTS_VOICE_GENDER=male|female|default).
369
369
  *
370
370
  * @param {string} [explicitVoice] voice passed by caller (takes priority)
371
+ * @param {string} [text] narration text for language detection
371
372
  * @returns {string}
372
373
  */
373
- selectNarratorVoice(explicitVoice) {
374
+ selectNarratorVoice(explicitVoice, text = '') {
374
375
  if (explicitVoice) return explicitVoice;
376
+
377
+ // Language detection for edge-tts
378
+ if (this.name === 'edge-tts' && text) {
379
+ const gender = (process.env.TTS_VOICE_GENDER || 'default').toLowerCase();
380
+ if (/[\u0D00-\u0D7F]/.test(text)) {
381
+ return gender === 'female' ? 'ml-IN-SobhanaNeural' : 'ml-IN-MidhunNeural';
382
+ }
383
+ if (/[\u0900-\u097F]/.test(text)) {
384
+ return gender === 'female' ? 'hi-IN-SwaraNeural' : 'hi-IN-MadhurNeural';
385
+ }
386
+ if (/[\u0B80-\u0BFF]/.test(text)) {
387
+ return gender === 'female' ? 'ta-IN-PallaviNeural' : 'ta-IN-ValluvarNeural';
388
+ }
389
+ if (/[\u0C00-\u0C7F]/.test(text)) {
390
+ return gender === 'female' ? 'te-IN-ShrutiNeural' : 'te-IN-MohanNeural';
391
+ }
392
+ if (/[\u4E00-\u9FFF]/.test(text)) {
393
+ return gender === 'female' ? 'zh-CN-XiaoxiaoNeural' : 'zh-CN-YunjianNeural';
394
+ }
395
+ if (/[\u3040-\u30FF]/.test(text)) {
396
+ return gender === 'female' ? 'ja-JP-NanamiNeural' : 'ja-JP-KeitaNeural';
397
+ }
398
+ if (/[\u0600-\u06FF]/.test(text)) {
399
+ return gender === 'female' ? 'ar-SA-ZariyahNeural' : 'ar-SA-HamedNeural';
400
+ }
401
+ }
402
+
375
403
  const voices = NARRATOR_VOICES[this.name] || NARRATOR_VOICES['edge-tts'];
376
404
  const gender = (process.env.TTS_VOICE_GENDER || 'default').toLowerCase();
377
405
  return voices[gender] || voices.default;
@@ -384,7 +412,7 @@ class TTSProvider {
384
412
  * @returns {Promise<string>} resolves with outputPath
385
413
  */
386
414
  async synthesize({ text, voice, outputPath }) {
387
- const resolvedVoice = this.selectNarratorVoice(voice);
415
+ const resolvedVoice = this.selectNarratorVoice(voice, text);
388
416
  switch (this.name) {
389
417
  case 'edge-tts': return this._edgeTTS({ text, voice: resolvedVoice, outputPath });
390
418
  case 'elevenlabs': return this._elevenLabs({ text, voice: resolvedVoice, outputPath });
@@ -464,27 +492,36 @@ class TTSProvider {
464
492
 
465
493
  async _edgeTTS({ text, voice, outputPath }) {
466
494
  const rawPath = outputPath.replace(/\.[^.]+$/, '_raw.mp3');
467
- const safeVoice = voice || NARRATOR_VOICES['edge-tts'].default;
495
+ const txtPath = rawPath + '.txt';
496
+ const safeVoice = voice || this.selectNarratorVoice(null, text);
468
497
  // No artificial rate manipulation — use the voice's natural cadence.
469
498
  // Only a mild rate adjustment from env (e.g. TTS_RATE='+0%') is supported.
470
499
  const rate = this._overrides.rate || process.env.TTS_RATE || '+0%';
471
500
  return withRetry(async () => {
472
- const _shell = process.platform === 'win32';
473
- await new Promise((resolve, reject) => {
474
- const proc = spawn('edge-tts', ['--voice', safeVoice, '--rate', rate, '--text', text, '--write-media', rawPath], { timeout: 40000, shell: _shell });
475
- proc.on('close', code => code === 0 ? resolve() : reject(new Error(`edge-tts exited ${code}`)));
476
- proc.on('error', reject);
477
- });
478
- // Normalise audio via FFmpeg — trim trailing silence, normalise levels
479
- await new Promise((resolve, reject) => {
480
- const proc = spawn('ffmpeg', ['-nostdin', '-y', '-threads', '2',
481
- '-i', rawPath,
482
- '-af', 'silenceremove=stop_periods=-1:stop_duration=0.3:stop_threshold=-40dB,highpass=f=80,volume=1.0,dynaudnorm=f=150:g=5:p=0.95',
483
- '-ar', '48000', '-ac', '2', '-c:a', 'aac', '-b:a', '192k', outputPath], { timeout: 30000, shell: _shell });
484
- proc.on('close', code => code === 0 ? resolve() : reject(new Error(`ffmpeg exited ${code}`)));
485
- proc.on('error', reject);
486
- });
487
- await fs.unlink(rawPath).catch(() => {});
501
+ await fs.writeFile(txtPath, text, 'utf8');
502
+ try {
503
+ await new Promise((resolve, reject) => {
504
+ const proc = spawn('edge-tts', ['--voice', safeVoice, '--rate', rate, '--file', txtPath, '--write-media', rawPath], { timeout: 40000, shell: false });
505
+ let stderr = '';
506
+ proc.stderr?.on('data', d => { stderr += d; });
507
+ proc.on('close', code => code === 0 ? resolve() : reject(new Error(`edge-tts exited ${code}: ${stderr}`)));
508
+ proc.on('error', reject);
509
+ });
510
+ // Normalise audio via FFmpeg — trim trailing silence, normalise levels
511
+ await new Promise((resolve, reject) => {
512
+ const proc = spawn('ffmpeg', ['-nostdin', '-y', '-threads', '2',
513
+ '-i', rawPath,
514
+ '-af', 'silenceremove=stop_periods=-1:stop_duration=0.3:stop_threshold=-40dB,highpass=f=80,volume=1.0,dynaudnorm=f=150:g=5:p=0.95',
515
+ '-ar', '48000', '-ac', '2', '-c:a', 'aac', '-b:a', '192k', outputPath], { timeout: 30000 });
516
+ let stderr = '';
517
+ proc.stderr?.on('data', d => { stderr += d; });
518
+ proc.on('close', code => code === 0 ? resolve() : reject(new Error(`ffmpeg exited ${code}: ${stderr}`)));
519
+ proc.on('error', reject);
520
+ });
521
+ } finally {
522
+ await fs.unlink(txtPath).catch(() => {});
523
+ await fs.unlink(rawPath).catch(() => {});
524
+ }
488
525
  return outputPath;
489
526
  }, { maxAttempts: 3, baseDelayMs: 3000, label: 'TTS(edge-tts)' });
490
527
  }