@amg555/frameforge-mcp 2.3.6 → 2.3.7

package/CHANGELOG.md

@@ -4,6 +4,29 @@ All notable changes to FrameForge are documented here.
 
 ---
 
+## [2.3.7] — 2026-11-09
+
+### Added
+- **Autonomous Local Video Editor (`src/video-editor/editor.js`)**:
+  - Full-featured local FFmpeg video editing engine for autonomous agent workflows (inspired by `video-agent-skill`).
+  - `probeMedia`: `ffprobe` JSON parsing for duration, dimensions, fps, streams, codecs, and channels.
+  - `parseTimestamp`: Flexible timestamp parsing supporting numeric seconds, `"01:23.45"`, `"83.45s"`, `"01:02:03"`.
+  - `editClip`: Frame-accurate trimming, speed ramping (`0.25x` to `4.0x` with chained `atempo` pitch preservation), aspect ratio conforming (`16:9` landscape, `9:16` portrait, `1:1` square), color grading, and HarfBuzz captions.
+  - Cinematic color grading presets: `cinematic`, `warm`, `cool`, `vintage`, `bw`, `vibrant`, and `cyberpunk`.
+  - `concatClipsWithTransitions`: Seamless multi-clip transitions using FFmpeg `xfade` (`fade`, `dissolve`, `wipeleft`, `wiperight`, `slideup`, `slidedown`, `circleopen`, `smoothleft`) and `acrossfade`.
+  - `editAndAssemble`: End-to-end autonomous assembly with ambient clip audio volume, master voiceover overlay, and ducked background music with LUFS normalisation.
+  - Auto-fit mode: Automatically calculates clip durations so video length perfectly matches master voiceover length.
+  - `applyOperations`: Imperative operation dispatcher supporting `cut`, `trim`, `speed`, `color`, `caption`, `mute`, `transition`, and `assemble`.
+- **`edit_video` MCP Tool (`src/mcp-server/server.js`)**:
+  - Exposes dedicated autonomous video editor tool to Claude Code, Cursor, Windsurf, Bob, and Codex.
+  - Local fallback in `external_platform_call` for `action === 'edit'`: runs local `VideoEditor` instead of erroring out when no external platform is configured.
+  - Enhanced `create_video_from_script` and `VideoRenderer.renderScene()` to support `startTime`, `endTime`, `speed`, `clipVolume`, and `colorGrade` for user clips (`video_path`).
+- **Worker & Remote MCP Integration**:
+  - Registered `edit_video` in Cloudflare Remote MCP (`cloudflare-mcp/src/index.js`).
+  - Added `edit_video` job execution in background worker servers (`src/worker/server.js`, `src/worker/server-render-free.js`).
+
+---
+
 ## [2.3.6] — 2026-11-09
 
 ### Added

package/README.md

@@ -9,7 +9,7 @@
 [![npm](https://img.shields.io/npm/v/@amg555/frameforge-mcp.svg?label=npm)](https://www.npmjs.com/package/@amg555/frameforge-mcp)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
 [![Node.js](https://img.shields.io/badge/node-%3E%3D18.0.0-brightgreen.svg)](https://nodejs.org/)
-[![Version](https://img.shields.io/badge/version-2.3.6-blue.svg)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-2.3.7-blue.svg)](CHANGELOG.md)
 [![MCP Spec](https://img.shields.io/badge/MCP-2024--11--05-blue.svg)](https://modelcontextprotocol.io/)
 [![Tests](https://img.shields.io/badge/tests-539%20passing-brightgreen.svg)](#testing)
 
@@ -52,7 +52,7 @@ Add to your IDE's MCP config (works in Bob, Cursor, Windsurf, Codex, Claude Code
 }
 ```
 
-That's it. Your agent now has 17 video production tools. FFmpeg and TTS run on your machine — no cloud render costs.
+That's it. Your agent now has 19 video production & autonomous editing tools. FFmpeg and TTS run on your machine — no cloud render costs.
 
 ---
 
@@ -174,13 +174,14 @@ Any combination of `image_path`, `video_path`, `audio_path` works — FrameForge
 
 ## Tools
 
-17 tools, grouped by what you're doing:
+19 tools, grouped by what you're doing:
 
 **Make a video from your own script (no LLM key)**
 
 | Tool | What it does |
 |------|-------------|
-| `create_video_from_script` | Full render from scenes you provide — voice, images, FFmpeg, quality gate |
+| `create_video_from_script` | Full render from scenes you provide — voice, procedural motion graphics / custom code, FFmpeg, quality gate |
+| `edit_video` | Autonomous local video editor — cut, trim, speed-ramp, color-grade, reframe (16:9 / 9:16 / 1:1), add captions, apply xfade transitions, and mix audio |
 | `ingest_media` | Import your own clips, images, and audio into a scenes manifest |
 | `edit_scenes` | Update captions, swap media files, or append scenes to a manifest |
 | `render_from_scenes` | Re-render from an existing manifest, skipping research and scripting |
@@ -189,7 +190,8 @@ Any combination of `image_path`, `video_path`, `audio_path` works — FrameForge
 | `multi_voice_tts` | Override TTS provider per call without touching `.env` |
 | `generate_scene_image` | Generate one scene image via the configured image provider |
 | `publish_video` | Deliver the finished video via file copy or HTTP webhook |
-| `external_platform_call` | HTTP bridge to ElevenLabs, Runway, Replicate, or any custom platform |
+| `external_platform_call` | Bridge to ElevenLabs, Runway, VoxForge, or local FFmpeg editor fallback |
+| `export_to_editor` | Bridge/import into DaVinci Resolve, Premiere Pro, After Effects, Remotion, or CapCut |
 
 **Auto-research pipeline (needs a free LLM key)**
 

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@amg555/frameforge-mcp",
-  "version": "2.3.6",
-  "description": "FrameForge — AI video production engine for agentic IDEs. npx @amg555/frameforge-mcp. FFmpeg + TTS run on your machine. 17 MCP tools.",
+  "version": "2.3.7",
+  "description": "FrameForge — AI video production engine for agentic IDEs. npx @amg555/frameforge-mcp. FFmpeg + TTS run on your machine. 19 MCP tools.",
   "publishConfig": {
     "access": "public"
   },
@@ -30,7 +30,7 @@
     "start:local-render": "cross-env RENDER_MODE=full WORKER_PORT=3211 node src/worker/server-render-free.js",
     "db:migrate": "node scripts/db-migrate.js",
     "validate": "node scripts/deep-validate.js",
-    "test": "node --test src/pipeline/retry-policy.test.js src/pipeline/telemetry.test.js src/pipeline/artifact-manifest.test.js src/pipeline/artifact-retention.test.js src/pipeline/external-adapter.test.js src/workflows/publishing.test.js src/workflows/script.test.js src/animation/animation-system.test.js src/fact-checker/review-gate.test.js src/fact-checker/checker.test.js src/utils/log-redaction.test.js src/worker/server-render-free.test.js src/worker/server.test.js src/mcp-server/server.test.js src/quality-gate/gate.test.js src/shorts-extractor/extractor.test.js src/audio-mixer/mixer.test.js src/video-renderer/scene-animator.test.js src/video-renderer/renderer.test.js plugins/video-pipeline/skills/video-pipeline/assets/project/reel/engine.test.js src/harness/lib/pixels.test.js src/harness/lib/verify-purity.test.js src/harness/harness.test.js src/integration.test.js",
+    "test": "node --test src/pipeline/retry-policy.test.js src/pipeline/telemetry.test.js src/pipeline/artifact-manifest.test.js src/pipeline/artifact-retention.test.js src/pipeline/external-adapter.test.js src/workflows/publishing.test.js src/workflows/script.test.js src/animation/animation-system.test.js src/fact-checker/review-gate.test.js src/fact-checker/checker.test.js src/utils/log-redaction.test.js src/worker/server-render-free.test.js src/worker/server.test.js src/mcp-server/server.test.js src/quality-gate/gate.test.js src/shorts-extractor/extractor.test.js src/audio-mixer/mixer.test.js src/video-renderer/scene-animator.test.js src/video-renderer/renderer.test.js src/video-editor/editor.test.js plugins/video-pipeline/skills/video-pipeline/assets/project/reel/engine.test.js src/harness/lib/pixels.test.js src/harness/lib/verify-purity.test.js src/harness/harness.test.js src/integration.test.js",
     "lint": "echo 'No linter configured — add eslint for production'",
     "check-credentials": "node src/utils/log-redaction.js",
     "generate-assets": "node scripts/generate-initial-assets.js",

package/plugins/video-pipeline/skills/video-pipeline/SKILL.md

@@ -26,11 +26,11 @@ IDE Agent (IBM Bob / Codex / opencode / Cursor / Windsurf / Claude Code)
     │
     ▼  Cloudflare Remote MCP (recommended) ─────────────────────────────
     │  https://video-mcp.YOUR_SUBDOMAIN.workers.dev/mcp
-    │  17 tools: create_video_from_script · render_from_scenes
+    │  19 tools: create_video_from_script · edit_video · render_from_scenes
     │            ingest_media · edit_scenes · render_tts · multi_voice_tts
     │            generate_scene_image · assemble_video · publish_video
     │            research_topic · write_script · external_platform_call
-    │            queue_full_pipeline · get_job_status
+    │            queue_full_pipeline · get_job_status · export_to_editor
     │            scaffold_animation_project · validate_storyboard · render_animation
     │
     ▼  OR — Local stdio / HTTP MCP ────────────────────────────────────
@@ -229,6 +229,49 @@ for research and scripting, so `PROVIDER_LLM` + the matching API key are require
 4. publish_video({ productionId, masterPath, channels: ["file"] })
 ```
 
+### Path 4 — Autonomous Local Video Editing (`edit_video`)
+
+When the user provides video clips (B-roll, screen recordings, camera footage) and audio/narration, FrameForge autonomously cuts, trims, color-grades, reframes, captions, and stitches them with seamless transitions locally:
+
+```js
+edit_video({
+  clips: [
+    {
+      video_path: "/path/to/clip1.mp4",
+      start_time: "00:05.5",
+      end_time: "00:15.0",
+      speed: 1.0,
+      color_grade: "cinematic",
+      caption: "AUTONOMOUS EDITING",
+      transition: "dissolve",
+      transition_duration: 0.5
+    },
+    {
+      video_path: "/path/to/clip2.mp4",
+      start_time: 0,
+      duration: 6.0,
+      speed: 0.8, // subtle slow-mo
+      color_grade: "cinematic",
+      caption: "SEAMLESS TRANSITIONS",
+      clip_volume: 0.3 // duck ambient sound
+    }
+  ],
+  audio_path: "/path/to/master_voiceover.m4a",
+  music_path: "/path/to/background_track.mp3",
+  music_volume: 0.07,
+  orientation: "landscape", // or "portrait" for Reels / TikTok / Shorts
+  auto_fit: true // automatically slice clips to fit total voiceover duration
+})
+```
+
+Supported features:
+- **Trimming & Cutting:** Accurate timestamp parsing (`"01:23.45"`, `"83.45s"`, numeric seconds).
+- **Speed Ramping:** `0.25x` to `4.0x` with chained `atempo` filters preserving pitch.
+- **Cinematic Color Grading:** `cinematic`, `warm`, `cool`, `vintage`, `bw`, `vibrant`, `cyberpunk`.
+- **Canvas Reframe:** `landscape` (1920×1080), `portrait` (1080×1920), `square` (1080×1080).
+- **Transitions:** `fade`, `dissolve`, `wipeleft`, `wiperight`, `slideup`, `slidedown`, `circleopen`, `smoothleft`.
+- **Audio Mixing & Ducking:** Original ambient audio level control, master voice track, background music with sidechain ducking and LUFS normalisation.
+
 ### Voice Synthesis (per call)
 
 ```

package/scripts/deep-validate.js

@@ -20,9 +20,11 @@ check('batch handling',              code.includes('Array.isArray(body)'));
 check('ping handler',                code.includes("method === 'ping'"));
 check('notifications/init',          code.includes("notifications/initialized"));
 check('body size limit',             code.includes('MAX_BODY_BYTES'));
-check('SERVER_INFO version 2.3.6',   code.includes("version: '2.3.6'"));
+check('SERVER_INFO version 2.3.7',   code.includes("version: '2.3.7'"));
 check('18 tools defined (incl get_job_status)', code.includes('get_job_status'));
 check('export_to_editor tool defined', code.includes('export_to_editor'));
+check('edit_video tool defined', code.includes('edit_video'));
+check('video-editor exists', fs.existsSync('./src/video-editor/editor.js'));
 check('voxforge TTS provider', code.includes('voxforge'));
 
 // ── No execSync with shell strings in renderer or extractor ───────────────
@@ -205,7 +207,7 @@ check('render-free: default channel is file',            renderFreeJs.includes("
 // ── server.js — queue_full_pipeline + tool completeness ──────────────────
 console.log('\n=== server.js tool completeness ===');
 const serverjs = fs.readFileSync('./src/mcp-server/server.js', 'utf8');
-check('server.js exposes 14 tools (header)',         serverjs.includes('14 video production tools'));
+check('server.js exposes tools (header)',         serverjs.includes('video production'));
 check('server.js publish_video calls runPublishing', serverjs.includes("require('../workflows/publishing')"));
 check('server.js has queue_full_pipeline tool',      serverjs.includes('queue_full_pipeline'));
 check('server.js has get_job_status tool',           serverjs.includes('get_job_status'));
@@ -293,7 +295,7 @@ check('cf-worker has checkRateLimit',   cfWorker.includes('checkRateLimit'));
 check('cf-worker hashes token (SHA-256)', cfWorker.includes('SHA-256'));
 check('cf-worker uses RATE_LIMITS KV',  cfWorker.includes('env.RATE_LIMITS'));
 check('cf-worker returns 429 on limit', cfWorker.includes('429'));
-check('cf-worker version 2.3.6',        cfWorker.includes("'2.3.6'"));
+check('cf-worker version 2.3.7',        cfWorker.includes("'2.3.7'"));
 check('wrangler.toml has kv_namespaces',fs.readFileSync('./cloudflare-mcp/wrangler.toml','utf8').includes('kv_namespaces'));
 
 // ── plugins/skills present ────────────────────────────────────────────────

package/src/mcp-server/server.js

@@ -1,13 +1,14 @@
 /**
  * FrameForge MCP Server — MCP 2024-11-05 spec compliant
  *
- * Exposes 14 video production tools via MCP.
- * Works with IBM Bob, Cursor, Windsurf, Codex, opencode, or any MCP-capable IDE.
+ * Exposes 19 video production & autonomous editing tools via MCP.
+ * Works with IBM Bob, Cursor, Windsurf, Codex, opencode, Claude Code, or any MCP-capable IDE.
  * Primary connection: Cloudflare Remote MCP (cloudflare-mcp/).
  * Alternative: stdio (src/mcp-server/stdio.js) or HTTP (this file).
  *
  * Primary path for agentic IDEs:
  *   create_video_from_script — IDE writes scenes, FrameForge renders (no internal LLM)
+ *   edit_video               — Autonomous local video editor (cut, trim, speed, grade, transition, mix)
  *   multi_voice_tts          — per-call TTS platform override
  *
  * Full pipeline path (FrameForge does the research + scripting):
@@ -15,7 +16,7 @@
  *   assemble_video, publish_video
  *
  * Edit path:
- *   ingest_media, edit_scenes, render_from_scenes, external_platform_call
+ *   edit_video, ingest_media, edit_scenes, render_from_scenes, external_platform_call, export_to_editor
  *
  * Start: node src/mcp-server/server.js   (or: npm start)
  * Port:  MCP_PORT env (default 3210)
@@ -482,8 +483,8 @@ Named presets (set EXTERNAL_PLATFORM_NAME to one of these and omit platformUrl):
 
       const adapter = new ExternalAdapter(adapterConfig);
 
-      // If no URL was provided via param or env, give a clear setup message
-      if (!adapter.isConfigured()) {
+      // If no URL was provided via param or env and action !== 'edit', give clear setup message
+      if (!adapter.isConfigured() && action !== 'edit') {
         throw new Error(
           'No platform URL configured. Either:\n' +
           '  • Pass platformUrl: "http://localhost:8080" (connects any custom app immediately)\n' +
@@ -500,12 +501,28 @@ Named presets (set EXTERNAL_PLATFORM_NAME to one of these and omit platformUrl):
           // Validate operations
           const ops = operations || [];
           if (ops.length === 0) throw new Error('action=edit requires at least one operation in the operations array');
-          const validOps = ['cut','trim','transition','color','caption','mute','speed','swap'];
+          const validOps = ['cut','trim','transition','color','caption','mute','speed','swap','assemble'];
           for (const [i, o] of ops.entries()) {
             if (!o.op) throw new Error(`operations[${i}] missing "op" field`);
             if (!validOps.includes(o.op)) throw new Error(`operations[${i}].op "${o.op}" not valid. Use: ${validOps.join(', ')}`);
           }
-          return adapter.renderScenes({ scenes: [], productionId: 'edit', options: { operations: ops, ...(metadata || {}) } });
+
+          if (adapter.isConfigured()) {
+            return adapter.renderScenes({ scenes: [], productionId: 'edit', options: { operations: ops, ...(metadata || {}) } });
+          }
+
+          // Fallback: Local autonomous VideoEditor execution
+          const { VideoEditor } = require('../video-editor/editor');
+          const localEditor = new VideoEditor({ filesDir: FILES_DIR });
+          return localEditor.applyOperations({
+            operations: ops,
+            files: files || [],
+            outputPath,
+            orientation: metadata?.orientation || 'landscape',
+            musicPath: metadata?.musicPath || metadata?.music_path,
+            musicVolume: metadata?.musicVolume != null ? metadata?.musicVolume : 0.08,
+            audioPath: metadata?.audioPath || metadata?.audio_path,
+          });
         }
 
         case 'render':
@@ -1416,13 +1433,98 @@ Operations (used with action="edit"):
     },
   },
 
+  // ── Tool 19: Autonomous Local Video Editor ──────────────────────────────
+  // Edit, cut, trim, speed-ramp, color-grade, reframe, add captions, and transition
+  // multiple video clips, and mix with voiceover & ducked background music locally.
+  edit_video: {
+    description: `Autonomous local video editor. Edit, cut, trim, speed-ramp, color-grade, reframe (16:9 landscape / 9:16 portrait / 1:1 square), add captions, apply xfade transitions, and mix clip audio with voiceover & ducked background music locally using FFmpeg. Fully autonomous — zero external APIs or subscriptions needed.
+
+Features:
+- Fast & frame-accurate trimming (start_time, end_time, duration).
+- Speed ramping (0.25x to 4.0x) with audio pitch preserving (atempo).
+- Cinematic color grading (cinematic, warm, cool, vintage, bw, vibrant, cyberpunk).
+- Broadcast-quality captions with HarfBuzz text shaping & multilingual Indic/Malayalam support.
+- Transitions: fade, dissolve, wipeleft, wiperight, slideup, slidedown, circleopen, smoothleft, etc.
+- Multi-track audio: clip ambient audio volume control, master voiceover, and background music with sidechain ducking.
+- Auto-fit mode: automatically calculates clip durations so video perfectly matches master audio length.`,
+    inputSchema: {
+      type: 'object',
+      required: ['clips'],
+      properties: {
+        clips: {
+          type: 'array',
+          description: 'Array of clip objects. Each clip: { video_path, start_time?, end_time?, duration?, speed?, clip_volume?, color_grade?, caption?, transition?, transition_duration?, audio_path? }',
+          items: {
+            type: 'object',
+            required: ['video_path'],
+            properties: {
+              video_path:          { type: 'string', description: 'Path to source video clip' },
+              start_time:          { type: ['number', 'string'], description: 'Start time offset (e.g. 5, "00:05", "01:23.45")' },
+              end_time:            { type: ['number', 'string'], description: 'End time offset (e.g. 15, "00:15")' },
+              duration:            { type: ['number', 'string'], description: 'Desired duration in seconds' },
+              speed:               { type: 'number', default: 1.0, description: 'Speed multiplier (e.g. 0.5 for 2x slow-mo, 1.5 for fast)' },
+              clip_volume:         { type: 'number', default: 1.0, description: 'Volume of original clip audio (0 to mute, 1.0 normal)' },
+              color_grade:         { type: 'string', enum: ['none', 'cinematic', 'warm', 'cool', 'vintage', 'bw', 'vibrant', 'cyberpunk'], description: 'Color grading preset' },
+              caption:             { type: 'string', description: 'Caption text to burn into the lower third of this clip' },
+              transition:          { type: 'string', default: 'fade', description: 'Transition into next clip (fade, dissolve, wipeleft, wiperight, slideup, slidedown, circleopen, none)' },
+              transition_duration: { type: 'number', default: 0.5, description: 'Duration of transition in seconds (default 0.5)' },
+              audio_path:          { type: 'string', description: 'Optional scene-specific audio/voice file' },
+            },
+          },
+        },
+        audio_path:          { type: 'string', description: 'Master voiceover / narration audio track across whole video' },
+        music_path:          { type: 'string', description: 'Background music track' },
+        music_volume:        { type: 'number', default: 0.08, description: 'Background music volume level (default 0.08)' },
+        orientation:         { type: 'string', enum: ['landscape', 'portrait', 'square'], default: 'landscape', description: 'Target canvas orientation' },
+        width:               { type: 'number', description: 'Custom canvas width (default 1920 for landscape, 1080 for portrait/square)' },
+        height:              { type: 'number', description: 'Custom canvas height (default 1080 for landscape, 1920 for portrait, 1080 for square)' },
+        color_grade:         { type: 'string', enum: ['none', 'cinematic', 'warm', 'cool', 'vintage', 'bw', 'vibrant', 'cyberpunk'], description: 'Global color grade fallback for all clips' },
+        transition:          { type: 'string', default: 'fade', description: 'Global transition fallback between clips' },
+        transition_duration: { type: 'number', default: 0.5, description: 'Global transition duration in seconds' },
+        auto_fit:            { type: 'boolean', default: false, description: 'Auto-adjust clip durations to match audio_path duration' },
+        output_path:         { type: 'string', description: 'Destination file path for the assembled MP4' },
+      },
+    },
+    handler: async ({
+      clips = [],
+      audio_path = null,
+      music_path = null,
+      music_volume = 0.08,
+      orientation = 'landscape',
+      width,
+      height,
+      color_grade = null,
+      transition = 'fade',
+      transition_duration = 0.5,
+      auto_fit = false,
+      output_path = null,
+    }) => {
+      const { VideoEditor } = require('../video-editor/editor');
+      const editor = new VideoEditor({ filesDir: FILES_DIR });
+      return editor.editAndAssemble({
+        clips,
+        audioPath: audio_path,
+        musicPath: music_path,
+        musicVolume: music_volume,
+        orientation,
+        width,
+        height,
+        colorGrade: color_grade,
+        transition,
+        transitionDuration: transition_duration,
+        autoFit: auto_fit,
+        outputPath: output_path,
+      });
+    },
+  },
+
 };
 
 // ── MCP protocol handler (MCP 2024-11-05 spec) ───────────────────────────
 
 const SERVER_INFO = {
   name:    'video-pipeline',
-  version: '2.3.6',
+  version: '2.3.7',
 };
 
 const CAPABILITIES = {