ffmpeg-skill 2.9.0 → 2.10.0

package/README.md

@@ -257,7 +257,7 @@ These are the rules the skill file gives the agent and the code enforces.
 
 | Tool | What it does |
 |---|---|
-| `render.py` | Render a whole edit from a declarative `project.json` (clips, transitions, captions, overlays including the social sticker/hook/meme graphics, music and stem levels, loudness, export, chapter markers, check); `--init`, `--dry-run`, `--stop-after`, `--cache DIR`/`--from STAGE` (reuse the stages that did not change); the captions block takes the fit-size policy (`fit_size`, `min_size`, `fit_size_scope`) and the run reports the caption stage's counts as `caption`; `--template NAME INPUT` renders a shipped delivery template (`--template all` writes the whole social pack plus its table); `--export-timeline edit.fcpxml\|.edl\|.otio` (2.1) writes the cut as an editor timeline for Final Cut, DaVinci Resolve or Premiere instead of rendering it — clips, speed, dissolves, the music bed and chapter markers carry over, and `not_exported` names what stays behind |
+| `render.py` | Render a whole edit from a declarative `project.json` (clips, transitions, captions, overlays including the social sticker/hook/meme graphics and a picture-in-picture or chroma-keyed `video` layer sized for the final frame, music and stem levels, loudness, export, chapter markers, check); `--init`, `--dry-run`, `--stop-after`, `--cache DIR`/`--from STAGE` (reuse the stages that did not change); the captions block takes the fit-size policy (`fit_size`, `min_size`, `fit_size_scope`) and the run reports the caption stage's counts as `caption`; `--template NAME INPUT` renders a shipped delivery template (`--template all` writes the whole social pack plus its table); `--export-timeline edit.fcpxml\|.edl\|.otio` (2.1) writes the cut as an editor timeline for Final Cut, DaVinci Resolve or Premiere instead of rendering it — clips, speed, dissolves, the music bed and chapter markers carry over, and `not_exported` names what stays behind |
 | `batch.py` | Apply a step recipe or a project to a folder with a content-hash cache; `--watch`; `--jobs N` processes several files at once under one shared `--timeout` |
 | `multicam.py` | Align any number of cameras and recorders by audio (with drift correction) and cut between them from a switch list; `--switch energy` auto-cuts to whichever camera is loudest/talking instead of a hand-built list, with `--min-shot` (minimum shot length) and `--edl` (write a plain START-END cut list, not a CMX EDL; for Premiere/Resolve use `--write-project` then `render.py --export-timeline`) |
 | `verify.py` | Run the toolchain on real device files and report PASS / FAIL per step |

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`2.9.0`) | any release |
+| `skill.version` | the npm / package.json version (`2.10.0`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -119,7 +119,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
   "contract_version": "1.0",
   "deprecated": [],
   "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "2.9.0", "execution_mode": "local", "kind": "execution",
+  "skill": {"id": "ffmpeg-skill", "version": "2.10.0", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "2.9.0",
+  "version": "2.10.0",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/references/scripts.md

@@ -735,6 +735,23 @@ stem wins, and a stems level with no file to apply it to (`effects` without
 `"audio": {"voice": "light"|"medium"|"strong"}` picks the voice strength
 (`true` is `medium`).
 
+An `overlays[]` entry draws one of `"logo": true`, `"image"`, `"text"` or
+`"video"` (#326: a picture-in-picture or green-screen layer, `overlay.py
+--video`), with `position`, `scale`, `opacity`, `start`, `end`, `margin`,
+`platform` (and `fade`, `box`, `font*` for the still/text kinds) mapping to
+overlay.py's flags. `"chromakey": "0x00ff00"`, `"chromakey_similarity"` and
+`"chromakey_blend"` key a colour out of the video layer. The stage runs after
+`fit`, so `"scale": 480` is 480 px of the delivered frame, and a template's
+safe margins apply as they do to a logo. Refused (`kind: input`) before the
+first stage: `video` together with `logo`/`image`/`text` in one entry (a
+logo, image and text together still draw the first of them, as before), a
+`chromakey*` key without `video`, `chromakey_similarity`/`_blend` without
+`chromakey`, and a layer file that does not exist (under `--dry-run` too: no
+earlier stage writes it). The layer's content is in the stage's `--cache`
+key, so replacing the file re-runs `overlays` and what follows it; paths are
+relative to the project file; `--export-timeline` lists the layer under
+`not_exported`.
+
 `"chapters"` is a chapters file path, or an inline list of `{"at": TIME,
 "title": STR}`; it runs `metadata.py` on the delivered file as the last stage
 before `check`, so the markers are in the file that ships (streams copied). The

package/scripts/_common/timeline.py

@@ -39,7 +39,7 @@ _NOT_EXPORTED = {
     "silence": "the silence cut (render.py measures it at render time; the timeline keeps the clips whole)",
     "captions": "captions (burn them with render.py, or import the .srt into the editor)",
     "graphics": "graphics (titles, lower thirds, social templates)",
-    "overlays": "overlays (logos and text)",
+    "overlays": "overlays (logos, images, text and video layers)",
     "fit": "fit / reframe to the delivery aspect (set the sequence frame in the editor)",
     "audiogram": "the audiogram picture",
     "loudness": "loudness normalisation",

package/scripts/render.py

@@ -22,7 +22,8 @@ Project format (all keys optional except clips):
   ],
   "overlays": [
     {"logo": true},
-    {"text": "Episode 12", "position": "bottom", "start": 1, "end": 5, "fade": 0.3, "box": true}
+    {"text": "Episode 12", "position": "bottom", "start": 1, "end": 5, "fade": 0.3, "box": true},
+    {"video": "webcam.mp4", "position": "bottom-right", "scale": 480, "chromakey": "0x00ff00"}
   ],
   "audio": {"voice": "medium", "music": "bed.mp3", "music_volume": -16, "duck": true, "music_fade_out": 2,
             "effects": "sfx.wav", "stems": {"dialogue": 0, "music": -18, "effects": -22}},
@@ -37,6 +38,9 @@ Stages run in this order: clips (cut) → join → silence → fit → captions
 graphics → overlays → audio → loudness → export → chapters → check. Missing stages are
 skipped. "brand" points caption/graphics/overlay at a brand.json (fonts,
 colours, logo, safe margin); {"logo": true} in overlays places the brand logo.
+An overlay entry draws one of logo, image, text or video (a picture-in-picture
+layer, overlay.py --video; "chromakey", "chromakey_similarity" and
+"chromakey_blend" key a colour out of that video and need "video").
 
 "check" mirrors check.py's own exit code: a delivery-spec FAIL (or check.py
 itself failing to run) exits 1, same as running check.py directly would --
@@ -140,8 +144,10 @@ OBJECT_KEYS: Dict[str, frozenset] = {
     "graphics[]": frozenset({"template", "name", "title", "subtitle", "start", "end", "position",
                              "from", "scale", "primary", "text_color", "lang",
                              "text", "top", "bottom", "duration", "margin", "platform"}),
+    # #326: "video" is a fourth layer kind (overlay.py --video), keyed with the chromakey* keys
     "overlays[]": frozenset({"logo", "image", "text", "position", "start", "end", "fade", "opacity",
-                             "scale", "font_size", "font", "font_file", "margin", "box", "platform"}),
+                             "scale", "font_size", "font", "font_file", "margin", "box", "platform",
+                             "video", "chromakey", "chromakey_similarity", "chromakey_blend"}),
     "audio": frozenset({"music", "replace", "music_volume", "fade_in", "fade_out", "music_fade_out",
                         "gain", "duck_amount", "duck_threshold", "duck_attack", "duck_release",
                         "voice", "denoise", "duck", "music_loop", "stereo", "mono", "downmix",
@@ -410,6 +416,28 @@ def check_keys(obj: Any, schema: str, label: str) -> None:
             else f" (valid keys: {', '.join(sorted(valid))})"))
 
 
+def check_overlay(ov: Any, label: str) -> None:
+    """#326: a video layer is one kind of overlay, as overlay.py's --image/--text/--logo/--video
+    are one mutually exclusive choice, and the chromakey* keys only act on it. A logo, image and
+    text in one entry still draw the first of them, as before; only a combination with the new
+    "video" key is refused, so no project 2.8.6 rendered is refused now."""
+    if not isinstance(ov, dict):
+        return
+    if ov.get("video"):
+        others = [k for k in ("logo", "image", "text") if ov.get(k)]
+        if others:
+            die(f"{label}: give one of logo, image, text or video per entry, not video and {others[0]} "
+                "(add a second overlays[] entry for the other layer)")
+    else:
+        keyed = [k for k in ("chromakey", "chromakey_similarity", "chromakey_blend") if ov.get(k) is not None]
+        if keyed:
+            die(f"{label}: {keyed[0]} keys a colour out of a video layer and needs \"video\" (overlay.py --chromakey needs --video)")
+    if ov.get("chromakey") is None:
+        tuned = [k for k in ("chromakey_similarity", "chromakey_blend") if ov.get(k) is not None]
+        if tuned:
+            die(f"{label}: {tuned[0]} tunes \"chromakey\", which this entry does not set")
+
+
 def require_object(obj: Any, label: str) -> None:
     """Refuse a value that is not an object where the run reads it with .get(): "export": "reels"
     in a render, a string frame, a clip written as a bare path. 2.2.1 died there with a traceback
@@ -470,6 +498,8 @@ def validate_project(proj: Dict[str, Any], timeline: bool = False, stop_after: O
                     die(f"{name}[{i}]: must be an object {{...}}, got {type(item).__name__} {item!r:.60}")
         for i, item in enumerate(items if isinstance(items, list) else []):
             check_keys(item, f"{name}[]", f"{name}[{i}]")
+            if name == "overlays":
+                check_overlay(item, f"overlays[{i}]")
             if name != "clips":
                 continue
             # every clip stage starts from rel(c["src"]): without one it was a KeyError traceback
@@ -954,6 +984,15 @@ def main() -> int:
     output = rel(proj.get("output") or "final.mp4")
     # the final stage is a copy from the work dir, so run()'s own guard never sees the sources (review 5)
     refuse_output_is_input(output, *[rel(c.get("src")) for c in clips if c.get("src")])
+    if _reaches("overlays", args.stop_after):
+        # #326: an overlay video is the caller's file, never an earlier stage's output, so a
+        # missing one is refused here, under --dry-run too, not after every stage before it ran
+        for i, ov in enumerate(proj.get("overlays") or []):
+            if ov.get("video"):
+                layer = rel(ov["video"])
+                refuse_output_is_input(output, layer)
+                if not os.path.isfile(layer):
+                    die(f"overlays[{i}].video: file not found: {layer}", kind="input")
     # 2.0 refuses an existing output without --overwrite; the final place_output() would, but
     # only after every stage had run -- say so before the first one instead
     _check_existing_output(["ffmpeg", output])
@@ -1248,9 +1287,13 @@ def main() -> int:
             argv += ["--image", rel(ov["image"])]
         elif ov.get("text"):
             argv += ["--text", ov["text"]]
+        elif ov.get("video"):
+            # #326: sized and placed here, after fit, against the frame that ships
+            argv += ["--video", rel(ov["video"])]
         else:
-            die(f"overlays[{i}] needs image or text")
-        for k, flag in (("position", "--position"), ("start", "--start"), ("end", "--end"), ("fade", "--fade"), ("opacity", "--opacity"), ("scale", "--scale"), ("font_size", "--font-size"), ("font", "--font"), ("font_file", "--font-file"), ("margin", "--margin"), ("platform", "--platform")):
+            die(f"overlays[{i}] needs logo, image, text or video")
+        for k, flag in (("position", "--position"), ("start", "--start"), ("end", "--end"), ("fade", "--fade"), ("opacity", "--opacity"), ("scale", "--scale"), ("font_size", "--font-size"), ("font", "--font"), ("font_file", "--font-file"), ("margin", "--margin"), ("platform", "--platform"),
+                        ("chromakey", "--chromakey"), ("chromakey_similarity", "--chromakey-similarity"), ("chromakey_blend", "--chromakey-blend")):
             if ov.get(k) is not None:
                 argv += [flag, str(ov[k])]
         if ov.get("box"):