ffmpeg-skill 2.10.6 → 2.10.8

package/README.md

@@ -395,9 +395,13 @@ The short list for humans. The agent-facing version, with the reasoning, is the
 
 Each operation is a script whose flags are its argparse parser, and that parser is what `contract --json` and the MCP `inputSchema` are built from. CI fails the build when those drift from the scripts. A caller-supplied filter graph, `-vf` string, or other ffmpeg option is not an argument any script accepts. `doctor` reads `ffmpeg -filters` (and `-encoders`, `-bsfs`) and reports a filter a tool needs before a job runs into it. MCP does accept an `argv` array, and that array is the named script's own CLI flags, still bound to that script: the script's flags, never an ffmpeg command line and never a shell.
 
-### How does this differ from Remotion or HyperFrames? Can they be used together?
+### How does ffmpeg-skill compare with Remotion?
 
-Remotion renders a video from React components. HyperFrames renders one from HTML. This repository does neither, and it does not call either of them. It edits a media file you already have, on the machine, with ffmpeg and ffprobe, then probes the file it wrote. A render from either tool is an ordinary input: `probe.py`, `cut.py`, `fit.py`, `export.py` and `check.py` take the file the same way they take any other clip.
+ffmpeg-skill edits footage you already have, on this machine, with FFmpeg tools whose flags are a machine-readable contract (`contract --json`). Remotion renders a video from React components. They complement each other: this repository does not call Remotion, and a file Remotion writes is an ordinary input (`probe.py`, `cut.py`, `fit.py`, `export.py`, `check.py`). HyperFrames renders a video from HTML and is the same kind of neighbour: a file it writes is an ordinary input too.
+
+### Does it validate the filter graph before running?
+
+No. `--dry-run` prints the ffmpeg command a writing tool would run and does not start ffmpeg, so ffmpeg never parses that filter graph. ffprobe still runs when the tool probes an input that exists. Unless `--overwrite` is passed, an existing output is refused (`kind: input`) before ffmpeg. An output that is the same file as an input is refused either way. No media file is written, and `verified` is false; `--plan FILE` does write the plan JSON. A path that is not on disk yet is not measured: the probe is a stub (0×0, duration 0), so a number taken from it is not a measurement. The measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) still run their measurements; only the artifact is skipped, and `silence` with `--filler --transcribe` also runs the speech engine, which may download a model. `verify` accepts `--dry-run` and ignores it: its steps run. `--plan FILE` is that dry run written down — the planned argv, the command lines, and a fingerprint of each existing input (size, plus a sha256 of the first and last 8 MiB). `render.py` and `verify.py` refuse `--plan`. `render.py FILE` runs a plan only when those fingerprints still match, by running the tool again with the planned argv, not by executing the recorded ffmpeg lines; `--dry-run` on that run only prints them. `doctor` reports whether the filters, encoders and bitstream filters the contract names are in this build (`ffmpeg -filters`, `-encoders`, `-bsfs`) and which fonts are installed. A name in that listing is not a check of the graph this command would build, and a listed GPU encoder is not a successful encode.
 
 ### Can it remove burned-in subtitles or logos?
 

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`2.10.6`) | any release |
+| `skill.version` | the npm / package.json version (`2.10.8`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -119,7 +119,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
   "contract_version": "1.0",
   "deprecated": [],
   "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "2.10.6", "execution_mode": "local", "kind": "execution",
+  "skill": {"id": "ffmpeg-skill", "version": "2.10.8", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
@@ -613,7 +613,7 @@ Per-tool keys added in 1.18.0, all additive:
 | `shots` | `scenes.py --shots` | `[{start, end, label, flow_magnitude}]` per detected scene, `label` one of `static`/`pan`/`motion` from a lightweight block-matching optical-flow proxy (frames decoded at 4 fps, 48x27, no external dependency). A shot too short to sample two frames is `static` with `flow_magnitude: 0` |
 | `audio_peaks_db` | `scenes.py --audio-peaks` | `[{time, level}]`, measured dBFS loudness peaks. A **new** key: the pre-existing `audio_peaks` (always reported, unrelated unitless RMS figures used for `--highlights` scoring) keeps its 1.0 meaning unchanged |
 | `speech` | `scenes.py --speech` | `[{time, speech_music_ratio}]`, a per-second zero-crossing-rate ratio against the file's own median — a measured proxy, not a speech/music classification |
-| `motion_centre` | `cropdetect.py --motion-centre` | `[{time, x, y, x_frac, y_frac, motion}]` per second, sampled over the same windows as the crop-bar detection. `x`/`y` are source pixels, `x_frac`/`y_frac` a 0..1 fraction of `source_width`/`source_height`; a window with no measured motion reports `x`/`y`/`x_frac`/`y_frac: null`. Report only — this tool never picks a reframe |
+| `motion_centre` | `cropdetect.py --motion-centre` | `[{time, x, y, x_frac, y_frac, motion}]` per second, sampled over the same windows as the crop-bar detection. `x`/`y` are source pixels, `x_frac`/`y_frac` a 0..1 fraction of `source_width`/`source_height`. Those two sizes are the displayed frame (a 90° or 270° rotation swaps the stored size), the same frame the crop rectangle and `crop.py` use. A window with no measured motion reports `x`/`y`/`x_frac`/`y_frac: null`. Report only — this tool never picks a reframe |
 | `speech_aware`, `speech_aware.breaths` | `silence.py --speech-aware` | `{min_silence, floor, breaths_kept, breaths_kept_seconds, breaths}`. `breaths` are the sub-`--min-silence` gaps kept because they sit inside a sentence; the removal list (`silences`, `keep`, `removed_seconds`) already reflects the speech-aware classification. Composes with `--filler` through the same `keep_ranges()`/`merge_spans()` pipeline, so `--speech-aware --filler` produces one removal list |
 | `sources` | `sync.py` | `[{path, offset_s, confidence, drift_ppm}]`, one entry per SOURCE. Present for every run, including the original single-SOURCE shape (where it mirrors the top-level `second`/`offset_seconds`/`confidence` additively). With 2+ SOURCEs it is the *only* per-source shape: there is no top-level `second`/`offset_seconds` because there is no single pair to put there |
 | `switch_mode`, `min_shot` | `multicam.py --switch energy` | `"energy"` and the `--min-shot` value used (default 1.5s), alongside the existing `cuts` (`[[start, end, camera], ...]`) which already carries the camera index for `--edl`'s companion cut list |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "2.10.6",
+  "version": "2.10.8",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/references/gotchas.md

@@ -281,8 +281,12 @@ see what the app covers. An explicit `--margin` always wins.
 
 ### Dimensions and rotation
 `yuv420p` needs even width/height; `fit.py` and `export.py` round to even values
-automatically. Phone footage often carries a `rotation` tag; `probe.py` reports
-it and `fit.py` accounts for it when computing the output frame.
+automatically. Phone footage often carries a `rotation` tag. `probe.py` reports
+the stored width and height plus that rotation. A tool that places or measures
+a rectangle (`fit.py`, `crop.py`, `cropdetect.py`, `broll.py`, `overlay.py
+--scale-percent` / `--platform`, `look.py --safe`) uses the displayed size,
+the two swapped for 90° and 270°, because ffmpeg applies the tag before the
+filter graph.
 
 ## Planning
 

package/references/scripts.md

@@ -242,7 +242,10 @@ cropdetect.py INPUT [--seconds N] [--samples N] [--limit F] [--round N] [--motio
 Measurement only -- writes no file. Samples `--samples` windows spread
 across the file (default 5, totalling `--seconds` 10s of footage) and
 reports the crop rectangle FFmpeg's `cropdetect` filter found most often, as
-`{x, y, width, height}` ready to hand to `crop.py`. Distinct from
+`{x, y, width, height}` ready to hand to `crop.py`. `source_width` /
+`source_height` are that same displayed frame: a 90° or 270° rotation swaps
+the stored size, because the filter measures the picture after ffmpeg applies
+the tag. Distinct from
 `fit.py --fit crop`, which crops to a target aspect ratio it computes
 itself with no black-bar measurement involved. A rectangle that matches the
 full source frame means no bars were found. Does not decide whether
@@ -501,7 +504,8 @@ broll.py A.mp4 --insert B.mp4 --at T [--duration D | --end T2] [--from T3]
                [--insert ... --at ...] [--audio a|b|mix] [--pad-color black] [-o OUT]
 ```
 A plays as it is; during each window B's picture is shown instead (scaled and
-padded to A's frame, A's fps), and A resumes at its own time when the window
+padded to A's displayed frame, A's fps; a 90° or 270° rotation swaps the
+stored size), and A resumes at its own time when the window
 ends -- a cutaway, not a splice, so the output is exactly as long as A. One
 `--insert`/`--at` pair per cutaway (`--duration`, `--end`, `--from` are per
 cutaway too, or given once for all; defaults 4 s and 0); windows may not
@@ -1220,7 +1224,8 @@ Outputs PNG. View it with the Read tool (or any image viewer) and judge the
 frame like an editor would. Use `--compare` to show before/after to the user. `--safe NAME` (1.14) shades the zones that
 destination's own UI covers -- TikTok's description block and like column, the Reels/Shorts
 chrome -- on the sheet or the frame, so "is the caption readable" can be answered about the app
-rather than about the file.
+rather than about the file. The "wrong aspect" note compares the displayed frame, so a phone
+file stored sideways is not flagged when the picture already matches the destination.
 
 ### caption.py — subtitles (static, animated, karaoke)
 ```
@@ -1466,6 +1471,9 @@ overlay.py INPUT --image PNG [--scale W | --scale-percent P] | --text "..." [--f
 `--platform NAME` (1.14) takes each edge's margin from that destination's safe zone
 (`scripts/_platforms.py`), so a template's top-left logo clears TikTok's status bar instead of
 sitting 24 px into it; an explicit `--margin` (or a brand `safe_margin`) wins.
+`--scale-percent` and those margins use the displayed frame width and height:
+a 90° or 270° rotation swaps the stored size, which is the frame the overlay
+is composited on.
 Alpha in PNGs is respected. Fades apply to the overlay only; the video keeps
 playing. `--scale` / `--scale-percent` set the width of an image, logo or
 `--video` layer. They are refused with `--text`: text size is `--font-size`.

package/scripts/_common/decision.py

@@ -244,7 +244,7 @@ def fmt_smpte_time(seconds: float, fps: float) -> str:
     return f"{h:02d}:{m:02d}:{s:02d}:{frame:02d}"
 
 
-def escape_filter_path(path: str) -> str:
+def escape_filter_path(path: str, *, bind: bool = True) -> str:
     """Escape a file path for use as a filter option value (subtitles=, ass=, lut3d=file=, fontfile=, fontsdir=).
 
     A filter option value is parsed twice: the graph parser splits filters on `,` / `;` and options
@@ -258,8 +258,11 @@ def escape_filter_path(path: str) -> str:
     token, so a single `\\'` is consumed by the first pass and "Ryo's Mac/cues.srt" reaches the
     filter as "Ryos Mac/cues.srt" (Unable to open ...). Three backslashes survive both passes
     (measured on 6.1 and 7.1 with subtitles=, ass= and lut3d=file=).
+
+    `bind=False` escapes without recording the path as a plan input. The colour-emoji probe
+    uses that: its SRT lives in a temp directory that is gone before any plan is written.
     """
-    if os.path.isfile(path) and path not in STATE.plan_inputs:
+    if bind and os.path.isfile(path) and path not in STATE.plan_inputs:
         STATE.plan_inputs.append(path)  # a plan binds subtitle/LUT/font files too (review 6)
     p = str(Path(path))
     p = p.replace("\\", "/")

package/scripts/_common/emoji.py

@@ -8,6 +8,7 @@ import os
 import shutil
 import subprocess
 from typing import Any, Dict, List, Tuple, Optional
+from _common.decision import escape_filter_path
 from _common.emit import die
 from _common.runner import ffmpeg_exe
 
@@ -165,6 +166,18 @@ EMOJI_ASSET_HINT = (
 _EMOJI_COLOR_FAMILIES = ("Noto Color Emoji", "Apple Color Emoji", "Segoe UI Emoji")
 
 
+def _fc_list_path(line: str) -> str:
+    """The file from one `fc-list ... file` line.
+
+    fontconfig prints `<file>: <families>`. The separator is a colon followed by a space.
+    A Windows drive letter is `C:` followed by a backslash, so splitting on the first colon
+    yields `C` and the font is reported missing.
+    """
+    if ": " in line:
+        return line.partition(": ")[0].strip()
+    return line.strip().rstrip(":")
+
+
 _EMOJI_SUPPORT_CACHE: "Dict[Tuple[Optional[str], bool], Dict[str, Any]]" = {}
 
 
@@ -182,7 +195,7 @@ def _emoji_color_font() -> "Tuple[Optional[str], Optional[str], bool]":
         if proc.returncode != 0:
             return None, None, False
         for line in proc.stdout.splitlines():
-            path = line.split(":", 1)[0].strip()
+            path = _fc_list_path(line)
             if path and os.path.exists(path):
                 return family, path, True
     return None, None, True
@@ -205,7 +218,7 @@ def _libass_color_probe() -> "Optional[bool]":
             proc = subprocess.run(
                 [exe, "-hide_banner", "-loglevel", "error", "-f", "lavfi",
                  "-i", "color=c=black:s=64x64:d=0.04",
-                 "-vf", "subtitles=" + srt.replace("\\", "/"), "-frames:v", "1",
+                 "-vf", "subtitles=" + escape_filter_path(srt, bind=False), "-frames:v", "1",
                  "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
                 stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, timeout=20)
         except (subprocess.TimeoutExpired, OSError):

package/scripts/_contract.py

@@ -1465,7 +1465,20 @@ def mcp_tool(spec: Dict[str, Any]) -> Dict[str, Any]:
     descriptions verbatim (the same ~250 characters repeated 42 times in every tools/list dump);
     it now goes once into the server's `initialize` response (`instructions`) instead, so a tool's
     own description stays its own one-line sentence."""
-    return {"name": spec["name"], "description": spec["description"], "inputSchema": mcp_input_schema(spec)}
+    return {"name": spec["name"], "description": spec["description"], "inputSchema": mcp_input_schema(spec),
+            "annotations": mcp_annotations(spec)}
+
+
+def mcp_annotations(spec: Dict[str, Any]) -> Dict[str, Any]:
+    """MCP ToolAnnotations (spec 2025-06-18) derived from the ToolSpec flags. Hints only: clients
+    must not treat them as a security boundary. Every tool runs locally (no open world); none edits
+    its input in place; a tool that writes nothing is read-only."""
+    writes = bool(spec.get("produces_artifact")) or bool(spec.get("mutates_input"))
+    return {"title": spec["name"],
+            "readOnlyHint": not writes,
+            "destructiveHint": bool(spec.get("mutates_input")),
+            "idempotentHint": spec.get("idempotency_hint") in ("bit_exact", "content_equivalent", "cached"),
+            "openWorldHint": False}
 
 
 def mcp_tools(detect: bool = False) -> List[Dict[str, Any]]:

package/scripts/broll.py

@@ -56,6 +56,9 @@ def main() -> int:
         die("A-roll has no video stream")
     dur_a = meta_a.get("duration") or 0.0
     w, h = meta_a["video"]["width"], meta_a["video"]["height"]
+    # the cutaway is scaled onto the frame ffmpeg has already rotated, not the stored one
+    if meta_a["video"].get("rotation") in (90, -90, 270, -270):
+        w, h = h, w
     fps = meta_a["video"].get("fps") or 30.0
     has_audio_a = bool(meta_a.get("audio"))
 

package/scripts/cropdetect.py

@@ -138,6 +138,11 @@ def main() -> int:
     if not meta.get("video"):
         die("input has no video stream")
     sw, sh = meta["video"]["width"], meta["video"]["height"]
+    # cropdetect and the grey decode both see the frame after ffmpeg applies rotation, so the
+    # rectangle and the motion points are in that frame. Reporting the stored size made a
+    # 176x320 crop look taller than a 320x180 source.
+    if meta["video"].get("rotation") in (90, -90, 270, -270):
+        sw, sh = sh, sw
     duration = meta.get("duration") or 0.0
 
     rects = detect(args.input, args.seconds, args.samples, args.limit, args.round_to, duration)

package/scripts/look.py

@@ -170,8 +170,11 @@ def main() -> int:
         z = PLATFORMS[args.safe]["safe"]
         frame = PLATFORMS[args.safe]["frame"]
         src = meta["video"]
-        if src.get("width") and src.get("height") and abs(src["width"] / src["height"] - frame["w"] / frame["h"]) > 0.02:
-            info(f"--safe {args.safe}: this source is {src['width']}x{src['height']}, not {args.safe}'s "
+        dw, dh = src.get("width") or 0, src.get("height") or 0
+        if src.get("rotation") in (90, -90, 270, -270):
+            dw, dh = dh, dw
+        if dw and dh and abs(dw / dh - frame["w"] / frame["h"]) > 0.02:
+            info(f"--safe {args.safe}: this source is {dw}x{dh}, not {args.safe}'s "
                  f"{frame['w']}x{frame['h']} -- the zones are drawn as fractions of the frame you gave, "
                  f"so reframe first (fit.py --aspect) to see what the app really covers")
         boxes = []

package/scripts/overlay.py

@@ -117,7 +117,9 @@ def main() -> int:
     ap.add_argument("--opacity", type=float, default=1.0, help="0..1 (default 1)")
     img = ap.add_argument_group("image options")
     img.add_argument("--scale", type=int, help="width in px for an image, logo or video layer (keeps aspect); text size is --font-size")
-    img.add_argument("--scale-percent", type=float, help="width as a %% of the video for an image, logo or video layer (keeps aspect); text size is --font-size")
+    img.add_argument("--scale-percent", type=float,
+                     help="width as a %% of the displayed frame for an image, logo or video layer (keeps aspect); "
+                          "a 90 or 270 degree rotation uses the stored height; text size is --font-size")
     txt = ap.add_argument_group("text options")
     txt.add_argument("--font", default="DejaVu Sans", help="fontconfig font name")
     txt.add_argument("--font-file", help="explicit .ttf/.otf/.ttc path (use this for CJK fonts)")
@@ -209,7 +211,12 @@ def main() -> int:
         die(f"--audio-stream {args.audio_stream}: input has {len(audio_streams)} audio stream(s), 0..{len(audio_streams) - 1}")
     if args.audio_stream and not audio_streams:
         die("--audio-stream needs an input with audio streams")
-    vw = meta["video"]["width"]
+    vw = meta["video"]["width"] or 0
+    vh = meta["video"].get("height") or 0
+    # ffmpeg applies display rotation before the filter graph, so the frame this overlay is
+    # composited on is the displayed one. A 320x180 phone file tagged 90 degrees is 180 wide here.
+    if meta["video"].get("rotation") in (90, -90, 270, -270):
+        vw, vh = vh, vw
     # --platform: the edges this destination's own UI covers, in pixels of this frame. An
     # explicit --margin (or a brand safe_margin, applied above) is the more specific statement
     # and wins; without either, the historical 24 px default is unchanged.
@@ -220,7 +227,7 @@ def main() -> int:
         # fall back to the destination's own delivery frame -- which is what the fitted
         # intermediate this stage runs on will be anyway
         frame = PLATFORMS[args.platform]["frame"]
-        pw, ph = vw or frame["w"], meta["video"].get("height") or frame["h"]
+        pw, ph = vw or frame["w"], vh or frame["h"]
         safe_margins = safe_margins_px(args.platform, pw, ph)
         info(f"--platform {args.platform}: safe margins top {safe_margins['top']} / bottom {safe_margins['bottom']} / "
              f"left {safe_margins['left']} / right {safe_margins['right']} px (clear of the app's own UI)")