ffmpeg-skill 2.22.5 → 2.23.0

package/README.md

@@ -270,7 +270,7 @@ These are the rules the skill file gives the agent and the code enforces.
 |---|---|
 | `export.py` | Presets `youtube`, `youtube4k`, `reels`, `tiktok`, `shorts`, `linkedin`, `facebook`, `x`, `youtube-hdr` (HEVC Main10, source HDR tags kept), `youtube-av1`, `prores`, `h265`, `gif`, `copy`, all tagged BT.709 unless they carry HDR (a BT.601-matrix or RGB source is converted to BT.709, not relabelled); `--normalize` meets the platform's loudness in the same call (`render.py` turns it on by default for platform presets) |
 | `proxy.py` | Small, low-bitrate proxy for downstream AI analysis/preview/editing decisions — resize by `--width`/`--scale`, proxy-grade `--quality` (default 30), `--fps`, `--no-audio`; not a delivery preset |
-| `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, Facebook, broadcast and podcast specs, from the same delivery table the export presets and templates read (podcast also reports chapter markers and channel count), with the fix for each failure and a `format` / `judgement` kind per row; `--content` adds black-frame, frozen-frame, silence and short audio-dropout rows |
+| `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, Facebook, broadcast and podcast specs, from the same delivery table the export presets and templates read (podcast also reports chapter markers and channel count), with the fix for each failure and a `format` / `judgement` kind per row; `--content` adds black-frame, frozen-frame, silence, short audio-dropout, single-frame-flash and flat-first-frame rows |
 | `report.py` | Single-file HTML delivery report: before/after sheets, media facts, loudness, compliance, the commands run; `--pack` renders a social pack table |
 
 **Orchestration**

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`2.22.5`) | any release |
+| `skill.version` | the npm / package.json version (`2.23.0`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -131,7 +131,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
   "deprecated": [{"what": "running on Python 3.9 (the requirements.python floor)", "since": "2.10.9", "replacement": "Python 3.10 or later; ...", "removed_in": "3.0.0", "where": "behaviour"},
                  {"what": "running on FFmpeg 5.x (the requirements.ffmpeg floor)", "since": "2.14.0", "replacement": "FFmpeg 6.1 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
   "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "2.22.5", "execution_mode": "local", "kind": "execution",
+  "skill": {"id": "ffmpeg-skill", "version": "2.23.0", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only; 3.9 deprecated since 2.10.9, removed in 3.0.0; 3.10+ recommended)", "ffmpeg": ">=5.0 (5.x deprecated since 2.14.0, removed in 3.0.0; 6.1+ recommended)", "ffprobe": ">=5.0"},
@@ -206,7 +206,8 @@ rows carry `kind: format` (fix it) or `kind: judgement` (decide with the user).
 The `audio` row FAILs (WARN when no loudness target applies) when the track is present but silent (peak at or below -50 dBFS, from
 the loudness pass or, when loudness is skipped, one volumedetect pass). `check --content`
 (opt-in; render.py `"check": {"content": true}`) adds `black`, `frozen` and `silence` rows
-from one decode pass, and `dropouts` (2.17) from one audio-only pass; without it the row set is unchanged. Thresholds:
+from one decode pass, `dropouts` (2.17) from one audio-only pass, and `flashes` and `first frame` (2.23) from one
+video-only pass and a one-frame decode; without it the row set is unchanged. Thresholds:
 `docs/design-decisions.md` ("check.py --content").
 
 ### Dry run

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "2.22.5",
+  "version": "2.23.0",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/references/scripts.md

@@ -1142,7 +1142,12 @@ duration), FAIL when it covers >= 95%, i.e. the whole video) and `silence` (shar
 -50 dBFS: WARN > 50%, FAIL >= 95%); since 2.17 one more audio-only pass adds `dropouts`
 (gaps of digital silence, <= -80 dBFS for 20 ms to 1 s with sound on both sides -- lost audio
 at a join, a glitched decode; WARN when there is any, never FAIL, because a noise gate makes
-them on purpose). A render project enables it with `"check": {"platform":
+them on purpose); since 2.23 one more video pass adds `flashes` (single frames that differ from
+both neighbours by >= 12/255 mean luma while the neighbours differ by at most a quarter of that
+-- a stray frame at a cut, a decode glitch; WARN when there is any, with the first five times,
+never FAIL, because a camera flash or a strobe is real) and a one-frame decode adds `first frame`
+(WARN when it is one flat colour, luma spread < 4/255: the frame a feed's autoplay shows first).
+A render project enables it with `"check": {"platform":
 "reels", "content": true}`.
 
 ### batch.py — same recipe over a folder, cached

package/scripts/_common/probe.py

@@ -12,7 +12,7 @@ import re
 import statistics
 from fractions import Fraction
 from typing import Any, Dict, List, Optional, Sequence, Tuple
-from _common.emit import die
+from _common.emit import die, warn
 from _common.runner import STATE, dry_run_input_pending, require_tool, run, run_analysis
 
 
@@ -605,6 +605,15 @@ CONTENT_DROPOUT_MIN, CONTENT_DROPOUT_MAX = 0.02, 1.0   # seconds
 CONTENT_DROPOUT_EDGE = 0.05      # a gap this close to the start or end is a lead-in/out, not a dropout
 CONTENT_DROPOUT_PAD = 2.0        # zeros appended before the dropout pass; > MAX, so a tail that runs into it never counts
 
+# 2.23: a flash is one frame that differs from both of its neighbours while they match each other
+# (a stray frame from another shot, a white or black frame at a join, a decode glitch). Measured on
+# 64x36 grey frames as the mean absolute luma difference (0-255): moving footage stays at a few
+# units, a cut or a flash moves the whole frame by tens.
+CONTENT_FLASH_JUMP = 12.0        # both steps into and out of the frame are at least this
+CONTENT_FLASH_RETURN = 0.25      # ...and the frames either side differ by at most this share of the smaller step
+CONTENT_FLAT_STD = 4.0           # first frame: luma standard deviation (0-255) below this is one flat colour
+_DIFF_W, _DIFF_H = 64, 36
+
 _BLACK_RE = re.compile(r"black_start:\s*([0-9.]+)\s+black_end:\s*([0-9.]+)")
 _FREEZE_RE = re.compile(r"lavfi\.freezedetect\.freeze_(start|end):\s*([0-9.]+)")
 
@@ -619,7 +628,7 @@ def detect_content(path: str, duration: float, *, video: bool = True, audio: boo
     ffmpeg = require_tool("ffmpeg")
     cmd = [ffmpeg, "-hide_banner", "-nostdin", "-i", path]
     if video:
-        cmd += ["-map", "0:v:0", "-vf", f"blackdetect=d={CONTENT_MIN_SPAN}:pix_th={CONTENT_BLACK_PIX_TH},"
+        cmd += ["-map", "0:V:0", "-vf", f"blackdetect=d={CONTENT_MIN_SPAN}:pix_th={CONTENT_BLACK_PIX_TH},"
                 f"freezedetect=n={CONTENT_FREEZE_NOISE}:d={CONTENT_FREEZE_MIN}"]
     if audio:
         cmd += ["-map", "0:a:0", "-af", f"silencedetect=noise={SILENT_PEAK_DBFS:g}dB:d={CONTENT_SILENCE_MIN}"]
@@ -684,6 +693,83 @@ def _detect_dropouts(path: str, end: float) -> "List[Tuple[float, float]]":
     return gaps  # a gap still open at the end is a tail, never a dropout
 
 
+def detect_flashes(path: str) -> "Optional[List[int]]":
+    """Frame indexes of single-frame flashes in the first video stream: frame n differs from n-1 and
+    from n+1 by at least CONTENT_FLASH_JUMP while n-1 and n+1 differ by at most
+    CONTENT_FLASH_RETURN of the smaller step. One video-only pass; ffmpeg reduces each frame to
+    three 16-bit numbers (|f[n]-f[n-1]| and the two clipped halves of f[n]-f[n-2]): 6 bytes a
+    frame in the captured output (1.3 MB for an hour at 60 fps), read in place. The caller turns an index into a time;
+    the comparison itself needs no frame rate. None when no frame decoded or the pass failed
+    (nothing was compared; a failure is also a flash_pass_failed warning)."""
+    if dry_run_input_pending(path):
+        return []
+    graph = (f"[0:V:0]scale={_DIFF_W}:{_DIFF_H}:flags=area,format=gray16le,split=3[a][b][c];"
+             "[a]tblend=all_mode=difference,scale=1:1:flags=area[d];"
+             "[b]tmix=frames=3:weights='-1 0 1':scale=1,scale=1:1:flags=area[p];"
+             "[c]tmix=frames=3:weights='1 0 -1':scale=1,scale=1:1:flags=area[q];"
+             "[d][p][q]hstack=inputs=3")
+    proc = run_analysis([require_tool("ffmpeg"), "-hide_banner", "-loglevel", "error", "-nostdin", "-i", path,
+                         "-filter_complex", graph, "-fps_mode", "passthrough", "-f", "rawvideo", "-"],
+                        check=False, text=False, record=True)
+    if proc.returncode != 0:
+        # an unmeasured row, not the end of the report: the other content rows stand
+        warn("flash detection failed: " + (proc.stderr.decode(errors="replace").strip().splitlines() or ["ffmpeg exited " + str(proc.returncode)])[-1][:300],
+             "flash_pass_failed")
+        return None
+    # tblend has no output for frame 0, so record r is frame r+1: a zero record in front puts
+    # every value at its own frame's index. passthrough keeps the rawvideo muxer from padding a
+    # constant rate, which repeated records over timestamp gaps (a repeat is a zero step, and a
+    # flash before a gap went unreported)
+    if len(proc.stdout) < 6:
+        return None
+    data = bytes(6) + proc.stdout
+    n = len(data) // 6
+
+    def record(i: int) -> "Tuple[float, float]":
+        """(|f[i] - f[i-1]|, |f[i] - f[i-2]|), 0-255; read from the buffer, never kept per frame."""
+        d, p, q = (int.from_bytes(data[i * 6 + k:i * 6 + k + 2], "little") / 256.0 for k in (0, 2, 4))
+        return d, p + q
+
+    flashes = []
+    into = record(1)[0] if n > 1 else 0.0
+    for i in range(1, n - 1):
+        out, back = record(i + 1)
+        if into >= CONTENT_FLASH_JUMP and out >= CONTENT_FLASH_JUMP and back <= CONTENT_FLASH_RETURN * min(into, out):
+            flashes.append(i)
+        into = out
+    return flashes
+
+
+def presentation_times(path: str, start: float) -> "List[float]":
+    """The presentation time of every frame of the first video stream (attached pictures
+    excluded), in order: its packets' pts sorted, those before the stream's own start (an edit
+    list's hidden pre-roll, never shown) dropped. From packets, so no decode. [] when unreadable,
+    also when one packet has no pts: skipping it would shift every later time onto the wrong frame."""
+    proc = run([require_tool("ffprobe"), "-v", "error", "-select_streams", "V:0", "-show_entries", "packet=pts_time",
+                "-of", "csv=p=0", path], quiet=True, check=False)
+    times = []
+    for line in (proc.stdout or "").split() if proc.returncode == 0 else []:
+        try:
+            times.append(float(line.split(",")[0]))
+        except ValueError:
+            return []
+    return sorted(t for t in times if t >= start - 1e-6)
+
+
+def first_frame_stats(path: str) -> "Optional[Tuple[float, float]]":
+    """(mean, standard deviation) of the first frame's luma, 0-255, at 64x36; None when it does
+    not decode or the input is a pending dry-run placeholder."""
+    if dry_run_input_pending(path):
+        return None
+    proc = run_analysis([require_tool("ffmpeg"), "-hide_banner", "-loglevel", "error", "-nostdin", "-i", path,
+                         "-map", "0:V:0", "-frames:v", "1", "-vf", f"scale={_DIFF_W}:{_DIFF_H}:flags=area,format=gray",
+                         "-f", "rawvideo", "-"], check=False, text=False, record=True)
+    px = list(proc.stdout[:_DIFF_W * _DIFF_H])
+    if proc.returncode != 0 or len(px) < _DIFF_W * _DIFF_H:
+        return None
+    return statistics.fmean(px), statistics.pstdev(px)
+
+
 SCORE_RE = re.compile(r"frame:(\d+)\s+pts:\d+\s+pts_time:([0-9.]+)")
 
 

package/scripts/_contract.py

@@ -495,6 +495,7 @@ DIAGNOSTIC_CODES = {
     # degraded or partial results (part 2)
     "colour_not_converted": ("warnings", "export.py: an HDR or wide-gamut source written by a preset that tags SDR BT.709 without converting it; {preset, hdr_format}"),
     "output_audio_silent": ("warnings", "export.py: the written audio is silent (-inf LUFS), so no gain meets the platform's loudness; {platform}"),
+    "flash_pass_failed": ("warnings", "check.py --content: the flash pass failed, so the flashes row is not measured (WARN, could not measure); the other rows stand"),
     "loudness_outside_spec": ("warnings", "export.py: the written file's loudness is outside the platform's spec (normalized: after --normalize too); {platform, lufs, tp, normalized}"),
     "camera_not_started": ("fallbacks", "multicam.py: a range asked for a camera that had not started; camera 0 played it ({start, end, requested_camera, camera, starts_at}); `cuts` names the camera rendered"),
     "loudness_not_measured": ("warnings", "export.py: the written file's loudness could not be read, so the platform's loudness was not verified (loudness.ok null, a failing loudness verification step)"),

package/scripts/check.py

@@ -25,17 +25,20 @@ duration that is black: WARN > 10%, FAIL >= 95%), frozen (longest frozen span: W
 max(3 s, 30% of the duration), FAIL when the whole video is frozen) and silence (share
 below -50 dBFS: WARN > 50%, FAIL >= 95%); and, from one more audio-only pass (2.17),
 dropouts (gaps of digital silence, <= -80 dBFS for 20 ms to 1 s with sound on both sides:
-WARN when there is any). Without it the row set is unchanged.
+WARN when there is any); and, from one more video pass and a one-frame decode (2.23), flashes
+(single frames that differ from both neighbours while they match each other: WARN when there is
+any) and first frame (one flat colour, the frame a feed's autoplay shows first: WARN). Without it
+the row set is unchanged.
 """
 import argparse
 import json
 import re
 import sys
 from fractions import Fraction
-from typing import Any, Dict, List
+from typing import Any, Dict, List, Optional
 
 from _platforms import PLATFORMS, PLATFORM_CHOICES, TP_TOLERANCE_DB, spec_of, resolve as resolve_platform
-from _common.probe import SILENT_PEAK_DBFS, detect_content, measured_level_dbfs
+from _common.probe import CONTENT_FLAT_STD, SILENT_PEAK_DBFS, detect_content, detect_flashes, first_frame_stats, presentation_times, measured_level_dbfs
 from _common import STATE, add_common, apply_common, die, emit, info, probe, require_tool, run, run_analysis, dry_run_input_pending
 
 # The one delivery table (scripts/_platforms.py): check.py's rows, export.py's presets and the
@@ -68,6 +71,22 @@ FROZEN_FAIL = 0.95                         # a frozen span covering this share =
 SILENCE_WARN, SILENCE_FAIL = 0.50, 0.95    # share of the duration below SILENT_PEAK_DBFS
 
 
+def flash_times(flashes: List[int], v: Dict[str, Any], meta: Dict[str, Any], pts: "Optional[List[float]]" = None) -> str:
+    """The flashes row's first five times on look.py's timeline (seconds from the container
+    start). A flash frame's own presentation time when `pts` has it (presentation_times(): exact,
+    gaps in the timestamps included); otherwise index / average rate plus the video's lead over
+    the container, marked "~" (approximate), and `frame N` when there is no frame rate either."""
+    origin = float(meta.get("start_time") or 0.0)
+    fps = float(v.get("fps") or 0.0)
+    lead = max(0.0, float(v.get("start_time") or 0.0) - origin)
+
+    def one(n: int) -> str:
+        if pts and n < len(pts):
+            return f"{max(0.0, pts[n] - origin):.2f}s"
+        return f"~{lead + n / fps:.2f}s" if fps else f"frame {n}"
+    return ", ".join(one(n) for n in flashes[:5]) + (" ..." if len(flashes) > 5 else "")
+
+
 def _span_total(spans: Any) -> float:
     return sum(max(0.0, e - s) for s, e in spans)
 
@@ -91,7 +110,7 @@ def main() -> int:
     ap.add_argument("--max-mb", type=float, help="override max file size in MB")
     ap.add_argument("--no-loudness", action="store_true", help="skip the loudness measurement (faster)")
     ap.add_argument("--content", action="store_true",
-                    help="also check the content (one decode pass, plus an audio-only one): black, frozen, silence and dropouts rows")
+                    help="also check the content (one decode pass, plus an audio-only and a video-only one): black, frozen, silence, dropouts, flashes and first frame rows")
     add_common(ap)
     args = ap.parse_args()
     apply_common(args)
@@ -117,7 +136,8 @@ def main() -> int:
     v, a = meta.get("video") or {}, meta.get("audio") or {}
     rows: List[Dict[str, Any]] = []
 
-    JUDGEMENT = {"duration", "aspect", "loudness", "true peak", "fps", "resolution"}
+    # first frame / flashes: a fade from black or a strobe can be the edit, so neither is a mechanical fix
+    JUDGEMENT = {"duration", "aspect", "loudness", "true peak", "fps", "resolution", "first frame", "flashes"}
 
     def row(name: str, status: str, value: Any, expect: Any, fix: str = "", reason: str = "") -> None:
         # "format" rows are safe to fix mechanically; "judgement" rows change the content
@@ -284,6 +304,32 @@ def main() -> int:
                 f"longest {longest:.1f}s", f"<= {limit:.1f}s frozen",
                 "look.py the frozen span: a still title card is fine, a stuck decode is not",
                 reason="a picture that never changes is usually a stuck frame, not the edit")
+        if v and not dry_run_input_pending(args.input):  # a plan's not-yet-written input: no rows, as above
+            first = first_frame_stats(args.input)
+            if first is None:
+                row("first frame", "WARN", "could not measure", f"spread >= {CONTENT_FLAT_STD:g} (not one flat colour)",
+                    "look.py the first frame", reason="the first frame did not decode, so whether it is flat is unknown")
+            else:
+                mean, spread = first
+                flat = spread < CONTENT_FLAT_STD
+                row("first frame", "WARN" if flat else "PASS", f"luma {mean:.0f}, spread {spread:.1f}",
+                    f"spread >= {CONTENT_FLAT_STD:g} (not one flat colour)",
+                    "a fade from black is a choice; where autoplay or the preview shows the first frame, start on "
+                    "a picture (cut.py --start a few frames in, or move a title card to frame 0)" if flat else "",
+                    reason="a feed shows the first frame before anything plays, and a flat one reads as a broken upload" if flat else "")
+            flashes = detect_flashes(args.input)
+            pts = presentation_times(args.input, float(v.get("start_time") or 0.0)) if flashes else None
+            at = flash_times(flashes or [], v, meta, pts)
+            if flashes is None:
+                row("flashes", "WARN", "could not measure", "no single frame unlike both its neighbours",
+                    "look.py the video: no frame decoded, so nothing was compared",
+                    reason="the video did not decode, so whether it has flashes is unknown")
+            else:
+                row("flashes", "WARN" if flashes else "PASS", f"{len(flashes)}" + (f": {at}" if flashes else ""),
+                    "no single frame unlike both its neighbours",
+                    "look.py at those times: a stray frame at a cut or join is an edit error (re-cut with cut.py "
+                    "--accurate, or join.py again); a camera flash or a strobe in the footage is fine" if flashes else "",
+                    reason="one frame that matches neither neighbour is usually a frame left over at a cut, or a decode glitch" if flashes else "")
         if "silence" in cd and span:
             share = _span_total(cd["silence"]) / span
             row("silence", "FAIL" if share >= SILENCE_FAIL else ("WARN" if share > SILENCE_WARN else "PASS"),