ffmpeg-skill 2.16.0 → 2.17.0

package/README.md

@@ -29,6 +29,7 @@ No cloud · No API keys · Python standard library</p>
   <a href="https://www.npmjs.com/package/ffmpeg-skill"><img src="https://img.shields.io/npm/v/ffmpeg-skill" alt="npm"></a>
   <a href="#ffmpeg-compatibility"><img src="https://img.shields.io/badge/ffmpeg-5.1%20%7C%206.1%20%7C%207.1%20%7C%208%20%7C%209%20tested-orange" alt="FFmpeg 5.1, 6.1, 7.1, 8 and 9 tested in CI"></a>
   <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-green" alt="MIT"></a>
+  <a href="https://github.com/sponsors/kajisho5"><img src="https://img.shields.io/badge/sponsor-%E2%9D%A4-ea4aaa?logo=githubsponsors" alt="Sponsor"></a>
 </p>
 
 ```bash
@@ -71,6 +72,7 @@ If `ffmpeg` and `python3` are on your PATH, it works: offline, on footage you wo
 | Cut / join / measure / export a file right now | **this repo** (`ffmpeg-skill`), standalone |
 | Decide cut points, approve a deliverable, plan a whole edit | [`video-production-agent`](https://github.com/kajisho5/video-production-agent) / [`AI-video-production-OS`](https://github.com/kajisho5/AI-video-production-OS) |
 | Build a typed editing graph across a workspace, without writing raw `ffmpeg` | [`video-editing-skill`](https://github.com/kajisho5/video-editing-skill) / [`audio-production-skill`](https://github.com/kajisho5/audio-production-skill) |
+| Put Remotion motion graphics (animated captions, silence jump cuts, audio-peak motion) on a real recording | [`remotion-ffmpeg-skill`](https://github.com/kajisho5/remotion-ffmpeg-skill): its compositions take their length, fps, frame, silences, scene cuts, audio peaks and beats from this repo's `probe.py`, `silence.py` and `scenes.py`. It verifies its render with `check.py` and `look.py`, composites a graphics layer with `overlay.py` (`--place` for its `--sparse` mode), and delivers through `render.py --template` |
 
 Other repos in the ecosystem — [`media-analysis-skill`](https://github.com/kajisho5/media-analysis-skill), [`transcription-skill`](https://github.com/kajisho5/transcription-skill), [`subtitle-skill`](https://github.com/kajisho5/subtitle-skill), [`thumbnail-skill`](https://github.com/kajisho5/thumbnail-skill), [`color-grading-skill`](https://github.com/kajisho5/color-grading-skill), [`motion-graphics-skill`](https://github.com/kajisho5/motion-graphics-skill), [`qc-skill`](https://github.com/kajisho5/qc-skill) — read this repo's `contract --json`, its tools' `--json` output and `doctor`, the same way any agent framework would; this repo does not call into any of them. The dependency runs one way.
 
@@ -257,7 +259,7 @@ These are the rules the skill file gives the agent and the code enforces.
 |---|---|
 | `export.py` | Presets `youtube`, `youtube4k`, `reels`, `tiktok`, `shorts`, `linkedin`, `facebook`, `x`, `youtube-hdr` (HEVC Main10, source HDR tags kept), `youtube-av1`, `prores`, `h265`, `gif`, `copy`, all tagged BT.709 unless they carry HDR (a BT.601-matrix or RGB source is converted to BT.709, not relabelled); `--normalize` meets the platform's loudness in the same call (`render.py` turns it on by default for platform presets) |
 | `proxy.py` | Small, low-bitrate proxy for downstream AI analysis/preview/editing decisions — resize by `--width`/`--scale`, proxy-grade `--quality` (default 30), `--fps`, `--no-audio`; not a delivery preset |
-| `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, Facebook, broadcast and podcast specs, from the same delivery table the export presets and templates read (podcast also reports chapter markers and channel count), with the fix for each failure and a `format` / `judgement` kind per row |
+| `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, Facebook, broadcast and podcast specs, from the same delivery table the export presets and templates read (podcast also reports chapter markers and channel count), with the fix for each failure and a `format` / `judgement` kind per row; `--content` adds black-frame, frozen-frame, silence and short audio-dropout rows |
 | `report.py` | Single-file HTML delivery report: before/after sheets, media facts, loudness, compliance, the commands run; `--pack` renders a social pack table |
 
 **Orchestration**
@@ -400,7 +402,7 @@ Each operation is a script whose flags are its argparse parser, and that parser
 
 ### How does ffmpeg-skill compare with Remotion?
 
-ffmpeg-skill edits footage you already have, on this machine, with FFmpeg tools whose flags are a machine-readable contract (`contract --json`). Remotion renders a video from React components. They complement each other: this repository does not call Remotion, and a file Remotion writes is an ordinary input (`probe.py`, `cut.py`, `fit.py`, `export.py`, `check.py`). HyperFrames renders a video from HTML and is the same kind of neighbour: a file it writes is an ordinary input too.
+ffmpeg-skill edits footage you already have, on this machine, with FFmpeg tools whose flags are a machine-readable contract (`contract --json`). Remotion renders a video from React components. They complement each other: this repository does not call Remotion, and a file Remotion writes is an ordinary input (`probe.py`, `cut.py`, `fit.py`, `export.py`, `check.py`). [`remotion-ffmpeg-skill`](https://github.com/kajisho5/remotion-ffmpeg-skill) joins the two. It measures the footage with this repo, draws the graphics in Remotion from those measurements, and checks and delivers the result with this repo again. Install both (`npx ffmpeg-skill`, then `npx remotion-ffmpeg-skill`). As with the other skills, the dependency runs one way: it calls these tools, and nothing here calls it. HyperFrames renders a video from HTML and is the same kind of neighbour: a file it writes is an ordinary input too.
 
 ### Does it validate the filter graph before running?
 

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`2.16.0`) | any release |
+| `skill.version` | the npm / package.json version (`2.17.0`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -131,7 +131,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
   "deprecated": [{"what": "running on Python 3.9 (the requirements.python floor)", "since": "2.10.9", "replacement": "Python 3.10 or later; ...", "removed_in": "3.0.0", "where": "behaviour"},
                  {"what": "running on FFmpeg 5.x (the requirements.ffmpeg floor)", "since": "2.14.0", "replacement": "FFmpeg 6.1 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
   "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "2.16.0", "execution_mode": "local", "kind": "execution",
+  "skill": {"id": "ffmpeg-skill", "version": "2.17.0", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only; 3.9 deprecated since 2.10.9, removed in 3.0.0; 3.10+ recommended)", "ffmpeg": ">=5.0 (5.x deprecated since 2.14.0, removed in 3.0.0; 6.1+ recommended)", "ffprobe": ">=5.0"},
@@ -205,7 +205,7 @@ rows carry `kind: format` (fix it) or `kind: judgement` (decide with the user).
 The `audio` row FAILs (WARN when no loudness target applies) when the track is present but silent (peak at or below -50 dBFS, from
 the loudness pass or, when loudness is skipped, one volumedetect pass). `check --content`
 (opt-in; render.py `"check": {"content": true}`) adds `black`, `frozen` and `silence` rows
-from one decode pass; without it the row set is unchanged. Thresholds:
+from one decode pass, and `dropouts` (2.17) from one audio-only pass; without it the row set is unchanged. Thresholds:
 `docs/design-decisions.md` ("check.py --content").
 
 ### Dry run
@@ -643,6 +643,7 @@ Per-tool keys added in 1.17, all additive:
 | `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
 | `timeline` | `render.py --export-timeline FILE` | `{format, rate, duration, frames, clips, transition, transition_frames, music, markers, notes, not_exported}`: the written timeline's own numbers (`rate` is exact, `"30000/1001"` for 29.97) and what the project asked for that an editor timeline cannot carry. Nothing is encoded (`commands: []`); `verified` means the file was written and reads back as its format (`verification` step `parse`) -- not that an editor opened it |
 | `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, miss_reasons, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. `miss_reasons` (unreleased, additive): `[{stage, reason, ...}]`, `reason` one of `not_cached`, `sidecar_missing`, `artifact_missing`, `sidecar_corrupt`, `artifact_truncated`, `no_output_path`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …), the GPU setting (`FFMPEG_SKILL_HW` and whether `--hw`/`--no-hw` made it explicit) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
+| `intermediate_preset` | `render.py` | The x264/x265 preset the stages before an encoding export ran at: `--intermediate-preset`, default `superfast` since 2.16 (`veryfast` in 2.15). `null` when no stage received it: no encoding export was reached, `--fast`, `--stop-after` before the export, a `copy` export, or nothing but the export encoded. Present on every render result, early stops included |
 
 Per-tool keys added in 1.17.1, all additive:
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "2.16.0",
+  "version": "2.17.0",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/references/scripts.md

@@ -1042,7 +1042,10 @@ pass when `--no-loudness` (or a spec with no loudness target) skips it.
 `--content` (opt-in, one decode pass) adds three rows: `black` (share of the duration that is
 black: WARN > 10%, FAIL >= 95%), `frozen` (longest frozen span: WARN > max(3 s, 30% of the
 duration), FAIL when it covers >= 95%, i.e. the whole video) and `silence` (share below
--50 dBFS: WARN > 50%, FAIL >= 95%). A render project enables it with `"check": {"platform":
+-50 dBFS: WARN > 50%, FAIL >= 95%); since 2.17 one more audio-only pass adds `dropouts`
+(gaps of digital silence, <= -80 dBFS for 20 ms to 1 s with sound on both sides -- lost audio
+at a join, a glitched decode; WARN when there is any, never FAIL, because a noise gate makes
+them on purpose). A render project enables it with `"check": {"platform":
 "reels", "content": true}`.
 
 ### batch.py — same recipe over a folder, cached

package/scripts/_common/probe.py

@@ -569,6 +569,14 @@ CONTENT_MIN_SPAN = 0.1           # blackdetect minimum run, seconds
 CONTENT_FREEZE_NOISE = "-60dB"   # freezedetect noise tolerance
 CONTENT_FREEZE_MIN = 0.5         # freezedetect minimum run, seconds
 CONTENT_SILENCE_MIN = 0.5        # silencedetect minimum run, seconds
+# 2.17: a dropout is a short run of (near) digital silence with sound on both sides -- a join
+# that lost its audio, a glitched decode. Far quieter than the silence row's -50 dB, so a
+# recording's own pauses (room tone) do not count; a noise gate does produce them, which is
+# why the row only ever WARNs.
+CONTENT_DROPOUT_DB = -80.0
+CONTENT_DROPOUT_MIN, CONTENT_DROPOUT_MAX = 0.02, 1.0   # seconds
+CONTENT_DROPOUT_EDGE = 0.05      # a gap this close to the start or end is a lead-in/out, not a dropout
+CONTENT_DROPOUT_PAD = 2.0        # zeros appended before the dropout pass; > MAX, so a tail that runs into it never counts
 
 _BLACK_RE = re.compile(r"black_start:\s*([0-9.]+)\s+black_end:\s*([0-9.]+)")
 _FREEZE_RE = re.compile(r"lavfi\.freezedetect\.freeze_(start|end):\s*([0-9.]+)")
@@ -618,9 +626,37 @@ def detect_content(path: str, duration: float, *, video: bool = True, audio: boo
         if start is not None:
             sil.append((start, end))
         out["silence"] = sil
+        out["dropouts"] = _detect_dropouts(path, end)
     return out
 
 
+def _detect_dropouts(path: str, end: float) -> "List[Tuple[float, float]]":
+    """Short (near) digital-silence gaps inside the first audio stream: one audio-only pass
+    (silencedetect at CONTENT_DROPOUT_DB). A second silencedetect in the main pass would log
+    under the same name as the silence row's, so its lines could not be told apart. The audio
+    is padded with CONTENT_DROPOUT_PAD s of zeros first: a silent tail of audio that ends before
+    the container does then runs into the pad and is too long to count, instead of closing at
+    the audio's EOF and passing the container-duration edge test."""
+    proc = run_analysis([require_tool("ffmpeg"), "-hide_banner", "-nostdin", "-i", path, "-map", "0:a:0", "-af",
+                         f"apad=pad_dur={CONTENT_DROPOUT_PAD:g},"
+                         f"silencedetect=noise={CONTENT_DROPOUT_DB:g}dB:d={CONTENT_DROPOUT_MIN}", "-f", "null", "-"],
+                        check=False, record=True)
+    if proc.returncode != 0:
+        die(f"dropout detection failed:\n{proc.stderr.strip()[-800:]}", kind="ffmpeg")
+    gaps: "List[Tuple[float, float]]" = []
+    start = None
+    for kind, val in SIL_RE.findall(proc.stderr):
+        if kind == "start":
+            start = float(val)
+        elif start is not None:
+            s, e = start, float(val)
+            start = None
+            inside = s > CONTENT_DROPOUT_EDGE and (not end or e < end - CONTENT_DROPOUT_EDGE)
+            if inside and CONTENT_DROPOUT_MIN <= e - s <= CONTENT_DROPOUT_MAX:
+                gaps.append((s, e))
+    return gaps  # a gap still open at the end is a tail, never a dropout
+
+
 SCORE_RE = re.compile(r"frame:(\d+)\s+pts:\d+\s+pts_time:([0-9.]+)")
 
 

package/scripts/_contract.py

@@ -576,6 +576,9 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str
     elif name == "render":
         extra = {"stages": {"type": "array", "items": {"type": "string"}}, "check": {"type": ["object", "null"], "description": "check.py's document for a \"check\" stage with a platform, \"content\": true, or both; when check.py printed none, a failure document with error.reason check_output_unparsed"},
                  "cache": {"type": ["object", "null"], "description": "--cache: {dir, ffmpeg, hits, misses, miss_reasons: [{stage, reason: not_cached|sidecar_missing|artifact_missing|sidecar_corrupt|artifact_truncated|no_output_path, ...}], saved_seconds, entries, would_hit (--dry-run)}"},
+                 "caption": {"type": ["object", "null"], "description": "the caption stage's own block, forwarded verbatim from caption.py (cue-layout counts and fit-size keys); null without a caption stage"},
+                 "snap": {"type": ["object", "null"], "description": "a project's \"snap\": cut.py's snap block for the first snapped clip, with every clip's in clips; null when nothing was snapped"},
+                 "intermediate_preset": {"type": ["string", "null"], "description": "2.15: the x264/x265 preset the stages before an encoding export ran at (--intermediate-preset, default superfast since 2.16); null when no stage received it (no encoding export reached, --fast, --stop-after before the export, a copy export, or nothing but the export encoded)"},
                  "notes": {"type": "array", "items": {"type": "string"}, "description": "each stage's own notes, prefixed with its tool (\"export: loudness ... is outside x's ...\")"},
                  "plan": {"type": "string", "description": "when the argument was a plan.json (written by <tool> --plan): its path"},
                  "plan_sha256": {"type": ["string", "null"], "description": "plan execution: sha256 of the plan file as executed"},

package/scripts/check.py

@@ -20,10 +20,12 @@ Examples:
   python3 check.py clip.mp4 --platform custom --max-duration 30 --aspect 1:1 --lufs -16
   python3 check.py final.mp4 --platform reels --content
 
---content (opt-in) adds three content rows from one decode pass: black (share of the
+--content (opt-in) adds content rows from one decode pass: black (share of the
 duration that is black: WARN > 10%, FAIL >= 95%), frozen (longest frozen span: WARN >
 max(3 s, 30% of the duration), FAIL when the whole video is frozen) and silence (share
-below -50 dBFS: WARN > 50%, FAIL >= 95%). Without it the row set is unchanged.
+below -50 dBFS: WARN > 50%, FAIL >= 95%); and, from one more audio-only pass (2.17),
+dropouts (gaps of digital silence, <= -80 dBFS for 20 ms to 1 s with sound on both sides:
+WARN when there is any). Without it the row set is unchanged.
 """
 import argparse
 import json
@@ -89,7 +91,7 @@ def main() -> int:
     ap.add_argument("--max-mb", type=float, help="override max file size in MB")
     ap.add_argument("--no-loudness", action="store_true", help="skip the loudness measurement (faster)")
     ap.add_argument("--content", action="store_true",
-                    help="also check the content (one decode pass): black, frozen and silence rows")
+                    help="also check the content (one decode pass, plus an audio-only one): black, frozen, silence and dropouts rows")
     add_common(ap)
     args = ap.parse_args()
     apply_common(args)
@@ -288,6 +290,14 @@ def main() -> int:
                 f"{share * 100:.0f}% silent", f"<= {SILENCE_WARN * 100:.0f}% below {SILENT_PEAK_DBFS:g} dBFS",
                 "silence.py to cut the gaps, or audio.py --replace when the track is empty",
                 reason="most of the running time has no audible sound")
+        if "dropouts" in cd:
+            gaps = cd["dropouts"]
+            at = ", ".join(f"{s:.2f}s ({(e - s) * 1000:.0f} ms)" for s, e in gaps[:5]) + (" ..." if len(gaps) > 5 else "")
+            row("dropouts", "WARN" if gaps else "PASS", f"{len(gaps)}" + (f": {at}" if gaps else ""),
+                "no digital-silence gap under 1 s with sound on both sides",
+                "listen at those times: a gap at a cut or join is lost audio (re-cut with cut.py --accurate, or join.py "
+                "again); a noise gate makes these on purpose" if gaps else "",
+                reason="a short stretch of digital silence in the middle of the sound is usually lost audio at a join or a glitched decode" if gaps else "")
 
     failed = [r for r in rows if r["status"] == "FAIL"]
     warned = [r for r in rows if r["status"] == "WARN"]