ffmpeg-skill 2.20.0 → 2.21.0

package/README.md

@@ -412,6 +412,33 @@ The short list for humans. The agent-facing version, with the reasoning, is the
 
 Each operation is a script whose flags are its argparse parser, and that parser is what `contract --json` and the MCP `inputSchema` are built from. CI fails the build when those drift from the scripts. A caller-supplied filter graph, `-vf` string, or other ffmpeg option is not an argument any script accepts. `doctor` reads `ffmpeg -filters` (and `-encoders`, `-bsfs`) and reports a filter a tool needs before a job runs into it. MCP does accept an `argv` array, and that array is the named script's own CLI flags, still bound to that script: the script's flags, never an ffmpeg command line and never a shell.
 
+### Why 42 tools? Isn't that overkill?
+
+One tool per job, because each job promises something different, and the tool measures that promise after the run. `loudness.py` re-measures integrated loudness and true peak. `cut.py` reports how far a cut snapped to a keyframe. `silence.py` returns the ranges it kept. `export.py` checks the preset's maximum length. A value a tool cannot read fails that step as `not_measured`; it does not pass. One tool with a mode flag would need as large a schema, and its checks would be vaguer.
+
+An agent does not read 42 schemas. Over MCP, `tools/list` advertises the core 12 (bold below). The other 30 are called by name through `tools/call` when a job needs one, and `FFMPEG_SKILL_MCP_FULL=1` lists all 42. The groups are the ones in [Tools](#tools).
+
+<!-- why42:start -->
+| Group | MCP default / all | Tools |
+|---|---|---|
+| Analysis and inspection | 2 / 3 | **probe**, **look**, scenes |
+| Editing | 3 / 24 | **cut**, **fit**, **silence**, join, crop, cropdetect, deinterlace, denoise, redact, sphere, straighten, insert, background, reverse, stabilize, sequence, waveform, freeze, pad, speedramp, loop, broll, metadata, grid |
+| Audio | 2 / 3 | **audio**, **loudness**, sync |
+| Picture | 2 / 4 | **caption**, **graphics**, overlay, color |
+| Delivery | 2 / 4 | **export**, **check**, proxy, report |
+| Orchestration | 1 / 4 | **render**, batch, multicam, verify |
+| **Total** | **12 / 42** | |
+<!-- why42:end -->
+
+Without the other 30, an agent writes raw ffmpeg for those jobs, and a mistake there often still exits 0:
+
+- `stabilize.py` runs vidstab in two passes, and the second pass needs the first pass's transforms file.
+- `color.py` builds the HDR-to-SDR chain. With a wrong colour tag the run still finishes, and the picture comes out washed out.
+- `join.py` puts transitions between clips that differ in size, frame rate or audio layout. Done by hand, those clips have to be normalised first or xfade fails.
+- `sync.py` and `multicam.py` find the offset between two recordings by audio cross-correlation. No ffmpeg filter measures it.
+
+Every command a tool ran is in its `--json` `commands`, shell-quoted so it can be pasted and run again, and `--dry-run` prints the commands without running anything. In the 100-prompt agent eval (1.17.0, [Tested on real footage](#tested-on-real-footage)) the agent made no raw ffmpeg calls and reported no false successes.
+
 ### How does ffmpeg-skill compare with Remotion?
 
 ffmpeg-skill edits footage you already have, on this machine, with FFmpeg tools whose flags are a machine-readable contract (`contract --json`). Remotion renders a video from React components. They complement each other: this repository does not call Remotion, and a file Remotion writes is an ordinary input (`probe.py`, `cut.py`, `fit.py`, `export.py`, `check.py`). [`remotion-ffmpeg-skill`](https://github.com/kajisho5/remotion-ffmpeg-skill) joins the two. It measures the footage with this repo, draws the graphics in Remotion from those measurements, and checks and delivers the result with this repo again. Install both (`npx ffmpeg-skill`, then `npx remotion-ffmpeg-skill`). As with the other skills, the dependency runs one way: it calls these tools, and nothing here calls it. HyperFrames renders a video from HTML and is the same kind of neighbour: a file it writes is an ordinary input too.
@@ -614,6 +641,7 @@ Contributing a change: see [CONTRIBUTING.md](CONTRIBUTING.md).
 | [SECURITY.md](SECURITY.md) | how to report a vulnerability privately |
 | [SKILL.md](SKILL.md) | what the agent reads: workflow, request → tool map, audio-only rules, report format, pitfalls |
 | [references/scripts.md](references/scripts.md) | per-flag reference for every tool |
+| [references/scripts.md § Verdict thresholds](references/scripts.md#verdict-thresholds-what-passes) | every number behind `verified` and `check.py`'s rows (tolerances, silence levels, per-platform loudness and length), each with the constant or flag it comes from |
 | [references/devices.md](references/devices.md) | real-device notes (iPhone HDR, GoPro, DJI, screen recordings) |
 | [references/ci-platform-pitfalls.md](references/ci-platform-pitfalls.md) | per-OS ffmpeg/CI behaviour differences already diagnosed once — read before re-diagnosing a Windows/macOS-only test failure |
 | [references/process-pitfalls.md](references/process-pitfalls.md) | process mistakes already made once (breaking a pinned test by narrowing a capability list, retrying a git/GitHub operation this environment can't do, re-designing a fixture instead of recognising a real platform difference) — a living record, add to it whenever one recurs |

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`2.20.0`) | any release |
+| `skill.version` | the npm / package.json version (`2.21.0`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -131,7 +131,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
   "deprecated": [{"what": "running on Python 3.9 (the requirements.python floor)", "since": "2.10.9", "replacement": "Python 3.10 or later; ...", "removed_in": "3.0.0", "where": "behaviour"},
                  {"what": "running on FFmpeg 5.x (the requirements.ffmpeg floor)", "since": "2.14.0", "replacement": "FFmpeg 6.1 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
   "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "2.20.0", "execution_mode": "local", "kind": "execution",
+  "skill": {"id": "ffmpeg-skill", "version": "2.21.0", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only; 3.9 deprecated since 2.10.9, removed in 3.0.0; 3.10+ recommended)", "ffmpeg": ">=5.0 (5.x deprecated since 2.14.0, removed in 3.0.0; 6.1+ recommended)", "ffprobe": ">=5.0"},

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "2.20.0",
+  "version": "2.21.0",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/references/scripts.md

@@ -14,7 +14,67 @@ yourself, which is the only way for a tool with no input file (`caption.py
 anywhere is `kind: input` naming the flag. Nothing else about times differs
 between tools.
 
+## Verdict thresholds (what passes)
+
+The numbers behind `verified`, `check.py`'s PASS/WARN/FAIL rows and the cuts the tools make. A
+result also carries the tolerance it used in each `verification[]` step (`tolerance`, since 2.18).
+Every row names the constant or flag it comes from, and a test holds the table to it
+(`test_the_thresholds_table_matches_the_code`).
+
+<!-- thresholds:start -->
+| what | value | source | used by |
+|---|---|---|---|
+| true peak may exceed the dBTP ceiling by | 0.1 dB | `_platforms.TP_TOLERANCE_DB` | `loudness.py`, `check.py`, `export.py` |
+| a normalised file is on its `-I` target within | 1.0 LU | `loudness.LUFS_TOLERANCE` | `loudness.py` |
+| a levels pass keeps the sound's length to (or one AAC frame, 1024 samples, if longer) | 0.03 s | `loudness.AUDIO_LENGTH_TOLERANCE` | `loudness.py` |
+| a joined picture is as long as the clips less the transitions, to (frames) | 1.5 | `join.DURATION_TOLERANCE_FRAMES` | `join.py` |
+| ... and at least | 0.05 s | `join.DURATION_TOLERANCE_MIN` | `join.py` |
+| a lossless cut may land this far from the asked time before it re-encodes | 0.5 s | `cut.py --tolerance` | `cut.py` |
+| silence is sound at or below | -35 dBFS | `silence.py --threshold` | `silence.py` |
+| ... lasting at least | 0.6 s | `silence.py --min-silence` | `silence.py` |
+| a removed silence is digital silence when 99% of it sits at or below | -90 dBFS | `silence.DIGITAL_SILENCE_DB` | `silence.py` (`silence_kinds`) |
+| a track is silent when its peak is at or below | -50 dBFS | `_common.probe.SILENT_PEAK_DBFS` | `check.py`; the `--silence-threshold` default of `join.py`, `audio.py`, `waveform.py` |
+| a filler word longer than this is a held vowel, kept | 1.2 s | `_common.decision.FILLER_MAX_WORD` | `silence.py --filler` |
+| a pause this long between two tokens of an unspaced script is a word boundary | 0.1 s | `_common.decision.FILLER_BOUNDARY_GAP` | `silence.py --filler` |
+| `--content` black: WARN above this share of the duration | 0.10 | `check.BLACK_WARN` | `check.py --content` |
+| `--content` black: FAIL at this share | 0.95 | `check.BLACK_FAIL` | `check.py --content` |
+| `--content` black: a pixel is black below this share of the luma range | 0.10 | `_common.probe.CONTENT_BLACK_PIX_TH` | `check.py --content` |
+| `--content` black: a black run shorter than this is not counted | 0.1 s | `_common.probe.CONTENT_MIN_SPAN` | `check.py --content` |
+| `--content` frozen: WARN when the longest span is above this, or above 30% | 3.0 s | `check.FROZEN_WARN_S` | `check.py --content` |
+| `--content` frozen: FAIL when one span covers this share | 0.95 | `check.FROZEN_FAIL` | `check.py --content` |
+| `--content` frozen: a picture unchanged for at least this long is frozen | 0.5 s | `_common.probe.CONTENT_FREEZE_MIN` | `check.py --content` |
+| `--content` frozen: frames this close count as unchanged (freezedetect noise) | -60 dB | `_common.probe.CONTENT_FREEZE_NOISE` | `check.py --content` |
+| `--content` silence: WARN above this share below the silent level | 0.50 | `check.SILENCE_WARN` | `check.py --content` |
+| `--content` silence: FAIL at this share | 0.95 | `check.SILENCE_FAIL` | `check.py --content` |
+| `--content` silence: a silent run shorter than this is not counted | 0.5 s | `_common.probe.CONTENT_SILENCE_MIN` | `check.py --content` |
+| `--content` dropout: a gap of sound at or below | -80 dBFS | `_common.probe.CONTENT_DROPOUT_DB` | `check.py --content` |
+| ... between (s) and 1 s long, with sound on both sides | 0.02 s | `_common.probe.CONTENT_DROPOUT_MIN` | `check.py --content` |
+| a beat grid below this confidence is not snapped to | 0.5 | `_common.decision.BEAT_MIN_CONFIDENCE` | `cut.py --snap beats`, `scenes.py --beats` |
+<!-- thresholds:end -->
+
+Delivery targets per platform (`scripts/_platforms.py`). A file passes loudness within the
+platform's tolerance of its target, and true peak at or below its ceiling plus 0.1 dB. `export.py`
+holds the written length to the maximum with no tolerance, as `check.py` does.
+
+<!-- platforms:start -->
+| platform | loudness | tolerance | true peak | max duration |
+|---|---|---|---|---|
+| `tiktok` | -14 LUFS | ±2 LU | -1 dBTP | 600 s |
+| `reels` | -14 LUFS | ±2 LU | -1 dBTP | 90 s |
+| `shorts` | -14 LUFS | ±2 LU | -1 dBTP | 180 s |
+| `youtube` | -14 LUFS | ±2 LU | -1 dBTP | 43200 s |
+| `youtube-hdr` | -14 LUFS | ±2 LU | -1 dBTP | 43200 s |
+| `youtube-av1` | -14 LUFS | ±2 LU | -1 dBTP | 43200 s |
+| `x` | -14 LUFS | ±3 LU | -1 dBTP | 140 s |
+| `linkedin` | -14 LUFS | ±3 LU | -1 dBTP | 600 s |
+| `facebook` | -14 LUFS | ±3 LU | -1 dBTP | 14400 s |
+| `podcast` | -16 LUFS | ±1 LU | -1 dBTP | — |
+| `broadcast` | -23 LUFS | ±1 LU | -1 dBTP | — |
+| `custom` | — | ±2 LU | -1 dBTP | — |
+<!-- platforms:end -->
+
 ## Contents
+- verdict thresholds — what passes, and where each number lives
 - probe.py — inspect
 - cut.py — cut / join segments
 - fit.py — target duration and/or aspect, rotate/flip

package/scripts/audio.py

@@ -25,6 +25,7 @@ import sys
 from pathlib import Path
 from typing import Any, Dict, List
 
+from _common.probe import SILENT_PEAK_DBFS
 from _common import STATE, add_common, dry_run_input_pending, require_tool, apply_common, audio_codec_for, db_to_linear, default_output, die, emit, ffmpeg_base, info, is_audio_output, measured_level_dbfs, probe, run, run_keeping_subtitles, fmt_secs
 from _common.emit import warn
 
@@ -216,7 +217,7 @@ def main() -> int:
     ap.add_argument("--on-silent", choices=["warn", "fail"], default="warn",
                     help="a --music/--replace/--effects file (or, under --duck, the voice) whose peak is at or below "
                          "--silence-threshold: warn (default: mix it, report it under `silent`) or fail (refuse before ffmpeg runs)")
-    ap.add_argument("--silence-threshold", type=float, default=-50.0, metavar="DBFS",
+    ap.add_argument("--silence-threshold", type=float, default=SILENT_PEAK_DBFS, metavar="DBFS",
                     help="peak level in dBFS at or below which a track counts as silent (default -50)")
     add_common(ap)
     args = ap.parse_args()

package/scripts/join.py

@@ -39,6 +39,7 @@ import os
 import sys
 from typing import Any, Dict, List, Optional, Tuple
 
+from _common.probe import SILENT_PEAK_DBFS
 from _common import STATE, AUDIO_MEDIA_EXT, draws_chroma, drawn_chain, label_chain, matrix_chain, video_args, aac_args, add_common, apply_common, audio_codec_for, default_output, die, dry_run_input_pending, emit, ffmpeg_base, info, measured_level_dbfs, is_audio_output, probe, require_tool, run, validate_color, X264_PRESETS, fmt_secs
 from _common.emit import warn
 
@@ -50,6 +51,9 @@ STATE_SKIPPED: List[Dict[str, Any]] = []
 # --dry-run: inputs that do not exist yet, planned on as an earlier step's output
 STATE_PENDING: List[Dict[str, Any]] = []
 STATE_NOTES: List[str] = []
+# the joined picture's length against the clips' sum less the transitions: 1.5 frames, at least 0.05 s
+DURATION_TOLERANCE_FRAMES = 1.5
+DURATION_TOLERANCE_MIN = 0.05
 # --on-silent warn: inputs whose audio peak is at or below --silence-threshold, joined anyway
 STATE_SILENT: List[Dict[str, Any]] = []
 # warnings, never refusals: a clip shorter than MIN_SEGMENT_FRAMES frames (audio-only: shorter than
@@ -316,7 +320,7 @@ def main() -> int:
     ap.add_argument("--allow-silent", metavar="N[,N...]",
                     help="input numbers (1-based) whose silence is intended, e.g. a planned pause: always joined, "
                          "listed under `silent` with intended: true, no warning, whatever --on-silent says")
-    ap.add_argument("--silence-threshold", type=float, default=-50.0, metavar="DB",
+    ap.add_argument("--silence-threshold", type=float, default=SILENT_PEAK_DBFS, metavar="DB",
                     help="peak level in dBFS at or below which an input counts as silent (default -50)")
     ap.add_argument("-o", "--output", help="output file (default: <first>_joined.mp4)")
     ap.add_argument("--transition", choices=TRANSITIONS, default="fade", help="transition between clips (default fade)")
@@ -579,7 +583,7 @@ def main() -> int:
     measured = (r.get("video") or {}).get("duration") or r.get("duration")
     verification = []
     if expected is not None and measured is not None and not STATE.dry_run:
-        tol = max(1.5 / fps, 0.05)
+        tol = max(DURATION_TOLERANCE_FRAMES / fps, DURATION_TOLERANCE_MIN)
         verification.append({"step": "duration", "property": "video_duration", "ok": abs(measured - expected) <= tol,
                              "measured": round(measured, 3), "expected": expected,
                              "delta": round(measured - expected, 3), "tolerance": round(tol, 3),

package/scripts/loudness.py

@@ -22,6 +22,7 @@ import sys
 from _platforms import TP_TOLERANCE_DB
 
 AUDIO_LENGTH_TOLERANCE = 0.03  # seconds: one AAC frame at 48 kHz is 0.021 s
+LUFS_TOLERANCE = 1.0  # LU: a normalised file within this of the requested -I is on target
 from _common import STATE, add_common, apply_common, emit, AUDIO_CODECS, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run, run_analysis, run_keeping_subtitles, dry_run_input_pending
 
 
@@ -159,7 +160,7 @@ def main() -> int:
     if ext not in AUDIO_CODECS or "aac" in AUDIO_CODECS[ext]:
         result["audio_bitrate_used"] = bitrate
     result["encodes"] = rounds + 1
-    if not after.get("silent") and abs(float(after["input_i"]) - args.lufs) > 1.0:
+    if not after.get("silent") and abs(float(after["input_i"]) - args.lufs) > LUFS_TOLERANCE:
         result["note"] = (f"integrated loudness is {float(after['input_i']) - args.lufs:+.1f} LU from the target because the "
                           f"true-peak ceiling had to absorb the encoder's overshoot; a lossless delivery (wav/flac) or a pinned "
                           f"higher --audio-bitrate keeps both")
@@ -169,12 +170,12 @@ def main() -> int:
             kind="verification", output=output, result=result,
             hint="raise --audio-bitrate (e.g. 256k) or deliver a lossless format (wav/flac) and let the platform encode")
     verification = [{"step": "loudness",
-                     "ok": bool(after.get("silent")) or (abs(float(after["input_i"]) - args.lufs) <= 1.0 and float(after["input_tp"]) <= args.tp + TP_TOLERANCE_DB),
+                     "ok": bool(after.get("silent")) or (abs(float(after["input_i"]) - args.lufs) <= LUFS_TOLERANCE and float(after["input_tp"]) <= args.tp + TP_TOLERANCE_DB),
                      "lufs": float(after["input_i"]), "tp": float(after["input_tp"]), "target_lufs": args.lufs, "target_tp": args.tp,
                      "property": "loudness", "expected": {"lufs": args.lufs, "tp_max": args.tp},
                      "measured": {"lufs": float(after["input_i"]), "tp": float(after["input_tp"])},
                      "delta": {"lufs": round(float(after["input_i"]) - args.lufs, 2), "tp": round(float(after["input_tp"]) - args.tp, 2)},
-                     "tolerance": {"lufs": 1.0, "tp": TP_TOLERANCE_DB}, "provenance": "user_specified"}]
+                     "tolerance": {"lufs": LUFS_TOLERANCE, "tp": TP_TOLERANCE_DB}, "provenance": "user_specified"}]
     written = probe(output, role="output")
     out_audio = audio_length(written)
     if in_audio and out_audio:

package/scripts/waveform.py

@@ -35,6 +35,7 @@ import os
 import sys
 
 from _platforms import PLATFORMS, PLATFORM_CHOICES, resolve as resolve_platform
+from _common.probe import SILENT_PEAK_DBFS
 from _common import (add_common, apply_common, aac_args, default_output, die, draws_chroma, emit, ffmpeg_base,
                      info, load_brand, pad_filters, probe, rgb_chain_to_bt709, run, STATE, validate_color, video_args, X264_PRESETS,
                      fmt_secs, child_args, run_tool, measured_level_dbfs)
@@ -60,7 +61,7 @@ def main() -> int:
     ap.add_argument("--on-silent", choices=["warn", "fail"], default="warn",
                     help="input audio whose peak is at or below --silence-threshold (the render would be a flat "
                          "line): warn (default: render it, report `silent: true`) or fail (refuse before ffmpeg runs)")
-    ap.add_argument("--silence-threshold", type=float, default=-50.0, metavar="DBFS",
+    ap.add_argument("--silence-threshold", type=float, default=SILENT_PEAK_DBFS, metavar="DBFS",
                     help="peak level in dBFS at or below which the input counts as silent (default -50)")
     ag = ap.add_argument_group("audiogram (1.16)",
                                "The visualisation over a picture, for an episode that has no video. The image is a "