ffmpeg-skill 1.19.3 → 1.21.0

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`1.19.3`) | any release |
+| `skill.version` | the npm / package.json version (`1.21.0`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
 
 | What 2.0 removes | Since | Replacement | To be ready today |
 |---|---|---|---|
-| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.19.3 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
-| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.19.3 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
-| `json` and `progress` in the MCP `inputSchema` | 1.19.3 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
-| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.19.3 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
-| Overwriting an existing output with only a warning | 1.19.3 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
+| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.21.0 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
+| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.21.0 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
+| `json` and `progress` in the MCP `inputSchema` | 1.21.0 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
+| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.21.0 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
+| Overwriting an existing output with only a warning | 1.21.0 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
 
 ## Skill
 
 ```json
 {
   "contract_version": "1.0",
-  "deprecated": [{"what": "...", "since": "1.19.3", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "1.19.3", "execution_mode": "local", "kind": "execution",
+  "deprecated": [{"what": "...", "since": "1.21.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
+  "skill": {"id": "ffmpeg-skill", "version": "1.21.0", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
@@ -120,6 +120,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
 | Field | Meaning |
 |---|---|
 | `id`, `name`, `version`, `executable` | `ffmpeg-skill/cut`, `cut`, skill version, `scripts/cut.py` |
+| `examples[]` | `{prompts[], command}` (1.21.0): SKILL.md's own "User says" / "Do" table rows for this tool, machine-readable — `prompts` is the row's phrasing(s), `command` the row's `Do` text with the markdown backticks stripped. Every tool has at least one; parsed from SKILL.md at read time, so it never drifts from the table a person reads |
 | `role` | `analysis`, `analysis_and_execution`, `execution` or `verification` (see below) |
 | `capabilities.required` | ffmpeg components the tool always needs |
 | `capabilities.optional[]` | `{capability, when}`: needed only for that flag or input |
@@ -128,7 +129,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
 | `output_schema` | what `--json` prints on stdout |
 | `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
 | `supports_json` | whether `--json` exists |
-| `supports_json_brief` | whether `--json-brief` exists (1.19.3): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
+| `supports_json_brief` | whether `--json-brief` exists (1.21.0): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
 | `mutates_input` | always `false`: no tool overwrites its input |
 | `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
 | `verification` | `{required, tools}`: which tools to run on the output afterwards |
@@ -434,7 +435,7 @@ given a different type):
 
 | key | tool | what it holds |
 |---|---|---|
-| `caption` | `caption.py` | the cue-layout counts the run only printed before (`shifted`, `wrapped`, `rebalanced`, `split`, `extended`, `dropped`) plus `wrap` (`"phrase"` or `"measured"`) and `phrase_breaks`, the number of cues a phrase rule broke somewhere the 1.15 width rule would not. `broken_inside_word` (1.19.3) counts atoms hard-sliced at the live column's edge because they did not fit alone even at the size floor — never a rewrite, the sliced pieces are the exact original characters. `overlong` is now residual: it fires only when even a single character is wider than the column |
+| `caption` | `caption.py` | the cue-layout counts the run only printed before (`shifted`, `wrapped`, `rebalanced`, `split`, `extended`, `dropped`) plus `wrap` (`"phrase"` or `"measured"`) and `phrase_breaks`, the number of cues a phrase rule broke somewhere the 1.15 width rule would not. `broken_inside_word` (1.21.0) counts atoms hard-sliced at the live column's edge because they did not fit alone even at the size floor — never a rewrite, the sliced pieces are the exact original characters. `overlong` is now residual: it fires only when even a single character is wider than the column |
 | `tracks`, `subtitle_tracks` | `caption.py --mode mux` | one entry per subtitle stream in the output — `{index, file, language, title, codec, default, cues, kept_from_input}`; a stream the input already carried has `file: null` and `kept_from_input: true`. `subtitle_tracks` is the total. Every field describes the file as written, not as asked for: an MPEG-4 output reports `title: null` (the muxer stores none) and `default: true` on its first track (the muxer always enables it), each with a `notes` line |
 | `auto_chapters` | `metadata.py --auto-chapters` | `{source, min_chapter, max_chapters, proposed, kept, titles, chapters, description_block, files}`. `titles` is always `"placeholder"`: the machine-readable form of "the skill did not name these". Each chapter carries its `evidence` (`start`, `silence`, `scene`, or `silence+scene` with the span, its length and the cut time) |
 | `audiogram` | `waveform.py` (every run) | `{style, background, image, position, vis_height, platform, captions, title, stages, verified}`. `background` is `"image"` or `"color"`; `verified` is true when the render probes at the asked-for frame size, frame rate and within 0.05 s of the source audio, and is `false` under `--dry-run`, where nothing was rendered to verify |
@@ -450,7 +451,7 @@ Per-tool keys added in 1.17, all additive:
 | `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
 | `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
 
-Per-tool keys added in 1.19.3, all additive:
+Per-tool keys added in 1.21.0, all additive:
 
 | key | tool | what it holds |
 |---|---|---|
@@ -458,7 +459,7 @@ Per-tool keys added in 1.19.3, all additive:
 | `text_unchanged` | `caption.py` | a sibling inside the `caption` block, **burn mode only** (`--mode mux` never touches the text and omits the key): `true` when the drawn text equals the cues that were handed in — nothing transcribed, no cue dropped, no cue **split** across two consecutive cues and no glyph stripped (`--emoji none`). Wrapping, line breaks and timing do not count: the words are the same. This tool never rewrites, shortens or translates a cue, so the key is a statement of what happened, not a judgement of the text |
 
 
-Per-tool keys added in 1.19.3, all additive:
+Per-tool keys added in 1.21.0, all additive:
 
 | key | tool | what it holds |
 |---|---|---|
@@ -588,7 +589,7 @@ most often across the corpus in `evals/agent_prompts*.json`. Every tool -- inclu
 -- is still callable by name through `tools/call` regardless of what `tools/list` advertised; the
 contract (`ffmpeg-skill contract --json`) still describes all 42 unconditionally. Set
 `FFMPEG_SKILL_MCP_FULL=1` (anything but "" or `0`) in the server's environment to make `tools/list`
-return all 42, as every version before 1.19.3 did.
+return all 42, as every version before 1.21.0 did.
 
 ## Consuming the contract from an agent
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "1.19.3",
+  "version": "1.21.0",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/scripts/_contract.py

@@ -18,6 +18,7 @@ The contract has its own version (CONTRACT_VERSION) that only changes when the s
 this document changes; the skill version comes from package.json.
 """
 import argparse
+import functools
 import importlib.util
 import json
 import os
@@ -462,6 +463,43 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
     required = ["status", "output", "dry_run", "commands"]
     return {"type": "object", "properties": props, "required": required, "additionalProperties": True}
 
+@functools.lru_cache(maxsize=1)
+def skill_examples() -> Dict[str, List[Dict[str, Any]]]:
+    """1.20.0: per-tool `examples`, parsed from SKILL.md's "User says" / "Do" table -- the
+    same facts that table already states, machine-readable instead of prose-only. A row
+    naming more than one tool (e.g. metadata.py's --auto-chapters row also names render.py)
+    attaches to each; a row naming none (prose only, no backtick command) attaches to none.
+    `prompts` is a list because a row often gives several phrasings ("cut from 1:20 to 2:05",
+    "trim the first 10 s") for the one `command`; `command` has the markdown backticks
+    stripped but otherwise keeps the row's own text (including "or"/"then" alternatives).
+    """
+    examples: Dict[str, List[Dict[str, Any]]] = {}
+    try:
+        lines = (ROOT / "SKILL.md").read_text(encoding="utf-8").splitlines()
+    except OSError:
+        return examples
+    in_table = False
+    esc_pipe = "\x00"  # a markdown-escaped "\|" (a literal pipe inside a cell, e.g. broll's --audio b\|mix) is not a column separator
+    for line in lines:
+        if line.startswith("| User says"):
+            in_table = True
+            continue
+        if not in_table:
+            continue
+        if not line.startswith("|"):
+            break
+        if line.startswith("|---") or line.startswith("|-----"):
+            continue
+        cells = line.strip().strip("|").replace("\\|", esc_pipe).split("|")
+        if len(cells) != 2:
+            continue
+        prompt_cell, do = (c.replace(esc_pipe, "|").strip() for c in cells)
+        prompts = re.findall(r'"([^"]+)"', prompt_cell) or [prompt_cell.strip('"')]
+        command = re.sub(r"`([^`]*)`", r"\1", do)
+        for tool in dict.fromkeys(re.findall(r"`([a-z][a-z0-9_]*)\.py", do)):
+            examples.setdefault(tool, []).append({"prompts": prompts, "command": command})
+    return examples
+
 
 # ----------------------------------------------------------------------------- environment
 def skill_version() -> str:
@@ -1012,6 +1050,8 @@ def tool_spec(name: str, version: str) -> Dict[str, Any]:
         "version": version,
         "description": (parser.description or "").strip().splitlines()[0] if parser.description else "",
         "executable": f"scripts/{name}.py",
+        # 1.20.0: SKILL.md's own request-table rows for this tool, machine-readable
+        "examples": skill_examples().get(name, []),
         "role": meta["role"],
         "capabilities": {"required": list(meta["required"]), "optional": _merge_optional(meta["optional"], CODEC_CAPS if "codec" in schema["properties"] else [])},
         "inputs": list(meta["inputs"]),

package/scripts/look.py

@@ -14,10 +14,10 @@ import argparse
 import os
 import sys
 from pathlib import Path
-from typing import List
+from typing import List, Optional
 
 from _platforms import PLATFORMS, PLATFORM_CHOICES, resolve as resolve_platform
-from _common import STATE, add_common, apply_common, default_font_file, die, emit, escape_drawtext, escape_filter_path, ffmpeg_base, info, parse_time, probe, run, time_arg
+from _common import STATE, add_common, apply_common, default_font_file, die, emit, escape_drawtext, escape_filter_path, ffmpeg_base, info, parse_time, probe, run, run_analysis, time_arg
 
 FONT = "fontcolor=white:fontsize=h/18:box=1:boxcolor=black@0.55:boxborderw=6:x=8:y=8"
 
@@ -32,6 +32,103 @@ def timecode_filter(font_prefix: str) -> str:
     return f"drawtext=text='%{{pts\\:hms}}':{font_prefix}{FONT}"
 
 
+def _png_size(png_path: str) -> "Optional[tuple]":
+    # run_analysis(): the same wall-clock ceiling and #234 UTF-8 decoding as every other
+    # ffprobe/ffmpeg measurement in this codebase, check=False since a probe failure here is
+    # an unknown-metric result (has_ink: None), not a reason to die -- a stall still does,
+    # exactly like every other measurement's timeout.
+    proc = run_analysis(["ffprobe", "-v", "error", "-select_streams", "v:0", "-show_entries",
+                         "stream=width,height", "-of", "csv=p=0", png_path], check=False)
+    parts = proc.stdout.strip().split(",")
+    if len(parts) != 2:
+        return None
+    try:
+        return int(parts[0]), int(parts[1])
+    except ValueError:
+        return None
+
+
+def _png_pixels(png_path: str, w: int, h: int) -> "Optional[bytes]":
+    proc = run_analysis(["ffmpeg", "-v", "error", "-i", png_path, "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
+                        check=False, text=False)
+    data = proc.stdout
+    return data if len(data) >= w * h * 3 else None
+
+
+def measure_ink(png_path: str, *, box=None, margin_frac: float = 0.03, threshold: int = 28) -> dict:
+    """Non-background ink in `png_path` (or the `box` = (x0, y0, w, h) sub-region of it): a pixel-only
+    measurement, never a judgement of legibility. The background is estimated as the median RGB of a
+    thin strip around the region's own edge (so it works on any frame, not just a black synthetic one);
+    a pixel is "ink" when any channel differs from that estimate by more than `threshold`. Returns
+    {has_ink, bbox (x0,y0,x1,y1 within the region, or None), ink_fraction, row_bands} -- row_bands are
+    gap-merged bands of rows carrying ink, the same convention tests/_fixtures.py's _ass_ink_rows uses,
+    so a caption wrapped onto two lines shows as two bands without measuring any font."""
+    size = _png_size(png_path)
+    if not size:
+        return {"has_ink": None, "bbox": None, "ink_fraction": None, "row_bands": []}
+    full_w, full_h = size
+    data = _png_pixels(png_path, full_w, full_h)
+    if data is None:
+        return {"has_ink": None, "bbox": None, "ink_fraction": None, "row_bands": []}
+    if box:
+        bx, by, bw, bh = box
+        bx, by = max(0, bx), max(0, by)
+        bw, bh = min(bw, full_w - bx), min(bh, full_h - by)
+    else:
+        bx, by, bw, bh = 0, 0, full_w, full_h
+    if bw <= 0 or bh <= 0:
+        return {"has_ink": None, "bbox": None, "ink_fraction": None, "row_bands": []}
+
+    def px(x, y):
+        off = ((by + y) * full_w + (bx + x)) * 3
+        return data[off], data[off + 1], data[off + 2]
+
+    margin = max(1, int(min(bw, bh) * margin_frac))
+    border = []
+    for x in range(0, bw, max(1, bw // 40 or 1)):
+        border.append(px(x, 0))
+        border.append(px(x, bh - 1))
+    for y in range(0, bh, max(1, bh // 40 or 1)):
+        border.append(px(0, y))
+        border.append(px(bw - 1, y))
+    if not border:
+        border = [px(0, 0)]
+    bg = tuple(sorted(c[i] for c in border)[len(border) // 2] for i in range(3))
+
+    lit_rows = []
+    min_x = min_y = None
+    max_x = max_y = None
+    ink_count = 0
+    for y in range(bh):
+        row_lit = False
+        for x in range(bw):
+            r, g, b = px(x, y)
+            if abs(r - bg[0]) > threshold or abs(g - bg[1]) > threshold or abs(b - bg[2]) > threshold:
+                row_lit = True
+                ink_count += 1
+                min_x = x if min_x is None else min(min_x, x)
+                max_x = x if max_x is None else max(max_x, x)
+                min_y = y if min_y is None else min(min_y, y)
+                max_y = y if max_y is None else max(max_y, y)
+        if row_lit:
+            lit_rows.append(y)
+
+    bands = []
+    for y in lit_rows:
+        if bands and y - bands[-1][1] <= margin:
+            bands[-1][1] = y
+        else:
+            bands.append([y, y])
+
+    has_ink = min_x is not None
+    return {
+        "has_ink": has_ink,
+        "bbox": [min_x, min_y, max_x, max_y] if has_ink else None,
+        "ink_fraction": round(ink_count / (bw * bh), 4) if bw * bh else 0.0,
+        "row_bands": [list(b) for b in bands],
+    }
+
+
 def main() -> int:
     ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
     ap.add_argument("input")
@@ -43,6 +140,9 @@ def main() -> int:
     ap.add_argument("--no-timecode", action="store_true")
     ap.add_argument("--safe", choices=PLATFORM_CHOICES, help="shade the zones this platform's UI covers (its description bar, "
                                                        "like column, status bar) so you can see whether anything readable is under them")
+    ap.add_argument("--ink", action="store_true", help="measure non-background pixels in each written PNG (bbox, "
+                                                 "ink_fraction, row_bands per tile for a contact sheet) instead of "
+                                                 "just eyeballing it -- a pixel-only signal, not a legibility judgement")
     add_common(ap)
     args = ap.parse_args()
     apply_common(args)
@@ -103,9 +203,15 @@ def main() -> int:
         if not args.at:
             die("--compare needs --at TIME")
         probe(args.compare)
-        for t in args.at:
+        for idx, t in enumerate(args.at):
             sec = time_arg(t, "--at", meta["video"].get("fps") if meta.get("video") else None)
-            out = args.output or os.path.join(outdir, f"{stem}_vs_{Path(args.compare).stem}_{sec:.3f}s.png")
+            if args.output and len(args.at) == 1:
+                out = args.output  # one frame, one named image file: the caller's -o is the contract
+            else:
+                # several frames, or -o given as a stem/prefix: an occurrence index keeps every
+                # requested --at its own file even when two round to the same millisecond
+                stem_part = (args.output and Path(args.output).stem) or stem
+                out = os.path.join(outdir, f"{stem_part}_vs_{Path(args.compare).stem}_{idx}_{sec:.3f}s.png")
             half = args.width // 2
             stamp = "" if args.no_timecode else f",drawtext=text='{escape_drawtext(fmt_hms(sec))}':{font_prefix}{FONT}"
             tcs = tc.replace("," + timecode_filter(font_prefix), "") + stamp
@@ -159,7 +265,29 @@ def main() -> int:
     for o in outputs:
         if STATE.dry_run or os.path.exists(o):
             info(f"wrote {o}")
-    emit(outputs[0] if len(outputs) == 1 else None, outputs=outputs)
+
+    ink = None
+    if args.ink and not STATE.dry_run:
+        if outputs and args.at is None and not args.compare:
+            # contact sheet: one PNG holding a cols x rows grid (tile_w:padding=2:margin=2, as built above)
+            # -- split it back into per-tile boxes so each tile is measured on its own, not the whole sheet.
+            sheet = outputs[0]
+            size = _png_size(sheet)
+            if size:
+                full_w, full_h = size
+                margin, padding = 2, 2
+                th = (full_h - 2 * margin - (rows - 1) * padding) // rows if rows else full_h
+                tiles = []
+                for r in range(rows):
+                    for c in range(cols):
+                        bx = margin + c * (tile_w + padding)
+                        by = margin + r * (th + padding)
+                        tiles.append(measure_ink(sheet, box=(bx, by, tile_w, th)))
+                ink = {"tiles": tiles}
+        else:
+            ink = {"frames": [measure_ink(o) for o in outputs]}
+
+    emit(outputs[0] if len(outputs) == 1 else None, outputs=outputs, **({"ink": ink} if ink is not None else {}))
     if len(outputs) > 1 and not args.json:
         for o in outputs:
             print(o)