ffmpeg-skill 1.19.2 → 1.20.0

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`1.19.2`) | any release |
+| `skill.version` | the npm / package.json version (`1.20.0`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
 
 | What 2.0 removes | Since | Replacement | To be ready today |
 |---|---|---|---|
-| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.19.2 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
-| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.19.2 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
-| `json` and `progress` in the MCP `inputSchema` | 1.19.2 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
-| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.19.2 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
-| Overwriting an existing output with only a warning | 1.19.2 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
+| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.20.0 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
+| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.20.0 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
+| `json` and `progress` in the MCP `inputSchema` | 1.20.0 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
+| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.20.0 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
+| Overwriting an existing output with only a warning | 1.20.0 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
 
 ## Skill
 
 ```json
 {
   "contract_version": "1.0",
-  "deprecated": [{"what": "...", "since": "1.19.2", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "1.19.2", "execution_mode": "local", "kind": "execution",
+  "deprecated": [{"what": "...", "since": "1.20.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
+  "skill": {"id": "ffmpeg-skill", "version": "1.20.0", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
@@ -128,7 +128,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
 | `output_schema` | what `--json` prints on stdout |
 | `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
 | `supports_json` | whether `--json` exists |
-| `supports_json_brief` | whether `--json-brief` exists (1.19.2): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
+| `supports_json_brief` | whether `--json-brief` exists (1.20.0): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
 | `mutates_input` | always `false`: no tool overwrites its input |
 | `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
 | `verification` | `{required, tools}`: which tools to run on the output afterwards |
@@ -434,7 +434,7 @@ given a different type):
 
 | key | tool | what it holds |
 |---|---|---|
-| `caption` | `caption.py` | the cue-layout counts the run only printed before (`shifted`, `wrapped`, `rebalanced`, `split`, `extended`, `dropped`) plus `wrap` (`"phrase"` or `"measured"`) and `phrase_breaks`, the number of cues a phrase rule broke somewhere the 1.15 width rule would not. `broken_inside_word` (1.19.2) counts atoms hard-sliced at the live column's edge because they did not fit alone even at the size floor — never a rewrite, the sliced pieces are the exact original characters. `overlong` is now residual: it fires only when even a single character is wider than the column |
+| `caption` | `caption.py` | the cue-layout counts the run only printed before (`shifted`, `wrapped`, `rebalanced`, `split`, `extended`, `dropped`) plus `wrap` (`"phrase"` or `"measured"`) and `phrase_breaks`, the number of cues a phrase rule broke somewhere the 1.15 width rule would not. `broken_inside_word` (1.20.0) counts atoms hard-sliced at the live column's edge because they did not fit alone even at the size floor — never a rewrite, the sliced pieces are the exact original characters. `overlong` is now residual: it fires only when even a single character is wider than the column |
 | `tracks`, `subtitle_tracks` | `caption.py --mode mux` | one entry per subtitle stream in the output — `{index, file, language, title, codec, default, cues, kept_from_input}`; a stream the input already carried has `file: null` and `kept_from_input: true`. `subtitle_tracks` is the total. Every field describes the file as written, not as asked for: an MPEG-4 output reports `title: null` (the muxer stores none) and `default: true` on its first track (the muxer always enables it), each with a `notes` line |
 | `auto_chapters` | `metadata.py --auto-chapters` | `{source, min_chapter, max_chapters, proposed, kept, titles, chapters, description_block, files}`. `titles` is always `"placeholder"`: the machine-readable form of "the skill did not name these". Each chapter carries its `evidence` (`start`, `silence`, `scene`, or `silence+scene` with the span, its length and the cut time) |
 | `audiogram` | `waveform.py` (every run) | `{style, background, image, position, vis_height, platform, captions, title, stages, verified}`. `background` is `"image"` or `"color"`; `verified` is true when the render probes at the asked-for frame size, frame rate and within 0.05 s of the source audio, and is `false` under `--dry-run`, where nothing was rendered to verify |
@@ -450,7 +450,7 @@ Per-tool keys added in 1.17, all additive:
 | `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
 | `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
 
-Per-tool keys added in 1.19.2, all additive:
+Per-tool keys added in 1.20.0, all additive:
 
 | key | tool | what it holds |
 |---|---|---|
@@ -458,7 +458,7 @@ Per-tool keys added in 1.19.2, all additive:
 | `text_unchanged` | `caption.py` | a sibling inside the `caption` block, **burn mode only** (`--mode mux` never touches the text and omits the key): `true` when the drawn text equals the cues that were handed in — nothing transcribed, no cue dropped, no cue **split** across two consecutive cues and no glyph stripped (`--emoji none`). Wrapping, line breaks and timing do not count: the words are the same. This tool never rewrites, shortens or translates a cue, so the key is a statement of what happened, not a judgement of the text |
 
 
-Per-tool keys added in 1.19.2, all additive:
+Per-tool keys added in 1.20.0, all additive:
 
 | key | tool | what it holds |
 |---|---|---|
@@ -588,7 +588,7 @@ most often across the corpus in `evals/agent_prompts*.json`. Every tool -- inclu
 -- is still callable by name through `tools/call` regardless of what `tools/list` advertised; the
 contract (`ffmpeg-skill contract --json`) still describes all 42 unconditionally. Set
 `FFMPEG_SKILL_MCP_FULL=1` (anything but "" or `0`) in the server's environment to make `tools/list`
-return all 42, as every version before 1.19.2 did.
+return all 42, as every version before 1.20.0 did.
 
 ## Consuming the contract from an agent
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "1.19.2",
+  "version": "1.20.0",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/scripts/graphics.py

@@ -260,6 +260,8 @@ def main() -> int:
         usable = W - left - right - 2 * pad
         return max(0.1, min(SAFE_WIDTH_FRACTION, usable / float(W))) if W else SAFE_WIDTH_FRACTION
 
+    sliced_atoms: List[int] = []
+
     def wrapped(text, size_px, frac=SAFE_WIDTH_FRACTION):
         """A label broken to the frame's safe width (1.16).
 
@@ -268,6 +270,12 @@ def main() -> int:
         the breaks in, at the same measured width and by the same four phrase rules caption.py
         uses. A label that already fits comes back unchanged, which is why this is additive: the
         only text it touches is text that used to run off the edge of the frame.
+
+        `slice_overlong=True` matches caption.py's burn path (1.18.4): a single atom that is
+        still wider than the column even alone -- a long hashtag/URL/name with no break point --
+        is hard-sliced at the column's edge (preferring an existing hyphen) rather than left to
+        render past the frame. An atom that already fits is never touched, so a fitting Thai
+        phrase or katakana run comes through exactly as before.
         """
         text = str(text or "")
         if not text or not size_px:
@@ -277,7 +285,8 @@ def main() -> int:
             return text
         out = []
         for para in text.split("\n"):
-            out.extend(wrap_text(para, max_em, mode=args.wrap, lang=args.lang) if para.strip() else [para])
+            out.extend(wrap_text(para, max_em, mode=args.wrap, lang=args.lang,
+                                  slice_overlong=True, sliced=sliced_atoms) if para.strip() else [para])
         return "\n".join(out)
 
     def add_text(text, drawtext, *, target=None, **el):
@@ -587,6 +596,8 @@ def main() -> int:
         extra["ass"] = ass_path
     if emoji_result:
         extra["emoji"] = emoji_result
+    if sliced_atoms:
+        extra["broken_inside_word"] = len(sliced_atoms)
     emit(output, **extra)
     return 0
 

package/scripts/look.py

@@ -14,10 +14,10 @@ import argparse
 import os
 import sys
 from pathlib import Path
-from typing import List
+from typing import List, Optional
 
 from _platforms import PLATFORMS, PLATFORM_CHOICES, resolve as resolve_platform
-from _common import STATE, add_common, apply_common, default_font_file, die, emit, escape_drawtext, escape_filter_path, ffmpeg_base, info, parse_time, probe, run, time_arg
+from _common import STATE, add_common, apply_common, default_font_file, die, emit, escape_drawtext, escape_filter_path, ffmpeg_base, info, parse_time, probe, run, run_analysis, time_arg
 
 FONT = "fontcolor=white:fontsize=h/18:box=1:boxcolor=black@0.55:boxborderw=6:x=8:y=8"
 
@@ -32,6 +32,103 @@ def timecode_filter(font_prefix: str) -> str:
     return f"drawtext=text='%{{pts\\:hms}}':{font_prefix}{FONT}"
 
 
+def _png_size(png_path: str) -> "Optional[tuple]":
+    # run_analysis(): the same wall-clock ceiling and #234 UTF-8 decoding as every other
+    # ffprobe/ffmpeg measurement in this codebase, check=False since a probe failure here is
+    # an unknown-metric result (has_ink: None), not a reason to die -- a stall still does,
+    # exactly like every other measurement's timeout.
+    proc = run_analysis(["ffprobe", "-v", "error", "-select_streams", "v:0", "-show_entries",
+                         "stream=width,height", "-of", "csv=p=0", png_path], check=False)
+    parts = proc.stdout.strip().split(",")
+    if len(parts) != 2:
+        return None
+    try:
+        return int(parts[0]), int(parts[1])
+    except ValueError:
+        return None
+
+
+def _png_pixels(png_path: str, w: int, h: int) -> "Optional[bytes]":
+    proc = run_analysis(["ffmpeg", "-v", "error", "-i", png_path, "-f", "rawvideo", "-pix_fmt", "rgb24", "-"],
+                        check=False, text=False)
+    data = proc.stdout
+    return data if len(data) >= w * h * 3 else None
+
+
+def measure_ink(png_path: str, *, box=None, margin_frac: float = 0.03, threshold: int = 28) -> dict:
+    """Non-background ink in `png_path` (or the `box` = (x0, y0, w, h) sub-region of it): a pixel-only
+    measurement, never a judgement of legibility. The background is estimated as the median RGB of a
+    thin strip around the region's own edge (so it works on any frame, not just a black synthetic one);
+    a pixel is "ink" when any channel differs from that estimate by more than `threshold`. Returns
+    {has_ink, bbox (x0,y0,x1,y1 within the region, or None), ink_fraction, row_bands} -- row_bands are
+    gap-merged bands of rows carrying ink, the same convention tests/_fixtures.py's _ass_ink_rows uses,
+    so a caption wrapped onto two lines shows as two bands without measuring any font."""
+    size = _png_size(png_path)
+    if not size:
+        return {"has_ink": None, "bbox": None, "ink_fraction": None, "row_bands": []}
+    full_w, full_h = size
+    data = _png_pixels(png_path, full_w, full_h)
+    if data is None:
+        return {"has_ink": None, "bbox": None, "ink_fraction": None, "row_bands": []}
+    if box:
+        bx, by, bw, bh = box
+        bx, by = max(0, bx), max(0, by)
+        bw, bh = min(bw, full_w - bx), min(bh, full_h - by)
+    else:
+        bx, by, bw, bh = 0, 0, full_w, full_h
+    if bw <= 0 or bh <= 0:
+        return {"has_ink": None, "bbox": None, "ink_fraction": None, "row_bands": []}
+
+    def px(x, y):
+        off = ((by + y) * full_w + (bx + x)) * 3
+        return data[off], data[off + 1], data[off + 2]
+
+    margin = max(1, int(min(bw, bh) * margin_frac))
+    border = []
+    for x in range(0, bw, max(1, bw // 40 or 1)):
+        border.append(px(x, 0))
+        border.append(px(x, bh - 1))
+    for y in range(0, bh, max(1, bh // 40 or 1)):
+        border.append(px(0, y))
+        border.append(px(bw - 1, y))
+    if not border:
+        border = [px(0, 0)]
+    bg = tuple(sorted(c[i] for c in border)[len(border) // 2] for i in range(3))
+
+    lit_rows = []
+    min_x = min_y = None
+    max_x = max_y = None
+    ink_count = 0
+    for y in range(bh):
+        row_lit = False
+        for x in range(bw):
+            r, g, b = px(x, y)
+            if abs(r - bg[0]) > threshold or abs(g - bg[1]) > threshold or abs(b - bg[2]) > threshold:
+                row_lit = True
+                ink_count += 1
+                min_x = x if min_x is None else min(min_x, x)
+                max_x = x if max_x is None else max(max_x, x)
+                min_y = y if min_y is None else min(min_y, y)
+                max_y = y if max_y is None else max(max_y, y)
+        if row_lit:
+            lit_rows.append(y)
+
+    bands = []
+    for y in lit_rows:
+        if bands and y - bands[-1][1] <= margin:
+            bands[-1][1] = y
+        else:
+            bands.append([y, y])
+
+    has_ink = min_x is not None
+    return {
+        "has_ink": has_ink,
+        "bbox": [min_x, min_y, max_x, max_y] if has_ink else None,
+        "ink_fraction": round(ink_count / (bw * bh), 4) if bw * bh else 0.0,
+        "row_bands": [list(b) for b in bands],
+    }
+
+
 def main() -> int:
     ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
     ap.add_argument("input")
@@ -43,6 +140,9 @@ def main() -> int:
     ap.add_argument("--no-timecode", action="store_true")
     ap.add_argument("--safe", choices=PLATFORM_CHOICES, help="shade the zones this platform's UI covers (its description bar, "
                                                        "like column, status bar) so you can see whether anything readable is under them")
+    ap.add_argument("--ink", action="store_true", help="measure non-background pixels in each written PNG (bbox, "
+                                                 "ink_fraction, row_bands per tile for a contact sheet) instead of "
+                                                 "just eyeballing it -- a pixel-only signal, not a legibility judgement")
     add_common(ap)
     args = ap.parse_args()
     apply_common(args)
@@ -103,9 +203,15 @@ def main() -> int:
         if not args.at:
             die("--compare needs --at TIME")
         probe(args.compare)
-        for t in args.at:
+        for idx, t in enumerate(args.at):
             sec = time_arg(t, "--at", meta["video"].get("fps") if meta.get("video") else None)
-            out = args.output or os.path.join(outdir, f"{stem}_vs_{Path(args.compare).stem}_{sec:.3f}s.png")
+            if args.output and len(args.at) == 1:
+                out = args.output  # one frame, one named image file: the caller's -o is the contract
+            else:
+                # several frames, or -o given as a stem/prefix: an occurrence index keeps every
+                # requested --at its own file even when two round to the same millisecond
+                stem_part = (args.output and Path(args.output).stem) or stem
+                out = os.path.join(outdir, f"{stem_part}_vs_{Path(args.compare).stem}_{idx}_{sec:.3f}s.png")
             half = args.width // 2
             stamp = "" if args.no_timecode else f",drawtext=text='{escape_drawtext(fmt_hms(sec))}':{font_prefix}{FONT}"
             tcs = tc.replace("," + timecode_filter(font_prefix), "") + stamp
@@ -159,7 +265,29 @@ def main() -> int:
     for o in outputs:
         if STATE.dry_run or os.path.exists(o):
             info(f"wrote {o}")
-    emit(outputs[0] if len(outputs) == 1 else None, outputs=outputs)
+
+    ink = None
+    if args.ink and not STATE.dry_run:
+        if outputs and args.at is None and not args.compare:
+            # contact sheet: one PNG holding a cols x rows grid (tile_w:padding=2:margin=2, as built above)
+            # -- split it back into per-tile boxes so each tile is measured on its own, not the whole sheet.
+            sheet = outputs[0]
+            size = _png_size(sheet)
+            if size:
+                full_w, full_h = size
+                margin, padding = 2, 2
+                th = (full_h - 2 * margin - (rows - 1) * padding) // rows if rows else full_h
+                tiles = []
+                for r in range(rows):
+                    for c in range(cols):
+                        bx = margin + c * (tile_w + padding)
+                        by = margin + r * (th + padding)
+                        tiles.append(measure_ink(sheet, box=(bx, by, tile_w, th)))
+                ink = {"tiles": tiles}
+        else:
+            ink = {"frames": [measure_ink(o) for o in outputs]}
+
+    emit(outputs[0] if len(outputs) == 1 else None, outputs=outputs, **({"ink": ink} if ink is not None else {}))
     if len(outputs) > 1 and not args.json:
         for o in outputs:
             print(o)