ffmpeg-skill 2.22.3 → 2.22.4

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`2.22.3`) | any release |
+| `skill.version` | the npm / package.json version (`2.22.4`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -131,7 +131,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
   "deprecated": [{"what": "running on Python 3.9 (the requirements.python floor)", "since": "2.10.9", "replacement": "Python 3.10 or later; ...", "removed_in": "3.0.0", "where": "behaviour"},
                  {"what": "running on FFmpeg 5.x (the requirements.ffmpeg floor)", "since": "2.14.0", "replacement": "FFmpeg 6.1 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
   "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "2.22.3", "execution_mode": "local", "kind": "execution",
+  "skill": {"id": "ffmpeg-skill", "version": "2.22.4", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only; 3.9 deprecated since 2.10.9, removed in 3.0.0; 3.10+ recommended)", "ffmpeg": ">=5.0 (5.x deprecated since 2.14.0, removed in 3.0.0; 6.1+ recommended)", "ffprobe": ">=5.0"},
@@ -730,6 +730,14 @@ pointer to cuts that may sit in a breath):
 | `cut_points` | `silence.py` (every run that cuts) | per cut, `{source_time, output_time, edge, rms_dbfs}`: the RMS level 50 ms either side of the cut on the source, the loudest channel (`null` for digital silence), `edge` `out` where the source is left and `in` where it is picked up; `filler: true` on an edge that sits on a removed filler word. The whole key is `null` when the source could not be decoded for the measurement (`cut_levels_unmeasured` warning): unmeasured, not silent |
 | `cut_warnings` | `silence.py` | the `cut_points` (not filler cuts) whose level is within `silence.CUT_SOUND_MARGIN_DB` (10 dB) of `--threshold`: a breath or a word's weak tail may be cut there. A `cut_near_sound` warning names them. `verified` is unchanged: this is where to listen, not a measured failure |
 
+Keys added after 2.22.1, additive (an agent muxing soft subtitles before a platform export found
+the track gone and nothing in the result saying so):
+
+| key | tool | what it holds |
+|---|---|---|
+| `non_av_streams` | `export.py` | the source's subtitle and data streams: `{kept: [{type, index, codec, language, output_codec}], dropped: [{type, index, codec, language, reason}]}` (`index` is the n-th stream of its type; `language` on subtitles). Read back from the written file; under `--dry-run`, as planned. A subtitle track is copied when the container holds it, else converted to the container's text track (`mov_text` in .mp4/.mov, SubRip in .mkv); a platform trim cuts it with the picture. `reason`: `write_failed` (the attempt that mapped it failed and the encode ran again without it; the error is in its `non_av_streams_dropped` fallback, whose `dropped` says data alone first, then subtitles too), `gif`, `not_in_output` |
+| `dropped_non_av_streams` | `export.py` | `true` when `non_av_streams.dropped` is not empty (a `notes` line too), as on the tools that use `run_keeping_subtitles()` |
+
 Per-tool keys added after 2.10.11 ("nothing happens silently"), all additive:
 
 | key | tool | what it holds |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "2.22.3",
+  "version": "2.22.4",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/references/scripts.md

@@ -1875,6 +1875,17 @@ a failing `{step: loudness, ok: false, reason: not_measured}` and a
 `loudness_not_measured` warning, so the export is `verified: false` rather than
 unchecked.
 
+Subtitle and data streams (a soft-subtitle track muxed upstream, a camera's timecode) are kept
+when the container can hold them: copied as they are, or a text track converted to the
+container's own (mov_text in .mp4/.mov, SubRip in .mkv). A platform trim cuts the track at the
+same point as the picture. A data stream the container refuses is dropped on its own retry, so
+the subtitles are not lost with it (`non_av_streams_dropped` fallback). `non_av_streams: {kept,
+dropped}` lists each source stream with its codec, its `output_codec` or the `reason` it was
+dropped (`write_failed`, `gif`, `not_in_output`), read back from the written file; `dropped_non_av_streams`
+is true when any was dropped, with a `notes` line. A GIF holds none. When the source has such a
+stream, the picture and the sound are mapped explicitly: the first video stream that is not a
+cover picture and the first audio stream, the ones `probe` describes.
+
 Since 1.14 each social destination is its own preset rather than an alias: `tiktok`
 (1080x1920, max 600 s), `shorts` (1080x1920, max 180 s), `reels` (1080x1920, max 90 s),
 `linkedin` (1080x1080), `facebook` (1920x1080), each with its platform's loudness spec from the

package/scripts/_contract.py

@@ -445,7 +445,7 @@ DIAGNOSTIC_CODES = {
     # the runner
     "odd_dimensions_scaled_even": ("fallbacks", "an encode that failed on odd dimensions was run again scaled to even ones (the frame loses a row or column); {output, scale}"),
     "measurement_input_pending": ("skipped", "--dry-run: a measurement skipped because its input is an earlier stage's output that does not exist yet; {path}"),
-    "non_av_streams_dropped": ("fallbacks", "the subtitle/data streams could not be copied, so the encode ran again without them (dropped_non_av_streams: true); {output, returncode, error: the first attempt's last stderr lines}"),
+    "non_av_streams_dropped": ("fallbacks", "the subtitle/data streams could not be copied, so the encode ran again without them (dropped_non_av_streams: true); {output, returncode, error: the first attempt's last stderr lines}; export.py adds dropped: the stream types this retry leaves out (data alone first, then subtitle too)"),
     "ffmpeg_list_unreadable": ("warnings", "`ffmpeg -encoders` / `-bsfs` could not be read; what needs the list says so instead of 'the build lacks it'; {list, tool, error}"),
     "timeout_env_invalid": ("warnings", "FFMPEG_SKILL_TIMEOUT was not a number (the default was used) or was negative (read as 0, no limit); {value, timeout}"),
     # encoder decisions
@@ -610,6 +610,8 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str
     elif name == "export":
         extra = {"loudness": {"type": "object", "description": "platform presets with audio: the written file's lufs/tp against the platform's target_lufs/target_tp, ok true when inside the spec; normalized true when --normalize ran loudness.py on the file; silent true (lufs \"-inf\", ok false, no normalize) when the written audio is silent; measured false (lufs, tp and ok null, reason not_measured) when the measurement could not be read"},
                  "trimmed": {"type": "object", "description": "present when the source was longer than the preset's platform maximum and the export was cut to it (no --allow-long): {from, to} in seconds; notes says so too"},
+                 "non_av_streams": {"type": "object", "description": "the source's subtitle and data streams: {kept: [{type: subtitle|data, index (n-th of its type), codec, language (subtitle), output_codec}], dropped: [{type, index, codec, language, reason: write_failed|gif|not_in_output}]}, read back from the written file (under --dry-run, as planned). A subtitle track is copied when the container holds it as it is, else converted to the container's text track (mov_text in .mp4/.mov, SubRip in .mkv); a trimmed export cuts it at the same point. Both lists are empty for a source with neither"},
+                 "dropped_non_av_streams": {"type": "boolean", "description": "true when non_av_streams.dropped is not empty; a notes line says so too"},
                  "notes": {"type": "array", "items": {"type": "string"}}}
     elif name == "loudness":
         extra = {"measured": {"type": "object", "description": "the loudnorm measurement of the input (input_i, input_tp, input_lra, input_thresh, target_offset); with --measure-only it is the whole result"},

package/scripts/export.py

@@ -36,9 +36,9 @@ import sys
 from pathlib import Path
 from typing import Dict, List
 
-from _common import STATE, add_common, apply_common, bt709_matrix_args, bt709_tag_args, child_args, emit, cfr_args, default_output, die, encoder_args, ffmpeg_base, info, probe, run, run_tool, validate_color, pad_filters, add_pad_fill_args, fmt_secs
+from _common import STATE, add_common, apply_common, bt709_matrix_args, bt709_tag_args, child_args, emit, cfr_args, default_output, die, encoder_args, ffmpeg_base, info, probe, require_tool, run, run_tool, validate_color, pad_filters, add_pad_fill_args, fmt_secs
 from _common.decision import hw_preset_video, restate_last_swap
-from _common.emit import note, record_skip, warn
+from _common.emit import note, record_fallback, record_skip, warn
 from check import SPECS as PLATFORMS, measure_loudness
 from _platforms import ALIASES as _ALIASES, PLATFORMS as PLATFORM_TABLE, TP_TOLERANCE_DB, resolve as resolve_platform
 # Frame and duration limit come from the one platform table (scripts/_platforms.py) rather than
@@ -137,6 +137,108 @@ def _loudness_unmeasured(extra: Dict[str, object], notes: List[str], platform: s
     warn(msg, "loudness_not_measured", platform=platform, normalized=normalized)
 
 
+# Text subtitle codecs ffmpeg can convert to the container's own text track: mov_text is the one
+# text track MP4/MOV hold, Matroska holds SubRip/ASS/WebVTT as they are but not mov_text.
+_TEXT_SUBTITLES = frozenset({"mov_text", "subrip", "srt", "ass", "ssa", "webvtt", "text"})
+_MP4_FAMILY = frozenset({"mp4", "m4v", "mov"})
+
+
+def _subtitle_codec(codec: str, out_ext: str) -> str:
+    """The -c:s for one source subtitle stream: copy when the output container holds it as it is,
+    else the container's own text format (a bitmap track is tried as a copy; a container that
+    refuses it is the fallback below)."""
+    if out_ext in _MP4_FAMILY and codec in _TEXT_SUBTITLES and codec != "mov_text":
+        return "mov_text"
+    if out_ext == "mkv" and codec == "mov_text":
+        return "srt"
+    return "copy"
+
+
+def _data_streams(path: str) -> List[Dict[str, object]]:
+    """The data streams of a file ({index, codec}), without its attachments (probe()'s
+    data_streams counts both, and -map 0:d maps only the data streams)."""
+    proc = run([require_tool("ffprobe"), "-v", "error", "-select_streams", "d", "-show_entries",
+                "stream=codec_name,codec_tag_string", "-of", "json", path], quiet=True)
+    try:
+        streams = json.loads(proc.stdout or "{}").get("streams") or []
+    except ValueError:
+        # an unread list would map nothing and report nothing dropped
+        die(f"ffprobe's data-stream list for {path} did not parse", kind="ffmpeg")
+    return [{"index": n, "codec": s.get("codec_name") or s.get("codec_tag_string")} for n, s in enumerate(streams)]
+
+
+def _run_keeping_non_av(cmd: List[str], output: str, sub_codecs: List[str], data: int) -> List[str]:
+    """run_keeping_subtitles() for export: the subtitle streams go in the codec the container takes
+    (_subtitle_codec), and a refused data stream does not cost the subtitles. The rungs are
+    subtitles + data, subtitles alone, neither; each retry is a `non_av_streams_dropped` fallback
+    with the attempt's error. `cmd` already maps the picture and the sound and has no output.
+    Returns the stream types dropped ([] when everything mapped was written)."""
+    sub_args: List[str] = ["-map", "0:s"] if sub_codecs else []
+    for n, codec in enumerate(sub_codecs):
+        sub_args += [f"-c:s:{n}", codec]
+    data_args = ["-map", "0:d", "-c:d", "copy"] if data else []
+    rungs = [(sub_args + data_args, [])]
+    if sub_codecs and data:
+        rungs.append((sub_args, ["data"]))
+    rungs.append(([], (["subtitle"] if sub_codecs else []) + (["data"] if data else [])))
+    for n, (extra, dropped) in enumerate(rungs):
+        if n == len(rungs) - 1:
+            run(cmd + extra + [output])
+            return dropped
+        attempt = run(cmd + extra + [output], check=False)
+        if attempt.returncode == 0:
+            return dropped
+        error = " | ".join([ln.strip() for ln in (attempt.stderr or "").splitlines() if ln.strip()][-3:])[-500:]
+        going = rungs[n + 1][1]
+        record_fallback(f"{os.path.basename(output)} could not hold the source's {' and '.join(t for t in ('subtitle', 'data') if t in going)} "
+                        f"stream(s) ({error or 'ffmpeg exit ' + str(attempt.returncode)}); encoding again without them",
+                        "non_av_streams_dropped", output=output, returncode=attempt.returncode, error=error, dropped=going)
+    return []  # not reached: the last rung either returns or exits
+
+
+def _written_data_codec(n: int, sources: List[Dict], written: List[Dict]) -> bool:
+    """True when the written file holds a data track for source data stream `n`: tracks are
+    matched by codec, never by position, so with k source streams of one codec and j tracks of it
+    written, the first min(j, k) of those sources count as kept (a muxer's own timecode track is
+    not credited to an unrelated stream, and two copied streams of one codec are both kept)."""
+    codec = sources[n].get("codec")
+    if not codec:
+        return False
+    rank = sum(1 for s in sources[:n] if s.get("codec") == codec)
+    return rank < sum(1 for w in written if w.get("codec") == codec)
+
+
+def _non_av_report(subs: List[Dict], sub_codecs: List[str], data: List[Dict], dropped: List[str],
+                   output: str, reason: str) -> Dict[str, List[Dict]]:
+    """`non_av_streams`: each source subtitle/data stream as kept (with the codec it was written in)
+    or dropped (with why), read back from the written file; under --dry-run, as planned."""
+    kept: List[Dict] = []
+    gone: List[Dict] = []
+    written_subs: List[Dict] = []
+    written_data: List[Dict] = []
+    if not STATE.dry_run and (subs or data):
+        written_subs = probe(output, role="output").get("subtitle_stream_details") or []
+        written_data = _data_streams(output)
+    for kind, streams, written in (("subtitle", subs, written_subs), ("data", data, written_data)):
+        for n, s in enumerate(streams):
+            row = {"type": kind, "index": n, "codec": s.get("codec")}
+            if kind == "subtitle":
+                row["language"] = s.get("language")
+            if STATE.dry_run and kind not in dropped:
+                planned = sub_codecs[n] if kind == "subtitle" else "copy"
+                kept.append(dict(row, output_codec=s.get("codec") if planned == "copy" else planned))
+            elif kind == "subtitle" and n < len(written):
+                kept.append(dict(row, output_codec=written[n].get("codec")))
+            elif kind == "data" and _written_data_codec(n, streams, written):
+                # read back, not assumed, and matched by codec: an MP4/MOV muxer writes a
+                # timecode (tmcd) track of its own from the source's timecode tag even when the
+                # copied one was refused
+                kept.append(dict(row, output_codec=s.get("codec")))
+            else:
+                gone.append(dict(row, reason=reason if kind in dropped else "not_in_output"))
+    return {"kept": kept, "dropped": gone}
+
+
 def main() -> int:
     ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
     ap.add_argument("input", nargs="?")
@@ -202,6 +304,15 @@ def main() -> int:
         else:
             vf.append(f"scale={p['w']}:-2")
     cmd = ffmpeg_base() + ["-i", args.input]
+    # Subtitle and data streams (a soft-subtitle track muxed upstream, a camera's timecode) are
+    # kept where the container can hold them: ffmpeg's default stream selection takes one video
+    # and one audio stream only, and through 2.22.1 every preset dropped the rest without a word.
+    # Mapping them means mapping the picture and the sound too, the same streams probe() describes.
+    subs = meta.get("subtitle_stream_details") or []
+    data = _data_streams(args.input) if meta.get("data_streams") else []
+    sub_codecs = [_subtitle_codec(s.get("codec") or "", out_ext) for s in subs]
+    if args.preset != "gif" and (subs or data):
+        cmd += ["-map", "0:V:0"] + (["-map", "0:a:0"] if has_audio else [])
 
     if args.preset == "gif":
         chain = ",".join(["fps=12"] + vf) if vf else "fps=12"
@@ -209,7 +320,10 @@ def main() -> int:
         cmd += ["-filter_complex", fc, "-loop", "0", output]
         run(cmd)
         info(f"wrote {output}")
-        emit(output, **({"notes": notes} if notes else {}))
+        non_av = _non_av_report(subs, [], data, ["subtitle", "data"], output, "gif")
+        if non_av["dropped"]:
+            notes.append(f"a GIF holds no subtitle or data stream: the source's {len(non_av['dropped'])} were not written")
+        emit(output, non_av_streams=non_av, dropped_non_av_streams=bool(non_av["dropped"]), **({"notes": notes} if notes else {}))
         return 0
 
     if vf:
@@ -259,13 +373,15 @@ def main() -> int:
     trimmed = None
     if p["max"] and not args.allow_long and (meta.get("duration") or 0) > p["max"]:
         # the preset's promise (the platform's maximum), but the caller loses the tail: say how much
-        # in the document (`trimmed`, `notes`), not only on stderr
+        # in the document (`trimmed`, `notes`), not only on stderr. -t is an output option, so a
+        # kept subtitle track is cut at the same point (a cue after it is not written)
         trimmed = {"from": round(float(meta["duration"]), 3), "to": round(float(p["max"]), 3)}
         note(f"trimmed from {trimmed['from']:.3f}s to the {args.preset} maximum of {p['max']:.0f}s "
              "(use --allow-long to keep full length)")
         cmd += ["-t", f"{p['max']:.3f}"]
-    cmd.append(output)
-    run(cmd)
+    dropped = _run_keeping_non_av(cmd, output, sub_codecs, len(data)) if (subs or data) else []
+    if not (subs or data):
+        run(cmd + [output])
     result = probe(output, role="output")
     v = result["video"]
     info(f"wrote {output} ({fmt_secs(result['duration'])}, {v['width']}x{v['height']}, {v['codec']})")
@@ -347,6 +463,13 @@ def main() -> int:
             step = {"step": "duration", "property": "duration", "ok": False, "expected": {"max": p["max"]},
                     "reason": "not_measured: the written file reports no duration"}
         extra.setdefault("verification", []).append(step)
+    # read back after --normalize, which rewrote the file
+    non_av = _non_av_report(subs, sub_codecs, data, dropped, output, "write_failed")
+    if non_av["dropped"]:
+        notes.append(f"{len(non_av['dropped'])} subtitle/data stream(s) of the source are not in {os.path.basename(output)} "
+                     f"({', '.join(sorted({d['type'] + ': ' + d['reason'] for d in non_av['dropped']}))}); non_av_streams lists them")
+    extra["non_av_streams"] = non_av
+    extra["dropped_non_av_streams"] = bool(non_av["dropped"])
     if notes:
         extra["notes"] = notes
     # the delivery keeps the source's picture and, except as a GIF, its sound