ffmpeg-skill 2.1.0 → 2.2.0

package/README.md

@@ -203,7 +203,7 @@ These are the rules the skill file gives the agent and the code enforces.
 | Tool | What it does |
 |---|---|
 | `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact video and sample-exact audio; reports `precision`; `--snap beats` moves the in/out points onto a measured beat, or refuses when there is no measurable pulse |
-| `join.py` | Concatenate clips with xfade transitions, normalising size, fps, sample rate and channel layout (the widest clip's, or `--channels`); audio-only inputs are joined as audio |
+| `join.py` | Concatenate clips with xfade transitions, normalising size, fps, sample rate and channel layout (the widest clip's, or `--channels`); audio-only inputs are joined as audio; `--list parts.txt` reads generated segments (TTS lines) and checks every one first — missing, empty or unreadable parts are all named in one refusal, or skipped and reported with `--on-missing skip` |
 | `silence.py` | Detect and remove dead air (jump cuts) with a margin around speech; list or export the cut list; `--filler` also removes filler words, but only where a speech engine timed them; `--speech-aware` keeps a breath inside a sentence and cuts only at sentence-boundary pauses, composing with `--filler` through the same `keep_ranges()` |
 | `fit.py` | Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad, crop or `--fit blur`'s blurred, darkened fill, with `--crop-x`/`--crop-y` to keep an off-centre subject) and/or exact `--width`/`--height`; rotate 90/180/270, flip h/v; force constant fps |
 | `crop.py` | Crop to an exact pixel rectangle (`--x --y --width --height`) — distinct from `fit.py --fit crop`, which crops to an aspect ratio it computes itself |

package/SKILL.md

@@ -21,7 +21,7 @@ Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `chec
 5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix. Judgement rows change content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix only when the request implies the answer, else state the choice and its cost. Mention WARNs; do not chase them.
 6. **Verify the output.** Confirm duration, resolution, fps and audio match the request, from the writing tool's `--json`/`--json-brief` probe or `probe.py`, and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure reported with the script's error message.
 7. **Keep the user's originals.** Never overwrite the source; write new files next to input or where asked. An existing output path is refused (`kind: input`); `--overwrite` is the one way to say "yes, replace it".
-8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` scaling a clip to the first clip's frame, `color.py --to-sdr`) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for whole-clip layout jobs. Not finished until `Look:` names that PNG — a probe cannot see a caption on someone's face. Audio-only jobs write `Look: not needed`. What to look for splits like `check.py`'s rows in step 5:
+8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` scaling a clip to the first clip's frame, `color.py --to-sdr`) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG (4x3 for whole-clip layout). Not finished until `Look:` names that PNG — a probe cannot see a caption on a face. Audio-only: `Look: not needed`. What to look for splits like `check.py`'s rows in step 5:
    - **Mechanical (this skill's own job to verify and report):** the specified text/logo is at the specified position, subtitles appear at the specified timestamps, dimensions are even. Letterboxing from `fit.py --fit pad` is the *correct* result of that mode, never a defect to flag.
    - **Judgement (report it, don't silently pass or fail):** whether a subject or face is cut off, text sits over a face, colours look washed out, a transition lands. These need deciding what the subject *is*, which belongs to the calling agent — say what you see in one line and let them judge it.
    With no vision capability, write `Look: PATH (pixels not inspected; agent has no image view)` — never claim a picture was inspected when it wasn't, without stalling for a capability that isn't there.
@@ -136,7 +136,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
 | "add chapters", "chapter markers for YouTube" | `metadata.py episode.mp4 --chapters chapters.txt` (`TIME TITLE` per line; streams copied; in a project: `"chapters"`). `--auto-chapters` proposes them from measured pauses/scene cuts (`Chapter N`) |
 | "set the title / artist / comment" | `metadata.py episode.mp4 --title "Episode 12" --artist "Studio"` |
 | "put these videos in a 4x2 grid" | `grid.py t1.mp4 ... t8.mp4 --cols 4 --rows 2` |
-| "stitch these clips", "add a crossfade" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` |
+| "stitch these clips", "add a crossfade" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` (TTS parts: `--list parts.txt`, `--on-missing skip`) |
 | "several changes to one edit", 3+ steps | `render.py --init project.json`, edit, `render.py project.json` |
 | "I changed one stage, don't redo the rest" | `render.py project.json --cache DIR` — identical stages reused (`--from STAGE` starts there) |
 | "open it in Premiere / Resolve / Final Cut" | `render.py project.json --export-timeline edit.fcpxml\|.edl\|.otio` — the cut as a timeline, nothing rendered; read `not_exported` |

package/docs/contract.md

@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
 | Field | Meaning | Changes when |
 |---|---|---|
 | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
-| `skill.version` | the npm / package.json version (`2.1.0`) | any release |
+| `skill.version` | the npm / package.json version (`2.2.0`) | any release |
 
 A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
 ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -119,7 +119,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
   "contract_version": "1.0",
   "deprecated": [],
   "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
-  "skill": {"id": "ffmpeg-skill", "version": "2.1.0", "execution_mode": "local", "kind": "execution",
+  "skill": {"id": "ffmpeg-skill", "version": "2.2.0", "execution_mode": "local", "kind": "execution",
             "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
             "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
   "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ffmpeg-skill",
-  "version": "2.1.0",
+  "version": "2.2.0",
   "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
   "keywords": [
     "ffmpeg",

package/references/scripts.md

@@ -554,7 +554,16 @@ writes the resulting cut list exactly as it does today.
 ```
 join.py CLIP1 CLIP2 [...] [--transition fade|dissolve|wipeleft|slideleft|fadeblack|fadewhite|circleopen|none]
         [--duration 0.5] [--width W --height H] [--fps N] [--fit pad|crop] [-o OUT]
-```
+join.py --list parts.txt [--on-missing fail|skip] [...]
+```
+`--list FILE` (2.2) reads the clips from a file, one per line in order, paths relative to the
+file (blank lines and `#` comments ignored; ffmpeg's `file 'x.wav'` lines accepted), for the
+segments a TTS or render step wrote. Every input -- listed or given as arguments -- is checked
+before anything runs: missing, empty (0 bytes) or unreadable as media. By default all of them
+are named in one refusal (`kind: input`, `problems: [{index, path, reason}]`, nothing encoded);
+`--on-missing skip` joins the usable ones and reports the rest under `skipped` (always present,
+`[]` when nothing was). A list with no entries is refused as such, and a skip that leaves fewer
+than two inputs is refused rather than "joining" one file.
 Normalises every clip to one frame size, fps, `yuv420p`, 48 kHz and one
 channel layout (the widest clip's -- a 5.1 clip keeps 5.1 -- or `--channels`;
 silent track generated for clips without audio), then chains `xfade` +

package/scripts/_contract.py

@@ -453,7 +453,8 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
     elif name == "join":
         extra = {"mode": {"enum": ["video", "audio"]}, "clips": {"type": "integer"}, "transition": {"type": "string"}, "expected_duration": {"type": "number"},
                  "sample_rate": {"type": "integer", "description": "audio mode only"}, "channels": {"type": "integer", "description": "audio mode only"},
-                 "video": {"type": "boolean", "description": "false in audio mode: the output has no video stream"}}
+                 "video": {"type": "boolean", "description": "false in audio mode: the output has no video stream"},
+                 "skipped": {"type": "array", "description": "inputs --on-missing skip left out: [{index, path, reason}] ([] when none were)"}}
     elif name == "audio":
         extra = {"video": {"type": "boolean", "description": "true when the input's video stream was copied; false for an audio output extension (extraction)"},
                  "audio_stream": {"type": "integer", "description": "which input audio stream was processed (--audio-stream)"},

package/scripts/join.py

@@ -18,16 +18,25 @@ Examples:
   python3 join.py a.mov b.mp4 --transition none --width 1920 --height 1080 --fps 30
   python3 join.py intro.wav talk.m4a outro.wav -o episode.flac            # audio join, 0.5 s crossfade
   python3 join.py part1.wav part2.wav --transition none -o full.wav       # butt join, sample rate of part1
+  python3 join.py --list parts.txt --transition none -o voice.wav          # the segments a TTS step wrote, in order
+  python3 join.py --list parts.txt --on-missing skip -o voice.wav          # join what is there, report the rest
+
+Every input is checked before anything is joined: a missing, empty or unreadable segment is
+refused with every problem named at once (kind input), never discovered one run at a time.
+--on-missing skip joins the usable segments instead and lists the others under `skipped`.
 """
 import argparse
+import os
 import sys
-from typing import List
+from typing import Any, Dict, List
 
-from _common import STATE, video_args, aac_args, add_common, apply_common, audio_codec_for, default_output, die, emit, ffmpeg_base, info, is_audio_output, probe, run, validate_color, X264_PRESETS, fmt_secs
+from _common import STATE, video_args, aac_args, add_common, apply_common, audio_codec_for, default_output, die, emit, ffmpeg_base, info, is_audio_output, probe, require_tool, run, validate_color, X264_PRESETS, fmt_secs
 
 TRANSITIONS = ["fade", "dissolve", "wipeleft", "wiperight", "wipeup", "wipedown", "slideleft", "slideright",
                "circleopen", "circleclose", "fadeblack", "fadewhite", "smoothleft", "smoothright", "radial", "none"]
 LAYOUTS = {1: "mono", 2: "stereo", 6: "5.1", 8: "7.1"}
+# inputs --on-missing skip dropped, reported in the success document (empty when nothing was)
+STATE_SKIPPED: List[Dict[str, Any]] = []
 
 
 def join_audio(args: argparse.Namespace, metas: List[dict]) -> int:
@@ -77,14 +86,74 @@ def join_audio(args: argparse.Namespace, metas: List[dict]) -> int:
             die(f"{output} is {a.get('sample_rate')} Hz {a.get('channels')} ch, expected {rate} Hz {channels} ch")
     info(f"wrote {output} ({fmt_secs(r['duration'])}, expected ~{expected:.3f}s, audio {a.get('codec')} {channels}ch {rate}Hz, {n} clips, "
          + ("crossfade" if d else "butt join") + ")")
-    emit(output, mode="audio", clips=n, transition=args.transition if d else "none", expected_duration=round(expected, 3),
+    emit(output, mode="audio", skipped=list(STATE_SKIPPED), clips=n, transition=args.transition if d else "none", expected_duration=round(expected, 3),
          sample_rate=rate, channels=channels, video=False)
     return 0
 
 
+def read_list(path: str) -> List[str]:
+    """--list FILE: one segment per line, in order, relative to the list file's own folder.
+    Blank lines and # comments are ignored, and ffmpeg's concat-demuxer spelling
+    (`file 'part 01.wav'`) is accepted, so a list written for `ffmpeg -f concat` works as is."""
+    try:
+        text = open(path, encoding="utf-8").read()
+    except OSError as exc:
+        die(f"cannot read --list {path}: {exc}")
+    base = os.path.dirname(os.path.abspath(path))
+    entries: List[str] = []
+    for line in text.splitlines():
+        line = line.strip()
+        if not line or line.startswith("#"):
+            continue
+        if line.startswith("file "):
+            line = line[5:].strip()
+            if len(line) >= 2 and line[0] == line[-1] and line[0] in "'\"":
+                line = line[1:-1].replace("'\\''", "'")
+        entries.append(line if os.path.isabs(line) else os.path.join(base, line))
+    if not entries:
+        # the failure this exists for: an upstream step produced nothing, and the join must
+        # say so plainly rather than hand ffmpeg an empty concat and die somewhere later
+        die(f"--list {path} names no segments (empty, or only blank lines and comments)",
+            hint="check the step that writes the list: it produced no files")
+    return entries
+
+
+def preflight(paths: List[str]) -> List[Dict[str, Any]]:
+    """Every problem with every input, found before anything runs: missing, empty (0 bytes,
+    the usual trace of a TTS or download step that failed after creating its file), or not
+    readable as media. One ffprobe per file; nothing is decoded. Under --dry-run a file that
+    does not exist yet is not a problem: in a planned pipeline it is an earlier step's output."""
+    ffprobe = require_tool("ffprobe")
+    problems: List[Dict[str, Any]] = []
+    for i, p in enumerate(paths):
+        if not os.path.exists(p):
+            if STATE.dry_run:
+                continue  # a dry-run pipeline plans on outputs earlier steps have not written yet
+            problems.append({"index": i, "path": p, "reason": "missing"})
+            continue
+        if os.path.isdir(p):
+            problems.append({"index": i, "path": p, "reason": "a directory, not a file"})
+            continue
+        if os.path.getsize(p) == 0:
+            problems.append({"index": i, "path": p, "reason": "empty (0 bytes)"})
+            continue
+        proc = run([ffprobe, "-v", "error", "-show_entries", "stream=codec_type", "-of", "csv=p=0", p],
+                   quiet=True, check=False)
+        kinds = {line.strip() for line in (proc.stdout or "").splitlines() if line.strip()}
+        if proc.returncode != 0 or not kinds & {"audio", "video"}:
+            first = ((proc.stderr or "").strip().splitlines() or ["no audio or video stream"])[-1]
+            problems.append({"index": i, "path": p, "reason": f"unreadable: {first}"})
+    return problems
+
+
 def main() -> int:
     ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
-    ap.add_argument("inputs", nargs="+", help="two or more clips in order")
+    ap.add_argument("inputs", nargs="*", help="two or more clips in order (or --list FILE)")
+    ap.add_argument("--list", metavar="FILE", help="read the clips from FILE, one per line in order (paths relative to FILE; "
+                                                   "ffmpeg concat lines `file 'x.wav'` also accepted); an empty list is refused")
+    ap.add_argument("--on-missing", choices=["fail", "skip"], default="fail",
+                    help="a missing, empty or unreadable input: fail (default; every problem named at once) or skip "
+                         "(join the rest; each skipped input is reported under `skipped`)")
     ap.add_argument("-o", "--output", help="output file (default: <first>_joined.mp4)")
     ap.add_argument("--transition", choices=TRANSITIONS, default="fade", help="transition between clips (default fade)")
     ap.add_argument("--duration", type=float, default=0.5, help="transition length in seconds (default 0.5)")
@@ -104,8 +173,29 @@ def main() -> int:
     if args.fps is not None and args.fps <= 0:
         die(f"--fps must be positive, got {args.fps:g}")
 
+    if args.list:
+        if args.inputs:
+            die("give the clips either as arguments or with --list, not both")
+        args.inputs = read_list(args.list)
+    if not args.inputs:
+        die("give at least two clips (or --list FILE)")
+    skipped: List[Dict[str, Any]] = []
+    problems = preflight(args.inputs)
+    if problems and args.on_missing == "fail":
+        listing = "; ".join(f"#{p['index'] + 1} {p['path']}: {p['reason']}" for p in problems)
+        die(f"{len(problems)} of {len(args.inputs)} inputs cannot be joined: {listing}",
+            hint="fix or regenerate them, or pass --on-missing skip to join the rest", problems=problems)
+    if problems:
+        bad = {p["index"] for p in problems}
+        skipped = problems
+        args.inputs = [p for i, p in enumerate(args.inputs) if i not in bad]
+        for p in problems:
+            info(f"skipping #{p['index'] + 1} {p['path']}: {p['reason']}")
+    STATE_SKIPPED[:] = skipped
     if len(args.inputs) < 2:
-        die("give at least two clips")
+        die("give at least two clips" if not skipped else
+            f"only {len(args.inputs)} usable input(s) left after skipping {len(skipped)}; nothing to join",
+            skipped=skipped)
     validate_color(args.pad_color, "--pad-color")
     metas = [probe(p) for p in args.inputs]
     if all(not m.get("video") for m in metas):
@@ -206,7 +296,7 @@ def main() -> int:
     expected = sum(durs) - d * (n - 1)
     r = probe(output, role="output")
     info(f"wrote {output} ({fmt_secs(r['duration'])}, expected ~{expected:.3f}s, {w}x{h} @ {fps:g}fps, {n} clips, {args.transition})")
-    emit(output, mode="video", clips=n, transition=args.transition, expected_duration=round(expected, 3),
+    emit(output, mode="video", skipped=list(STATE_SKIPPED), clips=n, transition=args.transition, expected_duration=round(expected, 3),
          dropped_non_av_streams=any(m.get("subtitle_streams") or m.get("data_streams") for m in metas))
     return 0