ffmpeg-skill 3.1.7 → 3.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/contract.md CHANGED
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
21
21
  | Field | Meaning | Changes when |
22
22
  |---|---|---|
23
23
  | `contract_version` | shape of this document (`2.0`) | a key is renamed, removed or changes meaning |
24
- | `skill.version` | the npm / package.json version (`3.1.7`) | any release |
24
+ | `skill.version` | the npm / package.json version (`3.1.8`) | any release |
25
25
 
26
26
  A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
27
27
  ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -149,7 +149,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
149
149
  "deprecated": [],
150
150
  "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"},
151
151
  {"what": "running on FFmpeg 5.x (the requirements.ffmpeg floor)", "since": "2.14.0", "replacement": "FFmpeg 6.1 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
152
- "skill": {"id": "ffmpeg-skill", "version": "3.1.7", "execution_mode": "local", "kind": "execution",
152
+ "skill": {"id": "ffmpeg-skill", "version": "3.1.8", "execution_mode": "local", "kind": "execution",
153
153
  "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
154
154
  "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
155
155
  "requirements": {"python": ">=3.10 (standard library only)", "ffmpeg": ">=6.1", "ffprobe": ">=6.1"},
@@ -204,6 +204,11 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
204
204
  | `execution` (writes a new artifact) | cut, fit, caption, overlay, graphics, multicam, audio, join, color, export, render, batch |
205
205
  | `verification` (checks or shows an artifact) | check, look, verify, report |
206
206
 
207
+ `batch` writes one artifact per input only when a recipe step names `{out}` (or the recipe is a
208
+ render project): a recipe in which no step does (a QC pass of `check.py`/`probe.py`) writes none,
209
+ and each `results` row carries `written: false` beside its usual `output` path, which stays a
210
+ string (#415).
211
+
207
212
  ### Verification policy
208
213
 
209
214
  The workflow in `SKILL.md` is "probe first, verify last". The contract states it per tool:
@@ -882,6 +887,14 @@ caller never parses the prose:
882
887
  A tool that runs another (`render.py` stages and plans) forwards the child's `reason` and
883
888
  `details` in its own `error`. Both are absent on a failure that is not a refusal, like `hint`.
884
889
 
890
+ A `timeout` or `interrupted` failure exits 124, or 130/143, whether the tool or a child tool it ran
891
+ was stopped (`export.py --normalize`'s loudness pass, `render.py`'s stages, check, executed plan and
892
+ pack destinations, `waveform.py`'s title and captions pass the child's code on). Its message ends "nothing was written" only when
893
+ that is so: once a tool has written its deliverable and runs a further step on it
894
+ (`export.py --normalize`, `render.py`'s chapters and check stages), the message names the file,
895
+ says it is kept without the step that was stopped, and the document carries it as `output`
896
+ (3.1.6, #415).
897
+
885
898
  **Diagnostics.** What a run decided on its own -- a warning, a fallback it took, a step it
886
899
  skipped -- is recorded as it happens (a stderr line too) and lands in the document as
887
900
  `diagnostics: {warnings, fallbacks, skipped}`, each entry `{code, message, ...facts}`; it is
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "3.1.7",
3
+ "version": "3.1.8",
4
4
  "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
5
  "keywords": [
6
6
  "ffmpeg",
@@ -1240,10 +1240,18 @@ above the cap is clamped with a note, not refused, and both `jobs` and
1240
1240
  earlier has no such record and is processed again once, `entry_unverifiable`).
1241
1241
  A `.ffskill_cache.json` that does not parse is started empty with a
1242
1242
  `batch_cache_reset` warning and a line in `output_dir/.ffskill-audit.jsonl`.
1243
- - With `--jobs > 1` each item gets its own work subdirectory
1244
- `<workdir>/<index>-<stem>/`, because step file names are stem-derived and two
1245
- sources sharing a stem would otherwise write over each other. `--jobs 1` (the
1246
- default) keeps the flat layout and is 1.16's path exactly.
1243
+ - Each item's intermediates go in a new directory of its own under the work
1244
+ directory (`<workdir>/<index>-<random>/`), removed with them when the item
1245
+ ends, whatever the outcome (3.1.6, #415): step file names are stem-derived, so
1246
+ `clip.mp4` and `clip.mov` sharing a directory, or a rerun meeting an earlier
1247
+ run's files, were refused as someone else's. A step that names no `{out}`
1248
+ (`check.py`) passes its input on; when the last step writes nothing, the last
1249
+ file written is delivered as the item's output, under its own extension (`.mp4`
1250
+ for a video source) unless the recipe sets `ext`. A recipe in which no step names
1251
+ `{out}` (a QC pass: `check.py`/`probe.py` on every file) writes nothing: each
1252
+ row has `written: false` (`output` keeps its path string), a note says so, and `ok` follows the steps' exit codes.
1253
+ Final names that differ only in case are a
1254
+ collision (they are one file on macOS and Windows).
1247
1255
  - `--watch` composes: each pass is parallel, passes are sequential.
1248
1256
 
1249
1257
  ### caption.py --transcribe — optional local speech-to-text
@@ -114,9 +114,21 @@ def frame_size(ratio: Optional[Fraction], width: Optional[int], height: Optional
114
114
  SVT_PRESET = {"ultrafast": 12, "superfast": 11, "veryfast": 10, "faster": 9, "fast": 8, "medium": 6, "slow": 4, "slower": 3, "veryslow": 2, "placebo": 1}
115
115
 
116
116
 
117
+ # Source containers the tools' own encoders (x264/x265/SVT-AV1 and AAC) cannot be written into: WebM
118
+ # takes only VP8/VP9/AV1 and Vorbis/Opus. A default output name keeps the source's extension
119
+ # except for these, whose default is .mp4 (#415): the dry run planned `clip_loudnorm.webm` and the
120
+ # real run then failed in the muxer.
121
+ NO_DEFAULT_ENCODE_EXT = {".webm"}
122
+
123
+
117
124
  def default_output(input_path: str, suffix: str, ext: Optional[str] = None) -> str:
118
125
  p = Path(input_path)
119
- new_ext = ext if ext else p.suffix.lstrip(".") or "mp4"
126
+ if ext:
127
+ new_ext = ext
128
+ elif p.suffix.lower() in NO_DEFAULT_ENCODE_EXT:
129
+ new_ext = "mp4"
130
+ else:
131
+ new_ext = p.suffix.lstrip(".") or "mp4"
120
132
  return str(p.with_name(f"{p.stem}_{suffix}.{new_ext}"))
121
133
 
122
134
 
@@ -14,7 +14,7 @@ import re
14
14
  import shutil
15
15
  import subprocess
16
16
  import sys
17
- from typing import Any, Dict, List, Optional, Sequence, Tuple
17
+ from typing import Any, Callable, Dict, List, Optional, Sequence, Tuple
18
18
 
19
19
 
20
20
  INSTALL_HINTS = {
@@ -108,6 +108,13 @@ _FFMPEG_VERSION: "Optional[Tuple[int, int]]" = None
108
108
  _FFMPEG_VERSION_IS_RELEASE = False # ffmpeg_version() read a release number, not the libavutil major
109
109
 
110
110
 
111
+ def _probe_limit() -> float:
112
+ """The ceiling for a short read-only ffprobe/ffmpeg query run outside run(): PROBE_TIMEOUT, or
113
+ the caller's --timeout when that is lower (#415). Never unlimited: a version or stream-list read
114
+ that hangs is a stall, whatever --timeout 0 asks of the encodes."""
115
+ return min(PROBE_TIMEOUT, STATE.timeout) if STATE.timeout else PROBE_TIMEOUT
116
+
117
+
111
118
  def ffmpeg_version() -> "Tuple[int, int]":
112
119
  """(major, minor) of the FFmpeg build on PATH, parsed once from `ffprobe -version`; (0, 0)
113
120
  when it cannot be read. ffprobe rather than ffmpeg because --dry-run promises never to run
@@ -120,7 +127,7 @@ def ffmpeg_version() -> "Tuple[int, int]":
120
127
  _FFMPEG_VERSION = (0, 0)
121
128
  try:
122
129
  out = subprocess.run([found_exe("ffprobe"), "-version"], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True, encoding="utf-8", errors="replace",
123
- timeout=PROBE_TIMEOUT).stdout
130
+ timeout=_probe_limit()).stdout
124
131
  m = re.search(r"ffprobe version\s+n?(\d+)\.(\d+)", out)
125
132
  if m:
126
133
  _FFMPEG_VERSION = (int(m.group(1)), int(m.group(2)))
@@ -158,7 +165,7 @@ def ffmpeg_release() -> "Optional[Tuple[int, int, int]]":
158
165
  _FFMPEG_RELEASE_READ = True
159
166
  try:
160
167
  out = subprocess.run([found_exe("ffprobe"), "-version"], stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True, encoding="utf-8", errors="replace",
161
- timeout=PROBE_TIMEOUT).stdout
168
+ timeout=_probe_limit()).stdout
162
169
  m = re.search(r"ffprobe version\s+n?(\d+)\.(\d+)(?:\.(\d+))?", out)
163
170
  if m:
164
171
  _FFMPEG_RELEASE = (int(m.group(1)), int(m.group(2)), int(m.group(3) or 0))
@@ -325,7 +332,7 @@ class Context:
325
332
  __slots__ = ("dry_run", "json", "json_brief", "progress", "fast", "duration_hint", "commands", "timeout", "overwrite", "written", "preexisting", "plan", "plan_written", "plan_inputs", "codec",
326
333
  "hw", "hw_source", "hw_notes", "hw_swaps", "hw_fallback", "hw_env_ignored", "hw_stages", "hw_auto",
327
334
  "notes", "warnings", "fallbacks", "skipped", "audit_log", "stage_verification", "first_input", "audio_inputs", "video_inputs",
328
- "source_timecode", "smpte_hint")
335
+ "source_timecode", "smpte_hint", "delivered")
329
336
 
330
337
  def __init__(self) -> None:
331
338
  self.reset()
@@ -337,6 +344,7 @@ class Context:
337
344
  self.video_inputs = set() # real paths of the inputs probed that have a picture: with two or more, which one the output shows is the tool's choice (sync --trim-second, a join)
338
345
  self.source_timecode = None # (start timecode, duration) of the first input probed that carries one (time_arg's SMPTE hint)
339
346
  self.smpte_hint = None # what die() says when an hh:mm:ss:ff flag read past the end looks like a source timecode (#411)
347
+ self.delivered: List[str] = [] # deliverables written before a later step (export --normalize, render's chapters/check): a stop then keeps them (#415)
340
348
  self.audio_inputs = set() # real paths of the inputs probed that have sound: with two or more, whose tracks an output should keep is the tool's choice (grid --audio-from, a mix)
341
349
  self.json_brief = False # --json-brief: the same document trimmed to the fields a caller acts on
342
350
  self.progress = False # run() streams percent / ETA to stderr for ffmpeg
@@ -533,12 +541,52 @@ def stop_children() -> None:
533
541
  _CHILDREN.clear()
534
542
 
535
543
 
544
+ def mark_delivered(path: str) -> None:
545
+ """Record that this run has written its deliverable and still has a step to run on it (export's
546
+ --normalize, render's chapters and check stages): a timeout or an interrupt from then on says the
547
+ file is there and kept, instead of "nothing was written" (#415)."""
548
+ if path and path not in STATE.delivered:
549
+ STATE.delivered.append(path)
550
+
551
+
552
+ def child_exit_code(doc: Any, returncode: Optional[int]) -> int:
553
+ """The exit code a tool passes on from a child tool that failed: the code its document states,
554
+ else its process's own, else 1. A child killed by a signal has a negative returncode (-9), which
555
+ as an exit code wraps (247); it is passed on as the shell reports it, 128 + the signal (137)."""
556
+ code = (doc.get("exit_code") if isinstance(doc, dict) else None) or returncode or 1
557
+ try:
558
+ code = int(code)
559
+ except (TypeError, ValueError):
560
+ return 1
561
+ return 128 - code if code < 0 else code
562
+
563
+
564
+ def kept_deliverables() -> List[str]:
565
+ """The deliverables mark_delivered() named that are on disk."""
566
+ return [p for p in STATE.delivered if os.path.isfile(p)]
567
+
568
+
569
+ def kept_clause() -> str:
570
+ """How a stopped run's message ends: "nothing was written", or which deliverable it had already
571
+ written and keeps (not the finished result: the step that was stopped did not run on it)."""
572
+ kept = kept_deliverables()
573
+ if not kept:
574
+ return "nothing was written"
575
+ return (f"{', '.join(kept)} had already been written by this run and {'is' if len(kept) == 1 else 'are'} kept, "
576
+ "without the step that was stopped (pass --overwrite to make it again)")
577
+
578
+
579
+ def _kept_fields() -> Dict[str, Any]:
580
+ kept = kept_deliverables()
581
+ return {"output": kept[-1]} if kept else {}
582
+
583
+
536
584
  def _on_signal(signum: int, frame: Any) -> None:
537
585
  import signal as _signal
538
586
  name = {getattr(_signal, "SIGINT", None): "SIGINT", getattr(_signal, "SIGTERM", None): "SIGTERM"}.get(signum, str(signum))
539
587
  stop_children()
540
- die(f"interrupted by {name}: the running command was stopped and its partial output removed; nothing was written",
541
- code=128 + signum, kind="interrupted")
588
+ die(f"interrupted by {name}: the running command was stopped and its partial output removed; {kept_clause()}",
589
+ code=128 + signum, kind="interrupted", **_kept_fields())
542
590
 
543
591
 
544
592
  def install_signal_handlers() -> None:
@@ -895,10 +943,13 @@ class _OutputLock:
895
943
  the hour ran from the lock's creation, so a legitimate run longer than an hour lost its lock
896
944
  to a second writer. A takeover is recorded (`diagnostics.fallbacks`, stderr, and the audit
897
945
  log); so is running without a lock where none can be created (`diagnostics.warnings`)."""
898
- def __init__(self, output: str) -> None:
946
+ def __init__(self, output: str, on_locked: "Optional[Callable[[str, str, Dict[str, Any]], None]]" = None) -> None:
947
+ # on_locked(message, hint, details): a caller that must not exit (batch.py's item worker)
948
+ # raises its own error for a held lock instead of the refusal below
899
949
  self.path: Optional[str] = None
900
950
  self.fd: Optional[int] = None
901
951
  self.output = output
952
+ self.on_locked = on_locked
902
953
  if output == "-" or output.startswith("pipe:") or output.startswith("-"):
903
954
  return
904
955
  d, base = os.path.split(os.path.abspath(output))
@@ -931,12 +982,14 @@ class _OutputLock:
931
982
  taken = holder
932
983
  continue
933
984
  name = os.path.basename(self.path)[1:-len(".ffskill-lock")]
934
- refuse("input", "output_locked",
935
- f"another run is writing {name!r} right now (lock {self.path}"
936
- + (f", pid {holder['pid']}" if holder.get("pid") else "") + "); wait for it or choose a different --output/-o path",
937
- hint=("wait for that run, or choose another -o path; if pid "
938
- f"{holder.get('pid')} is not an ffmpeg-skill run, delete {self.path}"),
939
- lock=self.path, output=os.path.abspath(self.output), **{k: v for k, v in holder.items() if k != "stale"})
985
+ msg = (f"another run is writing {name!r} right now (lock {self.path}"
986
+ + (f", pid {holder['pid']}" if holder.get("pid") else "") + "); wait for it or choose a different --output/-o path")
987
+ hint = ("wait for that run, or choose another -o path; if pid "
988
+ f"{holder.get('pid')} is not an ffmpeg-skill run, delete {self.path}")
989
+ details = dict(lock=self.path, output=os.path.abspath(self.output), **{k: v for k, v in holder.items() if k != "stale"})
990
+ if self.on_locked:
991
+ self.on_locked(msg, hint, details)
992
+ refuse("input", "output_locked", msg, hint=hint, **details)
940
993
  except OSError as exc:
941
994
  # an unlockable location (a read-only directory surfaces elsewhere): the run goes
942
995
  # on without a lock, and says so -- two runs could now both write this output
@@ -1110,11 +1163,28 @@ def _remember_output(cmd: Sequence[str]) -> None:
1110
1163
  pass
1111
1164
 
1112
1165
 
1166
+ def remember_written(*paths: Optional[str]) -> None:
1167
+ """Count files a child tool of this run wrote (render.py's stages, batch.py's steps) as this
1168
+ run's own, as run() counts its ffmpeg outputs: a later stage that replaces one is not refused
1169
+ as replacing someone else's file (#415)."""
1170
+ for path in paths:
1171
+ if path and os.path.isfile(path):
1172
+ try:
1173
+ STATE.written.add(os.path.realpath(path))
1174
+ except OSError:
1175
+ pass
1176
+
1177
+
1113
1178
  def _timed_out(cmd: Sequence[str], seconds: float) -> "None":
1114
1179
  _cleanup_partial_output(cmd)
1115
- die(f"{os.path.basename(cmd[0])} exceeded the {seconds:.0f} s time limit and was killed; nothing was written. "
1116
- f"Raise --timeout (or FFMPEG_SKILL_TIMEOUT) if the job is genuinely that long, or check the input for a stall",
1117
- code=124, kind="timeout")
1180
+ if _is_ffmpeg(cmd) or seconds < PROBE_TIMEOUT:
1181
+ advice = "Raise --timeout (or FFMPEG_SKILL_TIMEOUT) if the job is genuinely that long, or check the input for a stall"
1182
+ else:
1183
+ # a probe's own ceiling, which a higher --timeout does not raise: a read this long is a stall (#415)
1184
+ advice = (f"A probe is limited to {PROBE_TIMEOUT:g} s whatever --timeout says, so this is a stall, not a long "
1185
+ "job: check that the input is readable (a network share, a file still being written)")
1186
+ die(f"{os.path.basename(cmd[0])} exceeded the {seconds:g} s time limit and was killed; {kept_clause()}. {advice}",
1187
+ code=124, kind="timeout", **_kept_fields())
1118
1188
 
1119
1189
 
1120
1190
  def _stage_existing_output(cmd: Sequence[str]) -> Tuple[List[str], Optional[str], Optional[str]]:
@@ -1446,7 +1516,7 @@ def run_tool(argv: Sequence[str], *, per_call: Optional[float] = None) -> subpro
1446
1516
  child.communicate()
1447
1517
  _unwatch(child)
1448
1518
  name = os.path.basename(str(argv[0]))
1449
- msg = f"{name} did not finish within {limit:.0f} s (4x the per-ffmpeg --timeout plus 60 s) and was killed"
1519
+ msg = f"{name} did not finish within {limit:g} s (4x the per-ffmpeg --timeout plus 60 s) and was killed"
1450
1520
  doc = {"status": "failed", "exit_code": 124,
1451
1521
  "error": {"kind": "timeout", "message": msg, "code": ERROR_CODE["timeout"], "retryable": ERROR_RETRYABLE},
1452
1522
  "commands": []}
@@ -1479,13 +1549,22 @@ def _has_non_av_streams(cmd: Sequence[str]) -> bool:
1479
1549
  try:
1480
1550
  out = subprocess.run([found_exe("ffprobe"), "-v", "error", "-show_entries", "stream=codec_type", "-of", "csv=p=0", src],
1481
1551
  stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True, encoding="utf-8", errors="replace",
1482
- timeout=PROBE_TIMEOUT).stdout
1552
+ timeout=_probe_limit()).stdout
1483
1553
  except (OSError, subprocess.TimeoutExpired):
1484
1554
  return True
1485
1555
  kinds = {ln.strip() for ln in out.splitlines() if ln.strip()}
1486
1556
  return not kinds or bool(kinds & {"subtitle", "data", "attachment"})
1487
1557
 
1488
1558
 
1559
+ def fail_if_classified(proc: subprocess.CompletedProcess) -> None:
1560
+ """Stop with a failed attempt's own error when ffmpeg named a cause a second encode would hit
1561
+ the same way (a full disk, a missing encoder or filter, an unreadable input): the rule
1562
+ run_keeping_subtitles() follows since #389, for every copy-then-re-encode or keep-then-drop
1563
+ fallback (#415). Returns when the failure is unclassified, so the caller's retry runs."""
1564
+ if proc.returncode != 0 and classify_ffmpeg_failure(proc.stderr or ""):
1565
+ _fail(proc.args, proc.returncode, proc.stderr or "")
1566
+
1567
+
1489
1568
  # eia_608 (a QuickTime c608 caption track) decodes to text, so it converts like one (ffmpeg 6.1)
1490
1569
  TEXT_SUBTITLES = frozenset({"mov_text", "subrip", "srt", "ass", "ssa", "webvtt", "text", "eia_608"})
1491
1570
  MP4_FAMILY = frozenset({"mp4", "m4v", "mov"})
@@ -1550,10 +1629,11 @@ def run_keeping_subtitles(cmd: List[str], output: str) -> bool:
1550
1629
 
1551
1630
 
1552
1631
  def _limit_for(cmd: Sequence[str]) -> Optional[float]:
1553
- """The wall-clock ceiling for this command: ffprobe (and other read-only probes) get a fixed
1554
- short one, ffmpeg the configured one; None means unlimited."""
1632
+ """The wall-clock ceiling for this command: ffprobe (and other read-only probes) get a short
1633
+ one, PROBE_TIMEOUT or --timeout when that is lower (as _read_ff_list() does: a caller's
1634
+ --timeout 2 bounds every child, #415), ffmpeg the configured one; None means unlimited."""
1555
1635
  if not _is_ffmpeg(cmd):
1556
- return PROBE_TIMEOUT if STATE.timeout else None
1636
+ return min(PROBE_TIMEOUT, STATE.timeout) if STATE.timeout else None
1557
1637
  return STATE.timeout or None
1558
1638
 
1559
1639
 
@@ -1927,7 +2007,7 @@ def _read_ff_list(flag: str, pattern: str) -> set:
1927
2007
  tool = "ffprobe" if STATE.dry_run else "ffmpeg"
1928
2008
  # --timeout bounds this read too: on Apple Silicon every CPU re-encode asks for the list
1929
2009
  # (#314), so a deadlocked ffmpeg must not outlive the limit the caller set before the encode
1930
- limit = min(PROBE_TIMEOUT, STATE.timeout) if STATE.timeout else PROBE_TIMEOUT
2010
+ limit = _probe_limit()
1931
2011
  _LIST_ERRORS.pop(flag, None)
1932
2012
  names: set = set()
1933
2013
  why = None
@@ -219,7 +219,7 @@ TOOL_META: Dict[str, Dict[str, Any]] = {
219
219
  "render": dict(role="execution", inputs=["project.json (clips, transitions, captions, overlays, audio, loudness, export, check)", "media asset plus --template NAME (a shipped delivery template)"], outputs=["final video artifact", "one artifact per destination plus a pack table (--template all)", "work directory of stage outputs (--keep / --work)", "editor timeline (.fcpxml / .edl / .otio) instead of a render (--export-timeline)"],
220
220
  required=FF, optional=[{"capability": "delegated", "when": "each stage (and each destination of --template) runs cut / join / fit / caption / graphics / overlay / audio / loudness / export / check with their capabilities"}],
221
221
  video_required=False, audio_only=True, visual=True, verify=["probe", "check", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
222
- "batch": dict(role="execution", inputs=["folder of media", "batch.json recipe (steps or a render project)"], outputs=["one artifact per input file in the recipe's output_dir", "content-hash cache"],
222
+ "batch": dict(role="execution", inputs=["folder of media", "batch.json recipe (steps or a render project)"], outputs=["one artifact per input file in the recipe's output_dir (none for a steps recipe, not a render project, in which no step names {out}: its rows carry written: false)", "content-hash cache"],
223
223
  required=FF, optional=[{"capability": "delegated", "when": "each recipe step runs the named script with its capabilities"}],
224
224
  video_required=False, audio_only=True, visual=False, verify=["probe"], produces_artifact=True, idempotency="cached", deterministic=True),
225
225
  "verify": dict(role="verification", inputs=["media files and/or folders"], outputs=["PASS/FAIL JSON per step", "Markdown report (--report)", "step outputs (--out / --keep)"],
package/scripts/batch.py CHANGED
@@ -33,7 +33,7 @@ from pathlib import Path
33
33
  from typing import Any, Dict, List, Optional
34
34
 
35
35
  from _common import STATE, add_common, apply_common, child_args, die, emit, info, run_tool, MEDIA_EXT as _MEDIA_EXT
36
- from _common.runner import HW_FORCED_ENV, add_hw_orchestrator_args, env_hw, stop_children
36
+ from _common.runner import HW_FORCED_ENV, _OutputLock, add_hw_orchestrator_args, env_hw, stop_children
37
37
  from _common.emit import absorb_stage_hw, audit, note, shown, steps_encoder_report, warn
38
38
 
39
39
  HERE = Path(__file__).resolve().parent
@@ -59,6 +59,30 @@ def file_key(path: Path) -> str:
59
59
  return h.hexdigest()
60
60
 
61
61
 
62
+ def side_inputs_key(recipe: Dict[str, Any]) -> str:
63
+ """The files a recipe's steps name besides {in}/{out} (check.py --compare REF, a LUT, a logo),
64
+ keyed by their own content: editing or removing one is a change the cache must see, or a QC
65
+ pass is served its old verdict against a reference that changed or is gone (#428 review).
66
+ Read on every pass, so a --watch loop sees the edit too."""
67
+ h = hashlib.sha1(usedforsecurity=False)
68
+ for step in recipe.get("steps") or []:
69
+ for arg in step[1:] if isinstance(step, list) else []:
70
+ if not isinstance(arg, str) or "{" in arg:
71
+ continue
72
+ value = arg.split("=", 1)[1] if arg.startswith("-") and "=" in arg else arg
73
+ if value.startswith("-") or not value:
74
+ continue
75
+ path = Path(value)
76
+ try:
77
+ if path.is_file():
78
+ h.update(f"{value}\0{file_key(path)}\0".encode())
79
+ continue
80
+ except OSError:
81
+ pass
82
+ h.update(f"{value}\0-\0".encode()) # absent now: different from when it existed
83
+ return h.hexdigest()[:8]
84
+
85
+
62
86
  def recipe_key(recipe: Dict[str, Any]) -> str:
63
87
  # A "project" recipe is just {"project": "<path>", "clip_key": N} -- the actual settings
64
88
  # (export preset, captions, everything) live in the file at that path, not in this dict.
@@ -182,9 +206,80 @@ def final_path(src: Path, recipe: Dict[str, Any], outdir: Path) -> Path:
182
206
  # detection of its own; see one_pass()'s pre-flight check, which uses this same computation
183
207
  # to catch that before any file is actually processed (and the earlier one silently clobbered).
184
208
  final_ext = recipe.get("ext") or src.suffix.lstrip(".") or "mp4"
209
+ steps = recipe.get("steps") or []
210
+ if not recipe.get("ext") and not recipe.get("project") and steps and not step_writes(steps[-1]) and not writes_nothing(recipe):
211
+ # the last step writes nothing, so what is delivered is an intermediate: its name says its
212
+ # container (a .webm source's fit.py step writes MP4, which must not be named .webm)
213
+ final_ext = intermediate_ext(src)
185
214
  return outdir / f"{src.stem}{suffix}.{final_ext}"
186
215
 
187
216
 
217
+ def intermediate_ext(src: Path) -> str:
218
+ """The extension of an item's intermediate step files: the source's own for an audio file,
219
+ else mp4 (the tools' encoders write x264/AAC, which .webm and friends do not take)."""
220
+ return src.suffix.lstrip(".") if src.suffix.lower() in (".wav", ".mp3", ".m4a", ".flac") else "mp4"
221
+
222
+
223
+ def step_writes(step: Any) -> bool:
224
+ """Whether a recipe step writes a file: it names {out} (check.py and probe.py do not)."""
225
+ return isinstance(step, list) and any("{out}" in str(a) for a in step[1:])
226
+
227
+
228
+ def writes_nothing(recipe: Dict[str, Any]) -> bool:
229
+ """A steps recipe in which no step names {out}: it measures each file and writes none."""
230
+ return not recipe.get("project") and not any(step_writes(st) for st in recipe.get("steps") or [])
231
+
232
+
233
+ def row_output(src: Path, recipe: Dict[str, Any], outdir: Path) -> Dict[str, Any]:
234
+ """An item's output fields for its row: `output` is always the final path, a string as it has
235
+ been since 1.x (a --json key never changes type within a major), and a recipe that writes
236
+ nothing adds `written: false` (#415, #428 review)."""
237
+ fields: Dict[str, Any] = {"output": str(final_path(src, recipe, outdir))}
238
+ if writes_nothing(recipe):
239
+ fields["written"] = False
240
+ return fields
241
+
242
+
243
+ def deliver(src: str, final: Path) -> None:
244
+ """Move an item's last written intermediate to its final path, under the output guard every
245
+ tool applies: an existing file is replaced only with --overwrite. A dry run writes nothing."""
246
+ if STATE.dry_run:
247
+ # the refusal the real run would meet, as a step writing {out} plans it (#428 review)
248
+ if final.exists() and not STATE.overwrite:
249
+ raise ItemError({"step": None, "tool": None, "kind": "input", "reason": "output_exists",
250
+ "message": f"refusing to overwrite existing output {str(final)!r}: pass --overwrite to replace it"})
251
+ log(f" [dry-run] would deliver the last written file to {final}")
252
+ return
253
+
254
+ def locked(msg: str, hint: str, details: Dict[str, Any]) -> None:
255
+ raise ItemError({"step": None, "tool": None, "kind": "input", "reason": "output_locked",
256
+ "message": msg, "hint": hint, "details": details})
257
+ # under the output lock every tool's write holds, so the existence check and the move are one
258
+ # step for another run delivering or writing the same path
259
+ with _OutputLock(str(final), on_locked=locked):
260
+ if final.exists() and not STATE.overwrite:
261
+ raise ItemError({"step": None, "tool": None, "kind": "input", "reason": "output_exists",
262
+ "message": f"refusing to overwrite existing output {str(final)!r}: pass --overwrite to replace it"})
263
+ try:
264
+ os.replace(src, final)
265
+ except OSError:
266
+ # another filesystem: copy beside the final path and rename, so a copy that fails part
267
+ # way never leaves a truncated file where the caller's was
268
+ import shutil
269
+ tmp = final.with_name(f".{final.stem}.ffskill-{os.getpid()}-{threading.get_ident()}{final.suffix}")
270
+ try:
271
+ shutil.copyfile(src, tmp)
272
+ os.replace(tmp, final)
273
+ except OSError as exc:
274
+ try:
275
+ os.remove(tmp)
276
+ except OSError:
277
+ pass
278
+ raise ItemError({"step": None, "tool": None, "kind": "output",
279
+ "message": f"could not deliver {src} to {final}: {exc}"})
280
+ log(f" delivered the last written file to {final}")
281
+
282
+
188
283
  def process(src: Path, recipe: Dict[str, Any], outdir: Path, work: Path,
189
284
  deadline: "Optional[float]" = None) -> Dict[str, Any]:
190
285
  final = final_path(src, recipe, outdir)
@@ -211,7 +306,7 @@ def process(src: Path, recipe: Dict[str, Any], outdir: Path, work: Path,
211
306
  proj["clips"].append({})
212
307
  proj["clips"][idx]["src"] = str(src.resolve())
213
308
  proj["output"] = str(final.resolve())
214
- pj = work / f"{src.stem}_project.json"
309
+ pj = work / "project.json"
215
310
  pj.write_text(json.dumps(proj, indent=2), encoding="utf-8")
216
311
  ok, _doc = run_step(["render.py", str(pj)], budget())
217
312
  step_docs = [("render", _doc)]
@@ -231,7 +326,11 @@ def process(src: Path, recipe: Dict[str, Any], outdir: Path, work: Path,
231
326
  cut_reasons = []
232
327
  for i, step in enumerate(steps):
233
328
  last = i == len(steps) - 1
234
- out = str(final) if last else str(work / f"{src.stem}_step{i}.{'mp4' if src.suffix.lower() not in ('.wav', '.mp3', '.m4a', '.flac') else src.suffix.lstrip('.')}")
329
+ # the item's directory is its own, so the name needs no stem (a long one overflowed it)
330
+ out = str(final) if last else str(work / f"step{i}.{intermediate_ext(src)}")
331
+ # a step that names no {out} (check.py, probe.py) writes nothing: the next step reads
332
+ # what this one read, and when it is the last, the last file written is delivered (#415)
333
+ writes = step_writes(step)
235
334
  argv = [str(a).replace("{in}", cur).replace("{out}", out) for a in step]
236
335
  step_ok, doc = run_step(argv, budget())
237
336
  step_docs.append((Path(str(argv[0])).stem if argv else "step", doc))
@@ -246,8 +345,16 @@ def process(src: Path, recipe: Dict[str, Any], outdir: Path, work: Path,
246
345
  ok = False
247
346
  failure = step_error(i, str(argv[0]) if argv else None, doc)
248
347
  break
249
- cur = out
250
- result: Dict[str, Any] = {"file": str(src), "output": str(final), "ok": ok, "seconds": round(time.time() - t0, 1)}
348
+ if writes:
349
+ cur = out
350
+ if ok and cur != str(final) and not writes_nothing(recipe):
351
+ # the last step wrote nothing: the item's deliverable is the last file a step wrote,
352
+ # which is in this item's work directory and would be removed with it (#415)
353
+ try:
354
+ deliver(cur, final)
355
+ except ItemError as exc:
356
+ ok, failure = False, exc.error
357
+ result: Dict[str, Any] = {"file": str(src), **row_output(src, recipe, outdir), "ok": ok, "seconds": round(time.time() - t0, 1)}
251
358
  if failure:
252
359
  result["error"] = failure
253
360
  # `ok` is "every step completed" (design-decisions: a spec miss a tool cannot fix is completed +
@@ -291,6 +398,8 @@ def cached_output_changed(hit: Dict[str, Any]) -> "Optional[Dict[str, Any]]":
291
398
  output_missing, output_changed (size or mtime differ: something replaced or edited it) or
292
399
  entry_unverifiable (an entry written by 2.10.11 or earlier recorded no identity, so nothing can say
293
400
  the file is the one it describes)."""
401
+ if hit.get("written") is False or ("output" in hit and hit["output"] is None):
402
+ return None # a recipe that writes nothing (check.py over a folder): no file to have changed
294
403
  out = str(hit.get("output") or "")
295
404
  now = output_identity(out) if out else None
296
405
  if now is None:
@@ -344,9 +453,22 @@ def main() -> int:
344
453
  if not isinstance(name, str) or name not in ALLOWED_STEP_SCRIPTS:
345
454
  die(f"recipe step names a script that isn't one of this skill's own tools: {name!r} "
346
455
  f"(must be a bare filename like 'silence.py', found in scripts/)")
456
+ if writes_nothing(recipe):
457
+ # a QC batch (check.py / probe.py on every file): its rows carry written: false, and its
458
+ # cache entries claim no file, so they are served rather than missed as output_missing (#415)
459
+ note("no recipe step names {out}: the steps only read each file, so no output is written "
460
+ "(each row says written: false; its output path is not created)")
347
461
  outdir = Path(recipe.get("output_dir") or (folder / "out"))
348
462
  if not outdir.is_absolute():
349
463
  outdir = folder / outdir
464
+ # resolved, so an alias of the input folder ("sub/..", a symlink) is recognised as one: the
465
+ # input scan below excludes outdir's files by parentage, and the input check by resolved path
466
+ outdir = outdir.resolve()
467
+ if outdir == folder:
468
+ # the scan skips files under outdir, so it found nothing here, and with suffix "" a
469
+ # final path was the source itself, which --overwrite replaced (#428 review)
470
+ die(f"output_dir {recipe.get('output_dir')!r} is the input folder {folder}: its outputs would mix with "
471
+ "or replace the batch inputs -- set an \"output_dir\" outside it (the default is out/)", kind="input")
350
472
  work = Path(args.work) if args.work else outdir / ".work"
351
473
  # the children refuse an output whose directory does not exist, under --dry-run too, so the
352
474
  # directories are created for the plan as well -- and removed again afterwards when a dry
@@ -435,23 +557,47 @@ def main() -> int:
435
557
  # collapsing e.g. clip.mp4 and clip.mov to the same clip_out.mp4) -- catch that before
436
558
  # processing anything, rather than letting the later one silently overwrite the earlier
437
559
  # one's finished output with the cache still recording both as "ok".
438
- by_final: Dict[Path, List[Path]] = {}
560
+ # Compared case-folded: Clip_out.mp4 and clip_out.mp4 are one file on macOS's and
561
+ # Windows' default filesystems, so a batch that only works on Linux is refused everywhere (#415).
562
+ by_final: Dict[str, List[Path]] = {}
439
563
  for src in files:
440
- by_final.setdefault(final_path(src, recipe, outdir), []).append(src)
441
- collisions = {dst: srcs for dst, srcs in by_final.items() if len(srcs) > 1}
564
+ by_final.setdefault(str(final_path(src, recipe, outdir)).casefold(), []).append(src)
565
+ collisions = {key: srcs for key, srcs in by_final.items() if len(srcs) > 1 and not writes_nothing(recipe)}
442
566
  if collisions:
443
- detail = "; ".join(f"{dst.name} <- {', '.join(s.name for s in srcs)}" for dst, srcs in collisions.items())
444
- die(f"{len(collisions)} output filename collision(s) in this batch -- rename the sources, "
567
+ detail = "; ".join(" / ".join(sorted({final_path(s, recipe, outdir).name for s in srcs}))
568
+ + " <- " + ", ".join(s.name for s in srcs) for srcs in collisions.values())
569
+ die(f"{len(collisions)} output filename collision(s) in this batch (names that differ only in case are one "
570
+ f"file on macOS and Windows) -- rename the sources, "
445
571
  f"or add a distinguishing \"suffix\"/\"ext\" per run, or split into separate globs: {detail}")
572
+ # A final path that is one of the inputs would replace a source -- an output_dir that is
573
+ # the input folder under another name ("sub/..") with suffix "" -- and --overwrite does
574
+ # not license that (#428 review). A recipe that writes nothing delivers no file.
575
+ if not writes_nothing(recipe):
576
+ inputs = {str(p.resolve()).casefold() for p in files}
577
+ hits = [s for s in files if str(final_path(s, recipe, outdir).resolve()).casefold() in inputs]
578
+ if hits:
579
+ die(f"{len(hits)} final output path(s) would replace a batch input ({', '.join(s.name for s in hits)}); "
580
+ "the output_dir is the input folder -- set a different \"output_dir\" or a non-empty \"suffix\"",
581
+ kind="input")
446
582
  def item_work(i: int, src: Path) -> Path:
447
- """Where this item's intermediates go. Parallel items must not share one work dir:
448
- the step file names are stem-derived, so two globs holding the same stem would write
449
- over each other. Serial runs keep the flat layout 1.16 used, byte for byte."""
450
- if jobs == 1:
451
- return work
452
- sub = work / f"{i}-{src.stem}"
453
- sub.mkdir(parents=True, exist_ok=True)
454
- return sub
583
+ """Where this item's intermediates go: a new directory of its own under the work
584
+ directory, removed with everything in it when the item ends (run_item()). The step
585
+ file names are stem-derived, so two items with one stem (clip.mp4, clip.mov) sharing a
586
+ directory refused each other's files, and a later run refused the files an earlier one
587
+ left there as someone else's (#415); parallel items had their own directories since
588
+ 1.17, serial ones now do too."""
589
+ import tempfile
590
+ # no stem in the name: a stem near the 255-byte limit made the directory name too long
591
+ return Path(tempfile.mkdtemp(prefix=f"{i}-", dir=str(work)))
592
+
593
+ def run_item(i: int, src: Path) -> Dict[str, Any]:
594
+ """process() in the item's own work directory, which goes when it ends, however it ends."""
595
+ import shutil
596
+ iw = item_work(i, src)
597
+ try:
598
+ return process(src, recipe, outdir, iw, deadline)
599
+ finally:
600
+ shutil.rmtree(iw, ignore_errors=True)
455
601
 
456
602
  def store(key: str, r: Dict[str, Any]) -> None:
457
603
  if not (r["ok"] and not STATE.dry_run):
@@ -462,10 +608,10 @@ def main() -> int:
462
608
  with cache_lock:
463
609
  # this run just rewrote r["output"]: an entry under another key (another recipe,
464
610
  # the other GPU setting) that names the same file now describes bytes that are gone
465
- for stale in [k for k, v in cache.items() if k != key and isinstance(v, dict) and v.get("output") == r["output"]]:
611
+ for stale in [k for k, v in cache.items() if r["output"] and r.get("written", True) and k != key and isinstance(v, dict) and v.get("output") == r["output"]]:
466
612
  del cache[stale]
467
613
  # the output as this run left it: a later hit is served only while it is unchanged
468
- cache[key] = dict(r, output_identity=output_identity(r["output"]))
614
+ cache[key] = dict(r, output_identity=output_identity(r["output"]) if r["output"] and r.get("written", True) else None)
469
615
  # write_text isn't atomic -- a process killed mid-write (or a --watch loop racing
470
616
  # a concurrent manual run) could leave a truncated file that json.loads() above
471
617
  # then silently treats as "no cache" (a ValueError -> {}), discarding every prior
@@ -483,8 +629,9 @@ def main() -> int:
483
629
  slots: "List[Optional[Dict[str, Any]]]" = [None] * len(files)
484
630
  pending: List[tuple] = []
485
631
  cache_miss: Dict[int, Dict[str, Any]] = {}
632
+ pass_key = rkey + side_inputs_key(recipe)
486
633
  for i, src in enumerate(files):
487
- key = f"{file_key(src)}:{rkey}"
634
+ key = f"{file_key(src)}:{pass_key}"
488
635
  hit = cache.get(key)
489
636
  if isinstance(hit, dict) and not args.force:
490
637
  why = cached_output_changed(hit)
@@ -503,7 +650,7 @@ def main() -> int:
503
650
 
504
651
  def timed_out_row(src: Path) -> "Dict[str, Any]":
505
652
  timed_out["hit"] = True
506
- return {"file": str(src), "output": str(final_path(src, recipe, outdir)),
653
+ return {"file": str(src), **row_output(src, recipe, outdir),
507
654
  "ok": False, "seconds": 0.0, "skipped": "timeout"}
508
655
 
509
656
  def failed_row(src: Path, exc: BaseException) -> "Dict[str, Any]":
@@ -522,13 +669,13 @@ def main() -> int:
522
669
  error = {"step": None, "tool": None, "exception": exc.__class__.__name__,
523
670
  "message": (str(exc) or exc.__class__.__name__)[:300]}
524
671
  log(f" failed: {error['message']}")
525
- return {"file": str(src), "output": str(final_path(src, recipe, outdir)),
672
+ return {"file": str(src), **row_output(src, recipe, outdir),
526
673
  "ok": False, "verified": False, "seconds": 0.0, "error": error}
527
674
 
528
675
  def interrupted_row(src: Path, started: bool, r: "Optional[Dict[str, Any]]" = None) -> "Dict[str, Any]":
529
676
  """An item the interrupt reached: `interrupted: true` when it was running (its steps
530
677
  were stopped), `skipped: "interrupted"` when it had not started."""
531
- row = dict(r) if r else {"file": str(src), "output": str(final_path(src, recipe, outdir)), "seconds": 0.0}
678
+ row = dict(r) if r else {"file": str(src), **row_output(src, recipe, outdir), "seconds": 0.0}
532
679
  row.update(ok=False, verified=False)
533
680
  if started:
534
681
  row["interrupted"] = True
@@ -550,7 +697,7 @@ def main() -> int:
550
697
  continue
551
698
  info(f"=== {src.name}")
552
699
  try:
553
- r = process(src, recipe, outdir, item_work(i, src), deadline)
700
+ r = run_item(i, src)
554
701
  except KeyboardInterrupt:
555
702
  interrupted["hit"] = True
556
703
  # the running item, then every one not started: each keeps its row
@@ -572,7 +719,7 @@ def main() -> int:
572
719
  def work_one(i: int, src: Path, key: str) -> Dict[str, Any]:
573
720
  _LOG.buffer = [f"=== {src.name}"]
574
721
  try:
575
- r = process(src, recipe, outdir, item_work(i, src), deadline)
722
+ r = run_item(i, src)
576
723
  if not interrupted["hit"]: # a step the interrupt stopped is no result to keep
577
724
  store(key, r)
578
725
  return with_miss(i, r)
@@ -670,7 +817,7 @@ def main() -> int:
670
817
  info(f"{done}/{len(results)} processed, {sum(1 for r in results if r.get('cached'))} from cache")
671
818
  if not args.json:
672
819
  for r in results:
673
- print(f"{'OK ' if r['ok'] else 'FAIL'} {r['file']} -> {r['output']}" + (" (cached)" if r.get("cached") else ""))
820
+ print(f"{'OK ' if r['ok'] else 'FAIL'} {r['file']} -> {r['output'] or '(no output: the steps only read it)'}" + (" (cached)" if r.get("cached") else ""))
674
821
  if interrupted["hit"]:
675
822
  # Ctrl-C on a batch that already produced files: the user needs the partial table, not a
676
823
  # traceback and not "nothing was written". Exit 130 (143 for SIGTERM) with every item:
@@ -684,7 +831,7 @@ def main() -> int:
684
831
  timed_out=timed_out["hit"])
685
832
  if timed_out["hit"]:
686
833
  skipped = [r["file"] for r in results if r.get("skipped") == "timeout"]
687
- die(f"the batch's {STATE.timeout:.0f} s budget ran out with {len(skipped)} item(s) not "
834
+ die(f"the batch's {STATE.timeout:g} s budget ran out with {len(skipped)} item(s) not "
688
835
  f"started: {', '.join(os.path.basename(f) for f in skipped[:5])}"
689
836
  + (" ..." if len(skipped) > 5 else "")
690
837
  + ". --timeout is the whole run's limit, not each item's; raise it or split the folder.",
package/scripts/check.py CHANGED
@@ -32,6 +32,7 @@ the row set is unchanged.
32
32
  """
33
33
  import argparse
34
34
  import json
35
+ import os
35
36
  import re
36
37
  import sys
37
38
  from fractions import Fraction
@@ -164,6 +165,12 @@ def main() -> int:
164
165
 
165
166
  # read before anything is measured and again after: a file replaced mid-check is not the one graded
166
167
  identity_before = None if dry_run_input_pending(args.input) else file_identity(args.input)
168
+ if identity_before is None and STATE.dry_run and not os.path.exists(args.input):
169
+ # a render/batch plan's earlier stage would write this file: there is nothing to grade yet,
170
+ # and grading the probe's empty answer divided by a 0 height
171
+ emit(None, platform=args.platform, checks=[], failed=0, warnings=0, ok=False,
172
+ notes=[f"[dry-run] {args.input} does not exist yet, so nothing was checked"])
173
+ return 0
167
174
  meta = probe(args.input)
168
175
  v, a = meta.get("video") or {}, meta.get("audio") or {}
169
176
  rows: List[Dict[str, Any]] = []
package/scripts/color.py CHANGED
@@ -23,6 +23,7 @@ import os
23
23
  import sys
24
24
  from typing import List, Optional
25
25
 
26
+ from _common.runner import fail_if_classified
26
27
  from _common import STATE, add_common, analyze_levels, apply_common, emit, aac_args, cfr_args, default_output, die, escape_filter_path, ffmpeg_base, ffmpeg_bsfs, info, probe, require_tool, run, run_analysis, run_keeping_subtitles, x264_args, X264_PRESETS, fmt_secs
27
28
  from _common.emit import warn
28
29
 
@@ -320,6 +321,7 @@ def retag(args: argparse.Namespace, meta: dict, has_audio: bool) -> int:
320
321
  cmd += ["-movflags", "+faststart"]
321
322
  cmd.append(output)
322
323
  proc = run(cmd, check=False)
324
+ fail_if_classified(proc) # a full disk fails the re-encode too: its error, not two more encodes (#415)
323
325
  notes: List[str] = []
324
326
  reencoded = proc.returncode != 0
325
327
  if not reencoded and not STATE.dry_run:
@@ -345,6 +347,7 @@ def retag(args: argparse.Namespace, meta: dict, has_audio: bool) -> int:
345
347
  cmd = ffmpeg_base() + ["-i", args.input, "-map", "0:v:0", "-map", f"0:a:{args.audio_stream}?",
346
348
  "-map", "0:s?", "-map", "0:d?"] + label + enc + ["-c:s", "copy", "-c:d", "copy", output]
347
349
  proc2 = run(cmd, check=False)
350
+ fail_if_classified(proc2)
348
351
  if proc2.returncode != 0:
349
352
  info("re-encode with subtitles/data streams kept also failed; dropping them")
350
353
  dropped_streams = True
package/scripts/cut.py CHANGED
@@ -58,6 +58,7 @@ import tempfile
58
58
  import zlib
59
59
  from typing import List, Tuple
60
60
 
61
+ from _common.runner import fail_if_classified
61
62
  from _common import (beat_grid, snap_points, decode_pcm_mono, rms_envelope, BEAT_MIN_CONFIDENCE)
62
63
  from _common import require_tool
63
64
  from _common.probe import TS_EXTS, signatures_match, stream_signature
@@ -893,6 +894,7 @@ def cut_one(src: str, start: float, end: float, dst: str, reencode: bool, crf: i
893
894
  proc = run(cmd, check=False)
894
895
  if proc.returncode != 0:
895
896
  if not reencode:
897
+ fail_if_classified(proc) # a full disk fails the re-encode too: its error, not a retry (#415)
896
898
  info("stream copy failed, falling back to re-encode")
897
899
  return cut_one(src, start, end, dst, True, crf, preset, tolerance, meta, reasons + ["copy_failed"], edit_list_ok)
898
900
  die(f"ffmpeg failed:\n{proc.stderr.strip()}", kind="ffmpeg")
@@ -1604,6 +1606,7 @@ def main() -> int:
1604
1606
  cmd += ["-movflags", "+faststart"]
1605
1607
  cmd += [joined]
1606
1608
  proc = run(cmd, check=False)
1609
+ fail_if_classified(proc) # #415: a full disk is not a reason to re-cut every segment
1607
1610
  if proc.returncode != 0:
1608
1611
  info("concat with stream copy failed; re-cutting every segment from the source into one re-encode")
1609
1612
  compatible = False
package/scripts/export.py CHANGED
@@ -38,6 +38,7 @@ import tempfile
38
38
  from pathlib import Path
39
39
  from typing import Dict, List
40
40
 
41
+ from _common.runner import child_exit_code, fail_if_classified, kept_clause, mark_delivered
41
42
  from _common import STATE, run_analysis, add_common, apply_common, bt709_label_filter, bt709_matrix_args, bt709_tag_args, child_args, emit, cfr_args, default_output, die, encoder_args, ffmpeg_base, info, probe, require_tool, run, run_tool, validate_color, pad_filters, add_pad_fill_args, fmt_secs
42
43
  from _common.decision import hw_preset_video, restate_last_swap
43
44
  from _common.emit import note, record_fallback, record_skip, warn
@@ -183,6 +184,9 @@ def _run_keeping_non_av(cmd: List[str], output: str, sub_codecs: List[str], data
183
184
  attempt = run(cmd + extra + [output], check=False)
184
185
  if attempt.returncode == 0:
185
186
  return dropped
187
+ # a full disk or a missing encoder fails every rung the same way: that error, not a
188
+ # fallback that blames the subtitles and encodes again (#415, #389's rule)
189
+ fail_if_classified(attempt)
186
190
  error = " | ".join([ln.strip() for ln in (attempt.stderr or "").splitlines() if ln.strip()][-3:])[-500:]
187
191
  going = rungs[n + 1][1]
188
192
  record_fallback(f"{os.path.basename(output)} could not hold the source's {' and '.join(t for t in ('subtitle', 'data') if t in going)} "
@@ -347,7 +351,9 @@ def main() -> int:
347
351
  notes.append("source is %s (%s). This preset outputs SDR BT.709 tags without converting it; run color.py --to-sdr first for correct colours." % (kind, meta["video"].get("hdr_format")))
348
352
  warn(notes[-1], "colour_not_converted", preset=args.preset, hdr_format=meta["video"].get("hdr_format"))
349
353
  has_audio = bool(meta.get("audio"))
350
- output = args.output or default_output(args.input, args.preset, p["ext"])
354
+ # copy keeps the source's container, so its default name keeps the source's extension (as
355
+ # metadata.py's does): default_output()'s .webm -> .mp4 is for the re-encoding presets
356
+ output = args.output or default_output(args.input, args.preset, p["ext"] or Path(args.input).suffix.lstrip(".") or None)
351
357
  out_ext = Path(output).suffix.lstrip(".").lower()
352
358
 
353
359
  vf: List[str] = []
@@ -466,6 +472,10 @@ def main() -> int:
466
472
  dropped = _run_keeping_non_av(cmd, output, sub_codecs, len(data)) if (subs or data) else []
467
473
  if not (subs or data):
468
474
  run(cmd + [output])
475
+ if not STATE.dry_run:
476
+ # the export is on disk: a stop in the reads and the --normalize pass that follow keeps it,
477
+ # and says so (#415)
478
+ mark_delivered(output)
469
479
  result = probe(output, role="output")
470
480
  v = result["video"]
471
481
  info(f"wrote {output} ({fmt_secs(result['duration'])}, {v['width']}x{v['height']}, {v['codec']})")
@@ -508,7 +518,12 @@ def main() -> int:
508
518
  err = child.get("error") or {}
509
519
  if os.path.exists(tmp):
510
520
  os.remove(tmp)
511
- die(f"--normalize: loudness.py failed: {err.get('message') or proc.stderr.strip()[-300:]}",
521
+ # the child's "nothing was written" is about its own temp file; this run's export is
522
+ # on disk, and the child's exit code (124 timeout, 130/143 interrupted) is the
523
+ # run's, as render.py and waveform.py pass a stage's on (#415)
524
+ why = (err.get("message") or proc.stderr.strip()[-300:]).replace("; nothing was written", "")
525
+ die(f"--normalize: loudness.py failed: {why} ({kept_clause()})",
526
+ code=child_exit_code(child, proc.returncode) if proc.returncode else 1,
512
527
  kind=err.get("kind") or "ffmpeg", output=output, hint=err.get("hint"))
513
528
  os.replace(tmp, output)
514
529
  m = measure_loudness(output)
@@ -221,7 +221,8 @@ def main() -> int:
221
221
  die(f"chapters file not found: {args.chapters}")
222
222
 
223
223
  meta = probe(args.input)
224
- output = args.output or default_output(args.input, "meta")
224
+ # streams are copied, so the source's own container always holds them (a .webm stays .webm)
225
+ output = args.output or default_output(args.input, "meta", Path(args.input).suffix.lstrip(".") or "mp4")
225
226
  if os.path.abspath(output) == os.path.abspath(args.input):
226
227
  die("output must differ from the input (metadata.py never rewrites a file in place)")
227
228
  out_ext = Path(output).suffix.lower()
@@ -249,7 +250,13 @@ def main() -> int:
249
250
  cmd += ["-map", "0", "-map_metadata", "0", "-map_chapters", "0"]
250
251
  for key, value in tags.items():
251
252
  cmd += ["-metadata", f"{key}={value}"]
252
- cmd += ["-c", "copy", output]
253
+ cmd += ["-c", "copy"]
254
+ if out_ext in (".mp4", ".mov", ".m4v", ".m4a"):
255
+ # A stream copy into the MPEG-4 family writes the index (moov) after the media unless told
256
+ # otherwise, so without +faststart a faststart input -- render's export -- came out with
257
+ # its index at the end (#415; every other MP4 write here is faststart since #279)
258
+ cmd += ["-movflags", "+faststart"]
259
+ cmd.append(output)
253
260
  run(cmd)
254
261
  if tmpdir:
255
262
  tmpdir.cleanup()
package/scripts/render.py CHANGED
@@ -75,7 +75,7 @@ from typing import Any, Dict, List, Optional, Sequence
75
75
  from export import PRESETS, PLATFORM_OF
76
76
  from _platforms import PLATFORMS, caption_defaults, resolve as resolve_platform
77
77
  from _common import guard_sidecar, STATE, add_common, aspect_ratio, brand_caption_style, load_brand, apply_common, child_args, die, emit, info, probe, require_tool, run, run_tool, place_output, refuse_output_is_input, _check_existing_output, _check_output_path, fingerprint, PLAN_VERSION, ffmpeg_version, X264_PRESETS
78
- from _common.runner import HW_ENV, HW_FORCED_ENV, add_hw_orchestrator_args, found_exe
78
+ from _common.runner import HW_ENV, HW_FORCED_ENV, add_hw_orchestrator_args, child_exit_code, found_exe, kept_clause, kept_deliverables, mark_delivered, remember_written
79
79
  from _common.emit import absorb_stage, audit, record_fallback, record_skip, refuse, warn
80
80
  import subprocess
81
81
  from _contract import CONTRACT_VERSION
@@ -351,6 +351,7 @@ def render_pack(names: List[str], args) -> int:
351
351
  rows: List[Dict[str, Any]] = []
352
352
  outputs: List[str] = []
353
353
  failed: List[str] = []
354
+ stopped: List[Dict[str, Any]] = [] # destinations a timeout stopped: {kind, exit_code}
354
355
  for name in names:
355
356
  # the destination decides the container: an audio-only destination in a pack must not be
356
357
  # handed a .mp4 name the single-template form would never have written (review 12)
@@ -394,6 +395,9 @@ def render_pack(names: List[str], args) -> int:
394
395
  outputs.append(out)
395
396
  if not ok:
396
397
  failed.append(name)
398
+ err = doc.get("error") or {}
399
+ if err.get("kind") in ("timeout", "interrupted"):
400
+ stopped.append({"kind": err["kind"], "exit_code": child_exit_code(doc, proc.returncode)})
397
401
  pack = str(outdir / f"{stem}_pack.md")
398
402
  lines = [f"# Social pack — {stem}", "",
399
403
  "| platform | file | size | duration | check |", "|---|---|---|---|---|"]
@@ -410,8 +414,11 @@ def render_pack(names: List[str], args) -> int:
410
414
  Path(pack).write_text("\n".join(lines) + "\n", encoding="utf-8")
411
415
  info(("[dry-run] would write " if STATE.dry_run else "wrote ") + pack)
412
416
  if failed:
417
+ # a destination that was stopped makes the pack a stopped run: its kind and exit code (124,
418
+ # 130/143), as a stage's are render.py's (docs/contract.md); otherwise a verification failure
413
419
  die(f"pack: {len(failed)} of {len(rows)} destinations failed: {', '.join(failed)}",
414
- kind="verification", output=pack, dry_run=STATE.dry_run, pack=rows, outputs=outputs)
420
+ code=stopped[0]["exit_code"] if stopped else 1, kind=stopped[0]["kind"] if stopped else "verification",
421
+ output=pack, dry_run=STATE.dry_run, pack=rows, outputs=outputs)
415
422
  emit(pack, pack=rows, outputs=outputs, stages=["pack"],
416
423
  verification=[{"step": "check", "ok": r["ok"], "platform": r["platform"]} for r in rows])
417
424
  return 0
@@ -577,6 +584,8 @@ def sh(script: str, *argv: Any, extra: List[str] = None, stage: str = None) -> s
577
584
  # child's document standing, which a caller reading _LAST_DOC would misattribute.
578
585
  _LAST_DOC.clear()
579
586
  _LAST_DOC["cached"] = True
587
+ if not STATE.dry_run:
588
+ remember_written(dest) # placed by this run, as a stage's own write would be (#415)
580
589
  return dest
581
590
  CACHE["misses"].append(stage)
582
591
  _refuse_uncached_earlier_stage(stage)
@@ -603,12 +612,27 @@ def sh(script: str, *argv: Any, extra: List[str] = None, stage: str = None) -> s
603
612
  extra_fields = {"hint": err["hint"]} if err.get("hint") else {}
604
613
  error_extra = absorb_stage(Path(script).stem, doc) # the stage's reason/details and records ride along
605
614
  extra_fields.update(_stage_failure_hw(doc))
606
- die(f"{script} failed: {err.get('message') or (proc.stderr.strip().splitlines() or ['?'])[-1][:300]}",
615
+ why = err.get('message') or (proc.stderr.strip().splitlines() or ['?'])[-1][:300]
616
+ if kept_deliverables() and err.get("kind") in ("timeout", "interrupted"):
617
+ # a stage after the delivery (chapters, check): the stage's own "nothing was written"
618
+ # is about its own file, while the delivery is on disk (#415)
619
+ why = why.replace("; nothing was written", "") + f" ({kept_clause()})"
620
+ extra_fields.setdefault("output", kept_deliverables()[-1])
621
+ elif err.get("kind") in ("timeout", "interrupted") and doc.get("output") and os.path.isfile(str(doc["output"])):
622
+ # the stage was stopped after writing its own deliverable and kept it (export.py's
623
+ # --normalize pass): its message says so, and the file is this run's output too
624
+ extra_fields.setdefault("output", str(doc["output"]))
625
+ die(f"{script} failed: {why}",
607
626
  code=int(doc.get("exit_code") or 1), kind=err.get("kind") or "input", stage=script, error_extra=error_extra, **extra_fields)
608
627
  _LAST_DOC.clear()
609
628
  _LAST_DOC.update(doc if isinstance(doc, dict) else {})
610
629
  absorb_stage(Path(script).stem, doc) # a stage's GPU facts (ran on VideoToolbox, fell back and why) and its diagnostics reach this result
611
630
  out_path = str(doc.get("output") or "")
631
+ if not STATE.dry_run:
632
+ # A file a stage of this run wrote is this run's, not someone else's: the chapters stage
633
+ # places its tagged copy over the export's delivery, which the output guard would
634
+ # otherwise refuse as an existing file without --overwrite (#415).
635
+ remember_written(out_path, dest)
612
636
  if key:
613
637
  cache_store(stage, key, out_path or (dest or ""), time.time() - started, doc)
614
638
  return out_path
@@ -951,8 +975,10 @@ def execute_plan(plan: Dict[str, Any], path: str) -> int:
951
975
  if proc.returncode != 0 or doc.get("status") != "completed":
952
976
  err = doc.get("error") or {}
953
977
  error_extra = absorb_stage(tool, doc) # the tool's reason/details and records ride along
978
+ # the tool's exit code is the plan's (124 timeout, 130/143 interrupted), as sh() passes a stage's on
954
979
  die(f"{tool} failed while executing the plan: {err.get('message') or proc.stderr.strip()[-300:]}",
955
- kind=err.get("kind", "ffmpeg"), plan=path, tool=tool, error_extra=error_extra, **_stage_failure_hw(doc))
980
+ code=child_exit_code(doc, proc.returncode), kind=err.get("kind", "ffmpeg"), plan=path, tool=tool,
981
+ error_extra=error_extra, **_stage_failure_hw(doc))
956
982
  absorb_stage(tool, doc) # the tool's GPU facts (fell back and why, ran on VideoToolbox) and diagnostics reach this result
957
983
  output = doc.get("output") or plan.get("output")
958
984
  check_result = None
@@ -1611,6 +1637,8 @@ def main() -> int:
1611
1637
  place_output(current, output)
1612
1638
  info(("[dry-run] would copy" if STATE.dry_run else "copied") + f" final stage to {output}")
1613
1639
  current = output
1640
+ if not STATE.dry_run:
1641
+ mark_delivered(output) # the chapters and check stages run on it: a stop from here keeps it, and says so (#415)
1614
1642
 
1615
1643
  # ---- chapters (metadata.py on the delivered file: streams copied, markers written)
1616
1644
  ch = proj.get("chapters")
@@ -1648,6 +1676,13 @@ def main() -> int:
1648
1676
  exit_code = 1
1649
1677
  elif check_result.get("error") or check_result.get("status") == "failed":
1650
1678
  err = check_result.get("error")
1679
+ if isinstance(err, dict) and err.get("kind") in ("timeout", "interrupted"):
1680
+ # stopped, not failed: the run exits with check.py's own code (124, 130/143), as a
1681
+ # stage that stopped does, and says the delivery it had written is kept (#415)
1682
+ why = str(err.get("message") or "").replace("; nothing was written", "")
1683
+ die(f"check.py failed: {why} ({kept_clause()})",
1684
+ code=int(check_result.get("exit_code") or err.get("exit_code") or 1), kind=err["kind"],
1685
+ stage="check.py", output=output, stages=stages_done)
1651
1686
  info(f"check: could not run check.py — {err.get('message') if isinstance(err, dict) else err}")
1652
1687
  exit_code = 1
1653
1688
  else:
package/scripts/sync.py CHANGED
@@ -32,6 +32,7 @@ import os
32
32
  import sys
33
33
  from typing import Dict, List
34
34
 
35
+ from _common.runner import fail_if_classified
35
36
  from _common import video_args, add_common, apply_common, emit, aac_args, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run, run_analysis, x264_args, decode_pcm_mono, rms_envelope
36
37
  from _common.emit import record_skip, warn
37
38
  from _common.decision import stream_starts
@@ -377,6 +378,7 @@ def main() -> int:
377
378
  cmd += ["-af", ",".join(af_parts)]
378
379
  cmd += aac_args() + (["-t", f"{ref_dur:.3f}"] if ref_dur else ["-shortest"]) + [output]
379
380
  proc = run(cmd, check=False)
381
+ fail_if_classified(proc) # a cause the re-encode would hit too is reported, not retried (#415)
380
382
  if proc.returncode != 0:
381
383
  cmd = [c for c in cmd if c != "copy"]
382
384
  idx = cmd.index("-c:v"); del cmd[idx]
@@ -389,6 +391,7 @@ def main() -> int:
389
391
  cmd += ["-movflags", "+faststart"]
390
392
  cmd += ["-avoid_negative_ts", "make_zero", output]
391
393
  proc = run(cmd, check=False)
394
+ fail_if_classified(proc)
392
395
  if proc.returncode != 0:
393
396
  cmd = ffmpeg_base() + ["-ss", f"{head_trim:.4f}", "-i", args.second] + (video_args(probe(args.reference) if args.replace_audio else probe(args.second), args.crf) if has_video else []) + audio_codec_for(output) + [output]
394
397
  run(cmd)
@@ -284,6 +284,14 @@ def main() -> int:
284
284
  notes.append(f"the visualisation band and {args.platform}'s bottom safe zone overlap by "
285
285
  f"{overlap}px: the captions or the app's own UI will sit over the waveform "
286
286
  f"(lower --vis-height to {max(0.05, (args.height - safe_px) / args.height):.2f})")
287
+ if STATE.dry_run and (args.title or args.srt or args.text):
288
+ # graphics.py / caption.py plan on the render this dry run did not write, whose frame they
289
+ # cannot measure: their sizes and offsets come out 0 in the plan, and the real run takes
290
+ # them from the WxH render (#415). Said here, so a plan is not read as the real numbers.
291
+ later = " and ".join(n for n, on in (("title", args.title), ("captions", args.srt or args.text)) if on)
292
+ notes.append(f"the {later} stage was planned on the visualisation render this dry run did not write, so the "
293
+ f"font sizes and offsets in its planned command are placeholders (0); the real run sizes them "
294
+ f"from the {args.width}x{args.height} render")
287
295
  if silent_peak is not None:
288
296
  warn(f"{args.input} is silent (peak {silent_peak:g} dBFS); the visualisation is a flat line", "silent_input",
289
297
  path=args.input, peak_db=silent_peak)