ffmpeg-skill 2.10.11 → 2.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -3
- package/docs/contract.md +107 -11
- package/mcp/server.py +7 -1
- package/package.json +1 -1
- package/references/scripts.md +92 -20
- package/scripts/_common/__init__.py +6 -4
- package/scripts/_common/asr.py +2 -2
- package/scripts/_common/decision.py +57 -11
- package/scripts/_common/emit.py +198 -2
- package/scripts/_common/fonts.py +93 -28
- package/scripts/_common/runner.py +246 -68
- package/scripts/_common/timeline.py +22 -6
- package/scripts/_contract.py +168 -10
- package/scripts/audio.py +2 -1
- package/scripts/batch.py +240 -105
- package/scripts/caption.py +68 -18
- package/scripts/color.py +2 -1
- package/scripts/cropdetect.py +19 -4
- package/scripts/export.py +42 -10
- package/scripts/graphics.py +24 -18
- package/scripts/grid.py +7 -3
- package/scripts/join.py +5 -3
- package/scripts/loop.py +4 -2
- package/scripts/multicam.py +11 -3
- package/scripts/overlay.py +14 -10
- package/scripts/render.py +223 -74
- package/scripts/report.py +77 -23
- package/scripts/reverse.py +4 -2
- package/scripts/sequence.py +4 -2
- package/scripts/silence.py +19 -7
- package/scripts/sync.py +18 -7
- package/scripts/verify.py +98 -24
- package/scripts/waveform.py +3 -2
package/README.md
CHANGED
|
@@ -338,7 +338,7 @@ The contract is generated from the code that runs, not maintained beside it. For
|
|
|
338
338
|
|
|
339
339
|
Next to the tool list the document carries a top-level `deprecated` list (1.10): what 2.0.0 removes, since when, the replacement and the surface it lives on. `docs/contract.md` "What 2.0 changes" is written from it.
|
|
340
340
|
|
|
341
|
-
`contract_version` (1.0) is separate from the skill version, so a consumer can pin the shape and read the version for provenance. The document also states the invocation mapping (structured arguments → argv), the JSON shapes for success and failure (`{"status": "failed", "error": {"kind": "input | ffmpeg | output | missing_tool | timeout | verification | interrupted", "message": …}}`), and that no tool runs a shell or executes anything other than the named script, `ffmpeg` and `ffprobe`. Field-by-field reference: [docs/contract.md](docs/contract.md).
|
|
341
|
+
`contract_version` (1.0) is separate from the skill version, so a consumer can pin the shape and read the version for provenance. The document also states the invocation mapping (structured arguments → argv), the JSON shapes for success and failure (`{"status": "failed", "error": {"kind": "input | ffmpeg | output | missing_tool | timeout | verification | interrupted", "message": …}}`; a refusal adds `error.reason`, a stable code such as `output_exists` or `plan_inputs_changed`, and `error.details`, the facts behind it; warnings, fallbacks and skipped steps a run decided on its own are in `diagnostics`), and that no tool runs a shell or executes anything other than the named script, `ffmpeg` and `ffprobe`. Field-by-field reference: [docs/contract.md](docs/contract.md).
|
|
342
342
|
|
|
343
343
|
### MCP
|
|
344
344
|
|
|
@@ -388,7 +388,8 @@ The short list for humans. The agent-facing version, with the reasoning, is the
|
|
|
388
388
|
- **Non-Latin text picks a font by script (1.12).** Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari, Thai, Cyrillic and Greek cues, titles and overlays resolve a font file that covers them automatically, and a machine with no such font fails the job (`kind: input`) instead of rendering boxes. `doctor --json`'s `fonts.scripts` says which languages this machine can render; `--lang ja|ko` disambiguates Han-only text; an explicit `--font`/`--font-file` is always kept.
|
|
389
389
|
- **Silence detection finds nothing?** The default threshold is −35 dBFS. The tool prints a hint with the track's measured level; raise the threshold (`silence.py --threshold -25`) or shorten `--min-silence`.
|
|
390
390
|
- **Sync results carry a confidence.** Below 0.3, or an offset near the edge of the analysis window, is probably wrong: enlarge `--analyze-seconds` or find a clap. Recordings over ten minutes from separate devices need `sync.py --fix-drift`.
|
|
391
|
-
- **Outputs are never overwritten silently.** An existing output path is refused (`kind: input`, before anything runs); pass `--overwrite` where a replacement is intended.
|
|
391
|
+
- **Outputs are never overwritten silently.** An existing output path is refused (`kind: input`, `error.reason: output_exists`, before anything runs); pass `--overwrite` where a replacement is intended.
|
|
392
|
+
- **Nothing happens silently.** A refusal names its `error.reason` and `error.details`; a stale output lock taken over, a run without a lock, a plan input nobody could fingerprint, a fallback (an odd-sized frame scaled to even, a substituted font, libaom for SVT-AV1), a skipped step and every warning are recorded in `diagnostics` and on stderr (codes: `json_output.diagnostic_codes`); tools that draw text name the font they drew with (`font_used`). A refused plan and a lock takeover also append a line to an audit log (`<plan>.audit.jsonl`, or `render.py --audit-log FILE`). A degraded or partial result says so: `render.py`'s `verified` is every stage's, an unmeasured loudness or a report section that could not be built is a failing verification step, `export.py` reports `trimmed`, `multicam.py`'s `cuts` name the camera rendered, and `batch.py` rows carry the failing step's `error`, `verified`, and a row for every input after Ctrl-C.
|
|
392
393
|
- **Long chains belong in a plan.** Three hand-chained re-encodes lose quality and are hard to change; `render.py` runs the whole edit from one JSON file, and `--dry-run` shows every ffmpeg command before anything is written.
|
|
393
394
|
|
|
394
395
|
## FAQ
|
|
@@ -403,7 +404,7 @@ ffmpeg-skill edits footage you already have, on this machine, with FFmpeg tools
|
|
|
403
404
|
|
|
404
405
|
### Does it validate the filter graph before running?
|
|
405
406
|
|
|
406
|
-
No. `--dry-run` prints the ffmpeg command a writing tool would run and does not start ffmpeg, so ffmpeg never parses that filter graph. ffprobe still runs when the tool probes an input that exists. Unless `--overwrite` is passed, an existing output is refused (`kind: input`) before ffmpeg. An output that is the same file as an input is refused either way. No media file is written, and `verified` is false; `--plan FILE` does write the plan JSON. A path that is not on disk yet is not measured: the probe is a stub (0×0, duration 0), so a number taken from it is not a measurement. The measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) still run their measurements; only the artifact is skipped, and `silence` with `--filler --transcribe` also runs the speech engine, which may download a model. `overlay` measures the lines of a multi-line `--text` with one drawtext on a generated frame on FFmpeg 8.0, 8.0.1 and 8.1.0, where it draws them one by one. `verify` accepts `--dry-run` and ignores it: its steps run. `--plan FILE` is that dry run written down — the planned argv, the command lines, and a fingerprint of each existing input (size, plus a sha256 of the first and last 8 MiB). `render.py` and `verify.py` refuse `--plan`. `render.py FILE` runs a plan only when those fingerprints still match, by running the tool again with the planned argv, not by executing the recorded ffmpeg lines; `--dry-run` on that run only prints them. `doctor` reports whether the filters, encoders and bitstream filters the contract names are in this build (`ffmpeg -filters`, `-encoders`, `-bsfs`) and which fonts are installed. A name in that listing is not a check of the graph this command would build, and a listed GPU encoder is not a successful encode.
|
|
407
|
+
No. `--dry-run` prints the ffmpeg command a writing tool would run and does not start ffmpeg, so ffmpeg never parses that filter graph. ffprobe still runs when the tool probes an input that exists. Unless `--overwrite` is passed, an existing output is refused (`kind: input`) before ffmpeg. An output that is the same file as an input is refused either way. No media file is written, and `verified` is false; `--plan FILE` does write the plan JSON. A path that is not on disk yet is not measured: the probe is a stub (0×0, duration 0), so a number taken from it is not a measurement. The measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) still run their measurements; only the artifact is skipped, and `silence` with `--filler --transcribe` also runs the speech engine, which may download a model. `overlay` measures the lines of a multi-line `--text` with one drawtext on a generated frame on FFmpeg 8.0, 8.0.1 and 8.1.0, where it draws them one by one. `verify` accepts `--dry-run` and ignores it: its steps run. `--plan FILE` is that dry run written down — the planned argv, the command lines, and a fingerprint of each existing input (size, plus a sha256 of the first and last 8 MiB). `render.py` and `verify.py` refuse `--plan`. `render.py FILE` runs a plan only when those fingerprints still match (a mismatch is refused with each input's expected and actual fingerprint in `error.details`, and logged to `<plan>.audit.jsonl`), by running the tool again with the planned argv, not by executing the recorded ffmpeg lines; `--dry-run` on that run only prints them. `doctor` reports whether the filters, encoders and bitstream filters the contract names are in this build (`ffmpeg -filters`, `-encoders`, `-bsfs`) and which fonts are installed. A name in that listing is not a check of the graph this command would build, and a listed GPU encoder is not a successful encode.
|
|
407
408
|
|
|
408
409
|
### Can it remove burned-in subtitles or logos?
|
|
409
410
|
|
package/docs/contract.md
CHANGED
|
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
|
|
|
21
21
|
| Field | Meaning | Changes when |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
|
|
24
|
-
| `skill.version` | the npm / package.json version (`2.
|
|
24
|
+
| `skill.version` | the npm / package.json version (`2.12.0`) | any release |
|
|
25
25
|
|
|
26
26
|
A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
|
|
27
27
|
ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
|
|
@@ -125,7 +125,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
|
|
|
125
125
|
"contract_version": "1.0",
|
|
126
126
|
"deprecated": [{"what": "running on Python 3.9 (the requirements.python floor)", "since": "2.10.9", "replacement": "Python 3.10 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
|
|
127
127
|
"removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
|
|
128
|
-
"skill": {"id": "ffmpeg-skill", "version": "2.
|
|
128
|
+
"skill": {"id": "ffmpeg-skill", "version": "2.12.0", "execution_mode": "local", "kind": "execution",
|
|
129
129
|
"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
|
|
130
130
|
"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
|
|
131
131
|
"requirements": {"python": ">=3.9 (standard library only; 3.9 deprecated since 2.10.9, removed in 3.0.0; 3.10+ recommended)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
|
|
@@ -259,16 +259,23 @@ asks for the CPU and gets what ffmpeg says about it. What ran is in the result's
|
|
|
259
259
|
|
|
260
260
|
`--plan FILE` (1.6) is a dry run that also writes a plan document: `{"plan_version": 1,
|
|
261
261
|
"tool", "argv", "cwd", "inputs": [{"path", "size", "sha256_head_tail"}], "commands",
|
|
262
|
-
"output", "verify": [{"tool": "probe"}, {"tool": "check", "platform"}], "notes"}`. It
|
|
262
|
+
"unbound_inputs", "output", "verify": [{"tool": "probe"}, {"tool": "check", "platform"}], "notes"}`. It
|
|
263
263
|
implies `--dry-run`, so the same execution rules apply; `inputs` covers the `-i` files of the
|
|
264
264
|
planned commands, every existing file named in argv (a recipe, a still, an SRT) and the
|
|
265
|
-
subtitle/LUT/font files a filter reads
|
|
265
|
+
subtitle/LUT/font files a filter reads; `unbound_inputs` (absolute paths, additive to version 1)
|
|
266
|
+
names the inputs that did not exist yet when the plan was made (an earlier step's output), which
|
|
267
|
+
therefore carry no fingerprint. Tools that print their document without `--json`
|
|
266
268
|
(`probe`, the analysis tools) still write the plan at exit; `verify.py` and `render.py`
|
|
267
269
|
refuse `--plan` (their steps run for real; a project file is already a plan). `render.py FILE` executes a plan:
|
|
268
|
-
it refuses (`kind: input`) when an input's size or head/tail
|
|
270
|
+
it refuses (`kind: input`, `error.reason: plan_inputs_changed`) when an input's size or head/tail
|
|
271
|
+
hash differs from the plan, with every input's `{path, status, expected, actual}` in
|
|
272
|
+
`error.details.inputs` (`actual` is `"missing"` for a file that is gone) beside `plan_sha256` and
|
|
273
|
+
`checked_at`, and appends one record to the audit log (below); a wrong `plan_version`, an unknown
|
|
274
|
+
tool, a vanished `cwd` or a malformed `argv` is refused the same way under its own reason. It then
|
|
269
275
|
runs the tool with the planned `argv`, then the verify steps (a `check` step may also carry
|
|
270
|
-
`"content": true`, added by hand, for `check.py --content`), and reports `plan`, `
|
|
271
|
-
`tool_result` and `
|
|
276
|
+
`"content": true`, added by hand, for `check.py --content`), and reports `plan`, `plan_sha256`,
|
|
277
|
+
`tool`, `tool_result`, `check` and `unbound_inputs` (each one also a `diagnostics.warnings` entry:
|
|
278
|
+
nothing checked that file). `plan_version` is bumped when the document's shape changes.
|
|
272
279
|
|
|
273
280
|
### Repeatability
|
|
274
281
|
|
|
@@ -369,7 +376,9 @@ limit (`check.py` reports it instead). `_platforms.resolve()` is the one alias m
|
|
|
369
376
|
New in the same release, all additive: `export.py --preset tiktok|shorts|linkedin|facebook`
|
|
370
377
|
(real presets, not aliases of `reels`/`youtube`), `--preset youtube-hdr` (HEVC Main10 keeping
|
|
371
378
|
the source's HDR tags; `kind: input` on an SDR source) and `--preset youtube-av1`
|
|
372
|
-
(`kind: missing_tool` when the build has neither SVT-AV1 nor
|
|
379
|
+
(`kind: missing_tool`, `error.reason: encoder_missing`, when the build has neither SVT-AV1 nor
|
|
380
|
+
libaom; `kind: ffmpeg`, `encoder_list_unreadable`, when its encoder list could not be read at all,
|
|
381
|
+
which says nothing about the build); `caption.py --platform`
|
|
373
382
|
and `graphics.py --platform` / `--margin` and `overlay.py --platform` (margins from the safe
|
|
374
383
|
zone, an explicit `--margin`/`--position` wins); `look.py --safe NAME`; `fit.py --fit blur`; `report.py --pack`;
|
|
375
384
|
`graphics.py --template sticker|hook|meme`; and `render.py --template NAME INPUT`
|
|
@@ -539,7 +548,7 @@ the script, `missing` means fontconfig knows none, `unknown` means there is no w
|
|
|
539
548
|
ask (no `fc-list` on PATH, or it failed). `caption.py`, `graphics.py` and `overlay.py --text` resolve
|
|
540
549
|
a font by script automatically and fail with `kind: input` rather than render boxes **only for
|
|
541
550
|
`missing`**: `unknown` is not `missing` here any more than anywhere else in this document — the job
|
|
542
|
-
runs with the font as given and
|
|
551
|
+
runs with the font as given and a `font_coverage_unverified` warning says the coverage could not be verified. So a
|
|
543
552
|
`missing` script here is a job that will not run until a font is installed — but, like the default
|
|
544
553
|
font, it never affects `ok` or any tool's
|
|
545
554
|
`usable` (the tool works, this machine just has no glyphs for that language). The plain-text
|
|
@@ -627,7 +636,7 @@ Per-tool keys added in 1.17, all additive:
|
|
|
627
636
|
| `filler`, `removed_seconds_total` | `silence.py --filler` | `{lang, source, engine, words, removed, removed_count, removed_seconds, removed_words, word_timings, list, warnings}`. The existing `removed_seconds` is unchanged in name and meaning — the seconds of *silence* removed, which is what it has always held — and `removed_seconds_total` is the additive sibling covering silence plus filler |
|
|
628
637
|
| `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
|
|
629
638
|
| `timeline` | `render.py --export-timeline FILE` | `{format, rate, duration, frames, clips, transition, transition_frames, music, markers, notes, not_exported}`: the written timeline's own numbers (`rate` is exact, `"30000/1001"` for 29.97) and what the project asked for that an editor timeline cannot carry. Nothing is encoded (`commands: []`); `verified` means the file was written and reads back as its format (`verification` step `parse`) -- not that an editor opened it |
|
|
630
|
-
| `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …), the GPU setting (`FFMPEG_SKILL_HW` and whether `--hw`/`--no-hw` made it explicit) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
|
|
639
|
+
| `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, miss_reasons, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. `miss_reasons` (unreleased, additive): `[{stage, reason, ...}]`, `reason` one of `not_cached`, `sidecar_missing`, `artifact_missing`, `sidecar_corrupt`, `artifact_truncated`, `no_output_path`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …), the GPU setting (`FFMPEG_SKILL_HW` and whether `--hw`/`--no-hw` made it explicit) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
|
|
631
640
|
|
|
632
641
|
Per-tool keys added in 1.17.1, all additive:
|
|
633
642
|
|
|
@@ -669,6 +678,16 @@ Per-tool keys added after 2.2.2, all additive:
|
|
|
669
678
|
| `vfr_check` | `cut.py` (a video input that could still stream-copy) | the sampled frame timing, `{heuristic: true, measured, method: "sampled", guard, windows, deltas}`; `measured` is `sampled_cfr`, `vfr` or `inconclusive`. `--vfr-guard average` (the default) keeps the 2.x guard: a source whose nominal and average frame rates differ (`variable_frame_rate_suspected`) re-encodes as `--accurate` (reason `vfr`, stderr says "variable-frame-rate"), and only then is the timing sampled and reported, with a `notes` line naming `--vfr-guard sampled` when it measured `sampled_cfr`; the key is absent otherwise. `--vfr-guard sampled` samples before every copy and re-encodes on `vfr` (reason `vfr`) or `inconclusive` (fewer than 20 intervals; `vfr_inconclusive`); `--vfr-guard off` samples and keeps the copy, with a `notes` line. Absent for audio-only input or output, `--accurate`, `--codec` and a pending `--dry-run` input |
|
|
670
679
|
| `join_check`, `segment_end_snap_seconds` | `cut.py --segments` | `join_check`: a video stream-copy join measured by demuxing, `{packets, expected_packets, max_step_seconds, audio_offset_ms, audio_parts_checked, ok}`. `expected_packets` counts the source's video packets from each part's start keyframe to its end (or the end of the video); `max_step_seconds` is the largest presentation step, `null` when `vfr_check` measured anything but `sampled_cfr` (a `--vfr-guard off` join of variable timing: count only), and each step at a join must be the step its parts predict within 1 ms. `audio_offset_ms`: per part, its sound's shift minus its picture's in ms (+ = sound late), from audio packets matched by CRC32 to the source stream shaped like the joined audio (FFmpeg's default selection may copy one other than `a:0`; an MPEG-TS source's AAC is hashed without its ADTS headers); `null` for a part with no unique match (digital silence, a periodic tone), and as a whole when the source has no audio or the video check failed. `audio_parts_checked` counts the measured parts. `ok` is the conjunction (count, steps, every measured part within 5 ms); `ok: false` means the join was re-cut (`concat_fallback`) and this is the failed measurement. `null` for a single segment, audio, `--dry-run`, any re-encode forced for every segment, a segment that re-encoded, or a join re-cut before a copy was tried (open GOPs, a later B-frame segment starting between keyframes, parts that differ, a concat error). `segment_end_snap_seconds`: per segment, the signed seconds its end moved to the keyframe the copied part stops at, the first at or after the requested end (`null` for a part that reaches the end of the video, a part of a source with no B-frames, cut to its requested end as in 2.5.1, or a Matroska/MPEG-TS part, which is not snapped); `null` as a whole unless the output is those copied parts |
|
|
671
680
|
|
|
681
|
+
Per-tool keys added after 2.10.11 ("nothing happens silently"), all additive:
|
|
682
|
+
|
|
683
|
+
| key | tool | what it holds |
|
|
684
|
+
|---|---|---|
|
|
685
|
+
| `font_used` | `overlay.py --text`, `graphics.py`, `caption.py` burn mode, `grid.py --label auto` | the font the text was drawn with: `{asked, file, family, how, substituted, via?, fonts_dir?}`. `how`: `font_file` (the caller's file), `script` (picked to cover the text's script), `resolved` (the asked family resolved to a file by `fc-match` or the Windows Fonts folder; for libass, the file fontconfig picks for the family), `by_name` (no file resolved: ffmpeg or libass look the family up themselves, `file` null), `fonts_dir` (`caption.py`: the family is looked up in `--fonts-dir` first), `ass_styles` (`caption.py --ass`: the file's own styles name the fonts). `substituted: true` -- the file is not the asked family (a Windows machine without it gets a CJK system font or Arial; fontconfig answers every name) -- is also a `font_substituted` fallback |
|
|
686
|
+
| `windows`, `windows_failed` | `cropdetect.py` | the sampled windows and how many ffmpeg could not decode (skipped; a `cropdetect_windows_failed` warning when above 0). Every window failing stays a `kind: ffmpeg` failure |
|
|
687
|
+
| `drift_skipped_reason` | `sync.py --fix-drift` (top level for one SOURCE, and in each `sources` entry) | why the drift correction was not applied: `too_short` (no tail window far enough from the head) or `low_confidence` (the tail did not correlate); `null` when it was. Also a `drift_correction_skipped` entry under `diagnostics.skipped` |
|
|
688
|
+
| `ass_shifted`, `ass_unshifted`, `ass_malformed` | `caption.py --ass` (inside `caption`) | with `--offset`: the Dialogue lines moved, and those whose timestamps could not be read (copied as they were); every run: Dialogue lines that could not be read (too few fields, an unreadable start/end) and are not counted as cues. A count above 0 is a warning |
|
|
689
|
+
| `matched_untimed`, `matched_too_long` | `silence.py --filler` (inside `filler`) | listed words that were not cut: no usable timing in the transcript (also a `filler_words_untimed` warning), or longer than a filler (`FILLER_MAX_WORD`, a held vowel) |
|
|
690
|
+
|
|
672
691
|
`caption.py --transcribe` and `silence.py --filler --transcribe`: when a local speech engine ran
|
|
673
692
|
and produced no cue, the refusal is `"<engine> found no speech in <input>"` with `kind: input`,
|
|
674
693
|
`reason: "no_speech"` and `engine` in the failure document — distinct from the "no local
|
|
@@ -732,6 +751,78 @@ strings. `retryable` is currently always `false`: none of the kinds are distingu
|
|
|
732
751
|
today from a deterministic failure that would fail identically on a blind retry, so nothing
|
|
733
752
|
here claims otherwise until real exit-code/stderr sniffing exists to back that up.
|
|
734
753
|
|
|
754
|
+
A refusal also carries `error.reason`, a stable snake_case code for *why*
|
|
755
|
+
(`contract --json` → `json_output.error_reasons` lists them: `output_exists`, `output_is_input`,
|
|
756
|
+
`output_is_directory`, `output_dir_missing`, `output_dir_not_writable`, `output_locked`,
|
|
757
|
+
`render.py`'s `plan_*` reasons, and `--codec av1`/`prores`'s `encoder_missing` and
|
|
758
|
+
`encoder_list_unreadable`), and `error.details`, the facts the message is written from, so a
|
|
759
|
+
caller never parses the prose:
|
|
760
|
+
|
|
761
|
+
```
|
|
762
|
+
"error": {"kind": "input", "message": "refusing to overwrite existing output 'out.mp4': ...",
|
|
763
|
+
"code": "INPUT_INVALID", "retryable": false, "hint": "pass --overwrite, or choose another -o path",
|
|
764
|
+
"reason": "output_exists",
|
|
765
|
+
"details": {"outputs": [{"path": "out.mp4", "resolved": "/abs/out.mp4", "size": 48213, "mtime_ns": 1729660000000000000}]}}
|
|
766
|
+
```
|
|
767
|
+
|
|
768
|
+
A tool that runs another (`render.py` stages and plans) forwards the child's `reason` and
|
|
769
|
+
`details` in its own `error`. Both are absent on a failure that is not a refusal, like `hint`.
|
|
770
|
+
|
|
771
|
+
**Diagnostics.** What a run decided on its own -- a warning, a fallback it took, a step it
|
|
772
|
+
skipped -- is recorded as it happens (a stderr line too) and lands in the document as
|
|
773
|
+
`diagnostics: {warnings, fallbacks, skipped}`, each entry `{code, message, ...facts}`; it is
|
|
774
|
+
present on success and failure documents alike, only when one list is not empty, and an entry a
|
|
775
|
+
`render.py` stage reported carries `stage`. `contract --json` → `json_output.diagnostic_codes`
|
|
776
|
+
lists every code with its bucket and meaning (a test reads every `warn()`/`record_fallback()`/
|
|
777
|
+
`record_skip()` call against it). Among them: the runner's `odd_dimensions_scaled_even` (an encode
|
|
778
|
+
re-run with an even scale; the frame loses a row or column), `non_av_streams_dropped` (with the
|
|
779
|
+
first attempt's error beside `dropped_non_av_streams`), `measurement_input_pending` (skipped: a
|
|
780
|
+
dry-run measurement of an earlier stage's output), `ffmpeg_list_unreadable` (`ffmpeg -encoders` or
|
|
781
|
+
`-bsfs` could not be read; what needed it names the read failure, not a missing encoder) and
|
|
782
|
+
`timeout_env_invalid` (an `FFMPEG_SKILL_TIMEOUT` that is not a number, or is negative);
|
|
783
|
+
`av1_encoder_libaom` and `crf_clamped`; `font_substituted` (with `font_used`); `drift_correction_skipped`
|
|
784
|
+
(with `drift_skipped_reason`); `cropdetect_windows_failed` (with `windows_failed`);
|
|
785
|
+
`ass_lines_unshifted` / `ass_lines_malformed`; `timeline_parse_failed`; and every `warning:` line a
|
|
786
|
+
tool used to print to stderr only (emoji fallbacks, silent or short inputs, memory-heavy jobs,
|
|
787
|
+
`--strip-dovi` without Dolby Vision, `render.py`'s frame-from-preset and `--normalize` defaults).
|
|
788
|
+
`render.py`'s plan, check and cache records (`check_dry_run`, `check_output_unparsed`, `cache_entry_damaged`, ...),
|
|
789
|
+
`export.py`'s `loudness_not_measured`, `loudness_outside_spec` and `colour_not_converted`, `multicam.py`'s
|
|
790
|
+
`camera_not_started`, `report.py`'s `report_section_missing` and `batch.py`'s cache records
|
|
791
|
+
(`batch_cache_reset`, `batch_cache_output_changed`, ...) are listed there too. Every one of them
|
|
792
|
+
still prints its stderr line; the record is additive.
|
|
793
|
+
|
|
794
|
+
**A result that is not the one asked for says so.** `verified` is the conjunction of every
|
|
795
|
+
`verification` step, and a tool that runs others counts theirs: `render.py` lists each stage's own
|
|
796
|
+
steps beyond its intermediate's probe, tagged with `stage` (a cached stage's as the cache recorded
|
|
797
|
+
them), and its `notes` carry each stage's notes prefixed with the stage's tool, so an export that
|
|
798
|
+
missed its platform loudness leaves the render `verified: false`. A step that could not be
|
|
799
|
+
measured is a failing step with a `reason`, never an absent one: `export.py`'s unreadable loudness
|
|
800
|
+
is `{step: loudness, ok: false, reason: not_measured}` with `loudness.measured: false`, and
|
|
801
|
+
`report.py` lists each section it built (`loudness`, `compliance` with `passed`,
|
|
802
|
+
`contact_sheet`), one it could not with `ok: false` and the child's `reason`/`message`. A
|
|
803
|
+
deliberate change of result is a field: `export.py`'s `trimmed: {from, to}` when a platform
|
|
804
|
+
maximum cut the clip, `multicam.py`'s `cuts` naming the camera rendered (camera 0 where the one
|
|
805
|
+
asked for had not started, with a `camera_not_started` fallback). `batch.py` keeps `ok` as "every
|
|
806
|
+
step completed" (a spec miss is `completed` + `verified: false`, not a failure) and adds `verified`
|
|
807
|
+
and `unverified: [{step, tool, failed}]` to each row, one `item` verification step per row, a
|
|
808
|
+
failed row's `error: {step, tool, kind, message, code, reason, details, hint, exit_code}` from the
|
|
809
|
+
failing step's own document, and `cache_miss` when a cache entry was not served because its output
|
|
810
|
+
changed or could not be checked; an interrupted batch has a row for every input
|
|
811
|
+
(`interrupted: true`, or `skipped: "interrupted"` when it had not started) and `total` counts them
|
|
812
|
+
all. `verify.py`'s steps keep the tool's `kind`, `reason`, `message` and `exit_code`, and the steps
|
|
813
|
+
that do not apply are listed under each file's `skipped` with a reason.
|
|
814
|
+
|
|
815
|
+
**Audit log.** Events a later reader must be able to reconstruct are appended as JSON lines,
|
|
816
|
+
`{ts, tool, event, pid, plan_sha256, details}` (`plan_sha256` on plan events): `plan_rejected`,
|
|
817
|
+
`lock_takeover`, `output_unlocked`, `output_is_input_refused` when a log was named,
|
|
818
|
+
`cache_entry_damaged` (`render.py --cache`) and `batch_cache_reset`. The file
|
|
819
|
+
is `render.py --audit-log FILE` when given; otherwise a refused plan's `<plan>.audit.jsonl` next
|
|
820
|
+
to the plan, a lock event's hidden `.<output>.ffskill-audit.jsonl` beside the output, and a cache
|
|
821
|
+
event's `.ffskill-audit.jsonl` in the cache directory or the batch's `output_dir`. Nothing
|
|
822
|
+
is appended under `--dry-run`. A record that cannot be written is a `diagnostics.warnings` entry
|
|
823
|
+
(`audit_log_unwritable`) and never fails or blocks the run; the refusal's `details.audit_log`
|
|
824
|
+
says `{path, written}` either way.
|
|
825
|
+
|
|
735
826
|
## MCP relationship
|
|
736
827
|
|
|
737
828
|
`mcp/server.py` is a transport. It holds no tool table and no schema of its own:
|
|
@@ -804,7 +895,12 @@ returns one user-role text message naming the actual command lines to run and wh
|
|
|
804
895
|
afterward -- the same facts SKILL.md's own request table states, addressed by workflow instead of
|
|
805
896
|
by request phrasing. A missing required argument is a JSON-RPC error (`prompts/get` never guesses
|
|
806
897
|
a path or a platform); an unknown prompt name is a JSON-RPC error too, distinct from "method not
|
|
807
|
-
found" since `prompts/get` itself is a real method.
|
|
898
|
+
found" since `prompts/get` itself is a real method.
|
|
899
|
+
|
|
900
|
+
A line on stdin that is not JSON is answered with a JSON-RPC parse error (`-32700`, `id: null`)
|
|
901
|
+
rather than dropped, so a client whose request was mangled does not wait for an answer that
|
|
902
|
+
never comes; a JSON value that is not an object, and a notification, still get no response.
|
|
903
|
+
An unknown method is `-32601`, any other handler error `-32000`. These are recipes, not new tool calls: every
|
|
808
904
|
command line a prompt names is one `tools/call` (or the CLI) can already run.
|
|
809
905
|
|
|
810
906
|
## Consuming the contract from an agent
|
package/mcp/server.py
CHANGED
|
@@ -247,7 +247,13 @@ def main() -> int:
|
|
|
247
247
|
continue
|
|
248
248
|
try:
|
|
249
249
|
req = json.loads(line)
|
|
250
|
-
except ValueError:
|
|
250
|
+
except ValueError as exc:
|
|
251
|
+
# JSON-RPC 2.0: a line that is not JSON is answered with -32700 and id null, not
|
|
252
|
+
# dropped -- a client whose request was mangled in transit otherwise waits for an
|
|
253
|
+
# answer that never comes, with nothing to say why
|
|
254
|
+
resp = {"jsonrpc": "2.0", "id": None, "error": {"code": -32700, "message": f"parse error: {exc}"}}
|
|
255
|
+
stdout.write((json.dumps(resp) + "\n").encode("utf-8"))
|
|
256
|
+
stdout.flush()
|
|
251
257
|
continue
|
|
252
258
|
# json.loads accepts any valid JSON value, not just an object -- a bare `42`, `null`,
|
|
253
259
|
# `true` or `[1,2]` line parses fine but isn't a JSON-RPC request. Without this guard,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.12.0",
|
|
4
4
|
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ffmpeg",
|
package/references/scripts.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Script reference
|
|
2
2
|
|
|
3
|
-
Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), `overlay` measures a multi-line `--text`'s lines on FFmpeg 8.0, 8.0.1 and 8.1.0 (one drawtext on a generated frame), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
3
|
+
Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, `error.reason: output_exists`, before anything runs) to replace an existing output unless `--overwrite` is given. Refusals carry `error.reason` and `error.details` (the facts behind the message; `contract --json` → `json_output.error_reasons`), and what a run decided on its own -- a stale output lock taken over, no lock possible, a plan input nobody fingerprinted, an odd-sized frame scaled to even, subtitle streams dropped on a retry (with the first attempt's error), a font substituted, a measurement skipped under `--dry-run`, an encoder list that could not be read, an invalid `FFMPEG_SKILL_TIMEOUT`, and every `warning:` line -- is in `diagnostics: {warnings, fallbacks, skipped}` as well as on stderr (codes: `contract --json` → `json_output.diagnostic_codes`). `--codec av1`/`prores` with no encoder for it refuses `kind: missing_tool`, `error.reason: encoder_missing`; when the encoder list could not be read at all, `kind: ffmpeg`, `encoder_list_unreadable`. Tools that draw text (`overlay.py --text`, `graphics.py`, `caption.py`, `grid.py --label auto`) report `font_used: {asked, file, family, how, substituted}`. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), `overlay` measures a multi-line `--text`'s lines on FFmpeg 8.0, 8.0.1 and 8.1.0 (one drawtext on a generated frame), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
4
4
|
|
|
5
5
|
## Time grammar (every time-taking flag, 1.9)
|
|
6
6
|
|
|
@@ -251,7 +251,10 @@ itself with no black-bar measurement involved. A rectangle that matches the
|
|
|
251
251
|
full source frame means no bars were found. Does not decide whether
|
|
252
252
|
removing detected bars is wanted -- genuine letterboxed content (a
|
|
253
253
|
scope-ratio film in a 16:9 frame) "detects" the same way as accidental
|
|
254
|
-
bars; look at the frame before cropping it away.
|
|
254
|
+
bars; look at the frame before cropping it away. `windows` / `windows_failed`
|
|
255
|
+
say how many windows were sampled and how many ffmpeg could not decode (those
|
|
256
|
+
are skipped, with a `cropdetect_windows_failed` warning; all of them failing is
|
|
257
|
+
a `kind: ffmpeg` failure).
|
|
255
258
|
|
|
256
259
|
`--motion-centre` (1.18) reports the motion centroid once per second, sampled
|
|
257
260
|
across the same windows: `motion_centre: [{time, x, y, x_frac, y_frac, motion}]`,
|
|
@@ -595,6 +598,9 @@ per-word `start`/`end`) or `--transcribe` (the same local engine `caption.py`
|
|
|
595
598
|
uses, never required). There is no heuristic fallback — finding an "um" by
|
|
596
599
|
looking for short quiet blips would cut real speech, so the tool refuses
|
|
597
600
|
instead. `--filler-list` reports what would go and writes nothing.
|
|
601
|
+
`filler.matched_untimed` counts listed words the transcript gives no usable
|
|
602
|
+
timing for (not removed, and a `filler_words_untimed` warning), and
|
|
603
|
+
`matched_too_long` those longer than a filler (a held vowel, kept).
|
|
598
604
|
|
|
599
605
|
Built-in lists, one per language:
|
|
600
606
|
|
|
@@ -709,7 +715,7 @@ output; the result says so with `dropped_non_av_streams: true`.
|
|
|
709
715
|
render.py --init project.json # starter file
|
|
710
716
|
render.py project.json [--fast] [--dry-run] [--stop-after STAGE] [--work DIR --keep]
|
|
711
717
|
render.py project.json --cache DIR [--from STAGE] # reuse identical stages from a previous run
|
|
712
|
-
render.py plan.json
|
|
718
|
+
render.py plan.json [--audit-log FILE] # execute a plan written by <tool> --plan plan.json
|
|
713
719
|
render.py project.json --export-timeline edit.fcpxml|edit.edl|edit.otio # the cut for an editor, nothing rendered
|
|
714
720
|
```
|
|
715
721
|
`--export-timeline FILE` (2.1) writes the project's cut as an editor timeline instead of
|
|
@@ -734,13 +740,29 @@ few frames longer, because `join.py` offsets its crossfades by each part's conta
|
|
|
734
740
|
audio included.
|
|
735
741
|
A plan is a single tool's dry run as an artifact: `cut.py in.mp4 --start 2 --end 8
|
|
736
742
|
--plan cut.json` writes `{plan_version, tool, argv, inputs (path, size, sha256 of
|
|
737
|
-
head+tail), commands, output, verify}` and runs nothing
|
|
738
|
-
|
|
739
|
-
|
|
743
|
+
head+tail), unbound_inputs, commands, output, verify}` and runs nothing
|
|
744
|
+
(`unbound_inputs`: inputs that did not exist yet, so carry no fingerprint).
|
|
745
|
+
`render.py cut.json` re-fingerprints the inputs (refusing, `kind: input`,
|
|
746
|
+
`error.reason: plan_inputs_changed`, if any changed since the plan: every input's
|
|
747
|
+
`{path, status, expected, actual}` -- `actual` `"missing"` for a file that is gone --
|
|
748
|
+
plus `plan_sha256` and `checked_at` are in `error.details`, and one JSON line goes
|
|
749
|
+
to `<plan>.audit.jsonl`, or to `--audit-log FILE`; a log that cannot be written is
|
|
750
|
+
a `diagnostics.warnings` entry, never the failure, and a `--dry-run` writes none),
|
|
751
|
+
runs the tool with the planned argv, then the verify steps (probe; `check`
|
|
740
752
|
for a `--platform` or a platform export preset; add `"content": true` to a check
|
|
741
|
-
step for the black/frozen/silence rows), and reports `plan`, `tool`,
|
|
742
|
-
`tool_result` and `
|
|
753
|
+
step for the black/frozen/silence rows), and reports `plan`, `plan_sha256`, `tool`,
|
|
754
|
+
`tool_result`, `check` and `unbound_inputs` (each also a `diagnostics.warnings`
|
|
755
|
+
entry: nothing checked those files). Show the plan to the user, get the yes, execute:
|
|
743
756
|
one round trip instead of re-deriving the command.
|
|
757
|
+
Render's `verified` is every stage's: each stage's own verification steps (beyond
|
|
758
|
+
its intermediate's probe) are in the result's `verification`, tagged with `stage`,
|
|
759
|
+
and each stage's notes are in `notes`, prefixed with its tool -- an export that
|
|
760
|
+
misses its platform loudness leaves the render `completed` with `verified: false`
|
|
761
|
+
and the note naming the fix. `"check": {"platform": ...}`, `{"content": true}` or
|
|
762
|
+
both run `check.py` after the last stage (content alone: the black/frozen/silence
|
|
763
|
+
rows, youtube's spec assumed, as `check.py --content` without `--platform`); under
|
|
764
|
+
`--dry-run` the check is a `check_dry_run` entry in `diagnostics.skipped`.
|
|
765
|
+
|
|
744
766
|
`"export": {"preset": "reels", "normalize": true}` forwards `export.py --normalize`
|
|
745
767
|
so the rendered file meets the platform's loudness without a separate pass. Since
|
|
746
768
|
1.9.0 it is on by default when the preset is a platform (`youtube|youtube4k|reels|x`)
|
|
@@ -801,7 +823,14 @@ artifact the old one produced. Each entry is `DIR/<key><ext>` plus a
|
|
|
801
823
|
`DIR/<key>.json` sidecar (stage, versions, creation time, size, seconds). A hit
|
|
802
824
|
hardlinks the artifact into the work directory, or copies it where the
|
|
803
825
|
filesystem will not link — never moves it, since the cache has to outlive the
|
|
804
|
-
run's own cleanup.
|
|
826
|
+
run's own cleanup. A miss is work to do, not an error, but never a silent one:
|
|
827
|
+
`cache.miss_reasons` says why each stage missed (`not_cached`, `sidecar_missing`,
|
|
828
|
+
`artifact_missing`, `sidecar_corrupt`, `artifact_truncated`, `no_output_path`), and a
|
|
829
|
+
damaged entry (the last two of those that name a file this tool wrote) is also a
|
|
830
|
+
`cache_entry_damaged` warning and a line in `DIR/.ffskill-audit.jsonl`. A sidecar keeps
|
|
831
|
+
the stage's `verified`, `verification`, `notes` and `diagnostics`, and a hit replays
|
|
832
|
+
them, so a cached stage counts toward the render's `verified` as a live one does. A
|
|
833
|
+
store that fails is a `cache_store_failed` warning (the run is unaffected).
|
|
805
834
|
|
|
806
835
|
A stage whose flags changed misses the cache, so editing the captions block (for
|
|
807
836
|
example to `"fit_size": "off"`) re-runs captions and everything after it and
|
|
@@ -810,7 +839,7 @@ serves the earlier stages from the cache.
|
|
|
810
839
|
`stages_done` is unchanged: a cached stage is still a stage that happened.
|
|
811
840
|
Nothing is written under `--dry-run`, which instead reports `cache.would_hit`.
|
|
812
841
|
Result: `cache.dir`, `cache.ffmpeg`, `cache.hits`, `cache.misses`,
|
|
813
|
-
`cache.saved_seconds`, `cache.entries`. `--from STAGE` starts at that stage and
|
|
842
|
+
`cache.miss_reasons`, `cache.saved_seconds`, `cache.entries`. `--from STAGE` starts at that stage and
|
|
814
843
|
takes every earlier one from the cache; without `--cache`, or when an earlier
|
|
815
844
|
stage is not there, it refuses (`kind: input`) rather than quietly re-encoding
|
|
816
845
|
what it promised to skip. An unwritable cache directory is `kind: output`.
|
|
@@ -1040,10 +1069,28 @@ above the cap is clamped with a note, not refused, and both `jobs` and
|
|
|
1040
1069
|
the table in file order when the cache is only partially warm — and each item's
|
|
1041
1070
|
log lines are buffered and flushed in file order, so the per-item table and the
|
|
1042
1071
|
log read exactly as a serial run's whatever order the encodes finish in.
|
|
1043
|
-
- **A failed item is a row, not a dead run**: a
|
|
1044
|
-
`ok: false` with
|
|
1045
|
-
|
|
1046
|
-
`
|
|
1072
|
+
- **A failed item is a row, not a dead run**: a step that fails makes its row
|
|
1073
|
+
`ok: false` with `error: {step, tool, kind, message, code, reason, details,
|
|
1074
|
+
hint, exit_code}` taken from that step's own JSON document (a worker that raised:
|
|
1075
|
+
`exception` and its message), and the summary and the table are still printed.
|
|
1076
|
+
A recipe error that is the same for every item (a step script outside
|
|
1077
|
+
`scripts/`, no steps) is refused before any item runs.
|
|
1078
|
+
- **`ok` is "every step completed"; `verified` is beside it**: a step that
|
|
1079
|
+
completed with `verified: false` (an export's loudness miss) leaves the row
|
|
1080
|
+
`ok: true`, `verified: false`, with `unverified: [{step, tool, failed}]`; the
|
|
1081
|
+
batch's own `verification` has one `item` step per row and a note names the
|
|
1082
|
+
unverified items. A row also carries its steps' `diagnostics` (tagged with `step`).
|
|
1083
|
+
- **Ctrl-C** (or SIGTERM) stops the running steps and exits 130 (143)
|
|
1084
|
+
`kind: interrupted` with a row for every input: finished, `interrupted: true`
|
|
1085
|
+
(stopped while running) or `"skipped": "interrupted"` (not started), so `total`
|
|
1086
|
+
is the folder's count.
|
|
1087
|
+
- **The cache serves an output only while it is the file the batch wrote**: an
|
|
1088
|
+
entry keeps the output's size and mtime, and an output replaced or edited since
|
|
1089
|
+
is processed again (`cache_miss: {reason: output_changed, expected, actual}` on
|
|
1090
|
+
the row, a `batch_cache_output_changed` warning; an entry from 2.10.11 or
|
|
1091
|
+
earlier has no such record and is processed again once, `entry_unverifiable`).
|
|
1092
|
+
A `.ffskill_cache.json` that does not parse is started empty with a
|
|
1093
|
+
`batch_cache_reset` warning and a line in `output_dir/.ffskill-audit.jsonl`.
|
|
1047
1094
|
- With `--jobs > 1` each item gets its own work subdirectory
|
|
1048
1095
|
`<workdir>/<index>-<stem>/`, because step file names are stem-derived and two
|
|
1049
1096
|
sources sharing a stem would otherwise write over each other. `--jobs 1` (the
|
|
@@ -1175,7 +1222,12 @@ report.py --after FINAL [--before SOURCE] [--platform youtube] [--commands cmds.
|
|
|
1175
1222
|
```
|
|
1176
1223
|
One self-contained HTML: before/after facts and contact sheets, loudness,
|
|
1177
1224
|
compliance table with fixes, commands. Produce it for any multi-step job and
|
|
1178
|
-
hand the path to the user together with the numbers.
|
|
1225
|
+
hand the path to the user together with the numbers. Each section it builds is a
|
|
1226
|
+
`verification` step (`loudness` per file with audio, `compliance` for
|
|
1227
|
+
`--platform` with `passed` saying what the table found, `contact_sheet` per
|
|
1228
|
+
file); a section that could not be built is `ok: false` with the child's
|
|
1229
|
+
`reason` and `message` and a `report_section_missing` warning, so the report is
|
|
1230
|
+
`verified: false` rather than silently shorter.
|
|
1179
1231
|
|
|
1180
1232
|
### multicam.py — align several cameras and switch between them
|
|
1181
1233
|
```
|
|
@@ -1188,7 +1240,10 @@ All inputs are aligned to the first one by audio (same engine as `sync.py`,
|
|
|
1188
1240
|
each range of the reference timeline (gaps fall back to camera 0), `--auto N`
|
|
1189
1241
|
simply alternates every N seconds. Audio comes from the reference unless
|
|
1190
1242
|
`--audio` picks another input, e.g. an external recorder that has no video.
|
|
1191
|
-
`--offsets-only` reports offsets and confidence without rendering.
|
|
1243
|
+
`--offsets-only` reports offsets and confidence without rendering. `cuts` names
|
|
1244
|
+
the camera actually rendered: a range whose camera has not started yet plays
|
|
1245
|
+
camera 0, and a `camera_not_started` entry in `diagnostics.fallbacks` names the
|
|
1246
|
+
camera that was asked for.
|
|
1192
1247
|
|
|
1193
1248
|
**`--switch energy` (1.18)** auto-switches to whichever camera (of those with
|
|
1194
1249
|
a video stream) measures the loudest audio at each 0.25 s window on the
|
|
@@ -1215,6 +1270,11 @@ Runs the toolchain on the user's own files (phone HDR, GoPro, OBS, Log, Zoom)
|
|
|
1215
1270
|
and prints a PASS/FAIL table per step (probe, copy cut, accurate cut, fit,
|
|
1216
1271
|
caption, overlay, look, export, loudness, silence, plus `color --to-sdr` for
|
|
1217
1272
|
HDR and `audio --downmix` for >2 channels). Exit code 1 if anything fails.
|
|
1273
|
+
Each step runs with `--json`, and a failed one keeps its tool's own `kind`,
|
|
1274
|
+
`reason`, `message` and `exit_code` (`error` is the one-line table cell); a
|
|
1275
|
+
completed one carries the tool's `verified`. Steps that do not apply are listed
|
|
1276
|
+
under the file's `skipped` (`quick`, `no_video`, `no_audio`, `not_hdr`,
|
|
1277
|
+
`not_multichannel`, `input_unreadable`) and as SKIP rows in the report.
|
|
1218
1278
|
Run this first when a user hands over footage from a device you have not
|
|
1219
1279
|
seen before, and fix or report what fails.
|
|
1220
1280
|
|
|
@@ -1377,7 +1437,10 @@ and says so. A cue that would need more than `--max-lines` (default 2) is split
|
|
|
1377
1437
|
into consecutive cues sharing its time; a cue shorter than `--min-duration`
|
|
1378
1438
|
(default 1.0 s) is held longer, never past the next cue's start; `--offset
|
|
1379
1439
|
TIME` shifts every cue (seconds, `mm:ss`, `hh:mm:ss.ms` or `hh:mm:ss:ff`, a
|
|
1380
|
-
leading `-` for earlier) for `--text`, `--srt` and `--ass`. One `cues:` info line reports what changed
|
|
1440
|
+
leading `-` for earlier) for `--text`, `--srt` and `--ass`. One `cues:` info line reports what changed;
|
|
1441
|
+
with `--ass`, `caption.ass_shifted` / `ass_unshifted` count the Dialogue lines moved and those whose
|
|
1442
|
+
timestamps could not be read (copied as they were), and `ass_malformed` the lines that could not be
|
|
1443
|
+
read at all and are not counted as cues (each above 0 is a warning). A file you passed in is
|
|
1381
1444
|
never edited: the adjusted copy is written next to the output
|
|
1382
1445
|
(`<out>_adjusted.srt`, `<out>_offset.ass`) and burned instead — under `--dry-run`
|
|
1383
1446
|
/`--plan` the planned command names that same copy and the plan says where it
|
|
@@ -1518,7 +1581,11 @@ padded with silence, never allowed to cut the picture.
|
|
|
1518
1581
|
`--fix-drift` measures the offset again near the end of the overlap, reports
|
|
1519
1582
|
the clock difference in ppm, and resamples the source file so a 60-minute
|
|
1520
1583
|
take stays in sync (typical consumer devices drift 20-500 ppm = up to 1.8 s/h).
|
|
1521
|
-
Use it whenever the recording is longer than ~10 minutes.
|
|
1584
|
+
Use it whenever the recording is longer than ~10 minutes. A correction that
|
|
1585
|
+
could not be measured is skipped and says why: `drift_skipped_reason` is
|
|
1586
|
+
`too_short` (no tail window far enough from the head) or `low_confidence`
|
|
1587
|
+
(top level for one source, and in each `sources` entry; null when applied).
|
|
1588
|
+
Check `confidence`
|
|
1522
1589
|
(0–1, normalised correlation with a runner-up penalty); below 0.3 the match is
|
|
1523
1590
|
doubtful. Benchmark on real dialogue/music (±30 s offsets, gain, noise, EQ):
|
|
1524
1591
|
with the default 120 s window 40/40 within 10 ms (max 1.1 ms); with a 60 s
|
|
@@ -1669,13 +1736,18 @@ export.py INPUT --preset youtube|youtube4k|reels|tiktok|shorts|linkedin|facebook
|
|
|
1669
1736
|
export.py --list
|
|
1670
1737
|
```
|
|
1671
1738
|
Scales into the preset frame (pad by default), tags BT.709 (a BT.601-matrix or RGB source is converted to it, as by every re-encode), sets `+faststart`,
|
|
1672
|
-
trims to platform maximums (Reels 90 s, X 140 s) unless `--allow-long
|
|
1739
|
+
trims to platform maximums (Reels 90 s, X 140 s) unless `--allow-long`, and says
|
|
1740
|
+
so: `trimmed: {from, to}` and a `notes` line. It
|
|
1673
1741
|
does not touch levels: for youtube / youtube4k / reels / x the written file is
|
|
1674
1742
|
measured and the result's `loudness` (and a `notes` line) says when it is
|
|
1675
1743
|
outside the platform's LUFS / true-peak spec, naming the `loudness.py` call
|
|
1676
1744
|
that fixes it -- or pass `--normalize`, which runs that call on the written
|
|
1677
1745
|
file itself (audio re-encoded, video copied; `loudness.normalized: true`) so a
|
|
1678
|
-
platform export is one command instead of export, loudness, export again.
|
|
1746
|
+
platform export is one command instead of export, loudness, export again. A
|
|
1747
|
+
measurement that cannot be read is `loudness.measured: false` (values null) with
|
|
1748
|
+
a failing `{step: loudness, ok: false, reason: not_measured}` and a
|
|
1749
|
+
`loudness_not_measured` warning, so the export is `verified: false` rather than
|
|
1750
|
+
unchecked.
|
|
1679
1751
|
|
|
1680
1752
|
Since 1.14 each social destination is its own preset rather than an alias: `tiktok`
|
|
1681
1753
|
(1080x1920, max 600 s), `shorts` (1080x1920, max 180 s), `reels` (1080x1920, max 90 s),
|
|
@@ -56,15 +56,16 @@ from _common.runner import (
|
|
|
56
56
|
child_limit, _CHILDREN, _cleanup_partial_output, _cmdline, CODECS, Context, DEFAULT_TIMEOUT,
|
|
57
57
|
_DRAWTEXT_PENDING, _DRAWTEXT_TMPDIR, _drawtext_tmpdir, dry_run_input_pending, _ENCODERS, _env_timeout,
|
|
58
58
|
ERROR_CODE, ERROR_RETRYABLE, EVEN_SCALE, _execute, _fail, ffmpeg_base, ffmpeg_encoders, ffmpeg_exe, _FFMPEG_RELEASE, _FFMPEG_VERSION,
|
|
59
|
-
ffmpeg_bsfs, ffmpeg_release, ffmpeg_version, flush_drawtext_textfiles, INSTALL_HINTS, install_signal_handlers, _is_ffmpeg, _limit_for,
|
|
59
|
+
ffmpeg_bsfs, ffmpeg_list_error, ffmpeg_release, ffmpeg_version, flush_drawtext_textfiles, INSTALL_HINTS, install_signal_handlers, _is_ffmpeg, _limit_for,
|
|
60
60
|
_odd_dimension_retry, _on_signal, _OutputLock, _pid_dead, place_output, PROBE_TIMEOUT, _progress_line,
|
|
61
61
|
read_text_or_die, refuse_output_is_input, _remember_output, require_tool, run, run_analysis, _run_captured,
|
|
62
62
|
run_keeping_subtitles, run_tool, _run_with_progress, shell_quote, sibling_temp, _SIGNALS_INSTALLED, _stage_existing_output,
|
|
63
63
|
STATE, _timed_out, _unwatch, _watch, X264_PRESETS
|
|
64
64
|
)
|
|
65
65
|
from _common.emit import (
|
|
66
|
-
_brief, _BRIEF_DROP, _brief_summary, _CURRENT_CTX, die, emit, info, _plan_at_exit, _plan_inputs,
|
|
67
|
-
PLAN_VERSION, print_json, _set_current_ctx,
|
|
66
|
+
absorb_stage, audit, _brief, _BRIEF_DROP, _brief_summary, _CURRENT_CTX, die, emit, info, note, _plan_at_exit, _plan_inputs,
|
|
67
|
+
_plan_unbound_inputs, _PLAN_STRIP, PLAN_VERSION, print_json, record_fallback, record_skip, refuse, _set_current_ctx, warn,
|
|
68
|
+
write_plan
|
|
68
69
|
)
|
|
69
70
|
from _common.probe import (
|
|
70
71
|
analyze_levels, _aspect_string, _bit_depth, decode_pcm_mono, detect_scenes, detect_silences, fingerprint,
|
|
@@ -197,7 +198,7 @@ __all__ = [
|
|
|
197
198
|
"_emoji_name_candidates", "EMOJI_RANGES", "_EMOJI_REGIONAL", "emoji_support", "_EMOJI_SUPPORT_CACHE",
|
|
198
199
|
"_EMOJI_TAIL", "encoder_args", "_ENCODERS", "_env_timeout", "ERROR_CODE", "ERROR_RETRYABLE", "escape_drawtext",
|
|
199
200
|
"escape_filter_path", "EVEN_SCALE", "_execute", "_fail", "_family_rank", "FC_LANG", "_fc_list_fonts",
|
|
200
|
-
"FC_UNKNOWN", "ffmpeg_base", "ffmpeg_bsfs", "ffmpeg_encoders", "ffmpeg_exe", "_FFMPEG_RELEASE", "_FFMPEG_VERSION", "ffmpeg_release", "ffmpeg_version", "fingerprint",
|
|
201
|
+
"FC_UNKNOWN", "ffmpeg_base", "ffmpeg_bsfs", "ffmpeg_encoders", "ffmpeg_list_error", "ffmpeg_exe", "_FFMPEG_RELEASE", "_FFMPEG_VERSION", "ffmpeg_release", "ffmpeg_version", "fingerprint",
|
|
201
202
|
"flush_drawtext_textfiles", "fold_bt709_matrix", "fmt_secs", "fmt_smpte_time", "fmt_srt_time", "font_covers_script",
|
|
202
203
|
"font_family_for_script", "font_family_of_file", "FONT_FLAG_HINT", "font_for_script", "FONT_INSTALL_HINT",
|
|
203
204
|
"fonts_dir_covers_script", "_fraction", "has_emoji", "info", "INSTALL_HINTS", "install_signal_handlers",
|
|
@@ -216,6 +217,7 @@ __all__ = [
|
|
|
216
217
|
"time_arg", "_timed_out", "_to_float", "_to_int", "_unwatch", "validate_color", "verify_output",
|
|
217
218
|
"source_codec_video_args", "drawn_chain", "draws_chroma", "drawn_stream_format", "is_neutral_color", "rgb_island", "label_chain", "matrix_chain", "rgb_chain_to_bt709", "bt709_label_filter", "rgb_to_bt709_filter", "is_rgb", "video_args", "_VS15", "_VS16", "_watch", "WINDOWS_FONTS", "write_plan", "x264_args", "X264_PRESETS",
|
|
218
219
|
"_x264_raw", "_ZWJ",
|
|
220
|
+
"absorb_stage", "audit", "note", "_plan_unbound_inputs", "record_fallback", "record_skip", "refuse", "warn",
|
|
219
221
|
"ASR_ENGINES", "ASR_INSTALL_HINT", "_asr_run", "die_no_engine", "parse_srt", "transcribe",
|
|
220
222
|
"_transcribe_in", "transcribe_words", "_words_from_openai_whisper_json",
|
|
221
223
|
"_words_from_whisper_cpp_json", "whisper_word_timings", "write_srt",
|
package/scripts/_common/asr.py
CHANGED
|
@@ -16,7 +16,7 @@ from pathlib import Path
|
|
|
16
16
|
from typing import Any, Dict, List, NamedTuple, Optional, Tuple
|
|
17
17
|
|
|
18
18
|
from _common.decision import fmt_srt_time, parse_time
|
|
19
|
-
from _common.emit import die, info
|
|
19
|
+
from _common.emit import die, info, warn
|
|
20
20
|
from _common.runner import read_text_or_die
|
|
21
21
|
|
|
22
22
|
# The engines this skill knows how to drive, and how to install each -- one string, so every
|
|
@@ -296,7 +296,7 @@ def _parakeet_attempt(engines: "List[str]", facts: Dict[str, Any], wanted: str,
|
|
|
296
296
|
else:
|
|
297
297
|
run.notes.append(assumed_english_note(language_flag, whisper_failed=facts.get("routing") == ROUTING_LAST_RESORT,
|
|
298
298
|
parakeet_model=facts.get("routing") == ROUTING_PARAKEET_MODEL))
|
|
299
|
-
|
|
299
|
+
warn(run.notes[-1], "asr_language_assumed")
|
|
300
300
|
return run, os.path.basename(chosen)
|
|
301
301
|
return None
|
|
302
302
|
|