ffmpeg-skill 2.5.1 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/SKILL.md +3 -3
- package/docs/contract.md +30 -8
- package/package.json +1 -1
- package/references/gotchas.md +7 -1
- package/references/scripts.md +1 -1
- package/scripts/_common/decision.py +125 -2
- package/scripts/_common/emit.py +145 -1
- package/scripts/_common/runner.py +258 -13
- package/scripts/_contract.py +30 -5
- package/scripts/batch.py +30 -1
- package/scripts/export.py +10 -1
- package/scripts/render.py +23 -1
- package/scripts/waveform.py +19 -8
package/README.md
CHANGED
|
@@ -188,7 +188,7 @@ These are the rules the skill file gives the agent and the code enforces.
|
|
|
188
188
|
|
|
189
189
|
## Tools
|
|
190
190
|
|
|
191
|
-
42 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, `--plan FILE` (a dry run written as a plan `render.py` executes later), non-zero exit and a reason on stderr on failure. `--json-brief` (1.11.0) prints the same result document trimmed to what a caller acts on — status, output, `verified`, a compact `summary` of the output probe, the tool's own keys and the command count instead of the command lines — for roughly a third of the bytes; `--json` itself is unchanged. Every re-encoding tool takes `--codec h264|hevc|av1|prores` and `--quality N` (1.8), and every time flag takes seconds, `mm:ss`, `hh:mm:ss.fff` or SMPTE `hh:mm:ss:ff` with an optional `@fps` suffix (1.9).
|
|
191
|
+
42 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, `--plan FILE` (a dry run written as a plan `render.py` executes later), non-zero exit and a reason on stderr on failure. `--json-brief` (1.11.0) prints the same result document trimmed to what a caller acts on — status, output, `verified`, a compact `summary` of the output probe, the tool's own keys and the command count instead of the command lines — for roughly a third of the bytes; `--json` itself is unchanged. Every re-encoding tool takes `--codec h264|hevc|av1|prores` and `--quality N` (1.8) and, with `export.py`, `--hw`/`--no-hw` (Apple VideoToolbox on Apple Silicon, opt-in; see `doctor` below), and every time flag takes seconds, `mm:ss`, `hh:mm:ss.fff` or SMPTE `hh:mm:ss:ff` with an optional `@fps` suffix (1.9).
|
|
192
192
|
|
|
193
193
|
**Analysis and inspection**
|
|
194
194
|
|
|
@@ -356,7 +356,7 @@ npx ffmpeg-skill doctor --json # available / missing / missing_optional / unkn
|
|
|
356
356
|
|
|
357
357
|
`doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`. `doctor` also reports fonts: the default drawtext family, and `fonts.scripts` — one `available`/`missing`/`unknown` per writing system (ja, zh, ko, ar, he, hi, th, ru, el) with the file it would use — so "can this machine render Korean captions" is answered before the job, not after. `doctor --json`'s `tools` field folds that down to one answer per tool — `{"caption": {"usable": "no", "missing": ["filter:subtitles"], "fix": "..."}, ...}` — so "is `doctor` overall `ok`" and "can I run `caption.py` on this machine" are answered separately: a plain Homebrew `ffmpeg` is `ok` for tools that don't need `subtitles`/`drawtext`/`zscale`, while `caption`'s own `usable` is `"no"`.
|
|
358
358
|
|
|
359
|
-
`doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs).
|
|
359
|
+
`doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). It never affects `ok` or any tool's `usable`. The one GPU path a tool uses is Apple VideoToolbox, opt-in: `--hw` on every re-encoding tool (h264/hevc/prores on the GPU, AV1 stays on SVT-AV1), `FFMPEG_SKILL_HW=1` to make it this machine's default for everything except `export.py`'s delivery presets (those take `--hw` explicitly, or `render.py --hw` for a whole project). It runs only on Apple Silicon and never by default; it falls back to the CPU encoder — reported in the result's `hw.fallback`/`hw.notes` — only when VideoToolbox itself refuses a job (any other ffmpeg failure is reported as it happened, not retried); every result names the `encoder` that actually ran (`batch.py`, which encodes nothing itself: each `results` row), and a GPU encode adds a `notes` line that its quality is not CRF-equivalent. Measured on an M4 Max, 2–7× faster than x264/x265 `medium`; at matched SSIM (`tests/bench_vt.py`) 1.1–2.9× the bytes on SDR and 1.9–3.5× on HDR, which is why it is a flag and not the default (`docs/design-decisions.md`). `doctor --json`'s `hw` says whether this machine can run it (`platform_ok`) and whether it is the default here (`default_on`).
|
|
360
360
|
|
|
361
361
|
## Gotchas and best practices
|
|
362
362
|
|
package/SKILL.md
CHANGED
|
@@ -7,7 +7,7 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
|
|
|
7
7
|
|
|
8
8
|
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`, and delivery templates in `templates/`. This file is enough to do a job: the table below routes the request, and `--help` on the script about to run is the cheapest full flag list. A reference file costs as much as this one; open one only for a question you have: `references/scripts.md` (every flag of all 42 scripts), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end). MCP: `tools/list` shows only the core 12 by default; the other 30 (per this table) stay callable by name via `tools/call`, described by `contract --json`; set `FFMPEG_SKILL_MCP_FULL=1` for all 42.
|
|
9
9
|
|
|
10
|
-
Shared flags
|
|
10
|
+
Shared flags (every script): `--dry-run`; `--json` (output path, its probe, the commands run); `--json-brief` (status/output/verified + a `summary`; prefer on writing steps); `--fast` (preview quality); `--progress` (percent/ETA); `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (a dry run `render.py FILE` runs later; refuses if an input changed). Re-encoding tools also take `--hw` (Apple Silicon GPU, big files; `FFMPEG_SKILL_HW=1` = default, not export), `--codec h264|hevc|av1|prores` and `--quality N`: unset, SDR is x264, HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses HDR (`color.py --to-sdr` first).
|
|
11
11
|
|
|
12
12
|
Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) may still run ffmpeg/ffprobe, skipping artifacts/side files (`--edl`, `--sheet`, a generated `.ass`); `verify` ignores the flag. Per tool: `contract --json` `dry_run`.
|
|
13
13
|
|
|
@@ -16,7 +16,7 @@ Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `chec
|
|
|
16
16
|
0. **Environment, only on failure.** Never start a job with `doctor`: a broken machine fails with `kind: missing_tool` or an ffmpeg error naming the filter/encoder. Run `python3 <skill-dir>/scripts/_contract.py doctor` (or `npx ffmpeg-skill doctor`) after such a failure, or when asked what the machine can do: read `ok`/`usable`, report the missing capability. `contract --json`'s tool schema is for a *planning* agent, not this workflow.
|
|
17
17
|
1. **Probe what you must plan from.** Run `probe.py` on each input you plan from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks about a file. No separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output. Plan from real numbers, never assumptions.
|
|
18
18
|
2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; `--accurate` on `cut.py` only for frame-exact cuts.
|
|
19
|
-
3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use before long encodes, to report facts.
|
|
19
|
+
3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use before long encodes, to report facts. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
|
|
20
20
|
4. **Chain in a sensible order.** A delivery request with no other editing is one template run (`render.py --template NAME INPUT`), not a hand-built chain. Otherwise: colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes before captions, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18, `export.py` last. **Three or more steps: `render.py` with a project.json.**
|
|
21
21
|
5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix. Judgement rows change content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix only when the request implies the answer, else state the choice and its cost. Mention WARNs; do not chase them.
|
|
22
22
|
6. **Verify the output.** Confirm duration, resolution, fps and audio match the request, from the writing tool's `--json`/`--json-brief` probe or `probe.py`, and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure reported with the script's error message.
|
|
@@ -216,4 +216,4 @@ One line each; open the linked `references/gotchas.md` section when the job is i
|
|
|
216
216
|
- `--fit crop` 16:9→9:16 throws away 70% of the width; "60 seconds" by speed or by trim are different answers — say which and why. -> [#reframing-fps-and-duration](references/gotchas.md#reframing-fps-and-duration)
|
|
217
217
|
- TikTok/Reels cover the bottom fifth and right column with their own UI — templates keep text out of those zones; `look.py --safe tiktok` shows them. -> [#platform-safe-zones](references/gotchas.md#platform-safe-zones)
|
|
218
218
|
- `scenes.py --highlights` ranks by loudness (or duration), never by meaning: check the sheet before trusting picks. -> [#highlights](references/gotchas.md#highlights)
|
|
219
|
-
-
|
|
219
|
+
- CPU re-encodes: x264 `medium`; chain 3+ in one `render.py` project. -> [#chaining-and-speed](references/gotchas.md#chaining-and-speed)
|
package/docs/contract.md
CHANGED
|
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
|
|
|
21
21
|
| Field | Meaning | Changes when |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
|
|
24
|
-
| `skill.version` | the npm / package.json version (`2.
|
|
24
|
+
| `skill.version` | the npm / package.json version (`2.6.0`) | any release |
|
|
25
25
|
|
|
26
26
|
A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
|
|
27
27
|
ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
|
|
@@ -119,7 +119,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
|
|
|
119
119
|
"contract_version": "1.0",
|
|
120
120
|
"deprecated": [],
|
|
121
121
|
"removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
|
|
122
|
-
"skill": {"id": "ffmpeg-skill", "version": "2.
|
|
122
|
+
"skill": {"id": "ffmpeg-skill", "version": "2.6.0", "execution_mode": "local", "kind": "execution",
|
|
123
123
|
"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
|
|
124
124
|
"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
|
|
125
125
|
"requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
|
|
@@ -216,6 +216,15 @@ Main10 for HDR), so the flags add no behaviour to a caller that does not pass th
|
|
|
216
216
|
encoder each value needs is listed under the tool's optional capabilities (`--codec hevc` and
|
|
217
217
|
so on). `export.py` refuses `--codec`: its presets decide the codec.
|
|
218
218
|
|
|
219
|
+
`--hw` / `--no-hw` are on the same tools and on `export.py`, marked `common`;
|
|
220
|
+
`render.py` and `batch.py` take them for every stage they run. They are two booleans in the
|
|
221
|
+
input schema (`hw`, `no_hw`; MCP sends `--hw` for `hw: true` and `--no-hw` for `no_hw: true`),
|
|
222
|
+
mutually exclusive. `--hw` moves an h264/hevc/prores encode to Apple VideoToolbox (Apple Silicon
|
|
223
|
+
only; AV1 stays on the CPU); `--no-hw` keeps it on the CPU even when `FFMPEG_SKILL_HW=1` makes the
|
|
224
|
+
GPU this machine's default. Neither changes anything for a caller that passes neither on a
|
|
225
|
+
machine without the variable. What ran is in the result's `encoder` and `hw` (see the keys added
|
|
226
|
+
after 2.2.2, and `doctor`'s `gpu_encoders` below).
|
|
227
|
+
|
|
219
228
|
`--plan FILE` (1.6) is a dry run that also writes a plan document: `{"plan_version": 1,
|
|
220
229
|
"tool", "argv", "cwd", "inputs": [{"path", "size", "sha256_head_tail"}], "commands",
|
|
221
230
|
"output", "verify": [{"tool": "probe"}, {"tool": "check", "platform"}], "notes"}`. It
|
|
@@ -237,11 +246,17 @@ gives you:
|
|
|
237
246
|
|
|
238
247
|
| Hint | Tools |
|
|
239
248
|
|---|---|
|
|
240
|
-
| `bit_exact` | probe, check, scenes, look |
|
|
249
|
+
| `bit_exact` | probe, check, scenes, look, metadata, background |
|
|
241
250
|
| `content_equivalent` (same media, bytes may differ between encoder builds) | every encoding tool, cut, sync, report |
|
|
242
251
|
| `cached` | batch (content-hash cache, re-runs skip unchanged inputs); render **when `--cache DIR` is given** — the hint stays `content_equivalent` because that is what render is without the flag, and the cache is opt-in |
|
|
243
252
|
| `environment_dependent` | verify |
|
|
244
253
|
|
|
254
|
+
The hints describe the CPU encoders. Under `--hw`, or `FFMPEG_SKILL_HW=1` on the machine, a tool
|
|
255
|
+
that encodes video runs VideoToolbox, whose bytes are not promised identical from run to run:
|
|
256
|
+
treat such a result as `content_equivalent` at best. That includes `background`, whose hint stays
|
|
257
|
+
`bit_exact` within 2.x. The variable belongs to the machine, so the same flags can give x264 bytes
|
|
258
|
+
on one machine and VideoToolbox bytes on another; `encoder` in the result says which.
|
|
259
|
+
|
|
245
260
|
## `provides`
|
|
246
261
|
|
|
247
262
|
`provides` lists these 42 tools by a cross-repository Capability id, for
|
|
@@ -339,7 +354,8 @@ Names: `ffmpeg`, `ffprobe`, `encoder:<name>`, `filter:<name>`, `bsf:<name>`,
|
|
|
339
354
|
`available`, `missing` and `missing_optional` are added from `doctor`, which reads
|
|
340
355
|
`ffmpeg -encoders / -filters / -bsfs` and looks for a local whisper. Pass `--static`
|
|
341
356
|
to omit detection. Nothing from the environment other than those lists and the
|
|
342
|
-
ffmpeg/ffprobe/python versions is printed; no environment variables, no paths
|
|
357
|
+
ffmpeg/ffprobe/python versions is printed; no environment variables, no paths (`doctor`'s
|
|
358
|
+
`hw.default_on` is a boolean derived from `FFMPEG_SKILL_HW`, never its value).
|
|
343
359
|
|
|
344
360
|
`external:whisper` is **optional** for two tools since 1.17: `caption.py`
|
|
345
361
|
(`--transcribe`) and `silence.py` (`--filler --transcribe`). Neither requires
|
|
@@ -380,9 +396,14 @@ for tools that don't need `subtitles`/`drawtext`/`zscale`, but `caption.usable`
|
|
|
380
396
|
"parsed"|"unparsed"|"failed"|"missing", "present": [...]}`. It proves the build shipped the
|
|
381
397
|
capability, not that the GPU/driver on this machine will accept a job (that needs a real
|
|
382
398
|
encode, which this introspection never runs). No tool declares or requires a GPU encoder, so
|
|
383
|
-
`gpu_encoders` never affects `ok` or any tool's `usable`
|
|
384
|
-
|
|
385
|
-
|
|
399
|
+
`gpu_encoders` never affects `ok` or any tool's `usable`. `--hw` (and
|
|
400
|
+
`FFMPEG_SKILL_HW=1`, which `export.py`'s delivery presets ignore) puts h264/hevc/prores encodes on
|
|
401
|
+
VideoToolbox on Apple Silicon only, never by default. A job VideoToolbox refuses (the encoder could
|
|
402
|
+
not open, or its compression session failed) is re-encoded on the CPU and said so in the result's
|
|
403
|
+
`hw.fallback`/`hw.notes`; any other failure is not retried. When `FFMPEG_SKILL_HW=1` rather than `--hw` chose the GPU and the GPU ran,
|
|
404
|
+
`hw.notes` also says so, with the file-size cost and `--no-hw` as the remedy for a final
|
|
405
|
+
deliverable. `doctor`'s `hw` field is `{platform_ok, default_on}` — normalised
|
|
406
|
+
booleans, never the environment value.
|
|
386
407
|
|
|
387
408
|
`doctor`'s `fonts` field reports whether the default drawtext font (`caption.py`'s
|
|
388
409
|
`--animate`/`--karaoke`, `graphics.py`'s templates — `BRAND_DEFAULTS["font"]`, `"DejaVu Sans"`)
|
|
@@ -493,7 +514,7 @@ Per-tool keys added in 1.17, all additive:
|
|
|
493
514
|
| `filler`, `removed_seconds_total` | `silence.py --filler` | `{lang, source, engine, words, removed, removed_count, removed_seconds, removed_words, word_timings, list, warnings}`. The existing `removed_seconds` is unchanged in name and meaning — the seconds of *silence* removed, which is what it has always held — and `removed_seconds_total` is the additive sibling covering silence plus filler |
|
|
494
515
|
| `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
|
|
495
516
|
| `timeline` | `render.py --export-timeline FILE` | `{format, rate, duration, frames, clips, transition, transition_frames, music, markers, notes, not_exported}`: the written timeline's own numbers (`rate` is exact, `"30000/1001"` for 29.97) and what the project asked for that an editor timeline cannot carry. Nothing is encoded (`commands: []`); `verified` means the file was written and reads back as its format (`verification` step `parse`) -- not that an editor opened it |
|
|
496
|
-
| `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
|
|
517
|
+
| `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …), the GPU setting (`FFMPEG_SKILL_HW` and whether `--hw`/`--no-hw` made it explicit) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
|
|
497
518
|
|
|
498
519
|
Per-tool keys added in 1.17.1, all additive:
|
|
499
520
|
|
|
@@ -524,6 +545,7 @@ Per-tool keys added after 2.2.2, all additive:
|
|
|
524
545
|
| `silent` | `join.py` (every run) | `[{index, path, peak_db, at, end}]`: inputs whose whole-file audio peak (volumedetect) is at or below `--silence-threshold` (default -50 dBFS) and that `--on-silent warn` (default) joined anyway, with a `notes` line; `at`/`end` are where the input sits in the output, in seconds (a transition overlaps its neighbours). An input named by `--allow-silent N[,N]` (1-based) is a planned pause: always joined, listed with `intended: true`, no warning or note, whatever `--on-silent` says. `[]` otherwise. `--on-silent fail` names them in the `kind: input` refusal's `problems` (reason `silent (peak -91.0 dBFS)`), `skip` lists them under `skipped`. An input with no audio stream is never silent; `verified` is unaffected |
|
|
525
546
|
| `short_segments`, `duplicates` | `join.py` (every run) | warnings, never refusals, and the join command is unchanged: `short_segments: [{index, path, duration}]` names each measured input shorter than 2 frames at the join's fps (an audio-only join: shorter than 0.05 s); `duplicates: [{path, indices}]` names a path (compared resolved) listed more than once — repeating a clip can be intended. `[]` when none; each non-empty one adds a `notes` line |
|
|
526
547
|
| `silent` | `waveform.py` (every run) | `true` when the input audio's whole-file peak is at or below `--silence-threshold` (default -50 dBFS), which draws a flat line; `--on-silent warn` (default) renders it anyway with a `notes` line, `fail` refuses (`kind: input`) before ffmpeg runs. `false` when audible, `null` under `--dry-run` (nothing measured). `verified` is unaffected |
|
|
548
|
+
| `encoder`, `hw`, `notes` | every re-encoding tool, `export.py`, `render.py`, `batch.py` (and its rows), `waveform.py` | `encoder`: the video encoder the last command ran (after any GPU→CPU fallback; under `--dry-run`, the planned one), `copy` for a stream copy; absent when no command encoded video. A tool that runs other tools reports its stages': `render.py` its stages and an executed plan's tool, `waveform.py` the title/caption stage that writes its file (its `commands` include that stage's). `hw` is present when VideoToolbox was asked for (`--hw`, `FFMPEG_SKILL_HW=1`, or a stage or `batch.py` item that reported it) and on `export.py` when `FFMPEG_SKILL_HW=1` is set but its delivery preset does not take it: `{requested, source: flag\|env, used, fallback, notes}`. `requested` says whether this run's encodes were asked onto VideoToolbox (`false` for that export), `used` whether the encoder that ran last is VideoToolbox (`null` under `--dry-run`, which ran nothing, and when the tool encoded nothing itself: `batch.py`, whose top-level `hw` gathers its items' `requested`/`source`/`fallback`/`notes` while each `results` row carries the item's own `encoder`, and `hw`/`notes` when VideoToolbox was asked for), `fallback` whether a job VideoToolbox refused was re-encoded on the CPU, and `notes` why. Only a VideoToolbox failure (the encoder could not open, or its compression session failed) is retried on the CPU; any other failure is reported as it happened, once. Every VideoToolbox encode adds a top-level `notes` line: its quality is not CRF-equivalent (`--quality` / a preset's CRF maps to `-q:v` approximately), and `verified` stays what it is for every tool, the measured properties of the output. A `kind: ffmpeg` failure under `--hw` / `FFMPEG_SKILL_HW` carries `encoder` and `hw` for the command that ran last (a stage's, when `render.py`/`waveform.py` re-raise it); when the command itself failed inside the run (not a tool's own check afterwards, such as `cut.py`'s), its message also names that encoder |
|
|
527
549
|
|
|
528
550
|
`caption.py --transcribe` and `silence.py --filler --transcribe`: when a local speech engine ran
|
|
529
551
|
and produced no cue, the refusal is `"<engine> found no speech in <input>"` with `kind: input`,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.6.0",
|
|
4
4
|
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ffmpeg",
|
package/references/gotchas.md
CHANGED
|
@@ -267,7 +267,13 @@ the contact sheet (`--sheet`) before treating the picks as final.
|
|
|
267
267
|
Anything chained by hand through three re-encodes should be one `render.py`
|
|
268
268
|
project instead, so the plan is one file and the user can change one number.
|
|
269
269
|
Re-encodes use x264 `medium`; for long files add `--preset veryfast` to
|
|
270
|
-
intermediates and keep the default for the final export.
|
|
270
|
+
intermediates and keep the default for the final export. On Apple Silicon,
|
|
271
|
+
`--hw` (or `FFMPEG_SKILL_HW=1` for the machine) moves h264/hevc/prores encodes to
|
|
272
|
+
VideoToolbox: 2–7× faster, 1.1–2.9× the bytes at the same quality (1.9–3.5× on HDR). The variable
|
|
273
|
+
leaves `export.py`'s delivery presets on the CPU, which is the usual split (GPU
|
|
274
|
+
intermediates, CPU delivery); `render.py --hw` puts the export on the GPU too.
|
|
275
|
+
Check `encoder`/`hw` in the result: a job the GPU refuses (H.264 wider than 4096)
|
|
276
|
+
was re-encoded on the CPU.
|
|
271
277
|
|
|
272
278
|
## Audio-only files
|
|
273
279
|
|
package/references/scripts.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Script reference
|
|
2
2
|
|
|
3
|
-
Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither
|
|
3
|
+
Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, before anything runs) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
4
4
|
|
|
5
5
|
## Time grammar (every time-taking flag, 1.9)
|
|
6
6
|
|
|
@@ -16,7 +16,7 @@ from pathlib import Path
|
|
|
16
16
|
from typing import Any, Dict, List, Optional, Sequence, Tuple
|
|
17
17
|
from _common.color import bt709_tag_args, _sdr_bt709
|
|
18
18
|
from _common.emit import die
|
|
19
|
-
from _common.runner import CODECS, STATE, ffmpeg_encoders
|
|
19
|
+
from _common.runner import CODECS, STATE, ffmpeg_encoders, ffmpeg_version, hw_platform_reason
|
|
20
20
|
|
|
21
21
|
|
|
22
22
|
def pad_filters(out_w: int, out_h: int, fill: str, color: str, blur: int, darken: float = 0.0) -> str:
|
|
@@ -277,7 +277,126 @@ def cfr_args(meta: Optional[Dict[str, Any]], fps: Optional[float] = None) -> Lis
|
|
|
277
277
|
return ["-fps_mode", "cfr", "-r", f"{rate:g}"]
|
|
278
278
|
|
|
279
279
|
|
|
280
|
+
# ------------------------------------------------------- VideoToolbox (--hw)
|
|
281
|
+
#
|
|
282
|
+
# CRF -> VideoToolbox -q:v (1-100, higher = better): at each CRF the highest -q:v whose SSIM matched
|
|
283
|
+
# the CPU line's (x264/x265 medium) across the clips of tests/bench_vt.py -- synthetic CG, fractal
|
|
284
|
+
# and grain plus iPhone 17 Pro 1080p (M3, FFmpeg 9.0.2), so a GPU encode never measures below the
|
|
285
|
+
# CPU one. HDR has its own curve: its CPU line is x265 Main10 at CRF+2, and the Main10 VideoToolbox
|
|
286
|
+
# line needs ~11-15 less -q:v for that SSIM (the SDR curve wrote HLG phone footage at 6-8x x265's
|
|
287
|
+
# bytes). Matched quality costs 1.1-2.9x the bytes of x264/x265 on SDR and 1.9-3.5x on HDR --
|
|
288
|
+
# which is why --hw is opt-in and delivery presets stay on the CPU (docs/design-decisions.md).
|
|
289
|
+
|
|
290
|
+
|
|
291
|
+
def vt_quality(codec: str, crf: int, hdr: bool = False) -> int:
|
|
292
|
+
base, slope = (63.0, 1.6) if hdr else (75.0, 2.2) if codec == "h264" else (78.0, 2.0)
|
|
293
|
+
return max(1, min(100, int(round(base - slope * (crf - 18)))))
|
|
294
|
+
|
|
295
|
+
|
|
296
|
+
def _vt_bt709(codec: str) -> List[str]:
|
|
297
|
+
"""BT.709 tags for a VideoToolbox SDR encode. From FFmpeg 7.1 the -colorspace output options
|
|
298
|
+
drive a real matrix conversion on an untagged source (see bt709_tag_args), and VideoToolbox has
|
|
299
|
+
no -x264-params VUI route; a bitstream filter writes the VUI after encoding, which the filter
|
|
300
|
+
graph never sees (measured: 49.9 dB PSNR tag-neutral vs 23.8 dB through the output options).
|
|
301
|
+
It is used on every version, not only >= 7.1: it is tag-only everywhere, and a git build of
|
|
302
|
+
7.1 reads as (7, 0) from its libavutil major, which would otherwise take the converting path."""
|
|
303
|
+
return ["-bsf:v", f"{codec}_metadata=colour_primaries=1:transfer_characteristics=1:matrix_coefficients=1"]
|
|
304
|
+
|
|
305
|
+
|
|
306
|
+
def _vt_args(codec: str, crf: int, meta: Optional[Dict[str, Any]], keep_bt709: bool) -> Optional[List[str]]:
|
|
307
|
+
"""VideoToolbox encoder options for `codec`, or None (with a note in STATE.hw_notes) when the
|
|
308
|
+
request stays on the CPU: no VideoToolbox encoder for the codec, this machine cannot, or the
|
|
309
|
+
ffmpeg build lacks it."""
|
|
310
|
+
name = {"h264": "h264_videotoolbox", "hevc": "hevc_videotoolbox", "prores": "prores_videotoolbox"}.get(codec)
|
|
311
|
+
reason = None if name else f"no VideoToolbox encoder for {codec}"
|
|
312
|
+
reason = reason or hw_platform_reason()
|
|
313
|
+
if not reason and name not in ffmpeg_encoders():
|
|
314
|
+
reason = f"this ffmpeg build has no {name}"
|
|
315
|
+
if reason:
|
|
316
|
+
note = f"{reason}; encoded on the CPU"
|
|
317
|
+
if note not in STATE.hw_notes:
|
|
318
|
+
STATE.hw_notes.append(note)
|
|
319
|
+
return None
|
|
320
|
+
v = (meta or {}).get("video") or {}
|
|
321
|
+
hdr = bool(v.get("bt2020_or_hdr"))
|
|
322
|
+
hdr_tags = ["-colorspace", v.get("color_space") or "bt2020nc", "-color_primaries", v.get("color_primaries") or "bt2020",
|
|
323
|
+
"-color_trc", v.get("color_transfer") or "arib-std-b67"]
|
|
324
|
+
if codec == "prores":
|
|
325
|
+
return ["-c:v", name, "-profile:v", "hq", "-pix_fmt", "p210le"] + (hdr_tags if hdr else [])
|
|
326
|
+
# The HDR curve was fitted on the Main10 HEVC line; an 8-bit H.264 line (an export preset on
|
|
327
|
+
# an HDR source) replaces x264 at the same CRF and keeps the H.264 curve.
|
|
328
|
+
q = str(vt_quality(codec, crf, hdr and codec == "hevc"))
|
|
329
|
+
if codec == "h264":
|
|
330
|
+
return ["-c:v", name, "-q:v", q, "-profile:v", "high", "-pix_fmt", "yuv420p", "-movflags", "+faststart"] + \
|
|
331
|
+
(_vt_bt709("h264") if keep_bt709 else [])
|
|
332
|
+
if hdr:
|
|
333
|
+
# Measured on FFmpeg 9.0: HDR10 mastering-display and content-light side data carry through
|
|
334
|
+
# hevc_videotoolbox exactly as through libx265 (test_hw_hdr10_side_data_survives).
|
|
335
|
+
return ["-c:v", name, "-q:v", q, "-profile:v", "main10", "-pix_fmt", "p010le", "-tag:v", "hvc1"] + hdr_tags + ["-movflags", "+faststart"]
|
|
336
|
+
return ["-c:v", name, "-q:v", q, "-pix_fmt", "yuv420p", "-tag:v", "hvc1", "-movflags", "+faststart"] + \
|
|
337
|
+
(_vt_bt709("hevc") if keep_bt709 else [])
|
|
338
|
+
|
|
339
|
+
|
|
340
|
+
def _maybe_hw(codec: str, crf: int, meta: Optional[Dict[str, Any]], keep_bt709: bool, cpu: List[str]) -> List[str]:
|
|
341
|
+
"""`cpu` unchanged unless --hw / $FFMPEG_SKILL_HW is on and VideoToolbox can take `codec`; the
|
|
342
|
+
swap is remembered so run() can put the CPU line back if the GPU refuses the job."""
|
|
343
|
+
if not STATE.hw:
|
|
344
|
+
return cpu
|
|
345
|
+
vt = _vt_args(codec, crf, meta, keep_bt709)
|
|
346
|
+
if vt is None:
|
|
347
|
+
return cpu
|
|
348
|
+
# a copy: a caller that edits the returned line in place (export.py strips -movflags) must
|
|
349
|
+
# not edit the recorded slice with it, or restate_last_swap() never sees the line as built
|
|
350
|
+
STATE.hw_swaps.append((list(vt), cpu))
|
|
351
|
+
return vt
|
|
352
|
+
|
|
353
|
+
|
|
354
|
+
_CPU_TO_CODEC = {"libx264": "h264", "libx265": "hevc", "prores_ks": "prores"}
|
|
355
|
+
|
|
356
|
+
|
|
357
|
+
def hw_preset_video(video: List[str], meta: Optional[Dict[str, Any]], *, bt709: bool) -> List[str]:
|
|
358
|
+
"""export.py's fixed-preset encoder line moved to VideoToolbox (same codec, the preset's -crf
|
|
359
|
+
mapped to -q:v, its -r kept); unchanged -- with the reason noted -- when that cannot apply.
|
|
360
|
+
|
|
361
|
+
`bt709`: whether export.py tags this preset's CPU line BT.709 (every fixed preset but prores,
|
|
362
|
+
whose master keeps the source's tags). The VideoToolbox line writes the format the CPU preset
|
|
363
|
+
writes: an h264/hevc preset is 8-bit with BT.709 tags whatever the source (on an HDR source it
|
|
364
|
+
says so in a note, as without --hw), so it is built as for an SDR source -- not the Main10 HDR
|
|
365
|
+
line, which would make the GPU file and its CPU fallback different formats."""
|
|
366
|
+
enc = video[video.index("-c:v") + 1] if "-c:v" in video else None
|
|
367
|
+
codec = _CPU_TO_CODEC.get(enc or "")
|
|
368
|
+
if codec is None:
|
|
369
|
+
return video
|
|
370
|
+
crf = int(video[video.index("-crf") + 1]) if "-crf" in video else 18
|
|
371
|
+
vt = _vt_args(codec, crf, meta if codec == "prores" else None, bt709)
|
|
372
|
+
if vt is None:
|
|
373
|
+
return video
|
|
374
|
+
vt = strip_movflags(vt) # export adds it
|
|
375
|
+
if "-r" in video:
|
|
376
|
+
vt += ["-r", video[video.index("-r") + 1]]
|
|
377
|
+
# the CPU line export would have written: the preset's own options plus the BT.709 tags export
|
|
378
|
+
# adds to it (only to the CPU line), so a GPU->CPU fallback is tagged exactly like a CPU run
|
|
379
|
+
STATE.hw_swaps.append((list(vt), list(video) + (bt709_tag_args(enc) if bt709 else [])))
|
|
380
|
+
return vt
|
|
381
|
+
|
|
382
|
+
|
|
383
|
+
def strip_movflags(args: List[str]) -> List[str]:
|
|
384
|
+
return [a for i, a in enumerate(args) if not (a == "-movflags" or (i and args[i - 1] == "-movflags"))]
|
|
385
|
+
|
|
386
|
+
|
|
387
|
+
def restate_last_swap(before: List[str], after: List[str]) -> None:
|
|
388
|
+
"""A caller that edits an encoder line after encoder_args() returned it (export.py strips
|
|
389
|
+
-movflags) restates the recorded swap, so run()'s GPU->CPU fallback still finds the slice."""
|
|
390
|
+
if STATE.hw_swaps and STATE.hw_swaps[-1][0] == before:
|
|
391
|
+
STATE.hw_swaps[-1] = (list(after), strip_movflags(STATE.hw_swaps[-1][1]))
|
|
392
|
+
|
|
393
|
+
|
|
280
394
|
def encoder_args(codec: str, crf: int, preset: str, meta: Optional[Dict[str, Any]] = None, keep_bt709: bool = True) -> List[str]:
|
|
395
|
+
"""encoder options for `codec` (_cpu_encoder_args), on VideoToolbox when --hw asks and it can."""
|
|
396
|
+
return _maybe_hw(codec, crf, meta, keep_bt709, _cpu_encoder_args(codec, crf, preset, meta, keep_bt709))
|
|
397
|
+
|
|
398
|
+
|
|
399
|
+
def _cpu_encoder_args(codec: str, crf: int, preset: str, meta: Optional[Dict[str, Any]] = None, keep_bt709: bool = True) -> List[str]:
|
|
281
400
|
"""The one place that turns (--codec, --quality, --preset, source) into encoder options.
|
|
282
401
|
|
|
283
402
|
h264 -> x264 8-bit BT.709 (refuses HDR: 8-bit H.264 cannot carry it); hevc -> x265, Main10
|
|
@@ -340,7 +459,7 @@ def x264_args(crf: int = 18, preset: str = "medium", keep_bt709: bool = True) ->
|
|
|
340
459
|
(color.py's --to-sdr path builds its own H.264 line; the flag still has to reach it)."""
|
|
341
460
|
if STATE.codec and STATE.codec != "h264":
|
|
342
461
|
return encoder_args(STATE.codec, crf, preset, None, keep_bt709)
|
|
343
|
-
return _x264_raw(crf, preset, keep_bt709)
|
|
462
|
+
return _maybe_hw("h264", crf, None, keep_bt709, _x264_raw(crf, preset, keep_bt709))
|
|
344
463
|
|
|
345
464
|
|
|
346
465
|
def video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "medium") -> List[str]:
|
|
@@ -356,6 +475,10 @@ def video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "med
|
|
|
356
475
|
v = (meta or {}).get("video") or {}
|
|
357
476
|
if not v.get("bt2020_or_hdr"):
|
|
358
477
|
return x264_args(crf, preset)
|
|
478
|
+
return _maybe_hw("hevc", crf, meta, True, _x265_hdr_default(v, crf, preset))
|
|
479
|
+
|
|
480
|
+
|
|
481
|
+
def _x265_hdr_default(v: Dict[str, Any], crf: int, preset: str) -> List[str]:
|
|
359
482
|
cs = v.get("color_space") or "bt2020nc"
|
|
360
483
|
prim = v.get("color_primaries") or "bt2020"
|
|
361
484
|
trc = v.get("color_transfer") or "arib-std-b67"
|
package/scripts/_common/emit.py
CHANGED
|
@@ -9,7 +9,7 @@ import json
|
|
|
9
9
|
import math
|
|
10
10
|
import os
|
|
11
11
|
import sys
|
|
12
|
-
from typing import Any, Dict, List, Optional, Sequence
|
|
12
|
+
from typing import Any, Dict, List, Optional, Sequence, Tuple
|
|
13
13
|
from _common.runner import Context, ERROR_CODE, ERROR_RETRYABLE, STATE
|
|
14
14
|
|
|
15
15
|
|
|
@@ -28,6 +28,14 @@ def die(msg: str, code: int = 1, kind: str = "input", *, ctx: "Optional[Context]
|
|
|
28
28
|
ctx.plan = None # a failed run plans nothing (the exit hook must not write a plan for it)
|
|
29
29
|
STATE.plan = None # the hook falls back to STATE when nothing passed a ctx; a failed run plans nothing there either
|
|
30
30
|
sys.stderr.write(f"error: {msg}\n" + (f"hint: {hint}\n" if hint else ""))
|
|
31
|
+
if kind == "ffmpeg" and "hw" not in extra and (ctx.hw or ctx.hw_stages or ctx.hw_env_ignored):
|
|
32
|
+
# a tool's own ffmpeg failure (cut.py runs run(check=False) and dies itself; render.py
|
|
33
|
+
# re-raises a stage's) carries `encoder` and `hw` as run()'s _fail() does: under --hw /
|
|
34
|
+
# $FFMPEG_SKILL_HW two encoders may have run, and a caller must know whose failure it is
|
|
35
|
+
enc = extra.get("encoder") or _video_encoders(ctx.commands)[0]
|
|
36
|
+
if enc and "encoder" not in extra:
|
|
37
|
+
extra["encoder"] = enc
|
|
38
|
+
extra["hw"] = hw_report(ctx, enc)
|
|
31
39
|
if ctx.json:
|
|
32
40
|
doc: Dict[str, Any] = {
|
|
33
41
|
"status": "failed", "exit_code": code,
|
|
@@ -63,6 +71,138 @@ def _set_current_ctx(ctx: "Context") -> None:
|
|
|
63
71
|
_CURRENT_CTX = ctx
|
|
64
72
|
|
|
65
73
|
|
|
74
|
+
ENV_HW_NOTE = ("VideoToolbox chosen by FFMPEG_SKILL_HW=1 (machine default; ~1.1-3.5x the bytes of x264/x265 "
|
|
75
|
+
"at matched quality); rerun with --no-hw for a final deliverable")
|
|
76
|
+
ENV_NOT_FOR_DELIVERY_NOTE = ("FFMPEG_SKILL_HW=1 is not applied to export.py's delivery presets (a delivered file is where "
|
|
77
|
+
"VideoToolbox's larger files cost most); pass --hw to put this export on the GPU")
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
def _video_encoders(commands: Sequence[str]) -> "Tuple[Optional[str], Optional[str]]":
|
|
81
|
+
"""(the last video encoder the command lines name, or `copy` for a stream copy; that encoder's
|
|
82
|
+
command line). A later copy (loudness.py after export's encode) keeps the encode before it."""
|
|
83
|
+
import re as _re
|
|
84
|
+
enc, line, copied = None, None, False
|
|
85
|
+
for c in commands:
|
|
86
|
+
for name in _re.findall(r"(?:^|\s)-(?:c:v|vcodec|codec:v)\s+(\S+)", c):
|
|
87
|
+
if name == "copy":
|
|
88
|
+
copied = True
|
|
89
|
+
else:
|
|
90
|
+
enc, line = name, c
|
|
91
|
+
if _re.search(r"(?:^|\s)-(?:c|codec)\s+copy(?:\s|$)", c):
|
|
92
|
+
copied = True
|
|
93
|
+
if enc is None and copied:
|
|
94
|
+
return "copy", None
|
|
95
|
+
return enc, line
|
|
96
|
+
|
|
97
|
+
|
|
98
|
+
def hw_report(ctx: "Context", enc: Optional[str]) -> Dict[str, Any]:
|
|
99
|
+
"""The `hw` object for a run whose last video encoder was `enc` (None: this process recorded
|
|
100
|
+
no encode -- batch.py, whose stages are child processes). `requested`: VideoToolbox was asked
|
|
101
|
+
for this run's encodes (by --hw, by $FFMPEG_SKILL_HW where it applies, or for a stage);
|
|
102
|
+
`source`: flag|env; `used`: the encoder that ran last is VideoToolbox; `fallback`: a job
|
|
103
|
+
VideoToolbox refused was re-encoded on the CPU, here or in a stage; `notes`: why. Under
|
|
104
|
+
--dry-run `used` is null and `fallback` false: nothing ran."""
|
|
105
|
+
stages = list(ctx.hw_stages)
|
|
106
|
+
requested = bool(ctx.hw) or any(st.get("requested") for st in stages)
|
|
107
|
+
source = ctx.hw_source if ctx.hw else next((st.get("source") for st in stages if st.get("requested")), None)
|
|
108
|
+
if not requested and ctx.hw_env_ignored:
|
|
109
|
+
source = "env"
|
|
110
|
+
# a stage that reported its source without asking (export.py under the variable: requested
|
|
111
|
+
# false, source env) still names where the request came from
|
|
112
|
+
source = source or next((st.get("source") for st in stages if st.get("source")), None)
|
|
113
|
+
vt = enc is not None and enc.endswith("_videotoolbox")
|
|
114
|
+
# `used` is what ran; a dry run ran nothing (the planned encoder is `encoder`), so null --
|
|
115
|
+
# as waveform.py's `silent` is null when nothing was measured
|
|
116
|
+
used = None if enc is None or ctx.dry_run else vt
|
|
117
|
+
notes = list(ctx.hw_notes)
|
|
118
|
+
if vt and source == "env" and not any(n.endswith(ENV_HW_NOTE) for n in notes):
|
|
119
|
+
# the machine default, not this call, chose the GPU: say what it costs and how to opt out
|
|
120
|
+
# (once: a stage that ran on it already said so)
|
|
121
|
+
notes.append(ENV_HW_NOTE)
|
|
122
|
+
if ctx.hw_env_ignored and enc not in (None, "copy"):
|
|
123
|
+
notes.append(ENV_NOT_FOR_DELIVERY_NOTE)
|
|
124
|
+
return {"requested": requested, "source": source, "used": used,
|
|
125
|
+
"fallback": bool(ctx.hw_fallback) or any(st.get("fallback") for st in stages), "notes": notes}
|
|
126
|
+
|
|
127
|
+
|
|
128
|
+
def hw_quality_note(enc: Optional[str], line: Optional[str]) -> Optional[str]:
|
|
129
|
+
"""The `notes` line every VideoToolbox encode carries: its quality is not the CRF encode's."""
|
|
130
|
+
import re as _re
|
|
131
|
+
if not enc or not enc.endswith("_videotoolbox"):
|
|
132
|
+
return None
|
|
133
|
+
if enc == "prores_videotoolbox":
|
|
134
|
+
return ("prores_videotoolbox is not prores_ks: the same 422 HQ profile from a different encoder, so size and detail "
|
|
135
|
+
"differ from the CPU encode; `verified` covers the output's measured properties, not parity with the CPU "
|
|
136
|
+
"encode; --no-hw for prores_ks")
|
|
137
|
+
m = _re.search(r"(?:^|\s)-q:v\s+(\d+)", line or "")
|
|
138
|
+
return (f"{enc}{' -q:v ' + m.group(1) if m else ''} is not CRF-equivalent: --quality (or the preset's CRF) is mapped "
|
|
139
|
+
"to -q:v by an approximate SSIM fit (docs/design-decisions.md), so size and detail differ from the x264/x265 "
|
|
140
|
+
"encode (~1.1-3.5x the bytes at matched SSIM); `verified` covers the output's measured properties, not quality "
|
|
141
|
+
"parity with the CPU encode; --no-hw for the CRF encode")
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
def _encoder_report(ctx: "Context") -> Dict[str, Any]:
|
|
145
|
+
"""`encoder`: the last video encoder this run's ffmpeg commands name (after any GPU->CPU
|
|
146
|
+
fallback, since run() rewrites the recorded command), `copy` for a stream copy; absent when no
|
|
147
|
+
command encoded video. `hw` (hw_report): present whenever VideoToolbox was asked for (--hw or
|
|
148
|
+
$FFMPEG_SKILL_HW, a stage's, or the variable a delivery preset does not take)."""
|
|
149
|
+
enc, _line = _video_encoders(ctx.commands)
|
|
150
|
+
out: Dict[str, Any] = {"encoder": enc} if enc else {}
|
|
151
|
+
if ctx.hw or ctx.hw_stages or ctx.hw_env_ignored:
|
|
152
|
+
out["hw"] = hw_report(ctx, enc)
|
|
153
|
+
return out
|
|
154
|
+
|
|
155
|
+
|
|
156
|
+
def _hw_notes_for(ctx: "Context") -> List[str]:
|
|
157
|
+
"""Top-level `notes` lines for the GPU encodes this result stands on: this tool's last encode,
|
|
158
|
+
and (render.py) every stage that ran on VideoToolbox."""
|
|
159
|
+
enc, line = _video_encoders(ctx.commands)
|
|
160
|
+
out: List[str] = []
|
|
161
|
+
for note in [hw_quality_note(enc, line)] + [st.get("quality_note") for st in ctx.hw_stages]:
|
|
162
|
+
if note and note not in out:
|
|
163
|
+
out.append(note)
|
|
164
|
+
return out
|
|
165
|
+
|
|
166
|
+
|
|
167
|
+
def absorb_stage_hw(stage: str, doc: Any, ctx: "Optional[Context]" = None) -> None:
|
|
168
|
+
"""render.py runs each stage as a child process: carry the stage's `hw` facts (fell back, and
|
|
169
|
+
why; ran on VideoToolbox, with its quality note) into this result, which records only the
|
|
170
|
+
command lines."""
|
|
171
|
+
ctx = ctx or STATE
|
|
172
|
+
hw = doc.get("hw") if isinstance(doc, dict) else None
|
|
173
|
+
if not isinstance(hw, dict):
|
|
174
|
+
return
|
|
175
|
+
for n in hw.get("notes") or []:
|
|
176
|
+
# once: this process may have said it already (waveform.py's own encode), or another stage
|
|
177
|
+
if n not in ctx.hw_notes and not any(existing.endswith(": " + n) for existing in ctx.hw_notes):
|
|
178
|
+
ctx.hw_notes.append(f"{stage}: {n}")
|
|
179
|
+
enc, line = _video_encoders([str(c) for c in (doc.get("commands") or [])])
|
|
180
|
+
quality = hw_quality_note(enc, line) if hw.get("used") else None
|
|
181
|
+
ctx.hw_stages.append({"stage": stage, "requested": bool(hw.get("requested")), "source": hw.get("source"),
|
|
182
|
+
"used": hw.get("used"), "fallback": bool(hw.get("fallback")), "quality_note": quality})
|
|
183
|
+
|
|
184
|
+
|
|
185
|
+
def steps_encoder_report(steps: Sequence["Tuple[str, Any]"]) -> Dict[str, Any]:
|
|
186
|
+
"""batch.py's per-item `encoder`, `hw` and `notes`, from the --json documents of the steps
|
|
187
|
+
(child processes) that made the item: the last encoder a step ran (a later stream copy keeps
|
|
188
|
+
the encode before it), and their GPU facts as render.py's result carries its stages'."""
|
|
189
|
+
ctx = Context()
|
|
190
|
+
ctx.dry_run = STATE.dry_run
|
|
191
|
+
enc: Optional[str] = None
|
|
192
|
+
for name, doc in steps:
|
|
193
|
+
step_enc = doc.get("encoder") if isinstance(doc, dict) else None
|
|
194
|
+
if step_enc and (step_enc != "copy" or enc is None):
|
|
195
|
+
enc = step_enc
|
|
196
|
+
absorb_stage_hw(name, doc, ctx)
|
|
197
|
+
out: Dict[str, Any] = {"encoder": enc} if enc else {}
|
|
198
|
+
if ctx.hw_stages:
|
|
199
|
+
out["hw"] = hw_report(ctx, enc)
|
|
200
|
+
quality = [st["quality_note"] for st in ctx.hw_stages if st.get("quality_note")]
|
|
201
|
+
if quality:
|
|
202
|
+
out["notes"] = list(dict.fromkeys(quality))
|
|
203
|
+
return out
|
|
204
|
+
|
|
205
|
+
|
|
66
206
|
def emit(output: Optional[str], *, ctx: "Optional[Context]" = None, **extra: Any) -> None:
|
|
67
207
|
"""Final stdout line: the output path, or a JSON document with --json.
|
|
68
208
|
|
|
@@ -88,7 +228,11 @@ def emit(output: Optional[str], *, ctx: "Optional[Context]" = None, **extra: Any
|
|
|
88
228
|
steps.insert(0, {"step": "exists", "ok": True})
|
|
89
229
|
doc["verified"] = not ctx.dry_run and bool(steps) and all(s.get("ok") for s in steps)
|
|
90
230
|
doc["verification"] = steps
|
|
231
|
+
doc.update(_encoder_report(ctx))
|
|
91
232
|
doc.update(extra)
|
|
233
|
+
hw_notes = [n for n in _hw_notes_for(ctx) if n not in (doc.get("notes") or [])]
|
|
234
|
+
if hw_notes:
|
|
235
|
+
doc["notes"] = list(doc.get("notes") or []) + hw_notes
|
|
92
236
|
if ctx.plan:
|
|
93
237
|
doc["plan"] = write_plan(ctx.plan, output, extra, ctx=ctx)
|
|
94
238
|
print_json(_brief(doc, meta) if ctx.json_brief else doc)
|