ffmpeg-skill 2.11.0 → 2.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/docs/contract.md +69 -13
- package/mcp/server.py +7 -1
- package/package.json +1 -1
- package/references/scripts.md +79 -14
- package/scripts/_common/__init__.py +2 -2
- package/scripts/_common/asr.py +2 -2
- package/scripts/_common/decision.py +57 -11
- package/scripts/_common/emit.py +49 -5
- package/scripts/_common/fonts.py +93 -28
- package/scripts/_common/runner.py +112 -37
- package/scripts/_common/timeline.py +22 -6
- package/scripts/_contract.py +122 -12
- package/scripts/audio.py +2 -1
- package/scripts/batch.py +240 -105
- package/scripts/caption.py +68 -18
- package/scripts/color.py +2 -1
- package/scripts/cropdetect.py +19 -4
- package/scripts/export.py +42 -10
- package/scripts/graphics.py +24 -18
- package/scripts/grid.py +7 -3
- package/scripts/join.py +5 -3
- package/scripts/loop.py +4 -2
- package/scripts/multicam.py +11 -3
- package/scripts/overlay.py +14 -10
- package/scripts/render.py +133 -44
- package/scripts/report.py +77 -23
- package/scripts/reverse.py +4 -2
- package/scripts/sequence.py +4 -2
- package/scripts/silence.py +19 -7
- package/scripts/sync.py +18 -7
- package/scripts/verify.py +98 -24
- package/scripts/waveform.py +3 -2
package/README.md
CHANGED
|
@@ -389,7 +389,7 @@ The short list for humans. The agent-facing version, with the reasoning, is the
|
|
|
389
389
|
- **Silence detection finds nothing?** The default threshold is −35 dBFS. The tool prints a hint with the track's measured level; raise the threshold (`silence.py --threshold -25`) or shorten `--min-silence`.
|
|
390
390
|
- **Sync results carry a confidence.** Below 0.3, or an offset near the edge of the analysis window, is probably wrong: enlarge `--analyze-seconds` or find a clap. Recordings over ten minutes from separate devices need `sync.py --fix-drift`.
|
|
391
391
|
- **Outputs are never overwritten silently.** An existing output path is refused (`kind: input`, `error.reason: output_exists`, before anything runs); pass `--overwrite` where a replacement is intended.
|
|
392
|
-
- **Nothing happens silently.** A refusal names its `error.reason` and `error.details`; a stale output lock taken over, a run without a lock
|
|
392
|
+
- **Nothing happens silently.** A refusal names its `error.reason` and `error.details`; a stale output lock taken over, a run without a lock, a plan input nobody could fingerprint, a fallback (an odd-sized frame scaled to even, a substituted font, libaom for SVT-AV1), a skipped step and every warning are recorded in `diagnostics` and on stderr (codes: `json_output.diagnostic_codes`); tools that draw text name the font they drew with (`font_used`). A refused plan and a lock takeover also append a line to an audit log (`<plan>.audit.jsonl`, or `render.py --audit-log FILE`). A degraded or partial result says so: `render.py`'s `verified` is every stage's, an unmeasured loudness or a report section that could not be built is a failing verification step, `export.py` reports `trimmed`, `multicam.py`'s `cuts` name the camera rendered, and `batch.py` rows carry the failing step's `error`, `verified`, and a row for every input after Ctrl-C.
|
|
393
393
|
- **Long chains belong in a plan.** Three hand-chained re-encodes lose quality and are hard to change; `render.py` runs the whole edit from one JSON file, and `--dry-run` shows every ffmpeg command before anything is written.
|
|
394
394
|
|
|
395
395
|
## FAQ
|
package/docs/contract.md
CHANGED
|
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
|
|
|
21
21
|
| Field | Meaning | Changes when |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
|
|
24
|
-
| `skill.version` | the npm / package.json version (`2.
|
|
24
|
+
| `skill.version` | the npm / package.json version (`2.12.0`) | any release |
|
|
25
25
|
|
|
26
26
|
A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
|
|
27
27
|
ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
|
|
@@ -125,7 +125,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
|
|
|
125
125
|
"contract_version": "1.0",
|
|
126
126
|
"deprecated": [{"what": "running on Python 3.9 (the requirements.python floor)", "since": "2.10.9", "replacement": "Python 3.10 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
|
|
127
127
|
"removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
|
|
128
|
-
"skill": {"id": "ffmpeg-skill", "version": "2.
|
|
128
|
+
"skill": {"id": "ffmpeg-skill", "version": "2.12.0", "execution_mode": "local", "kind": "execution",
|
|
129
129
|
"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
|
|
130
130
|
"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
|
|
131
131
|
"requirements": {"python": ">=3.9 (standard library only; 3.9 deprecated since 2.10.9, removed in 3.0.0; 3.10+ recommended)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
|
|
@@ -376,7 +376,9 @@ limit (`check.py` reports it instead). `_platforms.resolve()` is the one alias m
|
|
|
376
376
|
New in the same release, all additive: `export.py --preset tiktok|shorts|linkedin|facebook`
|
|
377
377
|
(real presets, not aliases of `reels`/`youtube`), `--preset youtube-hdr` (HEVC Main10 keeping
|
|
378
378
|
the source's HDR tags; `kind: input` on an SDR source) and `--preset youtube-av1`
|
|
379
|
-
(`kind: missing_tool` when the build has neither SVT-AV1 nor
|
|
379
|
+
(`kind: missing_tool`, `error.reason: encoder_missing`, when the build has neither SVT-AV1 nor
|
|
380
|
+
libaom; `kind: ffmpeg`, `encoder_list_unreadable`, when its encoder list could not be read at all,
|
|
381
|
+
which says nothing about the build); `caption.py --platform`
|
|
380
382
|
and `graphics.py --platform` / `--margin` and `overlay.py --platform` (margins from the safe
|
|
381
383
|
zone, an explicit `--margin`/`--position` wins); `look.py --safe NAME`; `fit.py --fit blur`; `report.py --pack`;
|
|
382
384
|
`graphics.py --template sticker|hook|meme`; and `render.py --template NAME INPUT`
|
|
@@ -546,7 +548,7 @@ the script, `missing` means fontconfig knows none, `unknown` means there is no w
|
|
|
546
548
|
ask (no `fc-list` on PATH, or it failed). `caption.py`, `graphics.py` and `overlay.py --text` resolve
|
|
547
549
|
a font by script automatically and fail with `kind: input` rather than render boxes **only for
|
|
548
550
|
`missing`**: `unknown` is not `missing` here any more than anywhere else in this document — the job
|
|
549
|
-
runs with the font as given and
|
|
551
|
+
runs with the font as given and a `font_coverage_unverified` warning says the coverage could not be verified. So a
|
|
550
552
|
`missing` script here is a job that will not run until a font is installed — but, like the default
|
|
551
553
|
font, it never affects `ok` or any tool's
|
|
552
554
|
`usable` (the tool works, this machine just has no glyphs for that language). The plain-text
|
|
@@ -634,7 +636,7 @@ Per-tool keys added in 1.17, all additive:
|
|
|
634
636
|
| `filler`, `removed_seconds_total` | `silence.py --filler` | `{lang, source, engine, words, removed, removed_count, removed_seconds, removed_words, word_timings, list, warnings}`. The existing `removed_seconds` is unchanged in name and meaning — the seconds of *silence* removed, which is what it has always held — and `removed_seconds_total` is the additive sibling covering silence plus filler |
|
|
635
637
|
| `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
|
|
636
638
|
| `timeline` | `render.py --export-timeline FILE` | `{format, rate, duration, frames, clips, transition, transition_frames, music, markers, notes, not_exported}`: the written timeline's own numbers (`rate` is exact, `"30000/1001"` for 29.97) and what the project asked for that an editor timeline cannot carry. Nothing is encoded (`commands: []`); `verified` means the file was written and reads back as its format (`verification` step `parse`) -- not that an editor opened it |
|
|
637
|
-
| `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …), the GPU setting (`FFMPEG_SKILL_HW` and whether `--hw`/`--no-hw` made it explicit) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
|
|
639
|
+
| `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, miss_reasons, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. `miss_reasons` (unreleased, additive): `[{stage, reason, ...}]`, `reason` one of `not_cached`, `sidecar_missing`, `artifact_missing`, `sidecar_corrupt`, `artifact_truncated`, `no_output_path`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …), the GPU setting (`FFMPEG_SKILL_HW` and whether `--hw`/`--no-hw` made it explicit) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
|
|
638
640
|
|
|
639
641
|
Per-tool keys added in 1.17.1, all additive:
|
|
640
642
|
|
|
@@ -676,6 +678,16 @@ Per-tool keys added after 2.2.2, all additive:
|
|
|
676
678
|
| `vfr_check` | `cut.py` (a video input that could still stream-copy) | the sampled frame timing, `{heuristic: true, measured, method: "sampled", guard, windows, deltas}`; `measured` is `sampled_cfr`, `vfr` or `inconclusive`. `--vfr-guard average` (the default) keeps the 2.x guard: a source whose nominal and average frame rates differ (`variable_frame_rate_suspected`) re-encodes as `--accurate` (reason `vfr`, stderr says "variable-frame-rate"), and only then is the timing sampled and reported, with a `notes` line naming `--vfr-guard sampled` when it measured `sampled_cfr`; the key is absent otherwise. `--vfr-guard sampled` samples before every copy and re-encodes on `vfr` (reason `vfr`) or `inconclusive` (fewer than 20 intervals; `vfr_inconclusive`); `--vfr-guard off` samples and keeps the copy, with a `notes` line. Absent for audio-only input or output, `--accurate`, `--codec` and a pending `--dry-run` input |
|
|
677
679
|
| `join_check`, `segment_end_snap_seconds` | `cut.py --segments` | `join_check`: a video stream-copy join measured by demuxing, `{packets, expected_packets, max_step_seconds, audio_offset_ms, audio_parts_checked, ok}`. `expected_packets` counts the source's video packets from each part's start keyframe to its end (or the end of the video); `max_step_seconds` is the largest presentation step, `null` when `vfr_check` measured anything but `sampled_cfr` (a `--vfr-guard off` join of variable timing: count only), and each step at a join must be the step its parts predict within 1 ms. `audio_offset_ms`: per part, its sound's shift minus its picture's in ms (+ = sound late), from audio packets matched by CRC32 to the source stream shaped like the joined audio (FFmpeg's default selection may copy one other than `a:0`; an MPEG-TS source's AAC is hashed without its ADTS headers); `null` for a part with no unique match (digital silence, a periodic tone), and as a whole when the source has no audio or the video check failed. `audio_parts_checked` counts the measured parts. `ok` is the conjunction (count, steps, every measured part within 5 ms); `ok: false` means the join was re-cut (`concat_fallback`) and this is the failed measurement. `null` for a single segment, audio, `--dry-run`, any re-encode forced for every segment, a segment that re-encoded, or a join re-cut before a copy was tried (open GOPs, a later B-frame segment starting between keyframes, parts that differ, a concat error). `segment_end_snap_seconds`: per segment, the signed seconds its end moved to the keyframe the copied part stops at, the first at or after the requested end (`null` for a part that reaches the end of the video, a part of a source with no B-frames, cut to its requested end as in 2.5.1, or a Matroska/MPEG-TS part, which is not snapped); `null` as a whole unless the output is those copied parts |
|
|
678
680
|
|
|
681
|
+
Per-tool keys added after 2.10.11 ("nothing happens silently"), all additive:
|
|
682
|
+
|
|
683
|
+
| key | tool | what it holds |
|
|
684
|
+
|---|---|---|
|
|
685
|
+
| `font_used` | `overlay.py --text`, `graphics.py`, `caption.py` burn mode, `grid.py --label auto` | the font the text was drawn with: `{asked, file, family, how, substituted, via?, fonts_dir?}`. `how`: `font_file` (the caller's file), `script` (picked to cover the text's script), `resolved` (the asked family resolved to a file by `fc-match` or the Windows Fonts folder; for libass, the file fontconfig picks for the family), `by_name` (no file resolved: ffmpeg or libass look the family up themselves, `file` null), `fonts_dir` (`caption.py`: the family is looked up in `--fonts-dir` first), `ass_styles` (`caption.py --ass`: the file's own styles name the fonts). `substituted: true` -- the file is not the asked family (a Windows machine without it gets a CJK system font or Arial; fontconfig answers every name) -- is also a `font_substituted` fallback |
|
|
686
|
+
| `windows`, `windows_failed` | `cropdetect.py` | the sampled windows and how many ffmpeg could not decode (skipped; a `cropdetect_windows_failed` warning when above 0). Every window failing stays a `kind: ffmpeg` failure |
|
|
687
|
+
| `drift_skipped_reason` | `sync.py --fix-drift` (top level for one SOURCE, and in each `sources` entry) | why the drift correction was not applied: `too_short` (no tail window far enough from the head) or `low_confidence` (the tail did not correlate); `null` when it was. Also a `drift_correction_skipped` entry under `diagnostics.skipped` |
|
|
688
|
+
| `ass_shifted`, `ass_unshifted`, `ass_malformed` | `caption.py --ass` (inside `caption`) | with `--offset`: the Dialogue lines moved, and those whose timestamps could not be read (copied as they were); every run: Dialogue lines that could not be read (too few fields, an unreadable start/end) and are not counted as cues. A count above 0 is a warning |
|
|
689
|
+
| `matched_untimed`, `matched_too_long` | `silence.py --filler` (inside `filler`) | listed words that were not cut: no usable timing in the transcript (also a `filler_words_untimed` warning), or longer than a filler (`FILLER_MAX_WORD`, a held vowel) |
|
|
690
|
+
|
|
679
691
|
`caption.py --transcribe` and `silence.py --filler --transcribe`: when a local speech engine ran
|
|
680
692
|
and produced no cue, the refusal is `"<engine> found no speech in <input>"` with `kind: input`,
|
|
681
693
|
`reason: "no_speech"` and `engine` in the failure document — distinct from the "no local
|
|
@@ -741,8 +753,9 @@ here claims otherwise until real exit-code/stderr sniffing exists to back that u
|
|
|
741
753
|
|
|
742
754
|
A refusal also carries `error.reason`, a stable snake_case code for *why*
|
|
743
755
|
(`contract --json` → `json_output.error_reasons` lists them: `output_exists`, `output_is_input`,
|
|
744
|
-
`output_is_directory`, `output_dir_missing`, `output_dir_not_writable`, `output_locked`,
|
|
745
|
-
`render.py`'s `plan_*` reasons
|
|
756
|
+
`output_is_directory`, `output_dir_missing`, `output_dir_not_writable`, `output_locked`,
|
|
757
|
+
`render.py`'s `plan_*` reasons, and `--codec av1`/`prores`'s `encoder_missing` and
|
|
758
|
+
`encoder_list_unreadable`), and `error.details`, the facts the message is written from, so a
|
|
746
759
|
caller never parses the prose:
|
|
747
760
|
|
|
748
761
|
```
|
|
@@ -759,15 +772,53 @@ A tool that runs another (`render.py` stages and plans) forwards the child's `re
|
|
|
759
772
|
skipped -- is recorded as it happens (a stderr line too) and lands in the document as
|
|
760
773
|
`diagnostics: {warnings, fallbacks, skipped}`, each entry `{code, message, ...facts}`; it is
|
|
761
774
|
present on success and failure documents alike, only when one list is not empty, and an entry a
|
|
762
|
-
`render.py` stage reported carries `stage`.
|
|
763
|
-
|
|
764
|
-
(
|
|
775
|
+
`render.py` stage reported carries `stage`. `contract --json` → `json_output.diagnostic_codes`
|
|
776
|
+
lists every code with its bucket and meaning (a test reads every `warn()`/`record_fallback()`/
|
|
777
|
+
`record_skip()` call against it). Among them: the runner's `odd_dimensions_scaled_even` (an encode
|
|
778
|
+
re-run with an even scale; the frame loses a row or column), `non_av_streams_dropped` (with the
|
|
779
|
+
first attempt's error beside `dropped_non_av_streams`), `measurement_input_pending` (skipped: a
|
|
780
|
+
dry-run measurement of an earlier stage's output), `ffmpeg_list_unreadable` (`ffmpeg -encoders` or
|
|
781
|
+
`-bsfs` could not be read; what needed it names the read failure, not a missing encoder) and
|
|
782
|
+
`timeout_env_invalid` (an `FFMPEG_SKILL_TIMEOUT` that is not a number, or is negative);
|
|
783
|
+
`av1_encoder_libaom` and `crf_clamped`; `font_substituted` (with `font_used`); `drift_correction_skipped`
|
|
784
|
+
(with `drift_skipped_reason`); `cropdetect_windows_failed` (with `windows_failed`);
|
|
785
|
+
`ass_lines_unshifted` / `ass_lines_malformed`; `timeline_parse_failed`; and every `warning:` line a
|
|
786
|
+
tool used to print to stderr only (emoji fallbacks, silent or short inputs, memory-heavy jobs,
|
|
787
|
+
`--strip-dovi` without Dolby Vision, `render.py`'s frame-from-preset and `--normalize` defaults).
|
|
788
|
+
`render.py`'s plan, check and cache records (`check_dry_run`, `check_output_unparsed`, `cache_entry_damaged`, ...),
|
|
789
|
+
`export.py`'s `loudness_not_measured`, `loudness_outside_spec` and `colour_not_converted`, `multicam.py`'s
|
|
790
|
+
`camera_not_started`, `report.py`'s `report_section_missing` and `batch.py`'s cache records
|
|
791
|
+
(`batch_cache_reset`, `batch_cache_output_changed`, ...) are listed there too. Every one of them
|
|
792
|
+
still prints its stderr line; the record is additive.
|
|
793
|
+
|
|
794
|
+
**A result that is not the one asked for says so.** `verified` is the conjunction of every
|
|
795
|
+
`verification` step, and a tool that runs others counts theirs: `render.py` lists each stage's own
|
|
796
|
+
steps beyond its intermediate's probe, tagged with `stage` (a cached stage's as the cache recorded
|
|
797
|
+
them), and its `notes` carry each stage's notes prefixed with the stage's tool, so an export that
|
|
798
|
+
missed its platform loudness leaves the render `verified: false`. A step that could not be
|
|
799
|
+
measured is a failing step with a `reason`, never an absent one: `export.py`'s unreadable loudness
|
|
800
|
+
is `{step: loudness, ok: false, reason: not_measured}` with `loudness.measured: false`, and
|
|
801
|
+
`report.py` lists each section it built (`loudness`, `compliance` with `passed`,
|
|
802
|
+
`contact_sheet`), one it could not with `ok: false` and the child's `reason`/`message`. A
|
|
803
|
+
deliberate change of result is a field: `export.py`'s `trimmed: {from, to}` when a platform
|
|
804
|
+
maximum cut the clip, `multicam.py`'s `cuts` naming the camera rendered (camera 0 where the one
|
|
805
|
+
asked for had not started, with a `camera_not_started` fallback). `batch.py` keeps `ok` as "every
|
|
806
|
+
step completed" (a spec miss is `completed` + `verified: false`, not a failure) and adds `verified`
|
|
807
|
+
and `unverified: [{step, tool, failed}]` to each row, one `item` verification step per row, a
|
|
808
|
+
failed row's `error: {step, tool, kind, message, code, reason, details, hint, exit_code}` from the
|
|
809
|
+
failing step's own document, and `cache_miss` when a cache entry was not served because its output
|
|
810
|
+
changed or could not be checked; an interrupted batch has a row for every input
|
|
811
|
+
(`interrupted: true`, or `skipped: "interrupted"` when it had not started) and `total` counts them
|
|
812
|
+
all. `verify.py`'s steps keep the tool's `kind`, `reason`, `message` and `exit_code`, and the steps
|
|
813
|
+
that do not apply are listed under each file's `skipped` with a reason.
|
|
765
814
|
|
|
766
815
|
**Audit log.** Events a later reader must be able to reconstruct are appended as JSON lines,
|
|
767
816
|
`{ts, tool, event, pid, plan_sha256, details}` (`plan_sha256` on plan events): `plan_rejected`,
|
|
768
|
-
`lock_takeover`, `output_unlocked`,
|
|
817
|
+
`lock_takeover`, `output_unlocked`, `output_is_input_refused` when a log was named,
|
|
818
|
+
`cache_entry_damaged` (`render.py --cache`) and `batch_cache_reset`. The file
|
|
769
819
|
is `render.py --audit-log FILE` when given; otherwise a refused plan's `<plan>.audit.jsonl` next
|
|
770
|
-
to the plan,
|
|
820
|
+
to the plan, a lock event's hidden `.<output>.ffskill-audit.jsonl` beside the output, and a cache
|
|
821
|
+
event's `.ffskill-audit.jsonl` in the cache directory or the batch's `output_dir`. Nothing
|
|
771
822
|
is appended under `--dry-run`. A record that cannot be written is a `diagnostics.warnings` entry
|
|
772
823
|
(`audit_log_unwritable`) and never fails or blocks the run; the refusal's `details.audit_log`
|
|
773
824
|
says `{path, written}` either way.
|
|
@@ -844,7 +895,12 @@ returns one user-role text message naming the actual command lines to run and wh
|
|
|
844
895
|
afterward -- the same facts SKILL.md's own request table states, addressed by workflow instead of
|
|
845
896
|
by request phrasing. A missing required argument is a JSON-RPC error (`prompts/get` never guesses
|
|
846
897
|
a path or a platform); an unknown prompt name is a JSON-RPC error too, distinct from "method not
|
|
847
|
-
found" since `prompts/get` itself is a real method.
|
|
898
|
+
found" since `prompts/get` itself is a real method.
|
|
899
|
+
|
|
900
|
+
A line on stdin that is not JSON is answered with a JSON-RPC parse error (`-32700`, `id: null`)
|
|
901
|
+
rather than dropped, so a client whose request was mangled does not wait for an answer that
|
|
902
|
+
never comes; a JSON value that is not an object, and a notification, still get no response.
|
|
903
|
+
An unknown method is `-32601`, any other handler error `-32000`. These are recipes, not new tool calls: every
|
|
848
904
|
command line a prompt names is one `tools/call` (or the CLI) can already run.
|
|
849
905
|
|
|
850
906
|
## Consuming the contract from an agent
|
package/mcp/server.py
CHANGED
|
@@ -247,7 +247,13 @@ def main() -> int:
|
|
|
247
247
|
continue
|
|
248
248
|
try:
|
|
249
249
|
req = json.loads(line)
|
|
250
|
-
except ValueError:
|
|
250
|
+
except ValueError as exc:
|
|
251
|
+
# JSON-RPC 2.0: a line that is not JSON is answered with -32700 and id null, not
|
|
252
|
+
# dropped -- a client whose request was mangled in transit otherwise waits for an
|
|
253
|
+
# answer that never comes, with nothing to say why
|
|
254
|
+
resp = {"jsonrpc": "2.0", "id": None, "error": {"code": -32700, "message": f"parse error: {exc}"}}
|
|
255
|
+
stdout.write((json.dumps(resp) + "\n").encode("utf-8"))
|
|
256
|
+
stdout.flush()
|
|
251
257
|
continue
|
|
252
258
|
# json.loads accepts any valid JSON value, not just an object -- a bare `42`, `null`,
|
|
253
259
|
# `true` or `[1,2]` line parses fine but isn't a JSON-RPC request. Without this guard,
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.12.0",
|
|
4
4
|
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ffmpeg",
|
package/references/scripts.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Script reference
|
|
2
2
|
|
|
3
|
-
Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, `error.reason: output_exists`, before anything runs) to replace an existing output unless `--overwrite` is given. Refusals carry `error.reason` and `error.details` (the facts behind the message; `contract --json` → `json_output.error_reasons`), and what a run decided on its own -- a stale output lock taken over, no lock possible, a plan input nobody fingerprinted -- is in `diagnostics: {warnings, fallbacks, skipped}` as well as on stderr. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), `overlay` measures a multi-line `--text`'s lines on FFmpeg 8.0, 8.0.1 and 8.1.0 (one drawtext on a generated frame), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
3
|
+
Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every re-encoding tool and `export.py` also take `--hw`/`--no-hw` (h264/hevc/prores on Apple VideoToolbox, Apple Silicon only, CRF mapped to `-q:v`; AV1 stays on the CPU; a job VideoToolbox itself refuses -- the encoder cannot open or its compression session fails -- is re-encoded on the CPU and reported in `hw.fallback`/`hw.notes`, any other failure is not retried; a GPU encode adds a `notes` line that its quality is not CRF-equivalent; `FFMPEG_SKILL_HW=1` makes it the default for every tool except `export.py`, whose delivery presets need `--hw` itself; `render.py`/`batch.py --hw`/`--no-hw` apply to every stage, export included; results name the `encoder` that ran); every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, default 18 -- 30 for `proxy.py`; up to 63 for av1; ignored by prores). 2.0 removed `--crf` from these tools (it was `--quality`'s alias); `export.py` keeps its own `--crf`, which is not an alias. Every tool refuses (`kind: input`, `error.reason: output_exists`, before anything runs) to replace an existing output unless `--overwrite` is given. Refusals carry `error.reason` and `error.details` (the facts behind the message; `contract --json` → `json_output.error_reasons`), and what a run decided on its own -- a stale output lock taken over, no lock possible, a plan input nobody fingerprinted, an odd-sized frame scaled to even, subtitle streams dropped on a retry (with the first attempt's error), a font substituted, a measurement skipped under `--dry-run`, an encoder list that could not be read, an invalid `FFMPEG_SKILL_TIMEOUT`, and every `warning:` line -- is in `diagnostics: {warnings, fallbacks, skipped}` as well as on stderr (codes: `contract --json` → `json_output.diagnostic_codes`). `--codec av1`/`prores` with no encoder for it refuses `kind: missing_tool`, `error.reason: encoder_missing`; when the encoder list could not be read at all, `kind: ffmpeg`, `encoder_list_unreadable`. Tools that draw text (`overlay.py --text`, `graphics.py`, `caption.py`, `grid.py --label auto`) report `font_used: {asked, file, family, how, substituted}`. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither `--codec` nor `--quality`; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize`/`join` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), `overlay` measures a multi-line `--text`'s lines on FFmpeg 8.0, 8.0.1 and 8.1.0 (one drawtext on a generated frame), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
4
4
|
|
|
5
5
|
## Time grammar (every time-taking flag, 1.9)
|
|
6
6
|
|
|
@@ -251,7 +251,10 @@ itself with no black-bar measurement involved. A rectangle that matches the
|
|
|
251
251
|
full source frame means no bars were found. Does not decide whether
|
|
252
252
|
removing detected bars is wanted -- genuine letterboxed content (a
|
|
253
253
|
scope-ratio film in a 16:9 frame) "detects" the same way as accidental
|
|
254
|
-
bars; look at the frame before cropping it away.
|
|
254
|
+
bars; look at the frame before cropping it away. `windows` / `windows_failed`
|
|
255
|
+
say how many windows were sampled and how many ffmpeg could not decode (those
|
|
256
|
+
are skipped, with a `cropdetect_windows_failed` warning; all of them failing is
|
|
257
|
+
a `kind: ffmpeg` failure).
|
|
255
258
|
|
|
256
259
|
`--motion-centre` (1.18) reports the motion centroid once per second, sampled
|
|
257
260
|
across the same windows: `motion_centre: [{time, x, y, x_frac, y_frac, motion}]`,
|
|
@@ -595,6 +598,9 @@ per-word `start`/`end`) or `--transcribe` (the same local engine `caption.py`
|
|
|
595
598
|
uses, never required). There is no heuristic fallback — finding an "um" by
|
|
596
599
|
looking for short quiet blips would cut real speech, so the tool refuses
|
|
597
600
|
instead. `--filler-list` reports what would go and writes nothing.
|
|
601
|
+
`filler.matched_untimed` counts listed words the transcript gives no usable
|
|
602
|
+
timing for (not removed, and a `filler_words_untimed` warning), and
|
|
603
|
+
`matched_too_long` those longer than a filler (a held vowel, kept).
|
|
598
604
|
|
|
599
605
|
Built-in lists, one per language:
|
|
600
606
|
|
|
@@ -748,6 +754,15 @@ step for the black/frozen/silence rows), and reports `plan`, `plan_sha256`, `too
|
|
|
748
754
|
`tool_result`, `check` and `unbound_inputs` (each also a `diagnostics.warnings`
|
|
749
755
|
entry: nothing checked those files). Show the plan to the user, get the yes, execute:
|
|
750
756
|
one round trip instead of re-deriving the command.
|
|
757
|
+
Render's `verified` is every stage's: each stage's own verification steps (beyond
|
|
758
|
+
its intermediate's probe) are in the result's `verification`, tagged with `stage`,
|
|
759
|
+
and each stage's notes are in `notes`, prefixed with its tool -- an export that
|
|
760
|
+
misses its platform loudness leaves the render `completed` with `verified: false`
|
|
761
|
+
and the note naming the fix. `"check": {"platform": ...}`, `{"content": true}` or
|
|
762
|
+
both run `check.py` after the last stage (content alone: the black/frozen/silence
|
|
763
|
+
rows, youtube's spec assumed, as `check.py --content` without `--platform`); under
|
|
764
|
+
`--dry-run` the check is a `check_dry_run` entry in `diagnostics.skipped`.
|
|
765
|
+
|
|
751
766
|
`"export": {"preset": "reels", "normalize": true}` forwards `export.py --normalize`
|
|
752
767
|
so the rendered file meets the platform's loudness without a separate pass. Since
|
|
753
768
|
1.9.0 it is on by default when the preset is a platform (`youtube|youtube4k|reels|x`)
|
|
@@ -808,7 +823,14 @@ artifact the old one produced. Each entry is `DIR/<key><ext>` plus a
|
|
|
808
823
|
`DIR/<key>.json` sidecar (stage, versions, creation time, size, seconds). A hit
|
|
809
824
|
hardlinks the artifact into the work directory, or copies it where the
|
|
810
825
|
filesystem will not link — never moves it, since the cache has to outlive the
|
|
811
|
-
run's own cleanup.
|
|
826
|
+
run's own cleanup. A miss is work to do, not an error, but never a silent one:
|
|
827
|
+
`cache.miss_reasons` says why each stage missed (`not_cached`, `sidecar_missing`,
|
|
828
|
+
`artifact_missing`, `sidecar_corrupt`, `artifact_truncated`, `no_output_path`), and a
|
|
829
|
+
damaged entry (the last two of those that name a file this tool wrote) is also a
|
|
830
|
+
`cache_entry_damaged` warning and a line in `DIR/.ffskill-audit.jsonl`. A sidecar keeps
|
|
831
|
+
the stage's `verified`, `verification`, `notes` and `diagnostics`, and a hit replays
|
|
832
|
+
them, so a cached stage counts toward the render's `verified` as a live one does. A
|
|
833
|
+
store that fails is a `cache_store_failed` warning (the run is unaffected).
|
|
812
834
|
|
|
813
835
|
A stage whose flags changed misses the cache, so editing the captions block (for
|
|
814
836
|
example to `"fit_size": "off"`) re-runs captions and everything after it and
|
|
@@ -817,7 +839,7 @@ serves the earlier stages from the cache.
|
|
|
817
839
|
`stages_done` is unchanged: a cached stage is still a stage that happened.
|
|
818
840
|
Nothing is written under `--dry-run`, which instead reports `cache.would_hit`.
|
|
819
841
|
Result: `cache.dir`, `cache.ffmpeg`, `cache.hits`, `cache.misses`,
|
|
820
|
-
`cache.saved_seconds`, `cache.entries`. `--from STAGE` starts at that stage and
|
|
842
|
+
`cache.miss_reasons`, `cache.saved_seconds`, `cache.entries`. `--from STAGE` starts at that stage and
|
|
821
843
|
takes every earlier one from the cache; without `--cache`, or when an earlier
|
|
822
844
|
stage is not there, it refuses (`kind: input`) rather than quietly re-encoding
|
|
823
845
|
what it promised to skip. An unwritable cache directory is `kind: output`.
|
|
@@ -1047,10 +1069,28 @@ above the cap is clamped with a note, not refused, and both `jobs` and
|
|
|
1047
1069
|
the table in file order when the cache is only partially warm — and each item's
|
|
1048
1070
|
log lines are buffered and flushed in file order, so the per-item table and the
|
|
1049
1071
|
log read exactly as a serial run's whatever order the encodes finish in.
|
|
1050
|
-
- **A failed item is a row, not a dead run**: a
|
|
1051
|
-
`ok: false` with
|
|
1052
|
-
|
|
1053
|
-
`
|
|
1072
|
+
- **A failed item is a row, not a dead run**: a step that fails makes its row
|
|
1073
|
+
`ok: false` with `error: {step, tool, kind, message, code, reason, details,
|
|
1074
|
+
hint, exit_code}` taken from that step's own JSON document (a worker that raised:
|
|
1075
|
+
`exception` and its message), and the summary and the table are still printed.
|
|
1076
|
+
A recipe error that is the same for every item (a step script outside
|
|
1077
|
+
`scripts/`, no steps) is refused before any item runs.
|
|
1078
|
+
- **`ok` is "every step completed"; `verified` is beside it**: a step that
|
|
1079
|
+
completed with `verified: false` (an export's loudness miss) leaves the row
|
|
1080
|
+
`ok: true`, `verified: false`, with `unverified: [{step, tool, failed}]`; the
|
|
1081
|
+
batch's own `verification` has one `item` step per row and a note names the
|
|
1082
|
+
unverified items. A row also carries its steps' `diagnostics` (tagged with `step`).
|
|
1083
|
+
- **Ctrl-C** (or SIGTERM) stops the running steps and exits 130 (143)
|
|
1084
|
+
`kind: interrupted` with a row for every input: finished, `interrupted: true`
|
|
1085
|
+
(stopped while running) or `"skipped": "interrupted"` (not started), so `total`
|
|
1086
|
+
is the folder's count.
|
|
1087
|
+
- **The cache serves an output only while it is the file the batch wrote**: an
|
|
1088
|
+
entry keeps the output's size and mtime, and an output replaced or edited since
|
|
1089
|
+
is processed again (`cache_miss: {reason: output_changed, expected, actual}` on
|
|
1090
|
+
the row, a `batch_cache_output_changed` warning; an entry from 2.10.11 or
|
|
1091
|
+
earlier has no such record and is processed again once, `entry_unverifiable`).
|
|
1092
|
+
A `.ffskill_cache.json` that does not parse is started empty with a
|
|
1093
|
+
`batch_cache_reset` warning and a line in `output_dir/.ffskill-audit.jsonl`.
|
|
1054
1094
|
- With `--jobs > 1` each item gets its own work subdirectory
|
|
1055
1095
|
`<workdir>/<index>-<stem>/`, because step file names are stem-derived and two
|
|
1056
1096
|
sources sharing a stem would otherwise write over each other. `--jobs 1` (the
|
|
@@ -1182,7 +1222,12 @@ report.py --after FINAL [--before SOURCE] [--platform youtube] [--commands cmds.
|
|
|
1182
1222
|
```
|
|
1183
1223
|
One self-contained HTML: before/after facts and contact sheets, loudness,
|
|
1184
1224
|
compliance table with fixes, commands. Produce it for any multi-step job and
|
|
1185
|
-
hand the path to the user together with the numbers.
|
|
1225
|
+
hand the path to the user together with the numbers. Each section it builds is a
|
|
1226
|
+
`verification` step (`loudness` per file with audio, `compliance` for
|
|
1227
|
+
`--platform` with `passed` saying what the table found, `contact_sheet` per
|
|
1228
|
+
file); a section that could not be built is `ok: false` with the child's
|
|
1229
|
+
`reason` and `message` and a `report_section_missing` warning, so the report is
|
|
1230
|
+
`verified: false` rather than silently shorter.
|
|
1186
1231
|
|
|
1187
1232
|
### multicam.py — align several cameras and switch between them
|
|
1188
1233
|
```
|
|
@@ -1195,7 +1240,10 @@ All inputs are aligned to the first one by audio (same engine as `sync.py`,
|
|
|
1195
1240
|
each range of the reference timeline (gaps fall back to camera 0), `--auto N`
|
|
1196
1241
|
simply alternates every N seconds. Audio comes from the reference unless
|
|
1197
1242
|
`--audio` picks another input, e.g. an external recorder that has no video.
|
|
1198
|
-
`--offsets-only` reports offsets and confidence without rendering.
|
|
1243
|
+
`--offsets-only` reports offsets and confidence without rendering. `cuts` names
|
|
1244
|
+
the camera actually rendered: a range whose camera has not started yet plays
|
|
1245
|
+
camera 0, and a `camera_not_started` entry in `diagnostics.fallbacks` names the
|
|
1246
|
+
camera that was asked for.
|
|
1199
1247
|
|
|
1200
1248
|
**`--switch energy` (1.18)** auto-switches to whichever camera (of those with
|
|
1201
1249
|
a video stream) measures the loudest audio at each 0.25 s window on the
|
|
@@ -1222,6 +1270,11 @@ Runs the toolchain on the user's own files (phone HDR, GoPro, OBS, Log, Zoom)
|
|
|
1222
1270
|
and prints a PASS/FAIL table per step (probe, copy cut, accurate cut, fit,
|
|
1223
1271
|
caption, overlay, look, export, loudness, silence, plus `color --to-sdr` for
|
|
1224
1272
|
HDR and `audio --downmix` for >2 channels). Exit code 1 if anything fails.
|
|
1273
|
+
Each step runs with `--json`, and a failed one keeps its tool's own `kind`,
|
|
1274
|
+
`reason`, `message` and `exit_code` (`error` is the one-line table cell); a
|
|
1275
|
+
completed one carries the tool's `verified`. Steps that do not apply are listed
|
|
1276
|
+
under the file's `skipped` (`quick`, `no_video`, `no_audio`, `not_hdr`,
|
|
1277
|
+
`not_multichannel`, `input_unreadable`) and as SKIP rows in the report.
|
|
1225
1278
|
Run this first when a user hands over footage from a device you have not
|
|
1226
1279
|
seen before, and fix or report what fails.
|
|
1227
1280
|
|
|
@@ -1384,7 +1437,10 @@ and says so. A cue that would need more than `--max-lines` (default 2) is split
|
|
|
1384
1437
|
into consecutive cues sharing its time; a cue shorter than `--min-duration`
|
|
1385
1438
|
(default 1.0 s) is held longer, never past the next cue's start; `--offset
|
|
1386
1439
|
TIME` shifts every cue (seconds, `mm:ss`, `hh:mm:ss.ms` or `hh:mm:ss:ff`, a
|
|
1387
|
-
leading `-` for earlier) for `--text`, `--srt` and `--ass`. One `cues:` info line reports what changed
|
|
1440
|
+
leading `-` for earlier) for `--text`, `--srt` and `--ass`. One `cues:` info line reports what changed;
|
|
1441
|
+
with `--ass`, `caption.ass_shifted` / `ass_unshifted` count the Dialogue lines moved and those whose
|
|
1442
|
+
timestamps could not be read (copied as they were), and `ass_malformed` the lines that could not be
|
|
1443
|
+
read at all and are not counted as cues (each above 0 is a warning). A file you passed in is
|
|
1388
1444
|
never edited: the adjusted copy is written next to the output
|
|
1389
1445
|
(`<out>_adjusted.srt`, `<out>_offset.ass`) and burned instead — under `--dry-run`
|
|
1390
1446
|
/`--plan` the planned command names that same copy and the plan says where it
|
|
@@ -1525,7 +1581,11 @@ padded with silence, never allowed to cut the picture.
|
|
|
1525
1581
|
`--fix-drift` measures the offset again near the end of the overlap, reports
|
|
1526
1582
|
the clock difference in ppm, and resamples the source file so a 60-minute
|
|
1527
1583
|
take stays in sync (typical consumer devices drift 20-500 ppm = up to 1.8 s/h).
|
|
1528
|
-
Use it whenever the recording is longer than ~10 minutes.
|
|
1584
|
+
Use it whenever the recording is longer than ~10 minutes. A correction that
|
|
1585
|
+
could not be measured is skipped and says why: `drift_skipped_reason` is
|
|
1586
|
+
`too_short` (no tail window far enough from the head) or `low_confidence`
|
|
1587
|
+
(top level for one source, and in each `sources` entry; null when applied).
|
|
1588
|
+
Check `confidence`
|
|
1529
1589
|
(0–1, normalised correlation with a runner-up penalty); below 0.3 the match is
|
|
1530
1590
|
doubtful. Benchmark on real dialogue/music (±30 s offsets, gain, noise, EQ):
|
|
1531
1591
|
with the default 120 s window 40/40 within 10 ms (max 1.1 ms); with a 60 s
|
|
@@ -1676,13 +1736,18 @@ export.py INPUT --preset youtube|youtube4k|reels|tiktok|shorts|linkedin|facebook
|
|
|
1676
1736
|
export.py --list
|
|
1677
1737
|
```
|
|
1678
1738
|
Scales into the preset frame (pad by default), tags BT.709 (a BT.601-matrix or RGB source is converted to it, as by every re-encode), sets `+faststart`,
|
|
1679
|
-
trims to platform maximums (Reels 90 s, X 140 s) unless `--allow-long
|
|
1739
|
+
trims to platform maximums (Reels 90 s, X 140 s) unless `--allow-long`, and says
|
|
1740
|
+
so: `trimmed: {from, to}` and a `notes` line. It
|
|
1680
1741
|
does not touch levels: for youtube / youtube4k / reels / x the written file is
|
|
1681
1742
|
measured and the result's `loudness` (and a `notes` line) says when it is
|
|
1682
1743
|
outside the platform's LUFS / true-peak spec, naming the `loudness.py` call
|
|
1683
1744
|
that fixes it -- or pass `--normalize`, which runs that call on the written
|
|
1684
1745
|
file itself (audio re-encoded, video copied; `loudness.normalized: true`) so a
|
|
1685
|
-
platform export is one command instead of export, loudness, export again.
|
|
1746
|
+
platform export is one command instead of export, loudness, export again. A
|
|
1747
|
+
measurement that cannot be read is `loudness.measured: false` (values null) with
|
|
1748
|
+
a failing `{step: loudness, ok: false, reason: not_measured}` and a
|
|
1749
|
+
`loudness_not_measured` warning, so the export is `verified: false` rather than
|
|
1750
|
+
unchecked.
|
|
1686
1751
|
|
|
1687
1752
|
Since 1.14 each social destination is its own preset rather than an alias: `tiktok`
|
|
1688
1753
|
(1080x1920, max 600 s), `shorts` (1080x1920, max 180 s), `reels` (1080x1920, max 90 s),
|
|
@@ -56,7 +56,7 @@ from _common.runner import (
|
|
|
56
56
|
child_limit, _CHILDREN, _cleanup_partial_output, _cmdline, CODECS, Context, DEFAULT_TIMEOUT,
|
|
57
57
|
_DRAWTEXT_PENDING, _DRAWTEXT_TMPDIR, _drawtext_tmpdir, dry_run_input_pending, _ENCODERS, _env_timeout,
|
|
58
58
|
ERROR_CODE, ERROR_RETRYABLE, EVEN_SCALE, _execute, _fail, ffmpeg_base, ffmpeg_encoders, ffmpeg_exe, _FFMPEG_RELEASE, _FFMPEG_VERSION,
|
|
59
|
-
ffmpeg_bsfs, ffmpeg_release, ffmpeg_version, flush_drawtext_textfiles, INSTALL_HINTS, install_signal_handlers, _is_ffmpeg, _limit_for,
|
|
59
|
+
ffmpeg_bsfs, ffmpeg_list_error, ffmpeg_release, ffmpeg_version, flush_drawtext_textfiles, INSTALL_HINTS, install_signal_handlers, _is_ffmpeg, _limit_for,
|
|
60
60
|
_odd_dimension_retry, _on_signal, _OutputLock, _pid_dead, place_output, PROBE_TIMEOUT, _progress_line,
|
|
61
61
|
read_text_or_die, refuse_output_is_input, _remember_output, require_tool, run, run_analysis, _run_captured,
|
|
62
62
|
run_keeping_subtitles, run_tool, _run_with_progress, shell_quote, sibling_temp, _SIGNALS_INSTALLED, _stage_existing_output,
|
|
@@ -198,7 +198,7 @@ __all__ = [
|
|
|
198
198
|
"_emoji_name_candidates", "EMOJI_RANGES", "_EMOJI_REGIONAL", "emoji_support", "_EMOJI_SUPPORT_CACHE",
|
|
199
199
|
"_EMOJI_TAIL", "encoder_args", "_ENCODERS", "_env_timeout", "ERROR_CODE", "ERROR_RETRYABLE", "escape_drawtext",
|
|
200
200
|
"escape_filter_path", "EVEN_SCALE", "_execute", "_fail", "_family_rank", "FC_LANG", "_fc_list_fonts",
|
|
201
|
-
"FC_UNKNOWN", "ffmpeg_base", "ffmpeg_bsfs", "ffmpeg_encoders", "ffmpeg_exe", "_FFMPEG_RELEASE", "_FFMPEG_VERSION", "ffmpeg_release", "ffmpeg_version", "fingerprint",
|
|
201
|
+
"FC_UNKNOWN", "ffmpeg_base", "ffmpeg_bsfs", "ffmpeg_encoders", "ffmpeg_list_error", "ffmpeg_exe", "_FFMPEG_RELEASE", "_FFMPEG_VERSION", "ffmpeg_release", "ffmpeg_version", "fingerprint",
|
|
202
202
|
"flush_drawtext_textfiles", "fold_bt709_matrix", "fmt_secs", "fmt_smpte_time", "fmt_srt_time", "font_covers_script",
|
|
203
203
|
"font_family_for_script", "font_family_of_file", "FONT_FLAG_HINT", "font_for_script", "FONT_INSTALL_HINT",
|
|
204
204
|
"fonts_dir_covers_script", "_fraction", "has_emoji", "info", "INSTALL_HINTS", "install_signal_handlers",
|
package/scripts/_common/asr.py
CHANGED
|
@@ -16,7 +16,7 @@ from pathlib import Path
|
|
|
16
16
|
from typing import Any, Dict, List, NamedTuple, Optional, Tuple
|
|
17
17
|
|
|
18
18
|
from _common.decision import fmt_srt_time, parse_time
|
|
19
|
-
from _common.emit import die, info
|
|
19
|
+
from _common.emit import die, info, warn
|
|
20
20
|
from _common.runner import read_text_or_die
|
|
21
21
|
|
|
22
22
|
# The engines this skill knows how to drive, and how to install each -- one string, so every
|
|
@@ -296,7 +296,7 @@ def _parakeet_attempt(engines: "List[str]", facts: Dict[str, Any], wanted: str,
|
|
|
296
296
|
else:
|
|
297
297
|
run.notes.append(assumed_english_note(language_flag, whisper_failed=facts.get("routing") == ROUTING_LAST_RESORT,
|
|
298
298
|
parakeet_model=facts.get("routing") == ROUTING_PARAKEET_MODEL))
|
|
299
|
-
|
|
299
|
+
warn(run.notes[-1], "asr_language_assumed")
|
|
300
300
|
return run, os.path.basename(chosen)
|
|
301
301
|
return None
|
|
302
302
|
|
|
@@ -16,8 +16,8 @@ from pathlib import Path
|
|
|
16
16
|
from typing import Any, Dict, List, Optional, Sequence, Tuple
|
|
17
17
|
from _common.color import (bt709_label_filter, bt709_matrix_args, bt709_matrix_filter, bt709_tag_args, drawn_stream_format,
|
|
18
18
|
is_neutral_color, rgb_island, rgb_to_bt709_filter, _sdr_bt709)
|
|
19
|
-
from _common.emit import die
|
|
20
|
-
from _common.runner import CODECS, STATE, ffmpeg_encoders, ffmpeg_version, hw_platform_reason
|
|
19
|
+
from _common.emit import die, record_fallback, refuse, warn
|
|
20
|
+
from _common.runner import CODECS, STATE, ffmpeg_encoders, ffmpeg_list_error, ffmpeg_version, hw_platform_reason
|
|
21
21
|
|
|
22
22
|
|
|
23
23
|
def pad_filters(out_w: int, out_h: int, fill: str, color: str, blur: int, darken: float = 0.0,
|
|
@@ -321,7 +321,11 @@ def _vt_args(codec: str, crf: int, meta: Optional[Dict[str, Any]], keep_bt709: b
|
|
|
321
321
|
reason = None if name else f"no VideoToolbox encoder for {codec}"
|
|
322
322
|
reason = reason or hw_platform_reason()
|
|
323
323
|
if not reason and name not in ffmpeg_encoders():
|
|
324
|
-
|
|
324
|
+
unread = ffmpeg_list_error("encoders")
|
|
325
|
+
# an encoder list that could not be read says nothing about the build: name the read
|
|
326
|
+
# failure, not a missing encoder (the read itself is a ffmpeg_list_unreadable warning)
|
|
327
|
+
reason = (f"{name} could not be confirmed (this ffmpeg build's encoder list could not be read: {unread})" if unread
|
|
328
|
+
else f"this ffmpeg build has no {name}")
|
|
325
329
|
if reason:
|
|
326
330
|
note = f"{reason}; encoded on the CPU"
|
|
327
331
|
if note not in STATE.hw_notes:
|
|
@@ -471,7 +475,7 @@ def _cpu_encoder_args(codec: str, crf: int, preset: str, meta: Optional[Dict[str
|
|
|
471
475
|
if codec == "hevc":
|
|
472
476
|
if hdr:
|
|
473
477
|
x265 = f"log-level=error:colorprim={prim}:transfer={trc}:colormatrix={cs}:range=limited:hdr10-opt=1" if trc == "smpte2084" else f"log-level=error:colorprim={prim}:transfer={trc}:colormatrix={cs}"
|
|
474
|
-
return ["-c:v", "libx265", "-preset", preset, "-crf", str(
|
|
478
|
+
return ["-c:v", "libx265", "-preset", preset, "-crf", str(_clamped_crf("libx265", crf + 2, 51, crf)), "-pix_fmt", "yuv420p10le", "-tag:v", "hvc1",
|
|
475
479
|
"-x265-params", x265] + hdr_tags + ["-movflags", "+faststart"]
|
|
476
480
|
params, extra = _sdr_bt709("libx265") if keep_bt709 else ("", [])
|
|
477
481
|
return ["-c:v", "libx265", "-preset", preset, "-crf", str(crf), "-pix_fmt", "yuv420p", "-tag:v", "hvc1",
|
|
@@ -479,27 +483,56 @@ def _cpu_encoder_args(codec: str, crf: int, preset: str, meta: Optional[Dict[str
|
|
|
479
483
|
if codec == "av1":
|
|
480
484
|
pix = "yuv420p10le" if hdr else "yuv420p"
|
|
481
485
|
if "libsvtav1" in ffmpeg_encoders():
|
|
482
|
-
args = ["-c:v", "libsvtav1", "-preset", str(SVT_PRESET.get(preset, 6)), "-crf", str(
|
|
486
|
+
args = ["-c:v", "libsvtav1", "-preset", str(SVT_PRESET.get(preset, 6)), "-crf", str(_clamped_crf("libsvtav1", crf, 63, crf)), "-pix_fmt", pix]
|
|
483
487
|
if hdr:
|
|
484
488
|
args += hdr_tags
|
|
485
489
|
elif keep_bt709:
|
|
486
490
|
params, extra = _sdr_bt709("libsvtav1")
|
|
487
491
|
args += (["-svtav1-params", params] if params else []) + extra
|
|
488
492
|
elif "libaom-av1" in ffmpeg_encoders():
|
|
489
|
-
|
|
493
|
+
# recorded: libaom is the slower AV1 encoder with its own CRF scale and speed setting,
|
|
494
|
+
# so the same --quality does not give the file SVT-AV1 would have
|
|
495
|
+
record_fallback("--codec av1: this ffmpeg build has no libsvtav1 (SVT-AV1), so libaom-av1 encodes instead "
|
|
496
|
+
"(slower; -cpu-used 6, its own CRF scale)", "av1_encoder_libaom", wanted="libsvtav1", used="libaom-av1")
|
|
497
|
+
args = ["-c:v", "libaom-av1", "-crf", str(_clamped_crf("libaom-av1", crf, 63, crf)), "-b:v", "0", "-cpu-used", "6", "-row-mt", "1", "-pix_fmt", pix]
|
|
490
498
|
args += hdr_tags if hdr else (_sdr_bt709("libaom-av1")[1] if keep_bt709 else [])
|
|
491
499
|
else:
|
|
492
|
-
|
|
493
|
-
|
|
500
|
+
_refuse_missing_encoder("av1", ["libsvtav1", "libaom-av1"], "an AV1 encoder (libsvtav1 or libaom-av1)",
|
|
501
|
+
"install an ffmpeg built with SVT-AV1 (most distribution builds are), or use --codec hevc")
|
|
494
502
|
return args + ["-movflags", "+faststart"]
|
|
495
503
|
if codec == "prores":
|
|
496
504
|
if "prores_ks" not in ffmpeg_encoders():
|
|
497
|
-
|
|
505
|
+
_refuse_missing_encoder("prores", ["prores_ks"], "the prores_ks encoder", None)
|
|
498
506
|
return ["-c:v", "prores_ks", "-profile:v", "3", "-vendor", "apl0", "-pix_fmt", "yuv422p10le"] + (hdr_tags if hdr else [])
|
|
499
507
|
die(f"unknown --codec {codec!r} (one of {', '.join(CODECS)})")
|
|
500
508
|
return []
|
|
501
509
|
|
|
502
510
|
|
|
511
|
+
def _clamped_crf(encoder: str, value: int, top: int, requested: int) -> int:
|
|
512
|
+
"""`value` capped at the encoder's maximum CRF, the cap recorded (`crf_clamped`) when it
|
|
513
|
+
bites: the HEVC HDR line asks x265 for CRF+2, so --quality 50 or 51 there is encoded at 51."""
|
|
514
|
+
if value <= top:
|
|
515
|
+
return value
|
|
516
|
+
warn(f"{encoder}: CRF {value} (from {requested}) is above the encoder's maximum {top}; encoded at CRF {top}",
|
|
517
|
+
"crf_clamped", encoder=encoder, requested=requested, asked=value, used=top)
|
|
518
|
+
return top
|
|
519
|
+
|
|
520
|
+
|
|
521
|
+
def _refuse_missing_encoder(codec: str, encoders: List[str], what: str, hint: Optional[str]) -> None:
|
|
522
|
+
"""--codec av1/prores with no encoder for it -- or with an encoder list that could not be read,
|
|
523
|
+
which proves nothing about the build and is refused under its own reason rather than as a
|
|
524
|
+
missing encoder (ffmpeg_list_error)."""
|
|
525
|
+
unread = ffmpeg_list_error("encoders")
|
|
526
|
+
if unread:
|
|
527
|
+
refuse("ffmpeg", "encoder_list_unreadable",
|
|
528
|
+
f"--codec {codec} needs {what}, and whether this ffmpeg build has one is unknown: its encoder list "
|
|
529
|
+
f"could not be read ({unread})",
|
|
530
|
+
hint="check that ffmpeg runs (python3 scripts/_contract.py doctor), then retry",
|
|
531
|
+
codec=codec, encoders=encoders, error=unread)
|
|
532
|
+
refuse("missing_tool", "encoder_missing", f"--codec {codec} needs {what} and this ffmpeg build has "
|
|
533
|
+
+ ("neither" if len(encoders) > 1 else "none"), hint=hint, codec=codec, encoders=encoders)
|
|
534
|
+
|
|
535
|
+
|
|
503
536
|
def _x264_raw(crf: int, preset: str, keep_bt709: bool = True) -> List[str]:
|
|
504
537
|
args = ["-c:v", "libx264", "-preset", preset, "-crf", str(crf), "-pix_fmt", "yuv420p", "-movflags", "+faststart"]
|
|
505
538
|
if keep_bt709:
|
|
@@ -1159,17 +1192,22 @@ def normalise_filler_token(word: str) -> str:
|
|
|
1159
1192
|
|
|
1160
1193
|
|
|
1161
1194
|
def filler_spans(words, wordlist, *, pad: float = FILLER_PAD, min_gap: float = FILLER_MIN_GAP,
|
|
1162
|
-
max_word: float = FILLER_MAX_WORD) -> "List[Dict[str, Any]]":
|
|
1195
|
+
max_word: float = FILLER_MAX_WORD, stats: "Optional[Dict[str, int]]" = None) -> "List[Dict[str, Any]]":
|
|
1163
1196
|
"""Time spans to remove, from measured word timings. Pure: no subprocess, no I/O.
|
|
1164
1197
|
|
|
1165
1198
|
`words` is [{"word", "start", "end"}] as whisper emits. A word is removed only when its
|
|
1166
1199
|
normalised form is in `wordlist` AND it carries a real start < end pair AND its length is
|
|
1167
1200
|
<= max_word. Adjacent spans closer than min_gap merge. Returns [{"start", "end", "word"}]
|
|
1168
1201
|
sorted and non-overlapping.
|
|
1202
|
+
|
|
1203
|
+
`stats`, when given, receives the listed words that were kept anyway, so the caller can say
|
|
1204
|
+
so: `matched_untimed` (no usable start < end: nothing to cut on) and `matched_too_long`
|
|
1205
|
+
(longer than max_word: a held vowel someone meant).
|
|
1169
1206
|
"""
|
|
1170
1207
|
listed = {normalise_filler_token(w) for w in (wordlist or set())}
|
|
1171
1208
|
listed.discard("")
|
|
1172
1209
|
hits = []
|
|
1210
|
+
untimed = too_long = 0
|
|
1173
1211
|
for entry in words or []:
|
|
1174
1212
|
if not isinstance(entry, dict):
|
|
1175
1213
|
continue
|
|
@@ -1179,8 +1217,13 @@ def filler_spans(words, wordlist, *, pad: float = FILLER_PAD, min_gap: float = F
|
|
|
1179
1217
|
try:
|
|
1180
1218
|
start, end = float(entry["start"]), float(entry["end"])
|
|
1181
1219
|
except (KeyError, TypeError, ValueError):
|
|
1220
|
+
untimed += 1
|
|
1182
1221
|
continue # no measured timing: nothing to cut
|
|
1183
|
-
if not (end > start)
|
|
1222
|
+
if not (end > start):
|
|
1223
|
+
untimed += 1
|
|
1224
|
+
continue
|
|
1225
|
+
if (end - start) > max_word:
|
|
1226
|
+
too_long += 1
|
|
1184
1227
|
continue
|
|
1185
1228
|
hits.append({"start": max(0.0, start - pad), "end": end + pad, "word": token})
|
|
1186
1229
|
hits.sort(key=lambda h: (h["start"], h["end"]))
|
|
@@ -1194,6 +1237,9 @@ def filler_spans(words, wordlist, *, pad: float = FILLER_PAD, min_gap: float = F
|
|
|
1194
1237
|
for m in merged:
|
|
1195
1238
|
m["start"] = round(m["start"], 4)
|
|
1196
1239
|
m["end"] = round(m["end"], 4)
|
|
1240
|
+
if stats is not None:
|
|
1241
|
+
stats["matched_untimed"] = untimed
|
|
1242
|
+
stats["matched_too_long"] = too_long
|
|
1197
1243
|
return merged
|
|
1198
1244
|
|
|
1199
1245
|
|