ffmpeg-skill 2.7.0 → 2.8.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -202,7 +202,7 @@ These are the rules the skill file gives the agent and the code enforces.
202
202
 
203
203
  | Tool | What it does |
204
204
  |---|---|
205
- | `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact video and sample-exact audio; reports `precision`; `--snap beats` moves the in/out points onto a measured beat, or refuses when there is no measurable pulse |
205
+ | `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact video and sample-exact audio, `--keep-hevc` to re-encode an HEVC source as HEVC, `--edit-list` to keep an `.mp4` copy's start exact, `--vfr-guard` to choose the VFR check; reports `precision`; `--snap beats` moves the in/out points onto a measured beat, or refuses when there is no measurable pulse |
206
206
  | `join.py` | Concatenate clips with xfade transitions, normalising size, fps, sample rate and channel layout (the widest clip's, or `--channels`); audio-only inputs are joined as audio; `--list parts.txt` reads generated segments (TTS lines) and checks every one first — missing, empty or unreadable parts are all named in one refusal, or skipped and reported with `--on-missing skip` |
207
207
  | `silence.py` | Detect and remove dead air (jump cuts) with a margin around speech; list or export the cut list; `--filler` also removes filler words, but only where a speech engine timed them; `--speech-aware` keeps a breath inside a sentence and cuts only at sentence-boundary pauses, composing with `--filler` through the same `keep_ranges()` |
208
208
  | `fit.py` | Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad, crop or `--fit blur`'s blurred, darkened fill, with `--crop-x`/`--crop-y` to keep an off-centre subject) and/or exact `--width`/`--height`; rotate 90/180/270, flip h/v; force constant fps |
@@ -362,8 +362,8 @@ npx ffmpeg-skill doctor --json # available / missing / missing_optional / unkn
362
362
 
363
363
  The short list for humans. The agent-facing version, with the reasoning, is the "Things that look right but are wrong" and "Gotchas" sections of [SKILL.md](SKILL.md).
364
364
 
365
- - **Variable frame rate (phone and screen recordings).** `probe.py` flags it; every re-encoding tool conforms to a constant rate automatically, and `cut.py` switches to frame-accurate mode on its own because copy-cuts on VFR land on the wrong frame. Choose the rate yourself with `fit.py input.mp4 --fps 30` when the measured average is odd.
366
- - **Lossless cuts snap to keyframes.** A stream-copy cut can start up to one GOP earlier than asked. `cut.py` re-encodes when the snap exceeds 0.5 s (`--tolerance` changes the limit). For a strictly lossless file pass `--tolerance -1`, and expect the cut to land on the nearest earlier keyframe; the JSON result lists them under `nearest_keyframes`.
365
+ - **Variable frame rate (phone and screen recordings).** `probe.py` flags it; every re-encoding tool conforms to a constant rate automatically. `cut.py` switches to frame-accurate mode on it, because copy-cuts on VFR land on the wrong frame, and samples the frame timing to say whether that was needed (`--vfr-guard sampled` decides on the sample instead; `--vfr-guard off` keeps the copy). Choose the rate yourself with `fit.py input.mp4 --fps 30` when the measured average is odd.
366
+ - **Lossless cuts snap to keyframes.** A stream-copy cut can start up to one GOP earlier than asked. `cut.py` re-encodes when the snap exceeds 0.5 s (`--tolerance` changes the limit). For a strictly lossless file pass `--tolerance -1` (plus `--vfr-guard off` on a variable-frame-rate source), and expect the cut to land on the nearest earlier keyframe; the JSON result lists them under `nearest_keyframes`. A `--segments` join can still be re-cut: open-GOP video, a later segment of B-frame `.mp4`/`.mov` video that starts between keyframes (start it on a keyframe, or write `.mkv`), or a join that fails its check. For a single `.mp4`/`.mov` cut, `--edit-list` keeps the copy and starts the picture where asked: the keyframe's pre-roll is stored but hidden by an MP4 edit list, which a player that ignores edit lists shows.
367
367
  - **HDR stays HDR.** When the probe reports HDR (HDR10, HLG, Dolby Vision, BT.2020), the tools keep it rather than flatten it. Convert deliberately with `color.py --to-sdr` before H.264 deliverables or LUT work. `export.py` platform presets are SDR and warn on HDR input.
368
368
  - **Loudness targets.** −14 LUFS / −1 dBTP for YouTube and social platforms (the `loudness.py` default), `-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast. A clip measured at −40 LUFS or below is room tone, not content; raising it raises the noise. Check true peak as well as LUFS: `check.py file --platform podcast` measures both.
369
369
  - **Frame changes first, text second.** Captions and overlays burned before a crop or resize end up off-frame. Reframe, then caption.
@@ -401,7 +401,7 @@ FFmpeg 8 shortened the flag column of `ffmpeg -filters`. A parser anchored on th
401
401
  **What is tested where.** The contract and the test suite (`tests/test_contract.py`,
402
402
  `tests/test_all.py`, which aggregates one module per tool group — `test_analysis.py`,
403
403
  `test_editing.py`, `test_audio.py`, `test_picture.py`, `test_delivery.py`,
404
- `test_orchestration.py` — over the shared footage in `tests/_fixtures.py`) run on Linux, macOS
404
+ `test_orchestration.py`, `test_accel.py`, `test_asr.py`, `test_cut_copy.py` — over the shared footage in `tests/_fixtures.py`) run on Linux, macOS
405
405
  and Windows on every pull request, minus the handful of POSIX-shim tests listed under
406
406
  [Development](#development). The real-device media corpus
407
407
  (`tests/corpus.py`) has been run on Linux and macOS; the full corpus has **not** been run on
package/SKILL.md CHANGED
@@ -7,7 +7,7 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
7
7
 
8
8
  Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`, and delivery templates in `templates/`. This file is enough to do a job: the table below routes the request, and `--help` on the script about to run is the cheapest full flag list. A reference file costs as much as this one; open one only for a question you have: `references/scripts.md` (every flag of all 42 scripts), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end). MCP: `tools/list` shows only the core 12 by default; the other 30 (per this table) stay callable by name via `tools/call`, described by `contract --json`; set `FFMPEG_SKILL_MCP_FULL=1` for all 42.
9
9
 
10
- Shared flags (every script): `--dry-run`; `--json` (output path, its probe, the commands run); `--json-brief` (status/output/verified + a `summary`; prefer on writing steps); `--fast` (preview quality); `--progress` (percent/ETA); `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (a dry run `render.py FILE` runs later; refuses if an input changed). Re-encoding tools also take `--hw` (Apple Silicon GPU, big files; `FFMPEG_SKILL_HW=1` = default, not export), `--codec h264|hevc|av1|prores` and `--quality N`: unset, SDR is x264, HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses HDR (`color.py --to-sdr` first).
10
+ Shared flags (every script): `--dry-run`; `--json` (output path, its probe, the commands run); `--json-brief` (status/output/verified + a `summary`; prefer on writing steps); `--fast` (preview quality); `--progress` (percent/ETA); `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (a dry run `render.py FILE` runs later; refuses if an input changed). Re-encoding tools also take `--hw` (Apple Silicon GPU, big files; `FFMPEG_SKILL_HW=1` = default, not export), `--codec h264|hevc|av1|prores` and `--quality N`: unset, SDR is x264 (`cut.py --keep-hevc`: HEVC source stays HEVC), HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses HDR (`color.py --to-sdr` first).
11
11
 
12
12
  Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) may still run ffmpeg/ffprobe, skipping artifacts/side files (`--edl`, `--sheet`, a generated `.ass`); `verify` ignores the flag. Per tool: `contract --json` `dry_run`.
13
13
 
@@ -52,7 +52,7 @@ This skill cuts, joins, measures, syncs, exports and checks files — it execute
52
52
 
53
53
  The line: same input + same explicit parameters always producing the same verifiable output belongs here; anything depending on taste, understanding or what looks or sounds good belongs to whoever makes that judgement.
54
54
 
55
- If a request needs an FFmpeg feature none of the 42 scripts expose, say so and name the closest built-in option — never guess a raw `ffmpeg`/`ffprobe` invocation outside `scripts/*.py`. It bypasses every guarantee this skill makes, so never a fallback when a script's flag doesn't cover something.
55
+ If a request needs an FFmpeg feature none of the 42 scripts expose, say so and name the closest built-in option — never guess a raw `ffmpeg`/`ffprobe` invocation outside `scripts/*.py`. It bypasses every guarantee this skill makes, so it is never a fallback.
56
56
 
57
57
  ## Request → script
58
58
 
@@ -65,7 +65,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
65
65
  | "cut from 1:20 to 2:05", "trim the first 10 s" | `cut.py input.mp4 --start 1:20 --end 2:05` |
66
66
  | "keep only these parts", "remove the middle" | `cut.py input.mp4 --segments 0-1:00,1:30-2:00` |
67
67
  | "make it exactly 60 seconds" | `fit.py input.mp4 --duration 60` (speed) or `--method trim` |
68
- | "cut this and make it HEVC / AV1 / ProRes" (output codec named) | `cut.py input.mp4 --start 0:10 --end 0:40 --codec hevc` (`--codec`/`--quality` on any re-encoding tool; ProRes needs `-o NAME.mov`) |
68
+ | "cut this and make it HEVC / AV1 / ProRes" (output codec named) | `cut.py input.mp4 --start 0:10 --end 0:40 --codec hevc` (ProRes needs `-o NAME.mov`) |
69
69
  | "cut out the pauses", "jump cuts" | `silence.py input.mp4 [--threshold -40 --min-silence 0.8]` |
70
70
  | "cut the ums and uhs", "remove the filler words" | `silence.py input.mp4 --filler --words words.json` (measured word timings; `--transcribe` makes them) |
71
71
  | "don't cut inside a sentence, just the real pauses" | `silence.py input.mp4 --speech-aware` — a breath under `--min-silence` inside a sentence is kept, only sentence-boundary pauses cut; composes with `--filler` into one list |
@@ -206,8 +206,8 @@ One line each; open the linked `references/gotchas.md` section when the job is i
206
206
 
207
207
  - HDR (iPhone, HDR10) re-encoded through an SDR path goes flat; the scripts keep HDR; `hdr: true` is a real PQ/HLG/DV signal, `bt2020_or_hdr: true` also BT.2020 SDR. -> [#hdr-and-colour](references/gotchas.md#hdr-and-colour)
208
208
  - Log footage (S-Log/V-Log/C-Log) is tagged SDR and looks grey: `probe.py --analyze`, then `color.py --lut` first. -> [#log-footage](references/gotchas.md#log-footage)
209
- - A `-c copy` cut can start on a wrong or frozen frame; `cut.py` re-encodes past a 0.5 s snap, respect it. -> [#keyframe-cuts](references/gotchas.md#keyframe-cuts)
210
- - VFR phone/screen recordings: re-encodes conform to CFR, `cut.py` switches to `--accurate`; pick the rate with `fit.py --fps` when odd. -> [#variable-frame-rate](references/gotchas.md#variable-frame-rate)
209
+ - A `-c copy` cut can start on a wrong/frozen frame; `cut.py` re-encodes past a 0.5 s snap, respect it; `--edit-list` hides an .mp4's pre-roll. -> [#keyframe-cuts](references/gotchas.md#keyframe-cuts)
210
+ - VFR phone/screen clips: re-encodes go CFR; `cut.py` re-encodes (`--vfr-guard sampled`: only measured VFR); odd rate: `fit.py --fps`. -> [#variable-frame-rate](references/gotchas.md#variable-frame-rate)
211
211
  - Sync/multicam `confidence` under 0.3 (or a huge offset) is suspect — check every camera; these align audio, never lip sync. -> [#sync-multicam-and-drift](references/gotchas.md#sync-multicam-and-drift)
212
212
  - "Normalised" audio can still clip (check true peak); ambience at -40 LUFS or below must never be raised to a speech target. -> [#loudness-and-ambience](references/gotchas.md#loudness-and-ambience)
213
213
  - Captions burned before a crop/resize land off-frame; burned small then upscaled by `export.py` come out soft. -> [#captions-fonts-and-text-order](references/gotchas.md#captions-fonts-and-text-order)
package/docs/contract.md CHANGED
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
21
21
  | Field | Meaning | Changes when |
22
22
  |---|---|---|
23
23
  | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
24
- | `skill.version` | the npm / package.json version (`2.7.0`) | any release |
24
+ | `skill.version` | the npm / package.json version (`2.8.1`) | any release |
25
25
 
26
26
  A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
27
27
  ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -119,7 +119,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
119
119
  "contract_version": "1.0",
120
120
  "deprecated": [],
121
121
  "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
122
- "skill": {"id": "ffmpeg-skill", "version": "2.7.0", "execution_mode": "local", "kind": "execution",
122
+ "skill": {"id": "ffmpeg-skill", "version": "2.8.1", "execution_mode": "local", "kind": "execution",
123
123
  "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
124
124
  "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
125
125
  "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
@@ -216,7 +216,9 @@ BT.709 for SDR; av1 is SVT-AV1 with libaom as the fallback; prores is 422 HQ and
216
216
  `.mov`/`.mkv` output; h264 refuses an HDR source (`kind: input`). `--quality` is the CRF scale;
217
217
  its default is the tool's own (18, `proxy.py` 30). 1.x also accepted `--crf` as an alias; 2.0
218
218
  removed it (`export.py` keeps its own `--crf`). Without `--codec` the encoder is what it always was (x264 for SDR, x265
219
- Main10 for HDR), so the flags add no behaviour to a caller that does not pass them. The
219
+ Main10 for HDR), so the flags add no behaviour to a caller that does not pass them.
220
+ `cut.py --keep-hevc` (opt-in) re-encodes an SDR HEVC source as x265 8-bit BT.709 instead of x264; an HDR or
221
+ BT.2020 source keeps the Main10 line with its own colour tags, with or without the flag. The
220
222
  encoder each value needs is listed under the tool's optional capabilities (`--codec hevc` and
221
223
  so on). `export.py` refuses `--codec`: its presets decide the codec.
222
224
 
@@ -245,7 +247,7 @@ runs the tool with the planned `argv`, then the verify steps (a `check` step may
245
247
  ### Repeatability
246
248
 
247
249
  No tool keeps state or uses randomness. `deterministic_inputs` is `false` only for
248
- `verify`, whose output includes timings. `idempotency_hint` says what "same inputs"
250
+ `verify` and `cropdetect`, whose output includes timings or machine state. `idempotency_hint` says what "same inputs"
249
251
  gives you:
250
252
 
251
253
  | Hint | Tools |
@@ -253,7 +255,7 @@ gives you:
253
255
  | `bit_exact` | probe, check, scenes, look, metadata, background |
254
256
  | `content_equivalent` (same media, bytes may differ between encoder builds) | every encoding tool, cut, sync, report |
255
257
  | `cached` | batch (content-hash cache, re-runs skip unchanged inputs); render **when `--cache DIR` is given** — the hint stays `content_equivalent` because that is what render is without the flag, and the cache is opt-in |
256
- | `environment_dependent` | verify |
258
+ | `environment_dependent` | verify, cropdetect |
257
259
 
258
260
  The hints describe the CPU encoders. Under `--hw`, or `FFMPEG_SKILL_HW=1` on the machine, a tool
259
261
  that encodes video runs VideoToolbox, whose bytes are not promised identical from run to run:
@@ -566,13 +568,20 @@ Per-tool keys added after 2.2.2, all additive:
566
568
 
567
569
  | key | tool | what it holds |
568
570
  |---|---|---|
571
+ | `boomerang`, `notes` | `loop.py` | `boomerang` is `true` for `--boomerang` (frames 0..N-1 then N-2..1, repeated, so neither turnaround frame is shown twice); `notes` says the audio was dropped when the source had any (a boomerang is silent) |
569
572
  | `sdr_path`, `notes` | `color.py --to-sdr` | `sdr_path` is `"tonemap"` for PQ / HLG / Dolby Vision input (and `--force` on an untagged file) or `"gamut"` for BT.2020 primaries on an SDR transfer, which is converted to BT.709 without a tone map (`--tonemap`/`--peak`/`--desat` do not apply); `notes` says which path was taken and why |
570
573
  | `silent` | `join.py` (every run) | `[{index, path, peak_db, at, end}]`: inputs whose whole-file audio peak (volumedetect) is at or below `--silence-threshold` (default -50 dBFS) and that `--on-silent warn` (default) joined anyway, with a `notes` line; `at`/`end` are where the input sits in the output, in seconds (a transition overlaps its neighbours). An input named by `--allow-silent N[,N]` (1-based) is a planned pause: always joined, listed with `intended: true`, no warning or note, whatever `--on-silent` says. `[]` otherwise. `--on-silent fail` names them in the `kind: input` refusal's `problems` (reason `silent (peak -91.0 dBFS)`), `skip` lists them under `skipped`. An input with no audio stream is never silent; `verified` is unaffected |
571
574
  | `short_segments`, `duplicates` | `join.py` (every run) | warnings, never refusals, and the join command is unchanged: `short_segments: [{index, path, duration}]` names each measured input shorter than 2 frames at the join's fps (an audio-only join: shorter than 0.05 s); `duplicates: [{path, indices}]` names a path (compared resolved) listed more than once — repeating a clip can be intended. `[]` when none; each non-empty one adds a `notes` line |
572
575
  | `silent` | `waveform.py` (every run) | `true` when the input audio's whole-file peak is at or below `--silence-threshold` (default -50 dBFS), which draws a flat line; `--on-silent warn` (default) renders it anyway with a `notes` line, `fail` refuses (`kind: input`) before ffmpeg runs. `false` when audible, `null` under `--dry-run` (nothing measured). `verified` is unaffected |
573
576
  | `encoder`, `hw`, `notes` | every re-encoding tool, `export.py`, `render.py`, `batch.py` (and its rows), `waveform.py` | `encoder`: the video encoder the last command ran (after any GPU→CPU fallback; under `--dry-run`, the planned one), `copy` for a stream copy; absent when no command encoded video. A tool that runs other tools reports its stages': `render.py` its stages and an executed plan's tool, `waveform.py` the title/caption stage that writes its file (its `commands` include that stage's). `hw` is present when VideoToolbox was asked for (`--hw`, `FFMPEG_SKILL_HW=1`, or a stage or `batch.py` item that reported it) and on `export.py` when `FFMPEG_SKILL_HW=1` is set but its delivery preset does not take it: `{requested, source: flag\|env, used, fallback, notes}`. `requested` says whether this run's encodes were asked onto VideoToolbox (`false` for that export), `used` whether the encoder that ran last is VideoToolbox (`null` under `--dry-run`, which ran nothing, and when the tool encoded nothing itself: `batch.py`, whose top-level `hw` gathers its items' `requested`/`source`/`fallback`/`notes` while each `results` row carries the item's own `encoder`, and `hw`/`notes` when VideoToolbox was asked for), `fallback` whether a job VideoToolbox refused was re-encoded on the CPU, and `notes` why. Only a VideoToolbox failure (the encoder could not open, or its compression session failed) is retried on the CPU; any other failure is reported as it happened, once. Every VideoToolbox encode adds a top-level `notes` line: its quality is not CRF-equivalent (`--quality` / a preset's CRF maps to `-q:v` approximately), and `verified` stays what it is for every tool, the measured properties of the output. A `kind: ffmpeg` failure under `--hw` / `FFMPEG_SKILL_HW` carries `encoder` and `hw` for the command that ran last (a stage's, when `render.py`/`waveform.py` re-raise it); when the command itself failed inside the run (not a tool's own check afterwards, such as `cut.py`'s), its message also names that encoder |
574
- | `transcription` | `caption.py --transcribe` (every `--mode`); inside `filler` for `silence.py --filler --transcribe` | `{routing, detected_language?, engine, model, language}`: the engine and model that made the transcript; `language` is the language the engine was told -- the one named (`--language`, `--filler-lang`), else the one auto's detector named -- and `null` when the engine identified it itself; `detected_language` is present when auto's whisper.cpp detector named one; `routing` says why `--engine auto` chose the engine (`"requested"` for a named engine), ending `, no Parakeet engine transcribed it` when auto chose Parakeet and a Whisper engine made the transcript after every Parakeet engine failed |
577
+ | `transcription` | `caption.py --transcribe` (every `--mode`); inside `filler` for `silence.py --filler --transcribe` | `{routing, detected_language?, engine, model, language}`: the engine and model that made the transcript; `language` is the language the engine was told -- the one named (`--language`, `--filler-lang`), else the one auto's detector named -- and, when nothing was named, the language the engine decoded where it reports one (whisper.cpp's word-timing JSON, faster-whisper, openai-whisper's word-timing JSON), else `null`; `detected_language` is present when auto's whisper.cpp detector named one; `routing` says why `--engine auto` chose the engine (`"requested"` for a named engine), ending `, no Parakeet engine transcribed it` when auto chose Parakeet and a Whisper engine made the transcript after every Parakeet engine failed |
575
578
  | `notes` | `silence.py --filler --transcribe` (new for `silence.py`; `caption.py` already had it) | present when `--engine auto` ran the English-only Parakeet model on English *assumed* (no language given or detectable, and no Whisper engine installed or none that succeeded): the transcript is wrong if the speech is not English. `caption.py --transcribe` puts the same line in its `notes` |
579
+ | `reencode_reason`, `segment_precision`, `least_exact_precision`, `start_snapped` | `cut.py` (every run) | `reencode_reason`: why anything was re-encoded, distinct values in first-seen order, `[]` for a clean copy: `requested` (`--accurate`), `codec` (`--codec`), `vfr`, `vfr_inconclusive`, `pcm_container`, `copy_failed`, `tolerance`, `concat_fallback` (a `--segments` join that could not stay a stream copy, which now also sets `reencoded: true` and `mode: hybrid`; it reported `copy` before). `segment_precision`: each `--segments` segment's precision (`null` for one segment); `least_exact_precision`: the least exact of them, on every run. `precision` and `keyframe_snapped` keep their meaning (`keyframe_snapped` is `precision == "packet"`, a stream copy end to end); `least_exact_precision` differs from `precision` only for an audio-only join of copied and re-encoded parts. `start_snapped`: measured, `true` when a copied segment's picture starts more than one frame from its requested start (or its start could not be measured: an unknown start is never claimed exact), `false` when every start is where asked or everything re-encoded, `null` under `--dry-run`. `requested_segments`/`requested_duration` keep their meaning, the request clamped to the media's duration: a segment ended with the video (less than a frame of sound trimmed) shows in `duration_delta_seconds` and a `notes` line |
580
+ | `edit_list`, `stored_preroll_seconds`, `av_start_skew_seconds`, `notes` | `cut.py` (every run) | `edit_list`: `true` only for a single-segment `.mp4`/`.m4v`/`.mov` stream copy run with `--edit-list`, which keeps the MP4 edit list the copy writes instead of `-avoid_negative_ts make_zero` (the default, unchanged): the keyframe's pre-roll is stored but hidden, so the picture starts at `--start`. `stored_preroll_seconds`: with `edit_list`, the seconds decoded from the keyframe the copy's seek lands on (measured with the seek ffmpeg makes) up to the first presented frame; `null` otherwise. Under `--edit-list` a copy is judged by its video's length (a `notes` line says so when the container is longer), and one that starts where asked but ends past `--tolerance` re-encodes (`tolerance`) without a `lossless_alternative`. `av_start_skew_seconds`: the output's audio start minus its video start (`null` when either is missing or under `--dry-run`); on a stream copy, when it is more than max(2 frames, 0.1 s) away from the source's own audio/video offset where the copy's picture starts (the keyframe it began from; `--start` under `--edit-list`), a `notes` line names `--accurate`, and `--edit-list` for a single `.mp4`/`.mov` copy whose start snapped. A re-encode carries no such note. The skew is reported, never repaired. `notes`: what the run did that a caller should know, in plain sentences (`[]` when nothing) |
581
+ | `video.start_time`, `audio.start_time` | `probe.py` (and every `probe` block a tool returns) | where each stream starts, in seconds on the file's own clock (ffprobe's stream `start_time`; `null` when unknown). A video stream that starts after its audio, or the reverse, is how a lossless cut's A/V skew shows; `cut.py` reads them for `av_start_skew_seconds` |
582
+ | `cut_reencode_reasons` | `batch.py` (and its `results` rows) | a row whose recipe ran `cut.py` lists each `reencode_reason` its `cut.py` calls named, once per call, beside `cut_reencoded`; the top level counts them across the folder (`{}` when no call re-encoded, `null` when no `cut.py` step ran) on a completed batch, as `cut_stream_copy` does: a failure document carries `results`, whose rows hold each row's reasons; a re-encoded call in a row served from a cache an earlier 2.x batch wrote, which recorded no reasons, counts as `unknown`. `cut_stream_copy` is unchanged |
583
+ | `vfr_check` | `cut.py` (a video input that could still stream-copy) | the sampled frame timing, `{heuristic: true, measured, method: "sampled", guard, windows, deltas}`; `measured` is `sampled_cfr`, `vfr` or `inconclusive`. `--vfr-guard average` (the default) keeps the 2.x guard: a source whose nominal and average frame rates differ (`variable_frame_rate_suspected`) re-encodes as `--accurate` (reason `vfr`, stderr says "variable-frame-rate"), and only then is the timing sampled and reported, with a `notes` line naming `--vfr-guard sampled` when it measured `sampled_cfr`; the key is absent otherwise. `--vfr-guard sampled` samples before every copy and re-encodes on `vfr` (reason `vfr`) or `inconclusive` (fewer than 20 intervals; `vfr_inconclusive`); `--vfr-guard off` samples and keeps the copy, with a `notes` line. Absent for audio-only input or output, `--accurate`, `--codec` and a pending `--dry-run` input |
584
+ | `join_check`, `segment_end_snap_seconds` | `cut.py --segments` | `join_check`: a video stream-copy join measured by demuxing, `{packets, expected_packets, max_step_seconds, audio_offset_ms, audio_parts_checked, ok}`. `expected_packets` counts the source's video packets from each part's start keyframe to its end (or the end of the video); `max_step_seconds` is the largest presentation step, `null` when `vfr_check` measured anything but `sampled_cfr` (a `--vfr-guard off` join of variable timing: count only), and each step at a join must be the step its parts predict within 1 ms. `audio_offset_ms`: per part, its sound's shift minus its picture's in ms (+ = sound late), from audio packets matched by CRC32 to the source stream shaped like the joined audio (FFmpeg's default selection may copy one other than `a:0`; an MPEG-TS source's AAC is hashed without its ADTS headers); `null` for a part with no unique match (digital silence, a periodic tone), and as a whole when the source has no audio or the video check failed. `audio_parts_checked` counts the measured parts. `ok` is the conjunction (count, steps, every measured part within 5 ms); `ok: false` means the join was re-cut (`concat_fallback`) and this is the failed measurement. `null` for a single segment, audio, `--dry-run`, any re-encode forced for every segment, a segment that re-encoded, or a join re-cut before a copy was tried (open GOPs, a later B-frame segment starting between keyframes, parts that differ, a concat error). `segment_end_snap_seconds`: per segment, the signed seconds its end moved to the keyframe the copied part stops at, the first at or after the requested end (`null` for a part that reaches the end of the video, a part of a source with no B-frames, cut to its requested end as in 2.5.1, or a Matroska/MPEG-TS part, which is not snapped); `null` as a whole unless the output is those copied parts |
576
585
 
577
586
  `caption.py --transcribe` and `silence.py --filler --transcribe`: when a local speech engine ran
578
587
  and produced no cue, the refusal is `"<engine> found no speech in <input>"` with `kind: input`,
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "2.7.0",
3
+ "version": "2.8.1",
4
4
  "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
5
  "keywords": [
6
6
  "ffmpeg",
@@ -37,17 +37,39 @@ S-Log, V-Log and C-Log look grey and low-contrast but are tagged SDR. Run
37
37
 
38
38
  ### Keyframe cuts
39
39
  A lossless `cut.py` result may start up to one GOP (often 1–10 s) earlier than
40
- requested; the script re-encodes automatically when the deviation exceeds 0.5 s.
41
- If the user insists on lossless output, pass `--tolerance -1` and tell them the
42
- cut lands on the nearest earlier keyframe. A `-c copy` cut on VFR or a
40
+ requested (`start_snapped: true`); the script re-encodes automatically when the
41
+ deviation exceeds 0.5 s. With `--edit-list`, a single-segment `.mp4`/`.m4v`/`.mov` copy
42
+ keeps the MP4 edit list instead: the pre-roll back to the keyframe is stored but
43
+ hidden, so the picture starts at the requested time (`edit_list`,
44
+ `stored_preroll_seconds`). A player or tool that ignores edit lists shows that
45
+ pre-roll. The **end** still lands on a packet boundary, a few frames long, and
46
+ those extra frames can skip. `.mkv` output has no edit list, and a `--segments`
47
+ join shows each part from its keyframe. On B-frame video an `.mp4`/`.mov`
48
+ `--segments` part's end moves to the first keyframe at or after it, within the
49
+ same tolerance (`segment_end_snap_seconds`); no-B-frame and `.mkv`/`.ts` parts
50
+ are not snapped. Every copy join is checked (frames, steps, each part's sound
51
+ against its picture) and re-cut when wrong. Open-GOP HEVC (iPhone "High Efficiency")
52
+ always re-encodes a `--segments` join, even with `--tolerance -1`, and so does a B-frame `.mp4`/`.mov`
53
+ join whose later segment starts between keyframes (start it on a keyframe, or write `.mkv`). If the user insists on lossless output, pass
54
+ `--tolerance -1` and tell them the cut lands on the nearest earlier keyframe.
55
+ Where a copy begins is measured with the seek ffmpeg makes: on B-frame `.mp4`/`.mov`
56
+ a start a few frames before a keyframe can begin at that keyframe or one GOP
57
+ earlier (`start_snapped: true` either way).
58
+ `av_start_skew_seconds` warns when a copy's sound and picture start apart
59
+ (Core Media HEVC once gave 3.7 s of sound with no picture); it is reported, not
60
+ repaired: `--accurate` is the fix, or `--edit-list` for a lossless `.mp4`/`.m4v`/`.mov`. A `-c copy` cut on VFR or a
43
61
  non-keyframe boundary produces a file that "works" but starts on a frozen or
44
62
  wrong frame — respect the automatic re-encode rather than forcing the copy.
45
63
 
46
64
  ### Variable frame rate
47
65
  `probe.py` sets `variable_frame_rate_suspected` when `r_frame_rate` and
48
66
  `avg_frame_rate` disagree (phone and screen recordings). Every re-encoding
49
- script then adds `-fps_mode cfr` at the source's average rate, and `cut.py`
50
- switches itself to `--accurate` (copy-cuts on VFR are unreliable). Pick the rate
67
+ script then adds `-fps_mode cfr` at the source's average rate. That flag is a
68
+ whole-file average, and a phone clip at 29.98 against a nominal 30 trips it, so
69
+ `cut.py` re-encodes on it by default and reports packet timestamps sampled in up
70
+ to five windows (`vfr_check`), with a note when they are constant;
71
+ `--vfr-guard sampled` switches to `--accurate` only when they are irregular or too
72
+ few to judge, and `--vfr-guard off` keeps the copy anyway. Pick the rate
51
73
  explicitly with `fit.py --fps 30|60` when the average is odd (e.g. 23.4 fps from
52
74
  dropped frames).
53
75
 
@@ -56,24 +56,91 @@ between tools.
56
56
  ```
57
57
  probe.py INPUT... [--compact] [--field duration|video.fps|...]
58
58
  ```
59
- JSON with `duration`, `video{codec,width,height,fps,pix_fmt,color_space,rotation,variable_frame_rate_suspected,hdr,hdr_signal,bt2020_or_hdr,hdr_format}`
59
+ JSON with `duration`, `video{codec,width,height,fps,pix_fmt,color_space,rotation,variable_frame_rate_suspected,hdr,hdr_signal,bt2020_or_hdr,hdr_format,start_time}`
60
60
  (`hdr`, and `hdr_signal` equal to it, is true only for a PQ / HLG transfer or Dolby Vision -- since 2.0;
61
61
  `bt2020_or_hdr` also counts BT.2020 primaries on an SDR transfer, which `hdr_format` names "BT.2020 SDR"),
62
- `audio{codec,channels,sample_rate}`. `--compact` gives one line per file.
62
+ `audio{codec,channels,sample_rate,start_time}` (`start_time`: where the stream starts on the file's
63
+ own clock, `null` when unknown). `--compact` gives one line per file.
63
64
 
64
65
  ### cut.py — cut / join segments
65
66
  ```
66
- cut.py INPUT [--start T] [--end T | --duration T] [--segments A-B,C-D,...] [--accurate] [-o OUT]
67
+ cut.py INPUT [--start T] [--end T | --duration T] [--segments A-B,C-D,...] [--accurate] [--edit-list] [--keep-hevc] [--vfr-guard average|sampled|off] [-o OUT]
67
68
  cut.py INPUT --start T --end T --snap beats [--snap-tolerance 0.12] [--snap-source FILE] [--min-confidence 0.5]
68
69
  ```
69
70
  Times accept `12.5`, `1:30`, `00:01:30.250`. Default is `-c copy` (snaps to
70
71
  keyframes, instant, lossless); if the snapped result deviates more than
71
72
  `--tolerance` (0.5 s) from the request, that segment is re-encoded automatically
72
- (x264 CRF 18). `--accurate` always re-encodes; `--tolerance -1` never does.
73
+ (CRF 18: x264, or x265 Main10 with the source's own tags for an HDR or BT.2020 source;
74
+ `--keep-hevc` re-encodes an SDR HEVC source (not BT.2020: that keeps its Main10 line and colour tags) as x265 8-bit BT.709, on VideoToolbox under `--hw`;
75
+ `--codec` overrides all of them). `--accurate` always re-encodes; `--tolerance -1` never does for a keyframe snap
76
+ (a `--segments` join of open-GOP video, one whose later B-frame `.mp4`/`.mov` segment starts
77
+ between keyframes, or one that fails its check, still re-cuts).
73
78
  Multiple segments are concatenated in the order given. stderr reports whether
74
79
  the result was "lossless stream copy" or "re-encoded"; when the snap forced a
75
80
  re-encode, the result's `lossless_alternative` names the nearest keyframe
76
81
  `--start` that would stream-copy instead, so the trade can be offered.
82
+ A single `.mp4`/`.mov` stream copy shifts its timestamps to zero
83
+ (`-avoid_negative_ts make_zero`), so the picture starts at the keyframe.
84
+ `--edit-list` keeps its MP4 edit list instead, so the picture starts at
85
+ `--start`: `edit_list` and `stored_preroll_seconds` say so, `notes` warns that a
86
+ player which ignores edit lists shows the pre-roll, and the copy's length is
87
+ judged by its video (a copy that starts where asked but ends past `--tolerance`
88
+ re-encodes without offering another `--start`). `av_start_skew_seconds` is
89
+ audio start minus video start; a stream copy gets a note when it is more than
90
+ max(2 frames, 0.1 s) away from the source's own offset where the picture starts
91
+ (the keyframe the copy began from; `--start` under `--edit-list`). It is
92
+ reported, not repaired. `-ss`/`-t` are passed to the microsecond; `--accurate`
93
+ seeks a second early and trims on the output side, so frames just before a
94
+ keyframe are kept. `keyframe_snapped` is true for every stream copy (`precision`
95
+ `packet`); `start_snapped` is measured: true when a copied picture starts more
96
+ than a frame from `--start`, or when its start could not be measured (an unknown start is
97
+ never claimed exact); `null` under `--dry-run`.
98
+ `reencode_reason` lists every cause of a re-encode (`requested`, `codec`, `vfr`,
99
+ `vfr_inconclusive`, `pcm_container`, `copy_failed`, `tolerance`, `concat_fallback`);
100
+ `--segments` adds `segment_precision`, and `least_exact_precision` is the least exact one.
101
+ A `--segments` video copy join adds `join_check` (`packets`, `expected_packets`,
102
+ `max_step_seconds`, `audio_offset_ms` per part, `audio_parts_checked`, `ok`: the
103
+ written file measured against the source) and `segment_end_snap_seconds` (where
104
+ each part's end moved to its keyframe).
105
+
106
+ **VFR guard (`--vfr-guard`).** `average` (default): a source whose nominal and
107
+ average frame rates differ (`variable_frame_rate_suspected`) re-encodes as
108
+ `--accurate` (reason `vfr`), and `cut.py` then samples its packet timestamps (up
109
+ to five 6 s windows; no decoding) into `vfr_check` (`measured`: `sampled_cfr`,
110
+ `vfr` or `inconclusive`, plus `guard`); a `sampled_cfr` there adds a note naming
111
+ `--vfr-guard sampled`. `sampled` samples before every copy and re-encodes only on
112
+ `vfr` / `inconclusive` (reasons `vfr` / `vfr_inconclusive`), so a phone clip at
113
+ 29.98 against a nominal 30 copies; `off` samples and keeps the copy with a note.
114
+ Irregular timing between the windows is not seen; `--accurate` is always available.
115
+
116
+ A source with no B-frames (iPhone "Most Compatible" H.264) cuts its parts as
117
+ every 2.x release did: from the keyframe at (or before) each start to the end
118
+ asked for, each part within `--tolerance`. On B-frame video a copied
119
+ `.mp4`/`.mov` part runs from the keyframe at (or before) its start to the first
120
+ keyframe at or after its end (never an earlier one), each end within
121
+ `--tolerance`, so the join is the source's own frames with no gaps; a later
122
+ segment that starts between keyframes there is re-cut (its hidden pre-roll
123
+ cannot cross a join). A source with **open GOPs** (x265's default, iPhone "High
124
+ Efficiency" HEVC) cannot be copied exactly across a join and is re-cut
125
+ (`concat_fallback`, named in `notes`). Every copy join is measured after it is
126
+ written (`join_check`): the frame count, each step at a join against the step
127
+ its parts predict, and each part's sound against its picture (audio packets
128
+ matched to the source; more than 5 ms off fails, a part with no unique match
129
+ is unmeasured). One that fails is re-cut. Only windows around the segments
130
+ (about 10 s each side, widened as needed) are read from the source, so a long
131
+ source costs what its segments do.
132
+ The parts are joined by stream copy only when they all copied and match: the
133
+ same streams with the same codec parameters, rotation, colour tags and
134
+ extradata. Otherwise every segment is **re-cut from
135
+ the source** into one re-encode (`concat_fallback`): frame-exact boundaries, the
136
+ audio's offset from the video kept, subtitles dropped and reported, and at most
137
+ 32 segments per ffmpeg call. A segment shorter than one video frame is refused.
138
+ With `--accurate` the segments are always cut that way, in one encode (a part
139
+ encoded on its own ran an AAC frame past its picture, leaving a hole at every
140
+ join). A segment other than the last that runs past the end of the video is
141
+ ended with the video when it runs less than a frame past it, and otherwise keeps
142
+ its sound with the last frame held -- which forces the re-cut (`concat_fallback`,
143
+ named in `notes`). The last segment keeps the source's own sound-only tail.
77
144
 
78
145
  **`--snap beats` (1.17)** moves each in/out point to the nearest *measured*
79
146
  beat within `--snap-tolerance` seconds (default 0.12, about a quarter of a beat
@@ -550,7 +617,7 @@ Three refusals, all `kind: input`, all before any encode:
550
617
 
551
618
  Results: `filler.removed_count`, `filler.removed_seconds`, `filler.removed`
552
619
  (one entry per span), `filler.lang`, `filler.list`, `filler.word_timings` and
553
- `filler.warnings`. With `--transcribe`, `filler.source` is `whisper:ENGINE` or
620
+ `filler.warnings`; `--filler-lang auto` refuses (`kind: input`) a transcript in a language with no built-in list, so pass `--filler-words FILE` for it. With `--transcribe`, `filler.source` is `whisper:ENGINE` or
554
621
  `parakeet:ENGINE` and `filler.transcription` says which engine, model, language
555
622
  and routing made the words; a top-level `notes` line says when `--engine auto`
556
623
  ran the English-only Parakeet model on English assumed (no `--filler-lang`, and
@@ -905,6 +972,13 @@ batch.py FOLDER --recipe batch.json [--force] [--watch SECONDS] [--jobs N|auto]
905
972
  placeholders, chained) or `project` (a render project applied per file).
906
973
  Outputs land in `output_dir` with `suffix`; a content-hash cache skips files
907
974
  already done with the same recipe. Use `--dry-run` to preview the plan.
975
+ A recipe that runs `cut.py` rolls its calls up: each of those `results` rows carries
976
+ `cut_reencoded` (each call's `reencoded`) and `cut_reencode_reasons` (each
977
+ `reencode_reason` its calls named, once per call),
978
+ and the top level `cut_stream_copy` (`{calls, stream_copy, reencoded,
979
+ stream_copy_rate}`) and `cut_reencode_reasons` (counts per reason; `unknown` for a
980
+ re-encoded call cached by an earlier 2.x batch); both are `null`
981
+ when no `cut.py` step ran.
908
982
 
909
983
  **`--jobs N` (1.17)** processes N files at once (threads: the work is
910
984
  subprocess waiting). Capped at `min(N, cpu_count, 8)` — every item is itself an
@@ -59,7 +59,7 @@ from _common.runner import (
59
59
  ffmpeg_version, flush_drawtext_textfiles, INSTALL_HINTS, install_signal_handlers, _is_ffmpeg, _limit_for,
60
60
  _odd_dimension_retry, _on_signal, _OutputLock, _pid_dead, place_output, PROBE_TIMEOUT, _progress_line,
61
61
  read_text_or_die, refuse_output_is_input, _remember_output, require_tool, run, run_analysis, _run_captured,
62
- run_keeping_subtitles, run_tool, _run_with_progress, shell_quote, _SIGNALS_INSTALLED, _stage_existing_output,
62
+ run_keeping_subtitles, run_tool, _run_with_progress, shell_quote, sibling_temp, _SIGNALS_INSTALLED, _stage_existing_output,
63
63
  STATE, _timed_out, _unwatch, _watch, X264_PRESETS
64
64
  )
65
65
  from _common.emit import (
@@ -68,7 +68,7 @@ from _common.emit import (
68
68
  )
69
69
  from _common.probe import (
70
70
  analyze_levels, _aspect_string, _bit_depth, decode_pcm_mono, detect_scenes, detect_silences, fingerprint,
71
- _fraction, keyframes_near, SCORE_RE, SIL_RE, decode_gray_frames,
71
+ _fraction, keyframes_near, classify_frame_timing, measure_frame_timing, SCORE_RE, SIL_RE, decode_gray_frames,
72
72
  measured_level_dbfs, AUDIO_MEDIA_EXT, MEDIA_EXT, _output_failed, probe, rms_envelope, _to_float, _to_int, verify_output
73
73
  )
74
74
  from _common.decision import (
@@ -83,7 +83,7 @@ from _common.decision import (
83
83
  _onset_strength, _pick_onsets, _autocorrelation_peak, _grid_score,
84
84
  brand_states_font, cfr_args, concat_list_line, db_to_linear, default_output, encoder_args, escape_filter_path,
85
85
  fmt_secs, fmt_smpte_time, fmt_srt_time, frame_size, is_audio_output, load_brand, MissingFpsError, pad_filters, parse_time,
86
- signed_time_arg, SVT_PRESET, time_arg, video_args, x264_args, _x264_raw
86
+ signed_time_arg, source_codec_video_args, SVT_PRESET, time_arg, video_args, x264_args, _x264_raw
87
87
  )
88
88
  from _common.asr import (
89
89
  ASR_ENGINES, ASR_INSTALL_HINT, _asr_run, die_no_engine, parse_srt, transcribe, _transcribe_in,
@@ -200,7 +200,7 @@ __all__ = [
200
200
  "font_family_for_script", "font_family_of_file", "FONT_FLAG_HINT", "font_for_script", "FONT_INSTALL_HINT",
201
201
  "fonts_dir_covers_script", "_fraction", "has_emoji", "info", "INSTALL_HINTS", "install_signal_handlers",
202
202
  "is_audio_output", "_is_emoji_base", "_is_emoji_char", "_is_ffmpeg", "_is_mark", "_KEYCAP", "_KEYCAP_BASES",
203
- "keyframes_near", "LANGUAGE_NAMES", "LATIN_EM", "LEADING_VOWELS", "_libass_color_probe", "_limit_for",
203
+ "keyframes_near", "classify_frame_timing", "measure_frame_timing", "LANGUAGE_NAMES", "LATIN_EM", "LEADING_VOWELS", "_libass_color_probe", "_limit_for",
204
204
  "load_brand", "measured_level_dbfs", "AUDIO_MEDIA_EXT", "MEDIA_EXT", "MissingFpsError", "needs_shaping", "NO_SPACE_SCRIPTS",
205
205
  "_odd_dimension_retry", "_on_signal", "_output_failed", "_OutputLock", "pad_filters", "parse_time", "_pid_dead",
206
206
  "place_output", "_plan_at_exit", "_plan_inputs", "_PLAN_STRIP", "PLAN_VERSION", "PREFERRED_FAMILIES",
@@ -209,10 +209,10 @@ __all__ = [
209
209
  "decode_gray_frames", "frame_flow", "label_shot_flow", "MOTION_GRID", "MOTION_SEARCH", "MOTION_STATIC_PX", "MOTION_PAN_SPREAD",
210
210
  "_run_captured", "run_keeping_subtitles", "run_tool", "_run_with_progress", "_SCRIPT_FONT_CACHE",
211
211
  "_script_font_entry", "script_font_for_text", "script_font_status", "_script_font_uncached", "_SCRIPT_RANGES",
212
- "SCRIPTS", "_sdr_bt709", "_set_current_ctx", "_SHAPING_BUILD_CACHE", "SHAPING_SCRIPTS", "shell_quote",
212
+ "SCRIPTS", "_sdr_bt709", "_set_current_ctx", "_SHAPING_BUILD_CACHE", "SHAPING_SCRIPTS", "shell_quote", "sibling_temp",
213
213
  "_SIGNALS_INSTALLED", "signed_time_arg", "_stage_existing_output", "STATE", "SVT_PRESET", "text_width_em",
214
214
  "time_arg", "_timed_out", "_to_float", "_to_int", "_unwatch", "validate_color", "verify_output",
215
- "video_args", "_VS15", "_VS16", "_watch", "WINDOWS_FONTS", "write_plan", "x264_args", "X264_PRESETS",
215
+ "source_codec_video_args", "video_args", "_VS15", "_VS16", "_watch", "WINDOWS_FONTS", "write_plan", "x264_args", "X264_PRESETS",
216
216
  "_x264_raw", "_ZWJ",
217
217
  "ASR_ENGINES", "ASR_INSTALL_HINT", "_asr_run", "die_no_engine", "parse_srt", "transcribe",
218
218
  "_transcribe_in", "transcribe_words", "_words_from_openai_whisper_json",
@@ -301,6 +301,7 @@ def _skipped_parakeet(route: "Route", language: Optional[str], language_flag: st
301
301
  def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str], model: str, audio_stream: int,
302
302
  ffmpeg: str, shutil, subprocess, wanted: str = "auto") -> Transcription:
303
303
  from _common import run_analysis, STATE
304
+ language = engine_language(language)
304
305
  wav = os.path.join(tmpdir, "audio.wav")
305
306
  # A wav in our own temp dir: a measurement input for the engine, not a deliverable, so it
306
307
  # is not a run() call (no --dry-run gate, not recorded), but it keeps the time limit and
@@ -360,12 +361,14 @@ def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str
360
361
  import threading
361
362
  result: list = []
362
363
  crashed: list = []
364
+ heard: list = [] # the language faster-whisper decoded, when it names one
363
365
 
364
366
  def work() -> None:
365
367
  try:
366
368
  m = WhisperModel(model, device="cpu", compute_type="int8")
367
- segments, _ = m.transcribe(wav, language=lang, word_timestamps=False)
369
+ segments, decoded = m.transcribe(wav, language=lang, word_timestamps=False)
368
370
  result.extend((seg.start, seg.end, seg.text.strip()) for seg in segments if seg.text.strip())
371
+ heard.append(getattr(decoded, "language", None))
369
372
  except Exception as exc: # noqa: BLE001 -- a model that would not load or run: a failed engine
370
373
  crashed.append(exc)
371
374
 
@@ -388,7 +391,8 @@ def _transcribe_in(tmpdir: str, video: str, out_srt: str, language: Optional[str
388
391
  info("transcribed with faster-whisper")
389
392
  write_srt(cues, out_srt)
390
393
  return Transcription(cues=cues, engine="faster-whisper",
391
- facts=dict(facts, engine="faster-whisper", model=model, language=lang))
394
+ facts=dict(facts, engine="faster-whisper", model=model,
395
+ language=lang or _heard_language(heard)))
392
396
  except ImportError:
393
397
  pass
394
398
  # 3. openai-whisper CLI
@@ -508,6 +512,23 @@ def _words_from_whisper_cpp_json(path: str) -> "List[Dict[str, Any]]":
508
512
  return [w for w in out if w["word"].strip()]
509
513
 
510
514
 
515
+ def _whisper_json_language(path: str) -> Optional[str]:
516
+ """The language a Whisper engine's JSON says it decoded: whisper.cpp's `result.language`, or
517
+ openai-whisper's top-level `language`. None when the file has neither."""
518
+ try:
519
+ doc = json.loads(Path(path).read_text(encoding="utf-8")) or {}
520
+ lang = str(((doc.get("result") or {}).get("language")) or doc.get("language") or "").strip().lower()
521
+ except (OSError, ValueError, AttributeError):
522
+ return None
523
+ return lang or None
524
+
525
+
526
+ def _heard_language(heard: list) -> Optional[str]:
527
+ """The language an in-process engine decoded (faster-whisper's info.language), lower-cased, or None."""
528
+ lang = next((str(h).strip().lower() for h in heard if h), "")
529
+ return lang or None
530
+
531
+
511
532
  def _words_from_openai_whisper_json(path: str) -> "List[Dict[str, Any]]":
512
533
  """openai-whisper --word_timestamps True --output_format json: segments[].words[]."""
513
534
  try:
@@ -644,6 +665,22 @@ def _is_english(language: Optional[str]) -> bool:
644
665
  return (language or "").lower().split("-")[0].split("_")[0] in ("en", "eng", "english")
645
666
 
646
667
 
668
+ def engine_language(language: Optional[str]) -> Optional[str]:
669
+ """`--language` as the engines take it. "auto" (caption.py's documented default, spelled out)
670
+ is no language: None, the undetected route. Every spelling of English (en-US, eng, English)
671
+ is "en", which Parakeet accepts as English and Whisper accepts as a language code; any other
672
+ value is its two-letter primary language (ja-JP -> ja)."""
673
+ named = (language or "").strip()
674
+ if not named or named.lower() == "auto":
675
+ return None
676
+ if _is_english(named):
677
+ return "en"
678
+ # Whisper takes the two-letter code: a regional or script subtag (ja-JP, pt-BR, zh-Hant) names the
679
+ # same language to it, and an ISO 639-2 code (jpn) is passed as given
680
+ primary = named.replace("_", "-").split("-")[0].lower()
681
+ return primary if len(primary) == 2 else named
682
+
683
+
647
684
  def _whisper_cpp_cli(shutil) -> Optional[str]:
648
685
  cli = shutil.which("whisper-cli") or shutil.which("whisper-cpp")
649
686
  if not cli:
@@ -675,7 +712,7 @@ def detect_language(wav: str, shutil, subprocess) -> Optional[str]:
675
712
  if not cli:
676
713
  return None
677
714
  model = next((p for p in (_whisper_cpp_model(m) for m in ("tiny", "base", "small", "tiny.en"))
678
- if os.path.exists(p) and not p.endswith(".en.bin")), None)
715
+ if os.path.exists(p) and not _english_only_whisper_model(p)), None)
679
716
  if not model:
680
717
  return None
681
718
  proc = _asr_run([cli, "-m", model, "-f", wav, "-dl", "-l", "auto"], subprocess, "whisper.cpp language detection")
@@ -756,9 +793,10 @@ def _names_parakeet_model(name: str) -> bool:
756
793
 
757
794
 
758
795
  def _english_only_whisper_model(name: str) -> bool:
759
- """An English-only Whisper model: tiny.en, base.en, ... or a ggml-*.en.bin file."""
796
+ """An English-only Whisper model: tiny.en, base.en, ... or a ggml-*.en.bin file, in any
797
+ quantisation (ggml-base.en-q5_1.bin, tiny.en-q8_0) or variant (ggml-small.en-tdrz.bin)."""
760
798
  base = os.path.basename(name).lower()
761
- return (base[:-4] if base.endswith(".bin") else base).endswith(".en")
799
+ return re.search(r"\.en(-[a-z0-9_]+)*$", base[:-4] if base.endswith(".bin") else base) is not None
762
800
 
763
801
 
764
802
  def _whisper_ready(shutil, model: Optional[str]) -> bool:
@@ -913,6 +951,7 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
913
951
  from _common import require_tool, run_analysis, STATE
914
952
  ffmpeg = require_tool("ffmpeg")
915
953
  wanted = requested_engine(engine)
954
+ language = engine_language(language)
916
955
  parakeet_model = None if model == "base" else model
917
956
  tmpdir = _tempfile.mkdtemp(prefix="ffskill_asrw_")
918
957
  try:
@@ -962,7 +1001,8 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
962
1001
  words = _words_from_whisper_cpp_json(base + ".json")
963
1002
  info(f"word timings from whisper.cpp ({len(words)} words)")
964
1003
  return Transcription(words=words, engine="whisper.cpp",
965
- facts=dict(facts, engine="whisper.cpp", model=model_path, language=lang))
1004
+ facts=dict(facts, engine="whisper.cpp", model=model_path,
1005
+ language=lang or _whisper_json_language(base + ".json")))
966
1006
  line = _failure_line(proc)
967
1007
  info("whisper.cpp found but produced no word-timing JSON: " + line)
968
1008
  tried.append(_failed("whisper.cpp", "found but produced no word-timing JSON: " + line))
@@ -976,11 +1016,13 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
976
1016
  import threading
977
1017
  collected: list = []
978
1018
  crashed: list = []
1019
+ heard: list = []
979
1020
 
980
1021
  def work() -> None:
981
1022
  try:
982
1023
  m = WhisperModel(model, device="cpu", compute_type="int8")
983
- segments, _ = m.transcribe(wav, language=lang, word_timestamps=True)
1024
+ segments, decoded = m.transcribe(wav, language=lang, word_timestamps=True)
1025
+ heard.append(getattr(decoded, "language", None))
984
1026
  for seg in segments:
985
1027
  for w in (getattr(seg, "words", None) or []):
986
1028
  collected.append({"word": str(w.word).strip(),
@@ -1002,7 +1044,8 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
1002
1044
  else:
1003
1045
  info(f"word timings from faster-whisper ({len(collected)} words)")
1004
1046
  return Transcription(words=[w for w in collected if w["word"]], engine="faster-whisper",
1005
- facts=dict(facts, engine="faster-whisper", model=model, language=lang))
1047
+ facts=dict(facts, engine="faster-whisper", model=model,
1048
+ language=lang or _heard_language(heard)))
1006
1049
  except ImportError:
1007
1050
  pass
1008
1051
 
@@ -1018,7 +1061,8 @@ def transcribe_words_result(video: str, language: "Optional[str]" = None, model:
1018
1061
  words = _words_from_openai_whisper_json(doc)
1019
1062
  info(f"word timings from openai-whisper ({len(words)} words)")
1020
1063
  return Transcription(words=words, engine="openai-whisper",
1021
- facts=dict(facts, engine="openai-whisper", model=model, language=lang))
1064
+ facts=dict(facts, engine="openai-whisper", model=model,
1065
+ language=lang or _whisper_json_language(doc)))
1022
1066
  line = _failure_line(proc)
1023
1067
  info("openai-whisper found but produced no word-timing JSON: " + line)
1024
1068
  tried.append(_failed("openai-whisper", "found but produced no word-timing JSON: " + line))
@@ -478,6 +478,19 @@ def video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "med
478
478
  return _maybe_hw("hevc", crf, meta, True, _x265_hdr_default(v, crf, preset))
479
479
 
480
480
 
481
+ def source_codec_video_args(meta: Optional[Dict[str, Any]], crf: int = 18, preset: str = "medium",
482
+ keep_hevc: bool = False) -> List[str]:
483
+ """video_args(), except that with `keep_hevc` (cut.py --keep-hevc) an SDR HEVC source re-encodes
484
+ to HEVC (8-bit, BT.709-tagged, on VideoToolbox when --hw asks) instead of H.264: a cut is a trim,
485
+ so a re-encoded segment can come out in the codec the source (and any copied segment beside it)
486
+ is in. Opt-in within 2.x, where an SDR re-encode stays x264 unless asked. An HDR or BT.2020
487
+ source always takes video_args()'s Main10 line, and --codec always wins."""
488
+ v = (meta or {}).get("video") or {}
489
+ if keep_hevc and not STATE.codec and v.get("codec") == "hevc" and not v.get("bt2020_or_hdr"):
490
+ return encoder_args("hevc", crf, preset, meta)
491
+ return video_args(meta, crf, preset)
492
+
493
+
481
494
  def _x265_hdr_default(v: Dict[str, Any], crf: int, preset: str) -> List[str]:
482
495
  cs = v.get("color_space") or "bt2020nc"
483
496
  prim = v.get("color_primaries") or "bt2020"
@@ -115,7 +115,7 @@ def drawtext_text_opts(text: str, tmpdir: "Optional[str]" = None) -> str:
115
115
  `]`, `;` and `\` all reach the picture verbatim -- 1.15 fixes `overlay.py --text "it's 100%
116
116
  done"` losing both characters. A newline is kept (`\r\n` and `\r` become `\n`): drawtext
117
117
  draws it as a line break, and graphics.py's wrap puts them there on purpose (stripping them
118
- ran a wrapped title off both edges of the frame). A tab, vertical tab or form
118
+ ran a wrapped title off both edges of the frame); trailing newlines are dropped. A tab, vertical tab or form
119
119
  feed becomes a space, as the wrap measures it; other control characters are stripped (that
120
120
  dates from the inline-escape route, where they broke the graph parser).
121
121
 
@@ -127,6 +127,9 @@ def drawtext_text_opts(text: str, tmpdir: "Optional[str]" = None) -> str:
127
127
  """
128
128
  cleaned = (text or "").replace("\r\n", "\n").replace("\r", "\n")
129
129
  cleaned = re.sub(r"[\x00-\x08\x0e-\x1f\x7f]", "", re.sub(r"[\t\x0b\x0c]", " ", cleaned))
130
+ # drawtext ignores one trailing newline and draws a second as a blank line, which would
131
+ # centre the block half a line too high; a label that is only newlines stays as it is
132
+ cleaned = cleaned.rstrip("\n") or cleaned
130
133
  import hashlib
131
134
  name = "t_" + hashlib.sha256(cleaned.encode("utf-8")).hexdigest()[:16] + ".txt"
132
135
  if tmpdir is None: