ffmpeg-skill 1.16.0 → 1.17.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -82,7 +82,7 @@ An agent that "knows FFmpeg" still guesses: it assumes a frame rate, picks a cod
82
82
  - **Structured tools, not shell strings.** Each operation is a script with typed arguments. Nothing runs through a shell; no filter graph is accepted from the caller.
83
83
  - **A contract the agent can read.** `contract --json` states, for every tool, what it takes, what it writes, which FFmpeg components it needs and how the result is verified. The MCP surface is derived from it.
84
84
  - **Verification after execution.** The result is probed, checked against the destination's spec and, when the picture changed, looked at as a contact sheet.
85
- - **Local first.** No cloud, no API keys, no Python dependencies. Optional local transcription is used when a whisper is installed, never required.
85
+ - **Local first.** No cloud, no API keys, no Python dependencies. Optional local transcription is used when a whisper is installed, never required — by `caption.py --transcribe` and, since 1.17, by `silence.py --filler --transcribe`; both take a transcript you already have instead, and both refuse with the install lines rather than guessing.
86
86
 
87
87
  ## Quick start
88
88
 
@@ -190,16 +190,16 @@ These are the rules the skill file gives the agent and the code enforces.
190
190
  | Tool | What it does |
191
191
  |---|---|
192
192
  | `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, bit depth, HDR format incl. Dolby Vision (`hdr` for BT.2020 or PQ/HLG, `hdr_signal` for a real PQ/HLG/DV transfer only), colour space, rotation, every audio stream; `--analyze` flags Log footage |
193
- | `scenes.py` | Scene changes, audio peaks, highlight proposals (`--rank-by audio` loudest, or `--rank-by duration` longest — both proxies, not "best") and a per-scene sheet; cut list for `cut.py --segments` |
193
+ | `scenes.py` | Scene changes, audio peaks, highlight proposals (`--rank-by audio` loudest, or `--rank-by duration` longest — both proxies, not "best") and a per-scene sheet; cut list for `cut.py --segments`; `--beats` measures the music's beat grid (tempo, beat times, confidence) |
194
194
  | `look.py` | Contact sheet, single frames, side-by-side comparison as PNG so the agent can see what it made; `--safe NAME` shades the zones a platform's own UI covers |
195
195
 
196
196
  **Editing**
197
197
 
198
198
  | Tool | What it does |
199
199
  |---|---|
200
- | `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact video and sample-exact audio; reports `precision` |
200
+ | `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact video and sample-exact audio; reports `precision`; `--snap beats` moves the in/out points onto a measured beat, or refuses when there is no measurable pulse |
201
201
  | `join.py` | Concatenate clips with xfade transitions, normalising size, fps, sample rate and channel layout (the widest clip's, or `--channels`); audio-only inputs are joined as audio |
202
- | `silence.py` | Detect and remove dead air (jump cuts) with a margin around speech; list or export the cut list |
202
+ | `silence.py` | Detect and remove dead air (jump cuts) with a margin around speech; list or export the cut list; `--filler` also removes filler words, but only where a speech engine timed them |
203
203
  | `fit.py` | Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad, crop or `--fit blur`'s blurred, darkened fill, with `--crop-x`/`--crop-y` to keep an off-centre subject) and/or exact `--width`/`--height`; rotate 90/180/270, flip h/v; force constant fps |
204
204
  | `crop.py` | Crop to an exact pixel rectangle (`--x --y --width --height`) — distinct from `fit.py --fit crop`, which crops to an aspect ratio it computes itself |
205
205
  | `cropdetect.py` | Measure existing black letterbox/pillarbox bars and report the `crop.py`-ready rectangle that removes them — analysis only, writes no file |
@@ -234,7 +234,7 @@ These are the rules the skill file gives the agent and the code enforces.
234
234
 
235
235
  | Tool | What it does |
236
236
  |---|---|
237
- | `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; wraps to the safe area with a phrase-aware breaker (`--wrap phrase|measured`) and `--max-lines`/`--min-duration`/`--offset`; `--mode mux` takes a repeated `--srt file:lang` for several language-tagged, toggleable tracks in one file; picks a font by script for non-Latin text (`--lang`); animated and word-by-word karaoke timed to the speech energy or real word timings; optional local transcription |
237
+ | `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; wraps to the safe area with a phrase-aware breaker (`--wrap phrase|measured`) and `--max-lines`/`--min-duration`/`--offset`; `--mode mux` takes a repeated `--srt file:lang` for several language-tagged, toggleable tracks in one file; picks a font by script for non-Latin text (`--lang`); animated and word-by-word karaoke timed to the speech energy or real word timings; `--fit-size` shrinks the size until a cue fits `--max-lines` instead of splitting the sentence; optional local transcription |
238
238
  | `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades; `--platform NAME` keeps them clear of that destination's UI; `--video` for picture-in-picture, `--chromakey` for green-screen compositing |
239
239
  | `graphics.py` | Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs, social stickers, opening hook cards and meme captions drawn by FFmpeg from a brand kit; `--platform NAME` keeps them inside that destination's safe zone; `--text-render` routes shaping scripts through libass and `--emoji-assets` composites colour emoji |
240
240
  | `color.py` | HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting, typed primary correction (exposure/contrast/saturation/gamma/white balance/lift-gain/levels/curves) |
@@ -252,8 +252,8 @@ These are the rules the skill file gives the agent and the code enforces.
252
252
 
253
253
  | Tool | What it does |
254
254
  |---|---|
255
- | `render.py` | Render a whole edit from a declarative `project.json` (clips, transitions, captions, overlays including the social sticker/hook/meme graphics, music and stem levels, loudness, export, chapter markers, check); `--init`, `--dry-run`, `--stop-after`; `--template NAME INPUT` renders a shipped delivery template (`--template all` writes the whole social pack plus its table) |
256
- | `batch.py` | Apply a step recipe or a project to a folder with a content-hash cache; `--watch` |
255
+ | `render.py` | Render a whole edit from a declarative `project.json` (clips, transitions, captions, overlays including the social sticker/hook/meme graphics, music and stem levels, loudness, export, chapter markers, check); `--init`, `--dry-run`, `--stop-after`, `--cache DIR`/`--from STAGE` (reuse the stages that did not change); `--template NAME INPUT` renders a shipped delivery template (`--template all` writes the whole social pack plus its table) |
256
+ | `batch.py` | Apply a step recipe or a project to a folder with a content-hash cache; `--watch`; `--jobs N` processes several files at once under one shared `--timeout` |
257
257
  | `multicam.py` | Align any number of cameras and recorders by audio (with drift correction) and cut between them from a switch list |
258
258
  | `verify.py` | Run the toolchain on real device files and report PASS / FAIL per step |
259
259
 
@@ -361,7 +361,11 @@ The short list for humans. The agent-facing version, with the reasoning, is the
361
361
  - **Loudness targets.** −14 LUFS / −1 dBTP for YouTube and social platforms (the `loudness.py` default), `-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast. A clip measured at −40 LUFS or below is room tone, not content; raising it raises the noise. Check true peak as well as LUFS: `check.py file --platform podcast` measures both.
362
362
  - **Frame changes first, text second.** Captions and overlays burned before a crop or resize end up off-frame. Reframe, then caption.
363
363
  - **Cropping 16:9 to 9:16 discards 70 % of the width.** `fit.py --fit crop` centres by default; pass `--crop-x`/`--crop-y` toward the subject, or pad with `--fit pad --pad-fill blur`. Look at the contact sheet before deciding.
364
- - **Phrase-aware caption breaking (1.16).** `caption.py`/`graphics.py --wrap phrase` (the default) never breaks inside a word or on the wrong side of a hyphen, never leaves a lone digit, kana or punctuation pair on a line, prefers Japanese sentence ends and particles over a mid-word break, and never ends a line on an article or preposition. All four are penalties over break positions that already fit, so no line is widened and the line count never changes; `--wrap measured` restores 1.15's width-only wrap. The text itself is never rewritten or shortened.
364
+ - **Phrase-aware caption breaking (1.16).** `caption.py`/`graphics.py --wrap phrase` (the default) never breaks inside a word or on the wrong side of a hyphen, never leaves a lone digit, kana or punctuation pair on a line, prefers Japanese sentence ends and particles over a mid-word break, and never ends a line on an article or preposition. All four are penalties over break positions that already fit, so no line is widened and the line count never changes; `--wrap measured` restores 1.15's width-only wrap. The text itself is never rewritten or shortened. Since 1.16.1 a Thai run and a katakana word are never broken inside (Thai writes no space inside a phrase and there is no dictionary: the break goes where you put a space or `|`), and a line wider than the safe width is reported as `overlong` with the fix named.
365
+ - **Caption size fitted to the cue (1.17).** At a platform caption size a line holds about six em, so an ordinary sentence needs four lines and `--max-lines 2` used to cut it into consecutive cues — half the sentence arriving late. `caption.py --fit-size` (default `auto`) now walks the size down until every cue fits, *before* laying the cues out, with a legibility floor of 4.5 % of the frame height (`--min-size`, default 13 ASS units). `--fit-size off` is 1.16 byte for byte, an explicit `--size` or a `brand.json` size is never overridden, and the caption text is still never rewritten to make it fit.
366
+ - **Beat-synced cuts (1.17).** `scenes.py --beats` reports the measured grid — tempo, beat times, and a confidence built from how far the winning autocorrelation lag stands above the others and how many onsets land on it. `cut.py --snap beats` (and a `"snap"` block in a `render.py` project) moves in/out points to the nearest beat within `--snap-tolerance` — and only onto the grid points a measured onset actually marks, never onto the regular grid's continuation through a passage with no music in it. Below `--min-confidence` it **refuses**: a cut point may move to a measured beat and may not appear from one, so speech and ambience get an honest "no steady pulse here" instead of an invented grid.
367
+ - **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Whisper stays optional: `--transcribe` with no engine installed refuses and names the three installs.
368
+ - **Throughput (1.17).** `batch.py --jobs N` runs several files at once, capped at `min(N, cpu_count, 8)` and sharing one `--timeout` budget rather than one per item; the per-item table keeps its order. `render.py --cache DIR` reuses stages whose inputs and arguments did not change, so swapping an export preset re-runs export only. The cache is opt-in with no default directory, and the ffmpeg, skill and contract versions are part of every key, so a cache is never reused across them.
365
369
  - **Audiogram (1.16).** `waveform.py --image cover.png` (or `render.py --template audiogram`) puts the waveform over a still plate for an episode that has no picture, with `--platform` for the frame, `--title` and burnt-in captions. The image is a local file you give: nothing is fetched and no cover art is ever invented.
366
370
  - **Emoji in captions and titles (1.15).** `caption.py`/`graphics.py --emoji-assets DIR` composites a PNG per emoji (Twemoji/Noto naming, `1f389.png`) on top of the text, because drawtext cannot load a colour emoji font at all and an installed one does not prove libass will draw it in colour — `doctor --json`'s `fonts.emoji` answers that from a render probe. Without assets the run still succeeds and reports `mode: mono`. Nothing is ever downloaded.
367
371
  - **Indic and Thai text shaped correctly in titles and lower-thirds (1.15).** `graphics.py` renders Devanagari, Bengali, Tamil, Thai and Lao through libass automatically (`text_renderer: "ass"`), because drawtext never reorders matras or re-clusters marks; Arabic and Hebrew were already correct on a fribidi build. `--text-render drawtext` with such a script is refused, never rendered wrongly.
@@ -407,6 +411,7 @@ type on every OS.
407
411
  | **F1 0.97** | `scenes.py`, 53 hard cuts between single takes, precision 0.95, recall 1.00 at the default threshold |
408
412
  | **exact to the sample** | `cut.py --accurate` on WAV, FLAC (44.1 kHz) and AAC → WAV; WAV stream copy within 2 ms; AAC output +21 ms of encoder priming, reported as `codec_frame` (0.9.1) |
409
413
  | **72 / 72** | agent runs of 24 prompts (12 English edits, 8 Japanese, 4 that must be declined), three repeats, graded by an independent model: routing, honest refusals and user's language 72/72, report format 71/72, visual check whenever the picture changed 24/24 (0.8.4) |
414
+ | **90 / 90** | 1.16.0 run (2026-09-14, one pass per prompt, Sonnet agent, regex grader + focused Opus grader on 30 runs, chapters and subtitle streams re-probed, check.py re-run on every delivery output) on the set grown to 90 prompts (audiogram, auto chapters, multi-language tracks, caption breaking): routing 90/90, honest refusals and failures 90/90 with 0 false successes and 0 raw ffmpeg calls, report format 89/90 (one `Done (partially):`), user's language 90/90 by regex (89/90 by Opus), audiogram 2/2 with the cover behind the waveform and nothing fetched, auto chapters 2/2 with `Chapter N` titles only, delivery 16/16 platform pass, trigger set 45/45, Opus quality mean 4.17. The honest part: the phrase breaker never gets to act at the platform caption sizes (a five-word cue does not fit two lines at TikTok size, so the split is byte-identical to 1.15.1), Thai still breaks inside words, and a katakana word was split — 1.16.1 is the patch and the finding is written up in `evals/results/iteration-17.json` |
410
415
  | **82 / 82** | 1.15.0 run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader on 28 runs, stills extracted inside the emoji window, check.py re-run on every delivery output) on the set grown to 82 prompts (emoji captions and title cards, a Hindi and a Thai lower-third): routing 82/82, honest refusals and failures 82/82 with 0 false successes and 0 raw ffmpeg calls, report format 82/82 (both iteration-15 label defects closed: `dl8` and `he2` now carry one `Failed:`), user's language 82/82 by regex (81/82 by Opus: one Spanish report with three English labels), non-Latin glyphs 11/11 (Devanagari through `graphics.py` is fixed; Thai lower-third and captions correct), emoji visible in colour in 3/3 runs given PNG assets and reported monochrome in the one that was not, visual check 23/24, delivery 12/13 one encode and 13/13 platform pass, trigger set 40/40, Opus quality mean 4.68. Still open: the caption breaker splits phrases (`dl1`, `dl4` unchanged) — queued for 1.16.0. Tokens per run flat at 73.3k on the same 76. Details in `evals/results/iteration-16.json` |
411
416
  | **76 / 76** | 1.14.0 run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader on 26 runs, check.py re-run on every delivery output): routing 76/76, honest refusals and failures 76/76 with 0 false successes and 0 raw ffmpeg calls, report format 76/76, user's language 76/76 by regex (75/76 by Opus: one Spanish report with three English labels), visual check 18/18, trigger set 38/38, Opus quality mean 4.58. The delivery templates did their job: 12 of 13 delivery requests went through `render.py --template`, finished in one encode (was 3 of 7) and all 13 pass their platform check (was 7 of 8). Tokens per run flat at 73.4k. Details in `evals/results/iteration-15.json` |
412
417
  | **76 / 76** | 1.13.0 run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader) on the set grown to 76 prompts: 18 in Thai, Hindi, Hebrew, Russian, Greek, Vietnamese, Indonesian, Turkish and Italian, and 8 delivery requests (TikTok, Reels, Shorts, LinkedIn, Douyin, podcast): routing 76/76, honest refusals and failures 76/76 with 0 false successes and 0 raw ffmpeg calls, report format 76/76, user's language 76/76 across seventeen languages, visual check 18/18, trigger set 38/38, Opus quality mean 4.65 over the 26 new runs. One real defect found: Hindi through `graphics.py` (drawtext) comes out wrong-shaped even though the font covers Devanagari; captions through libass are fine (queued for 1.15.0). Four delivery runs spent a second encode for loudness, which 1.14.0's templates address. Tokens per run flat at 72.3k. Details in `evals/results/iteration-14.json` |
package/SKILL.md CHANGED
@@ -93,6 +93,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
93
93
  | "add subtitles from this SRT", "burn in captions" | `caption.py input.mp4 --srt subs.srt` |
94
94
  | "caption it with these lines" (text with times) | `caption.py input.mp4 --text cues.txt` |
95
95
  | "keep the subtitles toggleable", "mux in an SRT" | `caption.py input.mp4 --srt subs.srt --mode mux`; repeat `--srt file:lang` for several languages, `.mkv` for more than two |
96
+ | "the captions are tiny / three lines on a Short" | `caption.py` shrinks the size until the cue fits `--max-lines` before splitting it (`--fit-size off` for 1.16 behaviour, `--min-size` sets the floor) |
96
97
  | "our logo top-right", "a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` |
97
98
  | "a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` |
98
99
  | "webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` |
@@ -180,17 +181,17 @@ When a step fails, replace `Done:` with `Failed:` and keep the rest honest:
180
181
  Failed: color.py --lut grade.cube exited 1 — ffmpeg: "Unable to parse LUT file" (the .cube is not a valid LUT)
181
182
  Steps: probe -> color (failed); nothing written
182
183
  Check: nothing to verify
183
- Look: not needed (nothing written)
184
+ Look: not needed
184
185
  Notes: send a valid .cube, or say if you want the clip left as is
185
186
  ```
186
187
 
187
- A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure still gets all five labels, never prose headings. A refusal that still delivers something is `Failed:` — the label answers the request as asked; the alternative goes in `Notes:`. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change that makes a retry meaningful.
188
+ A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure still gets all five labels, never prose headings. A refusal that still delivers something is `Failed:`, never a third label like `Done (partially):` — the label answers the request as asked; the alternative goes in `Notes:`. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change that makes a retry meaningful.
188
189
 
189
190
  Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message, never paraphrase it.
190
191
 
191
192
  ## Things that look right but are wrong
192
193
 
193
- One line each, each enough to act on; open the linked `references/gotchas.md` section only when the job is in that area and the line leaves a question.
194
+ One line each; open the linked `references/gotchas.md` section when the job is in that area.
194
195
 
195
196
  - HDR (iPhone, HDR10) re-encoded through an SDR path goes flat; the scripts keep HDR, and `hdr: true` is wider than `hdr_signal: true` (a real PQ/HLG/DV transfer). Details: [#hdr-and-colour](references/gotchas.md#hdr-and-colour)
196
197
  - Log footage (S-Log/V-Log/C-Log) is tagged SDR and looks grey: `probe.py --analyze`, then `color.py --lut` before anything else. Details: [#log-footage](references/gotchas.md#log-footage)
@@ -198,7 +199,7 @@ One line each, each enough to act on; open the linked `references/gotchas.md` se
198
199
  - VFR phone/screen recordings: re-encodes conform to CFR, `cut.py` switches to `--accurate`; pick the rate with `fit.py --fps` when the average is odd. Details: [#variable-frame-rate](references/gotchas.md#variable-frame-rate)
199
200
  - Sync/multicam `confidence` under 0.3 (or a huge offset) is probably wrong — check every camera; these align audio, never lip sync. Details: [#sync-multicam-and-drift](references/gotchas.md#sync-multicam-and-drift)
200
201
  - "Normalised" audio can still clip (check true peak), and ambience at -40 LUFS or below must never be raised to a speech target. Details: [#loudness-and-ambience](references/gotchas.md#loudness-and-ambience)
201
- - Captions burned before a crop/resize land off-frame; burned small then upscaled by `export.py` they come out soft — fit to the delivery size first. Details: [#captions-fonts-and-text-order](references/gotchas.md#captions-fonts-and-text-order)
202
+ - Captions burned before a crop/resize land off-frame, and burned small then upscaled by `export.py` they come out soft. Details: [#captions-fonts-and-text-order](references/gotchas.md#captions-fonts-and-text-order)
202
203
  - Emoji need `--emoji-assets DIR` (a PNG per glyph) to render in colour; without it they come out monochrome and the run says so. Details: [#emoji](references/gotchas.md#emoji)
203
204
  - `graphics.py` renders Devanagari, Bengali, Tamil and Thai through libass automatically — drawtext cannot shape them.
204
205
  - Non-Latin text picks a font by script since 1.12; `doctor --json` `fonts.scripts` says which languages this machine renders; no font = failed job. Details: [#fonts-by-script](references/gotchas.md#fonts-by-script)
@@ -207,4 +208,3 @@ One line each, each enough to act on; open the linked `references/gotchas.md` se
207
208
  - `yuv420p` needs even dimensions and phone rotation tags are honoured, both automatically. Details: [#dimensions-and-rotation](references/gotchas.md#dimensions-and-rotation)
208
209
  - `scenes.py --highlights` ranks by loudness (or duration), never by meaning: check the sheet before treating picks as final. Details: [#highlights](references/gotchas.md#highlights)
209
210
  - Three hand-chained re-encodes should be one `render.py` project; re-encodes use x264 `medium`. Details: [#chaining-and-speed](references/gotchas.md#chaining-and-speed)
210
- - Windows drawtext crashes on some builds (#100): pass `--font-file` explicitly if one does. Details: `references/ci-platform-pitfalls.md`
package/docs/contract.md CHANGED
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
21
21
  | Field | Meaning | Changes when |
22
22
  |---|---|---|
23
23
  | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
24
- | `skill.version` | the npm / package.json version (`1.16.0`) | any release |
24
+ | `skill.version` | the npm / package.json version (`1.17.0`) | any release |
25
25
 
26
26
  A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
27
27
  ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
88
88
 
89
89
  | What 2.0 removes | Since | Replacement | To be ready today |
90
90
  |---|---|---|---|
91
- | The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.16.0 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
92
- | `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.16.0 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
93
- | `json` and `progress` in the MCP `inputSchema` | 1.16.0 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
94
- | `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.16.0 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
95
- | Overwriting an existing output with only a warning | 1.16.0 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
91
+ | The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.17.0 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
92
+ | `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.17.0 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
93
+ | `json` and `progress` in the MCP `inputSchema` | 1.17.0 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
94
+ | `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.17.0 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
95
+ | Overwriting an existing output with only a warning | 1.17.0 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
96
96
 
97
97
  ## Skill
98
98
 
99
99
  ```json
100
100
  {
101
101
  "contract_version": "1.0",
102
- "deprecated": [{"what": "...", "since": "1.16.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
103
- "skill": {"id": "ffmpeg-skill", "version": "1.16.0", "execution_mode": "local", "kind": "execution",
102
+ "deprecated": [{"what": "...", "since": "1.17.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
103
+ "skill": {"id": "ffmpeg-skill", "version": "1.17.0", "execution_mode": "local", "kind": "execution",
104
104
  "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
105
105
  "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
106
106
  "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
@@ -128,7 +128,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
128
128
  | `output_schema` | what `--json` prints on stdout |
129
129
  | `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
130
130
  | `supports_json` | whether `--json` exists |
131
- | `supports_json_brief` | whether `--json-brief` exists (1.16.0): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
131
+ | `supports_json_brief` | whether `--json-brief` exists (1.17.0): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
132
132
  | `mutates_input` | always `false`: no tool overwrites its input |
133
133
  | `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
134
134
  | `verification` | `{required, tools}`: which tools to run on the output afterwards |
@@ -212,7 +212,7 @@ gives you:
212
212
  |---|---|
213
213
  | `bit_exact` | probe, check, scenes, look |
214
214
  | `content_equivalent` (same media, bytes may differ between encoder builds) | every encoding tool, cut, sync, report |
215
- | `cached` | batch (content-hash cache, re-runs skip unchanged inputs) |
215
+ | `cached` | batch (content-hash cache, re-runs skip unchanged inputs); render **when `--cache DIR` is given** — the hint stays `content_equivalent` because that is what render is without the flag, and the cache is opt-in |
216
216
  | `environment_dependent` | verify |
217
217
 
218
218
  ## `provides`
@@ -314,6 +314,12 @@ Names: `ffmpeg`, `ffprobe`, `encoder:<name>`, `filter:<name>`, `bsf:<name>`,
314
314
  to omit detection. Nothing from the environment other than those lists and the
315
315
  ffmpeg/ffprobe/python versions is printed; no environment variables, no paths.
316
316
 
317
+ `external:whisper` is **optional** for two tools since 1.17: `caption.py`
318
+ (`--transcribe`) and `silence.py` (`--filler --transcribe`). Neither requires
319
+ it — both take a transcript the caller already has (`--srt`/`--words`), and
320
+ both refuse with the same three install lines when asked to make one with no
321
+ engine present. Whisper is never a dependency of this skill.
322
+
317
323
  `doctor` has three states per capability. `available` and `missing` come from a listing
318
324
  that was read; `unknown` means the listing that would prove the capability could not be
319
325
  read (`ffmpeg -filters` in a layout the parser does not recognise, or ffmpeg exiting
@@ -433,6 +439,18 @@ given a different type):
433
439
  | `auto_chapters` | `metadata.py --auto-chapters` | `{source, min_chapter, max_chapters, proposed, kept, titles, chapters, description_block, files}`. `titles` is always `"placeholder"`: the machine-readable form of "the skill did not name these". Each chapter carries its `evidence` (`start`, `silence`, `scene`, or `silence+scene` with the span, its length and the cut time) |
434
440
  | `audiogram` | `waveform.py` (every run) | `{style, background, image, position, vis_height, platform, captions, title, stages, verified}`. `background` is `"image"` or `"color"`; `verified` is true when the render probes at the asked-for frame size, frame rate and within 0.05 s of the source audio, and is `false` under `--dry-run`, where nothing was rendered to verify |
435
441
 
442
+ Per-tool keys added in 1.17, all additive:
443
+
444
+ | key | tool | what it holds |
445
+ |---|---|---|
446
+ | `fit_size`, `size_requested`, `size_used`, `size_floor`, `size_pct_height`, `shrunk`, `fit_scope`, `fit_exhausted` | `caption.py` | siblings inside the same `caption` block: which mode fitted the size (`auto`/`on`/`off`), the size asked for and the size used in ASS units, the floor (13 = 4.5 % of the frame height), that size as a percentage of the frame, how many cues the shrink rescued, `file` or `cue` scope, whether the floor was reached with cues still split, and `size_source` (`input`, or `platform-frame` when a plan was written before the input existed). `size_used` is `null` when there was no geometry to fit against at all |
447
+ | `beats`, `beat_grid` | `scenes.py --beats` | the measured beat times, and `{supported_beats, tempo_bpm, interval, confidence, phase, onsets, supported, unsupported, method, step_s, range_bpm, usable}`. `supported_beats` is the subset of the regular grid that a measured onset marks — the only list a tool that moves a cut may snap to. `usable` is `confidence >= --min-confidence`; a low confidence is reported, not refused — `scenes.py` measures, it does not act |
448
+ | `snap` | `cut.py --snap beats`, `render.py` | `{mode, tolerance, confidence, tempo_bpm, grid, grid_points, moved, snapped, unchanged, source}`. `grid` is `"supported"`: points are moved only onto grid points a measured onset marks, never onto the regular grid's continuation through a silent passage. `moved` has exactly one row per in/out point given (`from`, `to`, `delta`, `snapped`, `beat_index`) — a point is never added or dropped, and `to` is always either a measured beat or the caller's own value |
449
+ | `filler`, `removed_seconds_total` | `silence.py --filler` | `{lang, source, engine, words, removed, removed_count, removed_seconds, removed_words, word_timings, list, warnings}`. The existing `removed_seconds` is unchanged in name and meaning — the seconds of *silence* removed, which is what it has always held — and `removed_seconds_total` is the additive sibling covering silence plus filler |
450
+ | `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
451
+ | `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
452
+
453
+
436
454
  `check.py` also gains an informational `subtitles` row on **every** platform:
437
455
  `PASS` when every soft subtitle stream carries a language tag, `WARN` when one
438
456
  does not (or when there are none). Like `channels` and `chapters` it is never
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "1.16.0",
3
+ "version": "1.17.0",
4
4
  "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
5
  "keywords": [
6
6
  "ffmpeg",
@@ -122,6 +122,10 @@ printing one line — `font: /usr/share/fonts/.../wqy-zenhei.ttc (covers ko)`.
122
122
  `brew install --cask font-noto-sans-cjk font-noto-sans-arabic`, or point at a
123
123
  file with `--font-file` (`overlay.py`, `graphics.py`) / `--fonts-dir`
124
124
  (`caption.py`).
125
+ - Windows drawtext crashes on some builds (#100) when it has to resolve a font by
126
+ family name: pass `--font-file` explicitly if one does. `default_font_file()`
127
+ already resolves a file for you on Windows, so this bites only a hand-built
128
+ drawtext call. More: `references/ci-platform-pitfalls.md`.
125
129
  - Han characters alone (no kana, no hangul) are read as Chinese. Japanese or
126
130
  Korean hanja text with no kana needs `--lang ja` / `--lang ko`
127
131
  (`caption.py --language` is the same flag), or `"lang"` in brand.json.
@@ -64,6 +64,7 @@ BT.2020 primaries on an SDR transfer, which `hdr_format` names "BT.2020 SDR" --
64
64
  ### cut.py — cut / join segments
65
65
  ```
66
66
  cut.py INPUT [--start T] [--end T | --duration T] [--segments A-B,C-D,...] [--accurate] [-o OUT]
67
+ cut.py INPUT --start T --end T --snap beats [--snap-tolerance 0.12] [--snap-source FILE] [--min-confidence 0.5]
67
68
  ```
68
69
  Times accept `12.5`, `1:30`, `00:01:30.250`. Default is `-c copy` (snaps to
69
70
  keyframes, instant, lossless); if the snapped result deviates more than
@@ -74,6 +75,49 @@ the result was "lossless stream copy" or "re-encoded"; when the snap forced a
74
75
  re-encode, the result's `lossless_alternative` names the nearest keyframe
75
76
  `--start` that would stream-copy instead, so the trade can be offered.
76
77
 
78
+ **`--snap beats` (1.17)** moves each in/out point to the nearest *measured*
79
+ beat within `--snap-tolerance` seconds (default 0.12, about a quarter of a beat
80
+ at 120 BPM). The grid comes from the input's own audio, or from
81
+ `--snap-source` — either a `scenes.py --beats --json` document (no second
82
+ decode) or a separate music file. The snap happens *before* the
83
+ keyframe/tolerance decision, so lossless-vs-accurate is judged on where the cut
84
+ actually lands.
85
+
86
+ "Measured" is exact: the only points a cut may move onto are the
87
+ `supported_beats` — the grid points a measured onset actually marks — never the
88
+ full regular grid, which by construction runs on through a passage with no music
89
+ in it. A cut asked for inside that passage stays where it was asked for, and
90
+ `snap.grid` is `"supported"` with `snap.grid_points` saying how many there were.
91
+ `--min-confidence` must be greater than 0: at 0 every grid is "reliable",
92
+ including one measured from noise.
93
+
94
+ The number of in/out points never changes: a point with no beat inside the
95
+ tolerance is returned unchanged, and no point is ever invented. Three
96
+ `kind: input` refusals, nothing written:
97
+
98
+ - confidence below `--min-confidence` → *"no reliable beat grid in this audio
99
+ (confidence 0.21, needs 0.5): cutting to invented beats would move your in/out
100
+ points to times nothing in the audio supports. Re-run with `--snap none`, or
101
+ pass `--snap-source` from a music bed."*
102
+ - no audio stream → names `--snap none`.
103
+ - no in/out point at all (a whole-file copy) → there is nothing to snap, and
104
+ this tool never proposes cuts of its own.
105
+
106
+ Result: `snap.mode`, `snap.tolerance`, `snap.confidence`, `snap.tempo_bpm`,
107
+ `snap.moved` (one row per point, with `from`/`to`/`delta`/`snapped`),
108
+ `snap.snapped`, `snap.unchanged`, `snap.grid`, `snap.grid_points`, and
109
+ `snap.source` (`"measured"` or the path). In a `render.py` result, `snap.clips`
110
+ carries one entry per snapped clip with `clip` naming its index, and the first
111
+ entry's keys are repeated at the top level for a single-clip project. A clip
112
+ served from `--cache` was snapped when it was first rendered, and says so
113
+ (`source: "cache"`) rather than reporting `snap: null`. A `--snap-source`
114
+ document that does
115
+ not carry `beat_grid.supported_beats` (one written before 1.17) is refused
116
+ rather than treated as if every grid point were supported, as is one whose
117
+ `tempo_bpm` is null while it lists beats.
118
+ `render.py` forwards a project's `"snap": {"to": "beats", ...}` (project-wide
119
+ or per clip) to this flag and reports what came back.
120
+
77
121
  ### fit.py — target duration and/or aspect, rotate/flip
78
122
  ```
79
123
  fit.py INPUT [--duration T --method speed|trim [--from-center] [--max-speed 4]]
@@ -422,6 +466,9 @@ each shorter clip's last frame (with silence) out to the longest.
422
466
  ### silence.py — remove dead air / jump cuts
423
467
  ```
424
468
  silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [-o OUT]
469
+ silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|es|de|fr|pt|it]
470
+ [--filler-words FILE] [--filler-extra W,W] [--filler-keep W,W] [--filler-pad 0.02]
471
+ [--transcribe] [--filler-list] [--max-cuts 400]
425
472
  ```
426
473
  Runs `silencedetect`, keeps `--margin` seconds of air around speech, drops
427
474
  gaps shorter than `--min-silence`, and re-encodes once with `select`/`aselect`
@@ -430,6 +477,60 @@ without rendering; `--edl` saves the kept ranges in `cut.py --segments` format
430
477
  so the user can edit the list by hand. Quiet rooms need `--threshold -40`
431
478
  to `-45`; noisy ones `-30`. Always tell the user how many seconds were removed.
432
479
 
480
+ **`--filler` (1.17)** removes filler words as well, through the same
481
+ `keep_ranges()`/`aselect` graph with more, shorter ranges. It is **never**
482
+ applied without measured word timings: pass `--words` (a whisper JSON carrying
483
+ per-word `start`/`end`) or `--transcribe` (the same local engine `caption.py`
484
+ uses, never required). There is no heuristic fallback — finding an "um" by
485
+ looking for short quiet blips would cut real speech, so the tool refuses
486
+ instead. `--filler-list` reports what would go and writes nothing.
487
+
488
+ Built-in lists, one per language:
489
+
490
+ | lang | words |
491
+ |---|---|
492
+ | en | um, uh, erm, hmm, mm, mhm, er, ah |
493
+ | ja | えー, えーと, えっと, あの, あのー, その, そのー, まあ, なんか |
494
+ | es | eh, este, esto, mmm |
495
+ | de | äh, ähm, hm |
496
+ | fr | euh, hein |
497
+ | pt | é, hum |
498
+ | it | ehm |
499
+
500
+ **Discourse markers are not disfluencies.** `like`, `tipo` and `cioè` are
501
+ deliberately *not* in the default lists: they are grammatical words in most
502
+ sentences, and removing them cuts meaning rather than noise — a judgement about
503
+ content, which this skill does not make. They are reachable with
504
+ `--filler-extra like`, which says so in its own `--help`. Japanese `なんか` is
505
+ in the `ja` list because leaving it out makes the flag useless for Japanese,
506
+ and it is orthographically identical to the pronoun use — so every run that
507
+ removes one warns, and `--filler-keep なんか` takes it back out. Matching is
508
+ case-folded, punctuation-stripped and whole-token only: `umbrella` survives.
509
+
510
+ Three refusals, all `kind: input`, all before any encode:
511
+
512
+ - `--filler` with neither `--words` nor `--transcribe` → names both flags.
513
+ - `--transcribe` with no engine on PATH → the same message `caption.py` gives,
514
+ with the three install lines.
515
+ - `--transcribe` where the engine runs but its build produces no word-level
516
+ timings → names that engine, says some builds do not support word timestamps,
517
+ and points at `--words`. (`--transcribe` drives whichever engine is installed
518
+ with *its* word-timestamp option — whisper.cpp `--output-json-full`,
519
+ faster-whisper `word_timestamps=True`, openai-whisper `--word_timestamps
520
+ True` — because an SRT cannot answer this: a cue has a start and an end, a
521
+ word does not. `--words` is the tested path and the one to prefer.)
522
+ - a transcript with segments but no word-level timings → says that cutting on
523
+ segment boundaries would remove whole sentences, and how to re-run whisper.
524
+
525
+ Results: `filler.removed_count`, `filler.removed_seconds`, `filler.removed`
526
+ (one entry per span), `filler.lang`, `filler.list`, `filler.word_timings` and
527
+ `filler.warnings`. The existing `removed_seconds` keeps exactly the meaning it
528
+ has always had — the seconds of **silence** this run removed, the figure the
529
+ same run would report without `--filler` — and `removed_seconds_total` is the
530
+ additive sibling covering everything that went. A filler word quiet enough to
531
+ sit inside a detected silence is merged into it rather than counted twice, so
532
+ the two figures can be equal.
533
+
433
534
  ### join.py — concatenate with transitions
434
535
  ```
435
536
  join.py CLIP1 CLIP2 [...] [--transition fade|dissolve|wipeleft|slideleft|fadeblack|fadewhite|circleopen|none]
@@ -447,6 +548,7 @@ output; the result says so with `dropped_non_av_streams: true`.
447
548
  ```
448
549
  render.py --init project.json # starter file
449
550
  render.py project.json [--fast] [--dry-run] [--stop-after STAGE] [--work DIR --keep]
551
+ render.py project.json --cache DIR [--from STAGE] # reuse identical stages from a previous run
450
552
  render.py plan.json # execute a plan written by <tool> --plan plan.json
451
553
  ```
452
554
  A plan is a single tool's dry run as an artifact: `cut.py in.mp4 --start 2 --end 8
@@ -479,6 +581,40 @@ entries and the file path are validated before the first stage runs, and the
479
581
  stage plans its `metadata.py` command under `--dry-run`/`--plan` like every
480
582
  other stage, so the plan lists `chapters` and the run does the same work.
481
583
 
584
+ **`--cache DIR` (1.17)** reuses the artifact of a stage that already ran with
585
+ exactly the same arguments and the same inputs. Opt-in only: **there is no
586
+ default cache directory** — one appearing on someone's disk unasked would
587
+ contradict this tool's "a plan leaves nothing behind" posture. The key is a
588
+ sha1 over the stage name, the tool, its arguments (every existing path replaced
589
+ by its content hash), the input hashes, the **forwarded** flags (`--fast`,
590
+ `--overwrite`, `--timeout`, `--codec`), the output's extension, and **the ffmpeg
591
+ build banner, the skill version and the contract version**. `--fast` matters as
592
+ much as any of them: it rewrites every child's preset to `veryfast`, so without
593
+ it in the key a `--cache --fast` draft would be served back to a later run that
594
+ asked for the delivery. The banner rather than `major.minor` because two 7.1.x
595
+ builds with different libx264 produce different bytes from the same command. Those last three are in the key deliberately: a
596
+ different build simply *misses* rather than being asked to trust a file it did
597
+ not write, and a stage whose implementation changed cannot serve back an
598
+ artifact the old one produced. Each entry is `DIR/<key><ext>` plus a
599
+ `DIR/<key>.json` sidecar (stage, versions, creation time, size, seconds). A hit
600
+ hardlinks the artifact into the work directory, or copies it where the
601
+ filesystem will not link — never moves it, since the cache has to outlive the
602
+ run's own cleanup. Any mismatch is a silent miss.
603
+
604
+ `stages_done` is unchanged: a cached stage is still a stage that happened.
605
+ Nothing is written under `--dry-run`, which instead reports `cache.would_hit`.
606
+ Result: `cache.dir`, `cache.ffmpeg`, `cache.hits`, `cache.misses`,
607
+ `cache.saved_seconds`, `cache.entries`. `--from STAGE` starts at that stage and
608
+ takes every earlier one from the cache; without `--cache`, or when an earlier
609
+ stage is not there, it refuses (`kind: input`) rather than quietly re-encoding
610
+ what it promised to skip. An unwritable cache directory is `kind: output`.
611
+
612
+ `"snap": {"to": "beats", "tolerance": 0.12, "min_confidence": 0.5, "source":
613
+ "music.mp3"}` at the project root (or inside one clip) forwards `--snap beats`
614
+ to the clip cut for every clip that has `in`/`out`; the measured grid comes back
615
+ in the render result's `snap`. A project without `"snap"` builds the command
616
+ line 1.16 built.
617
+
482
618
  Stages: clips (cut, optional speed) → join (transition) → silence → fit →
483
619
  captions → graphics → overlays → audio → loudness → export → chapters → check. Keys mirror the
484
620
  CLI flags of each script (see the docstring); a key `render.py` does not read -- at
@@ -549,6 +685,7 @@ already has a picture.
549
685
  ### scenes.py — scene changes and highlight candidates
550
686
  ```
551
687
  scenes.py INPUT [--threshold 10] [--min-scene 1] [--highlights N [--target SECONDS] [--max-scene 15]] [--edl picks.txt] [--sheet scenes.png] [--json]
688
+ scenes.py INPUT --beats [--beat-step 0.01] [--beat-range 60-200] [--min-confidence 0.5] [--json]
552
689
  ```
553
690
  Lists scenes with audio energy, the loudest moments, and (with
554
691
  `--highlights`) proposes N ranges that add up to `--target` seconds, biased to
@@ -560,6 +697,45 @@ Dissolves and very slow fades are not cuts and will be missed. Highlights are
560
697
  a proposal engine, not a judgement of content: tell the user what it picked
561
698
  and why (energy, scene length).
562
699
 
700
+ **`--beats` (1.17)** measures the music's beat grid and reports it:
701
+
702
+ ```json
703
+ "beats": [0.0, 0.5, 1.0, ...], // 25 of them over a 12 s click track
704
+ "beat_grid": {"tempo_bpm": 120.0, "interval": 0.5, "confidence": 0.997,
705
+ "supported_beats": [0.0, 0.5, 1.0, ...], // the 24 an onset marks
706
+ "phase": 0.0, "onsets": 24, "supported": 24, "unsupported": 1,
707
+ "method": "rms-flux-autocorrelation", "step_s": 0.01,
708
+ "range_bpm": [60, 200], "usable": true}
709
+ ```
710
+
711
+ Method: onset strength as the half-wave-rectified first difference of
712
+ `log(envelope)`, peaks above `median + 1.5·MAD` over a ±1 s window with a 60 ms
713
+ refractory gap, tempo from the autocorrelation of the onset signal inside
714
+ `--beat-range` with its half and double checked, phase chosen to catch the most
715
+ onset strength. `confidence` is half how far the winning lag stands above the
716
+ other lags (in standard deviations) and half the fraction of onsets that land
717
+ on the grid. With `--beats` the file is decoded once, at 22050 Hz, and the
718
+ `supported + unsupported == len(beats)` always, and `supported_beats` is that
719
+ supported subset — the list `cut.py --snap beats` moves onto.
720
+
721
+ With `--beats` the file is decoded once, at 22050 Hz, and the 0.5 s scene
722
+ envelope is derived from that same pass rather than from a second 8 kHz decode.
723
+ One consequence worth knowing: `scenes.py X --json` and `scenes.py X --beats
724
+ --json` report very slightly different `audio_rms`/`audio_peak` figures for the
725
+ same file, because the two envelopes are built from PCM at different rates. The
726
+ scene boundaries and their ranking are unaffected; only the fourth decimal of
727
+ the level moves. `--beats` also holds ~2.75x the samples in memory, which is
728
+ worth knowing on a feature-length input.
729
+
730
+ **A beat grid is a measurement of the music's periodicity, not of where a cut
731
+ belongs.** A low confidence means the audio has no steady pulse — speech,
732
+ ambience, rubato — and the skill will not snap to a grid it cannot measure.
733
+ `scenes.py` is the analysis tool, so it *reports* a weak measurement
734
+ (`usable: false`) rather than refusing it; refusing belongs to the tools that
735
+ would change a file on the strength of it (`cut.py --snap beats`). `--edl` with
736
+ `--beats` is unchanged: beats are never written as an EDL, because a beat is
737
+ not a cut. No audio stream is a `kind: input` refusal.
738
+
563
739
  ### check.py — pre-delivery compliance
564
740
  ```
565
741
  check.py INPUT --platform youtube|shorts|reels|tiktok|x|linkedin|facebook|broadcast|podcast|custom [--no-loudness] [--json]
@@ -582,13 +758,42 @@ construction, not by two lists agreeing.
582
758
 
583
759
  ### batch.py — same recipe over a folder, cached
584
760
  ```
585
- batch.py FOLDER --recipe batch.json [--force] [--watch SECONDS] [--json]
761
+ batch.py FOLDER --recipe batch.json [--force] [--watch SECONDS] [--jobs N|auto] [--json]
586
762
  ```
587
763
  `batch.json` holds either `steps` (a list of script argv with `{in}`/`{out}`
588
764
  placeholders, chained) or `project` (a render project applied per file).
589
765
  Outputs land in `output_dir` with `suffix`; a content-hash cache skips files
590
766
  already done with the same recipe. Use `--dry-run` to preview the plan.
591
767
 
768
+ **`--jobs N` (1.17)** processes N files at once (threads: the work is
769
+ subprocess waiting). Capped at `min(N, cpu_count, 8)` — every item is itself an
770
+ ffmpeg that already threads across cores, so beyond a few concurrent encodes
771
+ the jobs contend and wall-clock stops improving while memory does not. A number
772
+ above the cap is clamped with a note, not refused, and both `jobs` and
773
+ `jobs_requested` are in the result. `--jobs auto` is `min(cpu_count, 4)`.
774
+
775
+ - **One timeout budget for the whole batch**, not one per item: `--timeout` is
776
+ computed into a deadline once, no new item starts after it, and the run exits
777
+ 124 `kind: timeout` with the items that never started marked
778
+ `"skipped": "timeout"`. That shared budget applies when a `--timeout` was
779
+ actually given, or when `--jobs > 1` asked for the batch to be treated as one
780
+ piece of work; the default sequential path with the default timeout is 1.16's
781
+ behaviour exactly, where a long folder was never cut off part-way.
782
+ - **Determinism**: every item is written into its own slot in one list indexed
783
+ by position in the sorted file list — cached hits included, which is what keeps
784
+ the table in file order when the cache is only partially warm — and each item's
785
+ log lines are buffered and flushed in file order, so the per-item table and the
786
+ log read exactly as a serial run's whatever order the encodes finish in.
787
+ - **A failed item is a row, not a dead run**: a worker that raises becomes
788
+ `ok: false` with its reason, and the summary and the table are still printed.
789
+ Ctrl-C cancels what has not started, keeps what finished, and exits 130
790
+ `kind: interrupted` with the partial table.
791
+ - With `--jobs > 1` each item gets its own work subdirectory
792
+ `<workdir>/<index>-<stem>/`, because step file names are stem-derived and two
793
+ sources sharing a stem would otherwise write over each other. `--jobs 1` (the
794
+ default) keeps the flat layout and is 1.16's path exactly.
795
+ - `--watch` composes: each pass is parallel, passes are sequential.
796
+
592
797
  ### caption.py --transcribe — optional local speech-to-text
593
798
  If `whisper-cli` (whisper.cpp), `faster-whisper` or `whisper` is installed,
594
799
  `caption.py input.mp4 --transcribe [--language ja] [--model base]` writes the
@@ -726,11 +931,62 @@ caption.py INPUT --srt FILE[:LANG] | --ass FILE | --text CUES.txt [--write-srt O
726
931
  [--audio-stream N] [--fps N] [--lang XX] [--offset TIME]
727
932
  [--max-lines N] [--min-duration S] [--wrap phrase|measured]
728
933
  [--font NAME] [--fonts-dir DIR] [--size N] [--color RRGGBB] [--outline N] [--outline-color RRGGBB]
934
+ [--fit-size auto|on|off] [--min-size N] [--fit-size-scope file|cue]
729
935
  [--bold] [--box] [--position bottom|top|center|top-left|...] [--margin N]
730
936
  [--animate none|fade|pop|slide] [--karaoke [--highlight-color RRGGBB]] [--write-ass OUT.ass]
731
937
  [--emoji auto|color|png|mono|none] [--emoji-assets DIR] [--emoji-scale 1.0] [--emoji-max 60] [-o OUT]
732
938
  caption.py --text CUES.txt --write-srt OUT.srt # generate the SRT only
733
939
  ```
940
+ **`--fit-size` (1.17): the size is fitted before a cue is split.** At the TikTok
941
+ caption size (24 ASS units against the 288-line script grid) a line has about
942
+ six em, so an ordinary sentence needs four lines — and `--max-lines 2` then cut
943
+ it into consecutive cues, so half of it arrived late. That was the size, not the
944
+ breaker. `--fit-size` walks the size down until every cue wraps within
945
+ `--max-lines`, and only then lays the cues out.
946
+
947
+ - `auto` (default) shrinks only a size the *skill* chose. An explicit `--size`,
948
+ or a `brand.json` `styles.caption.size`, is a statement about the look and is
949
+ never overridden.
950
+ - `on` always fits, `--size` or not. **`off` is 1.16.1 exactly, byte for byte**
951
+ (a pinned ASS fixture asserts it).
952
+ - `--min-size` is the floor, default `ass_units(0.045) = 13` — **4.5 % of the
953
+ frame height**, one floor for every destination (87 px of type on a 1920-tall
954
+ frame, above the ~3.5 % where mobile legibility and the platforms' own caption
955
+ UIs bottom out). Nothing per-platform is measured, so nothing is claimed.
956
+ - `--fit-size-scope file` (default) uses one size for the whole file; `cue`
957
+ writes a per-cue `{\fsN}` override. A size that changes from cue to cue reads
958
+ as a mistake, so it is opt-in — it exists for the one outlier cue that would
959
+ otherwise shrink a ten-minute file. Each cue is laid out at the size it will
960
+ be **drawn** at: a cue drawn larger has a narrower line in em, so wrapping
961
+ everything to the file-wide budget and then drawing some cues large would put
962
+ lines off the side of the frame.
963
+
964
+ The text is **never** touched: this skill does not rewrite, shorten or
965
+ paraphrase a caption to make it fit. Below the floor the cue is split exactly as
966
+ before and `fit_exhausted: true` says so, with `--min-size` named as the flag
967
+ that would go smaller and `|` as the manual break. `--min-size` above `--size`
968
+ is a `kind: input` refusal. `--mode mux` is unaffected: soft subtitles carry no
969
+ size, so the SRT is the one 1.16 wrote.
970
+
971
+ Results, alongside the existing caption stats:
972
+ `fit_size`, `size_requested`, `size_used`, `size_floor`, `size_pct_height`,
973
+ `shrunk`, `fit_scope`, `fit_exhausted`, and `size_source` (`input` or
974
+ `platform-frame`).
975
+
976
+ Under `--dry-run`/`--plan` on an input that does not exist yet there is no
977
+ geometry to measure. With `--platform` the destination's own frame is used —
978
+ that frame *is* what the real run will have, so the planned `FontSize` is the
979
+ one the run will burn. Without a platform nothing can stand in for the frame
980
+ and `size_used` is `null`, rather than presenting the requested size as a
981
+ fitted one.
982
+
983
+ | ASS size | px on 1920 | % frame h | em per line |
984
+ |---|---|---|---|
985
+ | 24 (default) | 160.0 | 8.3 % | 6.08 |
986
+ | 19 | 126.7 | 6.6 % | 7.67 |
987
+ | 16 | 106.7 | 5.6 % | 9.11 |
988
+ | 13 (floor) | 86.7 | 4.5 % | 11.22 |
989
+
734
990
  Text cue format, one per line: `0:00-0:03 Hello`, `00:00:03.500 --> 00:00:06 Two | lines`,
735
991
  or `00:00:03:15 --> 00:00:06:00 SMPTE non-drop-frame timecode` (`hh:mm:ss:ff`, frame count
736
992
  converted with `--fps`, or the input video's own fps when `--input` is given and `--fps` is
@@ -758,9 +1014,13 @@ frame width) at the chosen `--size`, measured per script — CJK and Thai count
758
1014
  full em per character, Latin per character from a table read off DejaVu Sans (so
759
1015
  an all-caps line measures as wide as it draws), Cyrillic/Greek about 0.55,
760
1016
  Arabic/Hebrew 0.6, Devanagari 0.7, and a combining mark nothing at all —
761
- breaking between characters for CJK/Thai and at spaces otherwise, but never
762
- between a character and the combining marks that belong to it (Thai tone marks
763
- and vowel signs, Devanagari matras, Arabic and Hebrew points). A cue that would need more than `--max-lines` (default 2) is split
1017
+ breaking between characters for CJK (a katakana word stays whole) and at spaces
1018
+ otherwise, but never between a character and the combining marks that belong
1019
+ to it (Devanagari matras, Arabic and Hebrew points). Thai (1.16.1) is never
1020
+ broken inside a run: it writes no space inside a phrase and the wrapper has no
1021
+ dictionary, so the break goes where you put a space or a `|`, and a run with
1022
+ none stays long on its own line — `caption.py` counts such lines as `overlong`
1023
+ and says so. A cue that would need more than `--max-lines` (default 2) is split
764
1024
  into consecutive cues sharing its time; a cue shorter than `--min-duration`
765
1025
  (default 1.0 s) is held longer, never past the next cue's start; `--offset
766
1026
  TIME` shifts every cue (seconds, `mm:ss`, `hh:mm:ss.ms` or `hh:mm:ss:ff`, a