ffmpeg-skill 1.17.2 → 1.18.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -57,7 +57,7 @@ If `ffmpeg` and `python3` are on your PATH, it works: offline, on footage you wo
57
57
 
58
58
  **Standalone**, this is a local FFmpeg engine: probe → edit → verify, `npx ffmpeg-skill` and nothing else. No API key, no account, no other repo required. Everything above and below this section describes that standalone tool, and none of it changes if you never read the rest of this one.
59
59
 
60
- **In [kajisho5](https://github.com/kajisho5)'s wider video-production ecosystem**, this repo is the *hands*: it cuts, measures and exports files, and reports back in structured JSON. It does not decide what to cut, whether a deliverable is approvable, or what makes a highlight interesting — those are a *brain*'s job, sitting in front of this engine, not inside it.
60
+ **In [kajisho5](https://github.com/kajisho5)'s wider video-production ecosystem**, this repo is the *hands*: it cuts, measures and exports files, and reports back in structured JSON. It does not decide what to cut, whether a deliverable is approvable, what makes a highlight interesting, or what a caption should say (the user's cue text is burned as written, never rewritten to fit) — those are a *brain*'s job, sitting in front of this engine, not inside it.
61
61
 
62
62
  | You want to... | Use |
63
63
  |---|---|
@@ -416,6 +416,7 @@ type on every OS.
416
416
  | **F1 0.97** | `scenes.py`, 53 hard cuts between single takes, precision 0.95, recall 1.00 at the default threshold |
417
417
  | **exact to the sample** | `cut.py --accurate` on WAV, FLAC (44.1 kHz) and AAC → WAV; WAV stream copy within 2 ms; AAC output +21 ms of encoder priming, reported as `codec_frame` (0.9.1) |
418
418
  | **72 / 72** | agent runs of 24 prompts (12 English edits, 8 Japanese, 4 that must be declined), three repeats, graded by an independent model: routing, honest refusals and user's language 72/72, report format 71/72, visual check whenever the picture changed 24/24 (0.8.4) |
419
+ | **20 / 20** | 1.17.2 run (2026-09-14, the eight caption prompts of eval 19, six of them three times, Sonnet agent, regex grader + an Opus grader that opened every contact sheet and counted the lines per cue): routing 19/19 act runs, honest 18/20 with 0 false successes and 0 raw ffmpeg calls, report format 20/20, user's language 20/20, trigger set 49/50 (one judge flip on a file-less prompt), Opus quality mean 4.25 (3.65 at eval 19). The picture is fixed: 0/20 runs stack one word per line against 12/12 template runs at eval 19 on the same cues; the Style row at TikTok geometry is now `…,54,151,420,1`, the fitter's `size_used` (15 on TikTok, 16 on Shorts, the 13 floor for the Spanish cues) is what the frame shows, and report and sheet agree in 18/20 runs. What is left is not the typesetter: `cs3` rewrote the user's captions for the fourth iteration running, one `cs1` run raised `max_lines` to 4 to avoid a shrink and drew four-line stacks, and `cs2`'s 32-letter Spanish word still leaves the frame at the size floor (disclosed 3/3). Written up in `evals/results/iteration-20.json` |
419
420
  | **26 / 26** | 1.17.1 run (2026-09-14, targeted re-run of the 18 prompts eval 18's follow-up named, plus three repeats each of the four caption-size prompts, Sonnet agent, regex grader + a full Opus grader over all 26 runs, every PNG opened and every written output re-probed): routing 23/26, honest refusals and failures 24/26 with 0 false successes and 0 raw ffmpeg calls, report format 26/26 with the third label gone, user's language 26/26, trigger set 50/50, Opus quality mean 3.65. 1.17.1's fix holds — `--fit-size` now fires on the `render.py --template` path in 12/12 caption runs (24 → 16, `dl4` to the 13-unit floor, `split` 0, `text_unchanged` true, identical across repeats), the beat, filler and `--jobs` prompts route on the first try, and `bt2` quotes its measured 0.184 confidence instead of denying the capability exists. The honest part: the picture is unchanged. `caption.py`'s `write_ass` writes the platform's *vertical* safe margin into `MarginL`, `MarginR` and `MarginV` alike (tiktok 63 ASS units → 420 px), so at `PlayResX` 1080 the text column is 240 px and libass wraps every word — the fitter budgets `play_w × 0.9`, which is why `split: 0` is true of the ASS text and false of the frame. It is the `--animate`/`--karaoke` path only, present since 1.14, and it explains eval 17's and eval 18's "one word per line" too; 1.17.2 is the patch and the finding is written up in `evals/results/iteration-19.json` |
420
421
  | **100 / 100** | 1.17.0 run (2026-09-14, one pass per prompt, Sonnet agent, regex grader + focused Opus grader on 28 runs, every written output re-probed, `check.py` re-run on every delivery output) on the set grown to 100 prompts (caption size fitting, beat-synced cuts, filler removal, batch `--jobs`, render `--cache`): routing 95% over the 64 act prompts, honest refusals and failures 22/25 with 0 false successes and 0 raw ffmpeg calls, report format 98/100 (two runs label an honest partial result with a third label), user's language 100/100 across seventeen languages, visual check 24/24, real execution 6/6 with honest failure 5/5, trigger set 50/50 including all five new 1.17 prompts, Opus quality mean 3.71. The honest part: `--fit-size` is unreachable on the template path (`render.py` forwards the platform table's caption size as an explicit `--size`, so the fitter declines to shrink a size it thinks the user chose, and the project schema rejects `fit_size` outright — only the one run that called `caption.py` by hand got 24 → 16, `split` 0), and SKILL.md names none of the 1.17 features, so beats, filler and `--cache` were each used in one run at most — 1.17.1 is the patch and the finding is written up in `evals/results/iteration-18.json` |
421
422
  | **90 / 90** | 1.16.0 run (2026-09-14, one pass per prompt, Sonnet agent, regex grader + focused Opus grader on 30 runs, chapters and subtitle streams re-probed, check.py re-run on every delivery output) on the set grown to 90 prompts (audiogram, auto chapters, multi-language tracks, caption breaking): routing 90/90, honest refusals and failures 90/90 with 0 false successes and 0 raw ffmpeg calls, report format 89/90 (one `Done (partially):`), user's language 90/90 by regex (89/90 by Opus), audiogram 2/2 with the cover behind the waveform and nothing fetched, auto chapters 2/2 with `Chapter N` titles only, delivery 16/16 platform pass, trigger set 45/45, Opus quality mean 4.17. The honest part: the phrase breaker never gets to act at the platform caption sizes (a five-word cue does not fit two lines at TikTok size, so the split is byte-identical to 1.15.1), Thai still breaks inside words, and a katakana word was split — 1.16.1 is the patch and the finding is written up in `evals/results/iteration-17.json` |
package/SKILL.md CHANGED
@@ -5,30 +5,30 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
5
5
 
6
6
  # ffmpeg-skill
7
7
 
8
- Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`, and delivery templates in `templates/`. This file is enough to do a job: the table below routes the request, and `--help` on the script you are about to run is the cheapest full flag list. A reference file costs as much to read as this file, so open one only when it answers a question you have: `references/scripts.md` (every flag of all 42 scripts), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end).
8
+ Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`, and delivery templates in `templates/`. This file is enough to do a job: the table below routes the request, and `--help` on the script you are about to run is the cheapest full flag list. A reference file costs as much to read as this file; open one only for a question you have: `references/scripts.md` (every flag of all 42 scripts), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end).
9
9
 
10
- Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (the same trimmed to status/output/verified plus a `summary` prefer it on writing steps); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (the dry run as a plan `render.py FILE` runs later, refusing if an input changed). Every re-encoding tool also takes `--codec h264|hevc|av1|prores` and `--quality N` (CRF scale, replaces the deprecated `--crf`): unset, SDR is x264 and HDR is x265 Main10; `prores` needs an explicit `-o NAME.mov`, `h264` refuses an HDR source (`color.py --to-sdr` first).
10
+ Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (the same trimmed to status/output/verified plus a `summary`; prefer it on writing steps); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (the dry run as a plan `render.py FILE` runs later; refuses if an input changed). Every re-encoding tool also takes `--codec h264|hevc|av1|prores` and `--quality N` (CRF scale): unset, SDR is x264 and HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses an HDR source (`color.py --to-sdr` first).
11
11
 
12
12
  Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) may still run ffmpeg/ffprobe — they just don't write their artifact, nor side files (`--edl`, `--sheet`, a generated `.ass`); `verify` ignores the flag. Per-tool: `contract --json`'s `dry_run` field.
13
13
 
14
14
  ## Workflow (always follow this order)
15
15
 
16
- 0. **Environment, only on failure.** Never start a job with `doctor`: a broken machine fails on its own with `kind: missing_tool` or an ffmpeg error naming the filter/encoder (`No such filter: 'subtitles'`). Run `python3 <skill-dir>/scripts/_contract.py doctor` (or `npx ffmpeg-skill doctor`) after such a failure, or when asked what the machine can do: read `ok` and the tool's `usable`, and report the missing capability (usually `libass`, `zscale` or an encoder). `contract --json`'s tool schema is for a *planning* agent, not this workflow.
16
+ 0. **Environment, only on failure.** Never start a job with `doctor`: a broken machine fails on its own with `kind: missing_tool` or an ffmpeg error naming the filter/encoder. Run `python3 <skill-dir>/scripts/_contract.py doctor` (or `npx ffmpeg-skill doctor`) after such a failure, or when asked what the machine can do: read `ok` and the tool's `usable`, report the missing capability. `contract --json`'s tool schema is for a *planning* agent, not this workflow.
17
17
  1. **Probe what you must plan from.** Run `probe.py` on each input you plan the edit from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks about a file. No separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output. Plan from real numbers, never assumptions.
18
18
  2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; `--accurate` on `cut.py` only for frame-exact cuts.
19
- 3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (dimensions there can be a placeholder — `docs/contract.md`). Use it before long encodes and to report exact facts. `--fast` is preview quality (x264 veryfast), `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
20
- 4. **Chain in a sensible order.** A delivery request with no other editing is one template run (`render.py --template NAME INPUT`), not a hand-built chain. Otherwise: colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes before captions and overlays, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18, `export.py` only last. **Three or more steps: `render.py` with a project.json** — one call, one JSON, one number for the user to change.
21
- 5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination the user named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix mechanically. Judgement rows change the content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix those only when the request implies the answer, otherwise state the choice and its cost in one line. Mention WARNs; do not chase them.
19
+ 3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use it before long encodes and to report exact facts. `--fast` is preview quality, `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
20
+ 4. **Chain in a sensible order.** A delivery request with no other editing is one template run (`render.py --template NAME INPUT`), not a hand-built chain. Otherwise: colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes before captions and overlays, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18, `export.py` last. **Three or more steps: `render.py` with a project.json** — one call, one JSON, one number for the user to change.
21
+ 5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination the user named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix. Judgement rows change the content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix those only when the request implies the answer, otherwise state the choice and its cost in one line. Mention WARNs; do not chase them.
22
22
  6. **Verify the output.** Confirm duration, resolution, fps and audio match the request — from the writing tool's own `--json`/`--json-brief` probe, or `probe.py` — and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure reported with the script's error message.
23
23
  7. **Keep the user's originals.** Never overwrite the source; write new files next to the input or where the user asked. Set `FFMPEG_SKILL_NO_OVERWRITE=1` in the environment you run these scripts in: an existing output path is then refused (`kind: input`) instead of warned about, with `--overwrite` the one way to say "yes, replace it". It is the recommended agent setting, and 2.0's default.
24
- 8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` that scaled a clip to the first clip's frame, a `color.py --to-sdr`) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for a job about layout across the whole clip. The job is not finished until the report's `Look:` line names that PNG — a probe cannot see a caption sitting on someone's face. Audio-only jobs write `Look: not needed`. What to look for splits like `check.py`'s rows in step 5:
24
+ 8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` that scaled a clip to the first clip's frame, a `color.py --to-sdr`) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for a job about layout across the whole clip. The job is not finished until the report's `Look:` line names that PNG — a probe cannot see a caption on someone's face. Audio-only jobs write `Look: not needed`. What to look for splits like `check.py`'s rows in step 5:
25
25
  - **Mechanical (this skill's own job to verify and report):** the specified text/logo is at the specified position, subtitles appear at the specified timestamps, dimensions are even. Letterboxing from `fit.py --fit pad` is the *correct* result of that mode, never a defect to flag.
26
26
  - **Judgement (report it, don't silently pass or fail):** whether a subject or face is cut off, text sits over a face, colours look washed out, a transition lands. These need deciding what the subject *is*, which belongs to the calling agent — say what you see in one line and let them judge it.
27
27
  With no vision capability, write `Look: PATH (pixels not inspected; agent has no image view)` — never claim a picture was inspected when it wasn't, and don't stall waiting for a capability that isn't there.
28
28
 
29
29
  ## Before you run anything: what to ask, what to assume
30
30
 
31
- Ask one short question only when the answer changes the output materially and the request does not imply it. When several things are open at once (a vague "make it for social media" leaves destination, aspect, length and captions open), don't ask them one per turn: propose one bundle with your defaults and let the user change any part ("Reels: 9:16 padded, 60 s, -14 LUFS, no captions — OK?"). One question, one answer, then the run. Never ask for what `probe.py` can tell you.
31
+ Ask one short question only when the answer changes the output materially and the request does not imply it. When several things are open at once (a vague "make it for social media" leaves destination, aspect, length and captions open), don't ask one per turn: propose one bundle with your defaults and let the user change any part ("Reels: 9:16 padded, 60 s, -14 LUFS, no captions — OK?"). One question, one answer, then the run. Never ask for what `probe.py` can tell you.
32
32
 
33
33
  - **Destination** decides aspect, length limit, loudness and codec; "for Reels" answers all four and names a template. No destination named and a plain cut/caption: keep the source format and say so. If the user says "export", "post" or "deliver", ask where.
34
34
  - **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, and say which. Ask when the content is a talk (trimming loses words) and the change is large.
@@ -47,6 +47,7 @@ This skill cuts, joins, measures, syncs, exports and checks files — it execute
47
47
  - **Thumbnail or cover composition** — a design decision, not a measurement.
48
48
  - **Understanding what a video is *about*** — there is no vision here beyond `look.py`'s contact sheets, which exist for the calling agent's eyes, not for this skill to interpret.
49
49
  - **Judging what looks good** — "apply this LUT", "correct exposure by +0.3 stops" (`color.py`) is mechanical; "grade this scene to look cinematic" belongs to a colour-grading skill ([`color-grading-skill`](https://github.com/kajisho5/color-grading-skill)) that decides the parameters and then calls `color.py`.
50
+ - **The words in a caption** — cue text is burned as written. Too long for the frame means a smaller size, `--max-lines`, or the user's own edit; never rewrite, shorten or paraphrase it, even when asked to "make it fit" — say so, offer `--max-lines 1` at a smaller size.
50
51
  - **Picking a subject or region you were not given** — "crop to x=200,y=0" is mechanical once the box is known; "crop to keep the speaker in frame" needs deciding *what* the speaker is — a judgement for the calling agent (from a `look.py` sheet).
51
52
 
52
53
  The line: same input + same explicit parameters always producing the same verifiable output belongs here; anything depending on taste, content understanding or what looks or sounds good belongs to whoever makes that judgement.
@@ -93,7 +94,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
93
94
  | "add subtitles from this SRT", "burn in captions" | `caption.py input.mp4 --srt subs.srt` |
94
95
  | "caption it with these lines" (text with times) | `caption.py input.mp4 --text cues.txt` |
95
96
  | "keep the subtitles toggleable", "mux in an SRT" | `caption.py input.mp4 --srt subs.srt --mode mux`; repeat `--srt file:lang` for several languages, `.mkv` for more than two |
96
- | "the captions are tiny / three lines on a Short" | `caption.py` shrinks the size until the cue fits `--max-lines` before splitting it (`--fit-size off` for 1.16 behaviour, `--min-size` sets the floor) |
97
+ | "the captions are tiny / three lines on a Short", "don't chop the sentence" | `caption.py` shrinks the size until the cue fits `--max-lines` before splitting it (`--fit-size off` for 1.16 behaviour, `--min-size` sets the floor). Keep the template's `--max-lines` (2 on a vertical) and let the size drop; raising it to dodge a shrink stacks two words per line |
97
98
  | "our logo top-right", "a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` |
98
99
  | "a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` |
99
100
  | "webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` |
@@ -102,7 +103,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
102
103
  | "sync the lav mic", "line up two cameras" | `sync.py camera.mp4 mic.wav --replace-audio` / `sync.py camA.mp4 camB.mp4 --trim-second` |
103
104
  | "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` podcast, `-I -23` broadcast; `--lra N` for the range) |
104
105
  | "cut this and make it HEVC / AV1 / ProRes" (output codec named) | `cut.py input.mp4 --start 0:10 --end 0:40 --codec hevc` (`--codec`/`--quality` on any re-encoding tool; ProRes needs `-o NAME.mov`) |
105
- | "make this a TikTok / Reel / Short / YouTube / X / LinkedIn / podcast" | `render.py --template tiktok\|reels\|shorts\|youtube-shorts\|youtube\|x\|linkedin\|facebook\|podcast input.mp4 [--cues cues.txt\|--srt subs.srt] [--title "..."] [--logo logo.png] [--brand brand.json]` — frame, captions in the safe area, loudness, export and that platform's check in one command (`--list-templates`, `--write-project` to edit first) |
106
+ | "make this a TikTok / Reel / Short / YouTube / X / LinkedIn / podcast" | `render.py --template tiktok\|reels\|shorts\|youtube-shorts\|youtube\|x\|linkedin\|facebook\|podcast input.mp4 [--cues cues.txt\|--srt subs.srt] [--title "..."] [--logo logo.png] [--brand brand.json]` — frame, safe-area captions, loudness, export and that platform's check in one command (`--list-templates`, `--write-project` to edit first) |
106
107
  | "post it everywhere", "one edit for every platform" | `render.py --template all input.mp4 --cues cues.txt` (or a comma list) → one file per destination plus `<name>_pack.md` (`report.py --pack` renders the HTML) |
107
108
  | "export for YouTube / Reels / X", "a ProRes master" | `export.py input.mp4 --preset youtube\|reels\|tiktok\|shorts\|linkedin\|facebook\|x\|prores\|h265` (`--normalize` hits the loudness spec in the same call; `youtube-hdr` keeps HDR, `youtube-av1` writes AV1) |
108
109
  | "make a GIF preview" | `export.py input.mp4 --preset gif` |
@@ -149,12 +150,12 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
149
150
 
150
151
  ## Audio-only files
151
152
 
152
- Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`, `audio.py`, `sync.py`, `check.py --platform podcast` and `render.py --template podcast` take WAV, FLAC, MP3, M4A/AAC, OGG and Opus; the output extension picks the format. `Check:` still applies. Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" — say so, do not force a video wrapper. Audio recipes, packet vs sample precision, joining, extracting a track: `references/gotchas.md#audio-only-files`.
153
+ Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`, `audio.py`, `sync.py`, `check.py --platform podcast` and `render.py --template podcast` take WAV, FLAC, MP3, M4A/AAC, OGG and Opus; the output extension picks the format. `Check:` still applies. Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" — say so, don't force a video wrapper. Audio recipes, packet vs sample precision, joining, extracting a track: `references/gotchas.md#audio-only-files`.
153
154
 
154
155
 
155
156
  ## Report format
156
157
 
157
- Reply in the language the request itself is written in — the user's own sentences, not a language the request merely talks about (subtitles in another language are still reported in the request's language). Keep the field labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English: they read as log fields in any language. Everything else is the user's language — the lines those labels head, any question, any judgement call explained. Never drift because the job was short or the report is a failure: even a one-line "file does not exist". A mid-conversation switch follows the user's latest message.
158
+ Reply in the language the request itself is written in — the user's own sentences, not a language the request merely talks about (subtitles in another language are still reported in the request's language). Keep the field labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English: they read as log fields in any language. Everything else is the user's language — the lines those labels head, any question, any judgement call. Never drift because the job was short or failed: even a one-line "file does not exist". A mid-conversation switch follows the user's latest message.
158
159
 
159
160
  Finish every job with this shape (numbers from `--json` or `probe.py`/`check.py`, not memory):
160
161
 
@@ -191,7 +192,7 @@ Notes: send a valid .cube, or say if you want the clip left as is
191
192
  Those filler lines are sentences, not labels: the same report for a Japanese request ends
192
193
  `Check: 検証するものなし` / `Look: 不要`.
193
194
 
194
- A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure gets all five labels, never prose headings. A partial result is `Done:` with the shortfall in `Notes:`, and a refusal that still delivers something is `Failed:` — never a third label like `Done (partially):`. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change that makes a retry meaningful.
195
+ A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure gets all five labels, never headings. A partial result is `Done:` with the shortfall in `Notes:`, and a refusal that still delivers something is `Failed:` — never a third label like `Done (partially):`. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change a retry needs.
195
196
 
196
197
  Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message, never paraphrase it.
197
198
 
@@ -205,10 +206,9 @@ One line each; open the linked `references/gotchas.md` section when the job is i
205
206
  - VFR phone/screen recordings: re-encodes conform to CFR, `cut.py` switches to `--accurate`; pick the rate with `fit.py --fps` when the average is odd. Details: [#variable-frame-rate](references/gotchas.md#variable-frame-rate)
206
207
  - Sync/multicam `confidence` under 0.3 (or a huge offset) is probably wrong — check every camera; these align audio, never lip sync. Details: [#sync-multicam-and-drift](references/gotchas.md#sync-multicam-and-drift)
207
208
  - "Normalised" audio can still clip (check true peak), and ambience at -40 LUFS or below must never be raised to a speech target. Details: [#loudness-and-ambience](references/gotchas.md#loudness-and-ambience)
208
- - Captions burned before a crop/resize land off-frame, and burned small then upscaled by `export.py` they come out soft. Details: [#captions-fonts-and-text-order](references/gotchas.md#captions-fonts-and-text-order)
209
+ - Captions burned before a crop/resize land off-frame; burned small then upscaled by `export.py` they come out soft. Details: [#captions-fonts-and-text-order](references/gotchas.md#captions-fonts-and-text-order)
209
210
  - Emoji need `--emoji-assets DIR` (a PNG per glyph) to render in colour; without it they come out monochrome and the run says so. Details: [#emoji](references/gotchas.md#emoji)
210
211
  - Non-Latin text picks a font by script (`graphics.py` shapes Devanagari, Bengali, Tamil and Thai through libass; drawtext cannot); no font = failed job. Details: [#fonts-by-script](references/gotchas.md#fonts-by-script)
211
- - `yuv420p` needs even dimensions and phone rotation tags are honoured, both automatically. Details: [#dimensions-and-rotation](references/gotchas.md#dimensions-and-rotation)
212
212
  - `--fit crop` 16:9 → 9:16 throws away 70 % of the width, and "60 seconds" by speed or by trim are different answers — say which and why. Details: [#reframing-fps-and-duration](references/gotchas.md#reframing-fps-and-duration)
213
213
  - TikTok/Reels cover the bottom fifth and the right column with their own UI — templates keep text out of those zones; `look.py --safe tiktok` shows them. Details: [#platform-safe-zones](references/gotchas.md#platform-safe-zones)
214
214
  - `scenes.py --highlights` ranks by loudness (or duration), never by meaning: check the sheet before treating picks as final. Details: [#highlights](references/gotchas.md#highlights)
package/docs/contract.md CHANGED
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
21
21
  | Field | Meaning | Changes when |
22
22
  |---|---|---|
23
23
  | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
24
- | `skill.version` | the npm / package.json version (`1.17.2`) | any release |
24
+ | `skill.version` | the npm / package.json version (`1.18.0`) | any release |
25
25
 
26
26
  A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
27
27
  ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
88
88
 
89
89
  | What 2.0 removes | Since | Replacement | To be ready today |
90
90
  |---|---|---|---|
91
- | The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.17.2 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
92
- | `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.17.2 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
93
- | `json` and `progress` in the MCP `inputSchema` | 1.17.2 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
94
- | `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.17.2 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
95
- | Overwriting an existing output with only a warning | 1.17.2 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
91
+ | The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.18.0 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
92
+ | `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.18.0 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
93
+ | `json` and `progress` in the MCP `inputSchema` | 1.18.0 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
94
+ | `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.18.0 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
95
+ | Overwriting an existing output with only a warning | 1.18.0 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
96
96
 
97
97
  ## Skill
98
98
 
99
99
  ```json
100
100
  {
101
101
  "contract_version": "1.0",
102
- "deprecated": [{"what": "...", "since": "1.17.2", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
103
- "skill": {"id": "ffmpeg-skill", "version": "1.17.2", "execution_mode": "local", "kind": "execution",
102
+ "deprecated": [{"what": "...", "since": "1.18.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
103
+ "skill": {"id": "ffmpeg-skill", "version": "1.18.0", "execution_mode": "local", "kind": "execution",
104
104
  "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
105
105
  "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
106
106
  "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
@@ -128,7 +128,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
128
128
  | `output_schema` | what `--json` prints on stdout |
129
129
  | `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
130
130
  | `supports_json` | whether `--json` exists |
131
- | `supports_json_brief` | whether `--json-brief` exists (1.17.2): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
131
+ | `supports_json_brief` | whether `--json-brief` exists (1.18.0): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
132
132
  | `mutates_input` | always `false`: no tool overwrites its input |
133
133
  | `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
134
134
  | `verification` | `{required, tools}`: which tools to run on the output afterwards |
@@ -450,7 +450,7 @@ Per-tool keys added in 1.17, all additive:
450
450
  | `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
451
451
  | `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
452
452
 
453
- Per-tool keys added in 1.17.2, all additive:
453
+ Per-tool keys added in 1.18.0, all additive:
454
454
 
455
455
  | key | tool | what it holds |
456
456
  |---|---|---|
@@ -458,6 +458,18 @@ Per-tool keys added in 1.17.2, all additive:
458
458
  | `text_unchanged` | `caption.py` | a sibling inside the `caption` block, **burn mode only** (`--mode mux` never touches the text and omits the key): `true` when the drawn text equals the cues that were handed in — nothing transcribed, no cue dropped, no cue **split** across two consecutive cues and no glyph stripped (`--emoji none`). Wrapping, line breaks and timing do not count: the words are the same. This tool never rewrites, shortens or translates a cue, so the key is a statement of what happened, not a judgement of the text |
459
459
 
460
460
 
461
+ Per-tool keys added in 1.18.0, all additive:
462
+
463
+ | key | tool | what it holds |
464
+ |---|---|---|
465
+ | `shots` | `scenes.py --shots` | `[{start, end, label, flow_magnitude}]` per detected scene, `label` one of `static`/`pan`/`motion` from a lightweight block-matching optical-flow proxy (frames decoded at 4 fps, 48x27, no external dependency). A shot too short to sample two frames is `static` with `flow_magnitude: 0` |
466
+ | `audio_peaks_db` | `scenes.py --audio-peaks` | `[{time, level}]`, measured dBFS loudness peaks. A **new** key: the pre-existing `audio_peaks` (always reported, unrelated unitless RMS figures used for `--highlights` scoring) keeps its 1.0 meaning unchanged |
467
+ | `speech` | `scenes.py --speech` | `[{time, speech_music_ratio}]`, a per-second zero-crossing-rate ratio against the file's own median — a measured proxy, not a speech/music classification |
468
+ | `motion_centre` | `cropdetect.py --motion-centre` | `[{time, x, y, x_frac, y_frac, motion}]` per second, sampled over the same windows as the crop-bar detection. `x`/`y` are source pixels, `x_frac`/`y_frac` a 0..1 fraction of `source_width`/`source_height`; a window with no measured motion reports `x`/`y`/`x_frac`/`y_frac: null`. Report only — this tool never picks a reframe |
469
+ | `speech_aware`, `speech_aware.breaths` | `silence.py --speech-aware` | `{min_silence, floor, breaths_kept, breaths_kept_seconds, breaths}`. `breaths` are the sub-`--min-silence` gaps kept because they sit inside a sentence; the removal list (`silences`, `keep`, `removed_seconds`) already reflects the speech-aware classification. Composes with `--filler` through the same `keep_ranges()`/`merge_spans()` pipeline, so `--speech-aware --filler` produces one removal list |
470
+ | `sources` | `sync.py` | `[{path, offset_s, confidence, drift_ppm}]`, one entry per SOURCE. Present for every run, including the original single-SOURCE shape (where it mirrors the top-level `second`/`offset_seconds`/`confidence` additively). With 2+ SOURCEs it is the *only* per-source shape: there is no top-level `second`/`offset_seconds` because there is no single pair to put there |
471
+ | `switch_mode`, `min_shot` | `multicam.py --switch energy` | `"energy"` and the `--min-shot` value used (default 1.5s), alongside the existing `cuts` (`[[start, end, camera], ...]`) which already carries the camera index for `--edl`'s companion cut list |
472
+
461
473
  `check.py` also gains an informational `subtitles` row on **every** platform:
462
474
  `PASS` when every soft subtitle stream carries a language tag, `WARN` when one
463
475
  does not (or when there are none). Like `channels` and `chapters` it is never
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "1.17.2",
3
+ "version": "1.18.0",
4
4
  "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
5
  "keywords": [
6
6
  "ffmpeg",
@@ -128,7 +128,7 @@ these had ever shown up before.
128
128
  are still spelled exactly that way: `_common` is a package since the refactor release after
129
129
  1.15.0, and its `__init__.py` re-exports every name the single module defined, so
130
130
  `_common.drawtext_boxborderw()`, `_common.ffmpeg_version()` and `_common.bt709_tag_args()`
131
- below resolve through the facade unchanged. The definitions are in `_common/text.py`,
131
+ below resolve through the facade unchanged. The definitions are in `_common/drawtext.py`,
132
132
  `_common/runner.py` and `_common/color.py` respectively.
133
133
  - **`showwaves` keeps emitting frames after the audio ends, `-shortest` notwithstanding.** A
134
134
  12 s source came out 14.08 s on 5.1.1. `waveform.py` now also passes `-t <source duration>`.
@@ -161,7 +161,7 @@ rounded, if they aren't.
161
161
 
162
162
  ### cropdetect.py — measure black bars, report the crop rectangle
163
163
  ```
164
- cropdetect.py INPUT [--seconds N] [--samples N] [--limit F] [--round N]
164
+ cropdetect.py INPUT [--seconds N] [--samples N] [--limit F] [--round N] [--motion-centre]
165
165
  ```
166
166
  Measurement only -- writes no file. Samples `--samples` windows spread
167
167
  across the file (default 5, totalling `--seconds` 10s of footage) and
@@ -174,6 +174,14 @@ removing detected bars is wanted -- genuine letterboxed content (a
174
174
  scope-ratio film in a 16:9 frame) "detects" the same way as accidental
175
175
  bars; look at the frame before cropping it away.
176
176
 
177
+ `--motion-centre` (1.18) reports the motion centroid once per second, sampled
178
+ across the same windows: `motion_centre: [{time, x, y, x_frac, y_frac, motion}]`,
179
+ `x`/`y` in source pixels and `x_frac`/`y_frac` as a 0..1 fraction of
180
+ `source_width`/`source_height`. A frame with no measured motion in a window
181
+ reports `x`/`y`/`x_frac`/`y_frac: null` and `motion: 0`. Report only, like the
182
+ crop rectangle above -- this hands the calling agent numbers to reframe a 9:16
183
+ crop with; it never picks the subject or the crop box itself.
184
+
177
185
  ### deinterlace.py — deinterlace interlaced footage
178
186
  ```
179
187
  deinterlace.py INPUT [--mode frame|field] [--parity auto|tff|bff] [--only-interlaced] [-o OUT]
@@ -465,7 +473,7 @@ each shorter clip's last frame (with silence) out to the longest.
465
473
 
466
474
  ### silence.py — remove dead air / jump cuts
467
475
  ```
468
- silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [-o OUT]
476
+ silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [--speech-aware] [-o OUT]
469
477
  silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|es|de|fr|pt|it]
470
478
  [--filler-words FILE] [--filler-extra W,W] [--filler-keep W,W] [--filler-pad 0.02]
471
479
  [--transcribe] [--filler-list] [--max-cuts 400]
@@ -531,6 +539,17 @@ additive sibling covering everything that went. A filler word quiet enough to
531
539
  sit inside a detected silence is merged into it rather than counted twice, so
532
540
  the two figures can be equal.
533
541
 
542
+ **`--speech-aware` (1.18)** keeps breaths shorter than `--min-silence` when
543
+ they sit inside a sentence, and only cuts at sentence-boundary pauses
544
+ (`--min-silence` or longer). It re-runs silence detection at a much shorter
545
+ floor to measure the short gaps at all, then classifies each one: shorter than
546
+ `--min-silence` is an in-sentence breath (kept, listed in
547
+ `speech_aware.breaths`); `--min-silence` or longer is a sentence boundary (cut,
548
+ same as the plain flag). It composes with `--filler` through the same
549
+ `keep_ranges()`/`merge_spans()` pipeline the filler-inside-a-silence fix
550
+ already uses, so `--speech-aware --filler` produces one removal list. `--edl`
551
+ writes the resulting cut list exactly as it does today.
552
+
534
553
  ### join.py — concatenate with transitions
535
554
  ```
536
555
  join.py CLIP1 CLIP2 [...] [--transition fade|dissolve|wipeleft|slideleft|fadeblack|fadewhite|circleopen|none]
@@ -701,6 +720,7 @@ already has a picture.
701
720
  ```
702
721
  scenes.py INPUT [--threshold 10] [--min-scene 1] [--highlights N [--target SECONDS] [--max-scene 15]] [--edl picks.txt] [--sheet scenes.png] [--json]
703
722
  scenes.py INPUT --beats [--beat-step 0.01] [--beat-range 60-200] [--min-confidence 0.5] [--json]
723
+ scenes.py INPUT [--shots] [--audio-peaks] [--speech] [--json]
704
724
  ```
705
725
  Lists scenes with audio energy, the loudest moments, and (with
706
726
  `--highlights`) proposes N ranges that add up to `--target` seconds, biased to
@@ -751,6 +771,29 @@ would change a file on the strength of it (`cut.py --snap beats`). `--edl` with
751
771
  `--beats` is unchanged: beats are never written as an EDL, because a beat is
752
772
  not a cut. No audio stream is a `kind: input` refusal.
753
773
 
774
+ **`--shots`, `--audio-peaks`, `--speech` (1.18)** are three independent
775
+ measurements that combine with each other and with `--beats`/`--highlights`:
776
+
777
+ - `--shots` labels each already-detected scene `static`/`pan`/`motion` by a
778
+ lightweight block-matching optical-flow proxy (frames decoded at 4 fps,
779
+ 48x27, no external dependency), reporting `shots: [{start, end, label,
780
+ flow_magnitude}]`. A shot too short to sample two frames is `static` with
781
+ `flow_magnitude: 0` — there is nothing to measure motion between. A proxy,
782
+ the same spirit as `--rank-by`: it reports what a coarse block match saw,
783
+ not what is interesting about the shot.
784
+ - `--audio-peaks` reports loudness peaks as measured dBFS: `audio_peaks_db:
785
+ [{time, level}]`. This is a new key, kept separate from the pre-existing
786
+ (always-on) `audio_peaks` list, whose entries are an unrelated unitless RMS
787
+ figure used for `--highlights` scoring — a different unit needed a
788
+ different key so `audio_peaks`'s meaning does not change underneath a
789
+ caller reading it since 1.0.
790
+ - `--speech` reports a per-second zero-crossing-rate ratio,
791
+ `speech: [{time, speech_music_ratio}]` — speech's rapid consonant
792
+ transients raise the zero-crossing rate; sustained tones (music, room
793
+ tone) cross zero at a steadier rate. `1.0` means "typical for this file's
794
+ own median," not an absolute threshold. This is a measured number, not a
795
+ speech/music classification — nothing here decides which stretch is which.
796
+
754
797
  ### check.py — pre-delivery compliance
755
798
  ```
756
799
  check.py INPUT --platform youtube|shorts|reels|tiktok|x|linkedin|facebook|broadcast|podcast|custom [--no-loudness] [--json]
@@ -905,7 +948,8 @@ hand the path to the user together with the numbers.
905
948
 
906
949
  ### multicam.py — align several cameras and switch between them
907
950
  ```
908
- multicam.py REF CAM2 [CAM3 ...] [--switch "START-END:CAM,..."] | [--auto N] [--audio IDX] [--fix-drift]
951
+ multicam.py REF CAM2 [CAM3 ...] [--switch "START-END:CAM,..."|energy [--min-shot 1.5]] | [--auto N]
952
+ [--audio IDX] [--fix-drift] [--edl cuts.txt]
909
953
  [--offsets-only] [--width W --height H --fps N] [-o OUT]
910
954
  ```
911
955
  All inputs are aligned to the first one by audio (same engine as `sync.py`,
@@ -915,6 +959,23 @@ simply alternates every N seconds. Audio comes from the reference unless
915
959
  `--audio` picks another input, e.g. an external recorder that has no video.
916
960
  `--offsets-only` reports offsets and confidence without rendering.
917
961
 
962
+ **`--switch energy` (1.18)** auto-switches to whichever camera (of those with
963
+ a video stream) measures the loudest audio at each 0.25 s window on the
964
+ reference timeline, then folds any run shorter than `--min-shot` (default
965
+ 1.5 s) into its neighbour so the cut never lingers on a shot too short to
966
+ read. A measured loudest-camera pick, the same spirit as `scenes.py
967
+ --rank-by audio`: a proxy for who is talking, not a judgement — a loud crowd
968
+ or a hot mic wins over a quiet subject exactly like the scene ranking does.
969
+ `--edl` (1.18, any switch mode) writes the resulting cut list as `cut.py
970
+ --segments`-format `START-END` lines; the camera index for each cut is
971
+ already in the JSON `cuts` field (`[[start, end, camera], ...]`), which is
972
+ what `--edl` leaves out on purpose — one file for an NLE's cut list, one
973
+ field for the camera it came from. The multicam timeline needs no dedicated
974
+ project format: each cut is a `render.py` clip (`{"src": ..., "in": ...,
975
+ "out": ...}` on that camera's own timeline, shifted by its measured offset),
976
+ so a switch list can be re-rendered with different `--min-shot` values by
977
+ editing `project.json`'s `clips`, not by inventing a second timeline schema.
978
+
918
979
  ### verify.py — real-footage verification kit
919
980
  ```
920
981
  verify.py FILES_OR_FOLDERS [--quick] [--report verify.md] [--out DIR --keep] [--seconds 6] [--json]
@@ -1185,18 +1246,18 @@ draws whatever glyph the text font has; `--emoji none` strips them).
1185
1246
 
1186
1247
  ### sync.py — offset detection, alignment, drift correction
1187
1248
  ```
1188
- sync.py REFERENCE SECOND [--json] [--max-offset 30] [--analyze-seconds 120] [--fix-drift [--drift-window 60]]
1189
- [--replace-audio | --trim-second] [-o OUT]
1249
+ sync.py REFERENCE SOURCE [SOURCE ...] [--json] [--max-offset 30] [--analyze-seconds 120]
1250
+ [--fix-drift [--drift-window 60]] [--replace-audio | --trim-second] [-o OUT]
1190
1251
  ```
1191
1252
  Cross-correlates loudness envelopes: coarse FFT search (20 ms), then a direct
1192
1253
  1 ms refinement (pure Python, a 2-minute window takes ~1-3 s). Positive offset
1193
- = the second recording started later. `--replace-audio` writes the reference
1194
- video with the second file's audio aligned (video stream copied); the output
1195
- keeps the reference's full length -- a shorter or head-trimmed second file is
1254
+ = the source recording started later. `--replace-audio` writes the reference
1255
+ video with the source file's audio aligned (video stream copied); the output
1256
+ keeps the reference's full length -- a shorter or head-trimmed source file is
1196
1257
  padded with silence, never allowed to cut the picture.
1197
- `--trim-second` writes the second file shifted to the reference timeline.
1258
+ `--trim-second` writes the source file shifted to the reference timeline.
1198
1259
  `--fix-drift` measures the offset again near the end of the overlap, reports
1199
- the clock difference in ppm, and resamples the second file so a 60-minute
1260
+ the clock difference in ppm, and resamples the source file so a 60-minute
1200
1261
  take stays in sync (typical consumer devices drift 20-500 ppm = up to 1.8 s/h).
1201
1262
  Use it whenever the recording is longer than ~10 minutes. Check `confidence`
1202
1263
  (0–1, normalised correlation with a runner-up penalty); below 0.3 the match is
@@ -1206,6 +1267,20 @@ window 95 %, misses flagged below 0.3. Keep `--analyze-seconds` at least 4×
1206
1267
  `--max-offset` (default 120 s vs 30 s): lags with under 35 % overlap are
1207
1268
  ignored, so an offset larger than ~60 % of the window cannot be found.
1208
1269
 
1270
+ **One reference, 1+ SOURCE (1.18).** With exactly one SOURCE the CLI keeps its
1271
+ original shape byte for byte -- `second`, `offset_seconds`, `confidence`,
1272
+ `meaning`, `drift` at the top level, `--replace-audio`/`--trim-second`
1273
+ available -- with the same measurement additively mirrored under a new
1274
+ `sources: [{path, offset_s, confidence, drift_ppm}]` list. With 2+ SOURCEs the
1275
+ result is one offsets JSON, `{reference, sources: [...]}`, and no flat
1276
+ `second`/`offset_seconds` (there is no single pair to put there);
1277
+ `--replace-audio`/`--trim-second` refuse with `kind: input`, since each writes
1278
+ one synced output and there is more than one source to choose from. This is
1279
+ the same offset/drift measurement `multicam.py` already loops over per camera
1280
+ internally (it imports `sync.measure_offset`); `sync.py`'s own N-source CLI
1281
+ now exposes that loop directly, for aligning cameras without cutting between
1282
+ them.
1283
+
1209
1284
  ### color.py — HDR to SDR, LUTs, colour tags, Dolby Vision
1210
1285
  ```
1211
1286
  color.py INPUT --to-sdr [--tonemap hable|mobius|reinhard|bt2390] [--peak 1000] [--desat 0] [-o OUT]
@@ -7,8 +7,9 @@ error reporting, and provides a compact media probe used by every script.
7
7
  Since the refactor release after 1.15.0 the helpers live in one module per responsibility --
8
8
  runner (process execution and timeouts), probe (ffprobe and the measured facts), decision (the
9
9
  pure copy-vs-re-encode and capability choices), emit (result documents, die(), info()), color
10
- (colour tags and the HDR paths) and text (fonts, scripts, emoji, drawtext) -- and this file is a
11
- facade that re-exports every name they define. `import _common` and `from _common import x` mean
10
+ (colour tags and the HDR paths) and, since the refactor after 1.17.3, fonts, emoji, drawtext and wrap
11
+ (text.py is a re-export shim over those four) -- and this file is a facade that re-exports every name
12
+ they define. `import _common` and `from _common import x` mean
12
13
  exactly what they meant when this was one 3072-line module; nothing else about the package is
13
14
  part of the contract.
14
15
  """
@@ -67,12 +68,13 @@ from _common.emit import (
67
68
  )
68
69
  from _common.probe import (
69
70
  analyze_levels, _aspect_string, _bit_depth, decode_pcm_mono, detect_scenes, detect_silences, fingerprint,
70
- _fraction, keyframes_near, SCORE_RE, SIL_RE,
71
+ _fraction, keyframes_near, SCORE_RE, SIL_RE, decode_gray_frames,
71
72
  measured_level_dbfs, MEDIA_EXT, _output_failed, probe, rms_envelope, _to_float, _to_int, verify_output
72
73
  )
73
74
  from _common.decision import (
74
75
  aac_args, add_pad_fill_args, audio_codec_for, AUDIO_CODECS, brand_caption_style, BRAND_DEFAULTS,
75
76
  description_block, _evidence_rank, fmt_chapter_time, propose_chapters,
77
+ frame_flow, label_shot_flow, MOTION_GRID, MOTION_SEARCH, MOTION_STATIC_PX, MOTION_PAN_SPREAD,
76
78
  filler_spans, FILLER_WORDS, FILLER_AMBIGUOUS, FILLER_DISCOURSE_MARKERS, FILLER_MAX_WORD,
77
79
  FILLER_MIN_GAP, FILLER_PAD, normalise_filler_token,
78
80
  beat_grid, snap_points, BEAT_MIN_CONFIDENCE, BEAT_ONSET_K, BEAT_OCTAVE_MARGIN,
@@ -112,7 +114,7 @@ from _common.text import (
112
114
  SHAPING_SCRIPTS, text_width_em, _VS15, _VS16, WINDOWS_FONTS, _ZWJ
113
115
  )
114
116
 
115
- from _common import asr, color, decision, runner, text # noqa: F401,E402
117
+ from _common import asr, color, decision, drawtext, emoji, fonts, runner, text, wrap # noqa: F401,E402
116
118
 
117
119
  # `_common.emit` and `_common.probe` are the FUNCTIONS, as they have always been -- the
118
120
  # from-imports above rebound the package attribute the submodule import had set. The two modules
@@ -123,7 +125,7 @@ from _common import asr, color, decision, runner, text # noqa: F401,E402
123
125
  _emit_module = sys.modules["_common.emit"]
124
126
  _probe_module = sys.modules["_common.probe"]
125
127
 
126
- _MODULES = (runner, _emit_module, _probe_module, decision, color, text, asr)
128
+ _MODULES = (runner, _emit_module, _probe_module, decision, color, text, fonts, emoji, drawtext, wrap, asr)
127
129
 
128
130
 
129
131
  class _Facade(_types.ModuleType):
@@ -204,6 +206,7 @@ __all__ = [
204
206
  "place_output", "_plan_at_exit", "_plan_inputs", "_PLAN_STRIP", "PLAN_VERSION", "PREFERRED_FAMILIES",
205
207
  "print_json", "probe", "PROBE_TIMEOUT", "_progress_line", "read_text_or_die", "refuse_output_is_input",
206
208
  "_remember_output", "require_tool", "resolve_emoji_assets", "_result_v2", "rms_envelope", "run", "run_analysis",
209
+ "decode_gray_frames", "frame_flow", "label_shot_flow", "MOTION_GRID", "MOTION_SEARCH", "MOTION_STATIC_PX", "MOTION_PAN_SPREAD",
207
210
  "_run_captured", "run_keeping_subtitles", "run_tool", "_run_with_progress", "_SCRIPT_FONT_CACHE",
208
211
  "_script_font_entry", "script_font_for_text", "script_font_status", "_script_font_uncached", "_SCRIPT_RANGES",
209
212
  "SCRIPTS", "_sdr_bt709", "_set_current_ctx", "_SHAPING_BUILD_CACHE", "SHAPING_SCRIPTS", "shell_quote",
@@ -864,3 +864,89 @@ def filler_spans(words, wordlist, *, pad: float = FILLER_PAD, min_gap: float = F
864
864
  m["start"] = round(m["start"], 4)
865
865
  m["end"] = round(m["end"], 4)
866
866
  return merged
867
+
868
+
869
+ # --- 1.18.0: lightweight block-matching motion estimate (scenes.py --shots, cropdetect.py
870
+ # --motion-centre) ---------------------------------------------------------------------------
871
+
872
+ MOTION_GRID = 4 # NxN anchor blocks per frame
873
+ MOTION_SEARCH = 3 # +/- pixels searched per block, at the decoded (low) resolution
874
+ MOTION_STATIC_PX = 0.35 # average per-frame displacement below this, at decode resolution, is "static"
875
+ MOTION_PAN_SPREAD = 0.6 # block-to-block direction agreement above this (0..1) reads as a pan
876
+
877
+
878
+ def _block_match(prev: bytes, cur: bytes, w: int, h: int, cx: int, cy: int, half: int, search: int) -> "Tuple[float, float]":
879
+ """(dx, dy) that best aligns a `half*2` square centred at (cx, cy) in `prev` to `cur`,
880
+ searched over +/- `search` px by sum-of-absolute-differences. Coordinates and the returned
881
+ offset are in decoded-frame pixels (a handful of pixels a side at 1.18.0's sample size)."""
882
+ x0, y0 = max(half, min(w - half - 1, cx)), max(half, min(h - half - 1, cy))
883
+ ref = [prev[(y0 + dy) * w + (x0 + dx)] for dy in range(-half, half + 1) for dx in range(-half, half + 1)]
884
+
885
+ def sad_at(xx: int, yy: int) -> "Optional[int]":
886
+ if xx - half < 0 or xx + half >= w or yy - half < 0 or yy + half >= h:
887
+ return None
888
+ sad = 0
889
+ for dy in range(-half, half + 1):
890
+ row = (yy + dy) * w
891
+ for dx in range(-half, half + 1):
892
+ sad += abs(ref[(dy + half) * (2 * half + 1) + (dx + half)] - cur[row + xx + dx])
893
+ return sad
894
+
895
+ # Zero shift is the tie-break candidate, not the search order's first cell: on a textureless
896
+ # block (a flat colour, sky, an out-of-focus background) every offset scores the same SAD, and
897
+ # without an explicit tie towards "no motion" the scan used to report the search window's
898
+ # first corner as the measured displacement -- a still frame with nothing to match against
899
+ # read as steady motion in one direction, every time.
900
+ best_sad, best = sad_at(x0, y0), (0.0, 0.0)
901
+ if best_sad is None:
902
+ best_sad = float("inf")
903
+ for sy in range(-search, search + 1):
904
+ for sx in range(-search, search + 1):
905
+ if sx == 0 and sy == 0:
906
+ continue
907
+ sad = sad_at(x0 + sx, y0 + sy)
908
+ if sad is not None and sad < best_sad:
909
+ best_sad, best = sad, (float(sx), float(sy))
910
+ return best
911
+
912
+
913
+ def frame_flow(prev: bytes, cur: bytes, w: int, h: int, *, grid: int = MOTION_GRID,
914
+ search: int = MOTION_SEARCH) -> "Dict[str, Any]":
915
+ """One measurement between two consecutive decoded grayscale frames: the mean block
916
+ displacement vector, its magnitude, and how consistently the blocks agree on direction
917
+ (0 = every block moved a different way, 1 = every block agrees -- a pan or dolly moves the
918
+ whole frame one way, on-screen motion inside a mostly-static frame does not)."""
919
+ half = max(1, min(w, h) // (grid * 3))
920
+ vecs: "List[Tuple[float, float]]" = []
921
+ for gy in range(grid):
922
+ for gx in range(grid):
923
+ cx = int((gx + 0.5) * w / grid)
924
+ cy = int((gy + 0.5) * h / grid)
925
+ vecs.append(_block_match(prev, cur, w, h, cx, cy, half, search))
926
+ mdx = sum(v[0] for v in vecs) / len(vecs)
927
+ mdy = sum(v[1] for v in vecs) / len(vecs)
928
+ magnitude = math.hypot(mdx, mdy)
929
+ mean_len = sum(math.hypot(*v) for v in vecs) / len(vecs)
930
+ agreement = (magnitude / mean_len) if mean_len > 1e-6 else 1.0 # 1.0 = every block agrees
931
+ return {"dx": mdx, "dy": mdy, "magnitude": magnitude, "agreement": min(1.0, agreement)}
932
+
933
+
934
+ def label_shot_flow(flows: "Sequence[Dict[str, Any]]") -> "Dict[str, Any]":
935
+ """{label, flow_magnitude} for one shot from its per-frame-pair flow measurements.
936
+
937
+ static: mean displacement below MOTION_STATIC_PX. pan: above it, and blocks agree on
938
+ direction (a camera move shifts the whole frame). motion: above it, blocks disagree (motion
939
+ inside an otherwise still frame -- handheld jitter, or a subject moving across a static
940
+ background). This is a measured proxy, the same spirit as scenes.py --rank-by: it reports
941
+ what a coarse block match saw, not what is interesting about the shot."""
942
+ if not flows:
943
+ return {"label": "static", "flow_magnitude": 0.0}
944
+ magnitude = sum(f["magnitude"] for f in flows) / len(flows)
945
+ agreement = sum(f["agreement"] for f in flows) / len(flows)
946
+ if magnitude < MOTION_STATIC_PX:
947
+ label = "static"
948
+ elif agreement >= MOTION_PAN_SPREAD:
949
+ label = "pan"
950
+ else:
951
+ label = "motion"
952
+ return {"label": label, "flow_magnitude": round(magnitude, 3)}