ffmpeg-skill 1.17.3 → 1.18.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/SKILL.md CHANGED
@@ -7,21 +7,21 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
7
7
 
8
8
  Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`, and delivery templates in `templates/`. This file is enough to do a job: the table below routes the request, and `--help` on the script you are about to run is the cheapest full flag list. A reference file costs as much to read as this file; open one only for a question you have: `references/scripts.md` (every flag of all 42 scripts), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end).
9
9
 
10
- Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (the same trimmed to status/output/verified plus a `summary`; prefer it on writing steps); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (the dry run as a plan `render.py FILE` runs later; refuses if an input changed). Every re-encoding tool also takes `--codec h264|hevc|av1|prores` and `--quality N` (CRF scale): unset, SDR is x264 and HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses an HDR source (`color.py --to-sdr` first).
10
+ Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (trimmed to status/output/verified plus a `summary`; prefer it on writing steps); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (step 7); `--plan FILE` (the dry run as a plan `render.py FILE` runs later; refuses if an input changed). Re-encoding tools also take `--codec h264|hevc|av1|prores` and `--quality N`: unset, SDR is x264, HDR is x265 Main10; `prores` needs `-o NAME.mov`, `h264` refuses HDR (`color.py --to-sdr` first).
11
11
 
12
12
  Writing tools run nothing under `--dry-run`; the measuring tools (`probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness`, `stabilize`) may still run ffmpeg/ffprobe — they just don't write their artifact, nor side files (`--edl`, `--sheet`, a generated `.ass`); `verify` ignores the flag. Per-tool: `contract --json`'s `dry_run` field.
13
13
 
14
14
  ## Workflow (always follow this order)
15
15
 
16
- 0. **Environment, only on failure.** Never start a job with `doctor`: a broken machine fails on its own with `kind: missing_tool` or an ffmpeg error naming the filter/encoder. Run `python3 <skill-dir>/scripts/_contract.py doctor` (or `npx ffmpeg-skill doctor`) after such a failure, or when asked what the machine can do: read `ok` and the tool's `usable`, report the missing capability. `contract --json`'s tool schema is for a *planning* agent, not this workflow.
17
- 1. **Probe what you must plan from.** Run `probe.py` on each input you plan the edit from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks about a file. No separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output. Plan from real numbers, never assumptions.
16
+ 0. **Environment, only on failure.** Never start a job with `doctor`: a broken machine fails on its own with `kind: missing_tool` or an ffmpeg error naming the filter/encoder. Run `python3 <skill-dir>/scripts/_contract.py doctor` (or `npx ffmpeg-skill doctor`) after such a failure, or when asked what the machine can do: read `ok`/`usable`, report the missing capability. `contract --json`'s tool schema is for a *planning* agent, not this workflow.
17
+ 1. **Probe what you must plan from.** Run `probe.py` on each input you plan from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks about a file. No separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output. Plan from real numbers, never assumptions.
18
18
  2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; `--accurate` on `cut.py` only for frame-exact cuts.
19
- 3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use it before long encodes and to report exact facts. `--fast` is preview quality, `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
20
- 4. **Chain in a sensible order.** A delivery request with no other editing is one template run (`render.py --template NAME INPUT`), not a hand-built chain. Otherwise: colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes before captions and overlays, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18, `export.py` last. **Three or more steps: `render.py` with a project.json** — one call, one JSON, one number for the user to change.
21
- 5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination the user named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix. Judgement rows change the content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix those only when the request implies the answer, otherwise state the choice and its cost in one line. Mention WARNs; do not chase them.
19
+ 3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's summary line, for any number in the plan (a dimension there can be a placeholder). Use it before long encodes, to report facts. `--fast` is preview quality, `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
20
+ 4. **Chain in a sensible order.** A delivery request with no other editing is one template run (`render.py --template NAME INPUT`), not a hand-built chain. Otherwise: colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes before captions, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18, `export.py` last. **Three or more steps: `render.py` with a project.json.**
21
+ 5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination named (a template run already does). Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix. Judgement rows change content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix only when the request implies the answer, else state the choice and its cost. Mention WARNs; do not chase them.
22
22
  6. **Verify the output.** Confirm duration, resolution, fps and audio match the request — from the writing tool's own `--json`/`--json-brief` probe, or `probe.py` — and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure reported with the script's error message.
23
- 7. **Keep the user's originals.** Never overwrite the source; write new files next to the input or where the user asked. Set `FFMPEG_SKILL_NO_OVERWRITE=1` in the environment you run these scripts in: an existing output path is then refused (`kind: input`) instead of warned about, with `--overwrite` the one way to say "yes, replace it". It is the recommended agent setting, and 2.0's default.
24
- 8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` that scaled a clip to the first clip's frame, a `color.py --to-sdr`) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for a job about layout across the whole clip. The job is not finished until the report's `Look:` line names that PNG — a probe cannot see a caption on someone's face. Audio-only jobs write `Look: not needed`. What to look for splits like `check.py`'s rows in step 5:
23
+ 7. **Keep the user's originals.** Never overwrite the source; write new files next to input or where asked. Set `FFMPEG_SKILL_NO_OVERWRITE=1` in the environment: an existing output path is then refused (`kind: input`) instead of warned about, `--overwrite` the one way to say "yes, replace it". Recommended agent setting, and 2.0's default.
24
+ 8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` scaling a clip to the first clip's frame, `color.py --to-sdr`) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for a job about layout across the whole clip. Not finished until `Look:` names that PNG — a probe cannot see a caption on someone's face. Audio-only jobs write `Look: not needed`. What to look for splits like `check.py`'s rows in step 5:
25
25
  - **Mechanical (this skill's own job to verify and report):** the specified text/logo is at the specified position, subtitles appear at the specified timestamps, dimensions are even. Letterboxing from `fit.py --fit pad` is the *correct* result of that mode, never a defect to flag.
26
26
  - **Judgement (report it, don't silently pass or fail):** whether a subject or face is cut off, text sits over a face, colours look washed out, a transition lands. These need deciding what the subject *is*, which belongs to the calling agent — say what you see in one line and let them judge it.
27
27
  With no vision capability, write `Look: PATH (pixels not inspected; agent has no image view)` — never claim a picture was inspected when it wasn't, and don't stall waiting for a capability that isn't there.
@@ -52,7 +52,7 @@ This skill cuts, joins, measures, syncs, exports and checks files — it execute
52
52
 
53
53
  The line: same input + same explicit parameters always producing the same verifiable output belongs here; anything depending on taste, content understanding or what looks or sounds good belongs to whoever makes that judgement.
54
54
 
55
- If a request needs an FFmpeg feature none of the 42 scripts expose, say so and name the closest built-in option — never guess a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards), so it is never the fallback when a script's flag doesn't cover something.
55
+ If a request needs an FFmpeg feature none of the 42 scripts expose, say so and name the closest built-in option — never guess a raw `ffmpeg`/`ffprobe` invocation outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes, so it is never the fallback when a script's flag doesn't cover something.
56
56
 
57
57
  ## Request → script
58
58
 
@@ -100,21 +100,24 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
100
100
  | "webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` |
101
101
  | "remove the green screen" | `overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00` |
102
102
  | "turn this podcast into a video", "audiogram" | `render.py --template audiogram ep.m4a --image cover.png` — waveform over a still or colour plate; give an image or colour, nothing is fetched |
103
- | "sync the lav mic", "line up two cameras" | `sync.py camera.mp4 mic.wav --replace-audio` / `sync.py camA.mp4 camB.mp4 --trim-second` |
103
+ | "sync the lav mic", "line up two cameras" | `sync.py camera.mp4 mic.wav --replace-audio` / `sync.py camA.mp4 camB.mp4 --trim-second`; a third+ recorder is another positional (`sync.py ref.mp4 mic.wav cam2.mp4`), one offsets JSON |
104
104
  | "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` podcast, `-I -23` broadcast; `--lra N` for the range) |
105
105
  | "cut this and make it HEVC / AV1 / ProRes" (output codec named) | `cut.py input.mp4 --start 0:10 --end 0:40 --codec hevc` (`--codec`/`--quality` on any re-encoding tool; ProRes needs `-o NAME.mov`) |
106
- | "make this a TikTok / Reel / Short / YouTube / X / LinkedIn / podcast" | `render.py --template tiktok\|reels\|shorts\|youtube-shorts\|youtube\|x\|linkedin\|facebook\|podcast input.mp4 [--cues cues.txt\|--srt subs.srt] [--title "..."] [--logo logo.png] [--brand brand.json]` — frame, safe-area captions, loudness, export and that platform's check in one command (`--list-templates`, `--write-project` to edit first) |
106
+ | "make this a TikTok / Reel / Short / YouTube / X / LinkedIn / podcast" | `render.py --template tiktok\|reels\|shorts\|youtube-shorts\|youtube\|x\|linkedin\|facebook\|podcast input.mp4 [--cues cues.txt] [--title "..."]` — frame, captions, loudness, export and check in one command (`--list-templates`, `--write-project` to edit first) |
107
107
  | "post it everywhere", "one edit for every platform" | `render.py --template all input.mp4 --cues cues.txt` (or a comma list) → one file per destination plus `<name>_pack.md` (`report.py --pack` renders the HTML) |
108
108
  | "export for YouTube / Reels / X", "a ProRes master" | `export.py input.mp4 --preset youtube\|reels\|tiktok\|shorts\|linkedin\|facebook\|x\|prores\|h265` (`--normalize` hits the loudness spec in the same call; `youtube-hdr` keeps HDR, `youtube-av1` writes AV1) |
109
109
  | "make a GIF preview" | `export.py input.mp4 --preset gif` |
110
110
  | "a small proxy / cheap preview file" | `proxy.py input.mp4 [--width 640 --no-audio]` — not a delivery preset (that is `export.py`) |
111
111
  | "cut out the pauses", "jump cuts" | `silence.py input.mp4 [--threshold -40 --min-silence 0.8]` |
112
112
  | "cut the ums and uhs", "remove the filler words" | `silence.py input.mp4 --filler --words words.json` (measured word timings; `--transcribe` makes them) |
113
+ | "don't cut inside a sentence, just the real pauses" | `silence.py input.mp4 --speech-aware` — a breath under `--min-silence` inside a sentence is kept, only sentence-boundary pauses cut; composes with `--filler` into one list |
113
114
  | "stitch these clips", "add a crossfade" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` |
114
115
  | "show me what it looks like", "are the captions readable" | `look.py output.mp4 --tiles 3x2`, then view the PNG |
115
116
  | "what would you run?", "don't render yet" | any script with `--dry-run` |
116
117
  | "a 60 s highlight from this hour" | `scenes.py long.mp4 --highlights 6 --target 60 --edl picks.txt` → `cut.py --segments` |
117
118
  | "cut on the beat", "edit it to the music" | `scenes.py track.mp4 --beats --json > beats.json`, then `cut.py input.mp4 --segments ... --snap beats --snap-source beats.json` (`--snap-source` carries the measured grid over) |
119
+ | "which shots are static vs moving", "volume peaks per second", "is this speech or music" | `scenes.py input.mp4 --shots` (static/pan/motion per shot, measured flow) / `--audio-peaks` (dBFS list) / `--speech` (a speech-vs-music ratio, not a classification) — combine, or alone |
120
+ | "where does the subject move, so I can crop it myself" | `cropdetect.py input.mp4 --motion-centre` — motion centroid per second, report-only; the calling agent picks the crop |
118
121
  | "is this OK to upload?" | `check.py final.mp4 --platform reels` |
119
122
  | "a podcast episode with chapters" | `loudness.py ep.wav -I -16 --tp -1.5` → `metadata.py ep.m4a --chapters chapters.txt` → `check.py ep.m4a --platform podcast` (chapters and channels rows) |
120
123
  | "several changes to the same edit", 3+ steps | `render.py --init project.json`, edit, `render.py project.json` |
@@ -126,7 +129,7 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
126
129
  | "send me a summary of what you did" | `report.py --before raw.mov --after final.mp4 --platform youtube -o report.html` |
127
130
  | "do this to every file in the folder", "use all the cores" | `batch.py FOLDER --recipe batch.json --jobs auto` (steps or a render project; cached) |
128
131
  | "transcribe it and caption it" | `caption.py input.mp4 --transcribe --animate pop --karaoke` (needs a local whisper; else `--text`) |
129
- | "three cameras, cut between them" | `multicam.py camA.mp4 camB.mp4 camC.mp4 --switch "0-20:0,20-40:1,40-60:2"` |
132
+ | "three cameras, cut between them" | `multicam.py camA.mp4 camB.mp4 camC.mp4 --switch "0-20:0,20-40:1,40-60:2"` (manual) or `--switch energy` (auto-cuts to the loudest camera, `--min-shot`, `--edl`) |
130
133
  | "iPhone Dolby Vision clip looks wrong" | `color.py clip.mov --to-sdr` or `--strip-dovi` (keep HDR, drop the DV layer) |
131
134
  | "does it look like Log / S-Log?" | `probe.py clip.mp4 --analyze` (`looks_like_log`) then `color.py --lut` |
132
135
  | "test the tool on my real files" | `verify.py ~/Footage --report verify.md` |
@@ -150,12 +153,12 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
150
153
 
151
154
  ## Audio-only files
152
155
 
153
- Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`, `audio.py`, `sync.py`, `check.py --platform podcast` and `render.py --template podcast` take WAV, FLAC, MP3, M4A/AAC, OGG and Opus; the output extension picks the format. `Check:` still applies. Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" — say so, don't force a video wrapper. Audio recipes, packet vs sample precision, joining, extracting a track: `references/gotchas.md#audio-only-files`.
156
+ Audio is a first-class input: `probe`, `cut`, `silence`, `loudness`, `audio`, `sync`, `check --platform podcast`, `render --template podcast` take WAV, FLAC, MP3, M4A/AAC, OGG, Opus; output extension picks the format. Scripts needing a picture (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse audio with "input has no video stream" — don't force a video wrapper. Recipes: `references/gotchas.md#audio-only-files`.
154
157
 
155
158
 
156
159
  ## Report format
157
160
 
158
- Reply in the language the request itself is written in — the user's own sentences, not a language the request merely talks about (subtitles in another language are still reported in the request's language). Keep the field labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English: they read as log fields in any language. Everything else is the user's language — the lines those labels head, any question, any judgement call. Never drift because the job was short or failed: even a one-line "file does not exist". A mid-conversation switch follows the user's latest message.
161
+ Reply in the language the request is written in — subtitles in another language are still reported in the request's language. Keep the labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English; everything else is the user's language. Never drift because the job was short or failed: even a one-line "file does not exist". A mid-conversation switch follows the user's latest message.
159
162
 
160
163
  Finish every job with this shape (numbers from `--json` or `probe.py`/`check.py`, not memory):
161
164
 
@@ -192,7 +195,7 @@ Notes: send a valid .cube, or say if you want the clip left as is
192
195
  Those filler lines are sentences, not labels: the same report for a Japanese request ends
193
196
  `Check: 検証するものなし` / `Look: 不要`.
194
197
 
195
- A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure gets all five labels, never headings. A partial result is `Done:` with the shortfall in `Notes:`, and a refusal that still delivers something is `Failed:` — never a third label like `Done (partially):`. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change a retry needs.
198
+ A refusal uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure gets all five labels, never headings. A partial result is `Done:` with the shortfall in `Notes:`; a refusal that still delivers something is `Failed:` — never a third label like `Done (partially):`. Quote a failure's `error.hint` in `Notes:`: it names the flag change a retry needs.
196
199
 
197
200
  Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message, never paraphrase it.
198
201
 
package/docs/contract.md CHANGED
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
21
21
  | Field | Meaning | Changes when |
22
22
  |---|---|---|
23
23
  | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
24
- | `skill.version` | the npm / package.json version (`1.17.3`) | any release |
24
+ | `skill.version` | the npm / package.json version (`1.18.1`) | any release |
25
25
 
26
26
  A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
27
27
  ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
88
88
 
89
89
  | What 2.0 removes | Since | Replacement | To be ready today |
90
90
  |---|---|---|---|
91
- | The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.17.3 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
92
- | `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.17.3 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
93
- | `json` and `progress` in the MCP `inputSchema` | 1.17.3 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
94
- | `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.17.3 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
95
- | Overwriting an existing output with only a warning | 1.17.3 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
91
+ | The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.18.1 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
92
+ | `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.18.1 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
93
+ | `json` and `progress` in the MCP `inputSchema` | 1.18.1 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
94
+ | `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.18.1 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
95
+ | Overwriting an existing output with only a warning | 1.18.1 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
96
96
 
97
97
  ## Skill
98
98
 
99
99
  ```json
100
100
  {
101
101
  "contract_version": "1.0",
102
- "deprecated": [{"what": "...", "since": "1.17.3", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
103
- "skill": {"id": "ffmpeg-skill", "version": "1.17.3", "execution_mode": "local", "kind": "execution",
102
+ "deprecated": [{"what": "...", "since": "1.18.1", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
103
+ "skill": {"id": "ffmpeg-skill", "version": "1.18.1", "execution_mode": "local", "kind": "execution",
104
104
  "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
105
105
  "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
106
106
  "requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
@@ -128,7 +128,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
128
128
  | `output_schema` | what `--json` prints on stdout |
129
129
  | `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
130
130
  | `supports_json` | whether `--json` exists |
131
- | `supports_json_brief` | whether `--json-brief` exists (1.17.3): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
131
+ | `supports_json_brief` | whether `--json-brief` exists (1.18.1): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
132
132
  | `mutates_input` | always `false`: no tool overwrites its input |
133
133
  | `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
134
134
  | `verification` | `{required, tools}`: which tools to run on the output afterwards |
@@ -450,7 +450,7 @@ Per-tool keys added in 1.17, all additive:
450
450
  | `jobs`, `jobs_requested`, `wall_seconds`, `item_seconds_total`, `timed_out` | `batch.py` | the parallelism actually applied and the number asked for, the batch's wall clock, the sum of the per-item times (so the speed-up can be quoted), and whether the shared timeout budget ran out. A timed-out item carries `"skipped": "timeout"` in its result row |
451
451
  | `cache` | `render.py --cache` | `{dir, ffmpeg, hits, misses, saved_seconds, entries}`, plus `would_hit` under `--dry-run`. The ffmpeg build banner, the skill version, the contract version, the forwarded flags (`--fast`, `--codec`, …) and the output's extension are all part of every key, so a cache is never reused across any of them — a `--fast` draft is never served to a run that did not ask for one |
452
452
 
453
- Per-tool keys added in 1.17.3, all additive:
453
+ Per-tool keys added in 1.18.1, all additive:
454
454
 
455
455
  | key | tool | what it holds |
456
456
  |---|---|---|
@@ -458,6 +458,18 @@ Per-tool keys added in 1.17.3, all additive:
458
458
  | `text_unchanged` | `caption.py` | a sibling inside the `caption` block, **burn mode only** (`--mode mux` never touches the text and omits the key): `true` when the drawn text equals the cues that were handed in — nothing transcribed, no cue dropped, no cue **split** across two consecutive cues and no glyph stripped (`--emoji none`). Wrapping, line breaks and timing do not count: the words are the same. This tool never rewrites, shortens or translates a cue, so the key is a statement of what happened, not a judgement of the text |
459
459
 
460
460
 
461
+ Per-tool keys added in 1.18.1, all additive:
462
+
463
+ | key | tool | what it holds |
464
+ |---|---|---|
465
+ | `shots` | `scenes.py --shots` | `[{start, end, label, flow_magnitude}]` per detected scene, `label` one of `static`/`pan`/`motion` from a lightweight block-matching optical-flow proxy (frames decoded at 4 fps, 48x27, no external dependency). A shot too short to sample two frames is `static` with `flow_magnitude: 0` |
466
+ | `audio_peaks_db` | `scenes.py --audio-peaks` | `[{time, level}]`, measured dBFS loudness peaks. A **new** key: the pre-existing `audio_peaks` (always reported, unrelated unitless RMS figures used for `--highlights` scoring) keeps its 1.0 meaning unchanged |
467
+ | `speech` | `scenes.py --speech` | `[{time, speech_music_ratio}]`, a per-second zero-crossing-rate ratio against the file's own median — a measured proxy, not a speech/music classification |
468
+ | `motion_centre` | `cropdetect.py --motion-centre` | `[{time, x, y, x_frac, y_frac, motion}]` per second, sampled over the same windows as the crop-bar detection. `x`/`y` are source pixels, `x_frac`/`y_frac` a 0..1 fraction of `source_width`/`source_height`; a window with no measured motion reports `x`/`y`/`x_frac`/`y_frac: null`. Report only — this tool never picks a reframe |
469
+ | `speech_aware`, `speech_aware.breaths` | `silence.py --speech-aware` | `{min_silence, floor, breaths_kept, breaths_kept_seconds, breaths}`. `breaths` are the sub-`--min-silence` gaps kept because they sit inside a sentence; the removal list (`silences`, `keep`, `removed_seconds`) already reflects the speech-aware classification. Composes with `--filler` through the same `keep_ranges()`/`merge_spans()` pipeline, so `--speech-aware --filler` produces one removal list |
470
+ | `sources` | `sync.py` | `[{path, offset_s, confidence, drift_ppm}]`, one entry per SOURCE. Present for every run, including the original single-SOURCE shape (where it mirrors the top-level `second`/`offset_seconds`/`confidence` additively). With 2+ SOURCEs it is the *only* per-source shape: there is no top-level `second`/`offset_seconds` because there is no single pair to put there |
471
+ | `switch_mode`, `min_shot` | `multicam.py --switch energy` | `"energy"` and the `--min-shot` value used (default 1.5s), alongside the existing `cuts` (`[[start, end, camera], ...]`) which already carries the camera index for `--edl`'s companion cut list |
472
+
461
473
  `check.py` also gains an informational `subtitles` row on **every** platform:
462
474
  `PASS` when every soft subtitle stream carries a language tag, `WARN` when one
463
475
  does not (or when there are none). Like `channels` and `chapters` it is never
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "1.17.3",
3
+ "version": "1.18.1",
4
4
  "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
5
  "keywords": [
6
6
  "ffmpeg",
@@ -128,7 +128,7 @@ these had ever shown up before.
128
128
  are still spelled exactly that way: `_common` is a package since the refactor release after
129
129
  1.15.0, and its `__init__.py` re-exports every name the single module defined, so
130
130
  `_common.drawtext_boxborderw()`, `_common.ffmpeg_version()` and `_common.bt709_tag_args()`
131
- below resolve through the facade unchanged. The definitions are in `_common/text.py`,
131
+ below resolve through the facade unchanged. The definitions are in `_common/drawtext.py`,
132
132
  `_common/runner.py` and `_common/color.py` respectively.
133
133
  - **`showwaves` keeps emitting frames after the audio ends, `-shortest` notwithstanding.** A
134
134
  12 s source came out 14.08 s on 5.1.1. `waveform.py` now also passes `-t <source duration>`.
@@ -161,7 +161,7 @@ rounded, if they aren't.
161
161
 
162
162
  ### cropdetect.py — measure black bars, report the crop rectangle
163
163
  ```
164
- cropdetect.py INPUT [--seconds N] [--samples N] [--limit F] [--round N]
164
+ cropdetect.py INPUT [--seconds N] [--samples N] [--limit F] [--round N] [--motion-centre]
165
165
  ```
166
166
  Measurement only -- writes no file. Samples `--samples` windows spread
167
167
  across the file (default 5, totalling `--seconds` 10s of footage) and
@@ -174,6 +174,14 @@ removing detected bars is wanted -- genuine letterboxed content (a
174
174
  scope-ratio film in a 16:9 frame) "detects" the same way as accidental
175
175
  bars; look at the frame before cropping it away.
176
176
 
177
+ `--motion-centre` (1.18) reports the motion centroid once per second, sampled
178
+ across the same windows: `motion_centre: [{time, x, y, x_frac, y_frac, motion}]`,
179
+ `x`/`y` in source pixels and `x_frac`/`y_frac` as a 0..1 fraction of
180
+ `source_width`/`source_height`. A frame with no measured motion in a window
181
+ reports `x`/`y`/`x_frac`/`y_frac: null` and `motion: 0`. Report only, like the
182
+ crop rectangle above -- this hands the calling agent numbers to reframe a 9:16
183
+ crop with; it never picks the subject or the crop box itself.
184
+
177
185
  ### deinterlace.py — deinterlace interlaced footage
178
186
  ```
179
187
  deinterlace.py INPUT [--mode frame|field] [--parity auto|tff|bff] [--only-interlaced] [-o OUT]
@@ -465,7 +473,7 @@ each shorter clip's last frame (with silence) out to the longest.
465
473
 
466
474
  ### silence.py — remove dead air / jump cuts
467
475
  ```
468
- silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [-o OUT]
476
+ silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [--speech-aware] [-o OUT]
469
477
  silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|es|de|fr|pt|it]
470
478
  [--filler-words FILE] [--filler-extra W,W] [--filler-keep W,W] [--filler-pad 0.02]
471
479
  [--transcribe] [--filler-list] [--max-cuts 400]
@@ -531,6 +539,17 @@ additive sibling covering everything that went. A filler word quiet enough to
531
539
  sit inside a detected silence is merged into it rather than counted twice, so
532
540
  the two figures can be equal.
533
541
 
542
+ **`--speech-aware` (1.18)** keeps breaths shorter than `--min-silence` when
543
+ they sit inside a sentence, and only cuts at sentence-boundary pauses
544
+ (`--min-silence` or longer). It re-runs silence detection at a much shorter
545
+ floor to measure the short gaps at all, then classifies each one: shorter than
546
+ `--min-silence` is an in-sentence breath (kept, listed in
547
+ `speech_aware.breaths`); `--min-silence` or longer is a sentence boundary (cut,
548
+ same as the plain flag). It composes with `--filler` through the same
549
+ `keep_ranges()`/`merge_spans()` pipeline the filler-inside-a-silence fix
550
+ already uses, so `--speech-aware --filler` produces one removal list. `--edl`
551
+ writes the resulting cut list exactly as it does today.
552
+
534
553
  ### join.py — concatenate with transitions
535
554
  ```
536
555
  join.py CLIP1 CLIP2 [...] [--transition fade|dissolve|wipeleft|slideleft|fadeblack|fadewhite|circleopen|none]
@@ -701,6 +720,7 @@ already has a picture.
701
720
  ```
702
721
  scenes.py INPUT [--threshold 10] [--min-scene 1] [--highlights N [--target SECONDS] [--max-scene 15]] [--edl picks.txt] [--sheet scenes.png] [--json]
703
722
  scenes.py INPUT --beats [--beat-step 0.01] [--beat-range 60-200] [--min-confidence 0.5] [--json]
723
+ scenes.py INPUT [--shots] [--audio-peaks] [--speech] [--json]
704
724
  ```
705
725
  Lists scenes with audio energy, the loudest moments, and (with
706
726
  `--highlights`) proposes N ranges that add up to `--target` seconds, biased to
@@ -751,6 +771,29 @@ would change a file on the strength of it (`cut.py --snap beats`). `--edl` with
751
771
  `--beats` is unchanged: beats are never written as an EDL, because a beat is
752
772
  not a cut. No audio stream is a `kind: input` refusal.
753
773
 
774
+ **`--shots`, `--audio-peaks`, `--speech` (1.18)** are three independent
775
+ measurements that combine with each other and with `--beats`/`--highlights`:
776
+
777
+ - `--shots` labels each already-detected scene `static`/`pan`/`motion` by a
778
+ lightweight block-matching optical-flow proxy (frames decoded at 4 fps,
779
+ 48x27, no external dependency), reporting `shots: [{start, end, label,
780
+ flow_magnitude}]`. A shot too short to sample two frames is `static` with
781
+ `flow_magnitude: 0` — there is nothing to measure motion between. A proxy,
782
+ the same spirit as `--rank-by`: it reports what a coarse block match saw,
783
+ not what is interesting about the shot.
784
+ - `--audio-peaks` reports loudness peaks as measured dBFS: `audio_peaks_db:
785
+ [{time, level}]`. This is a new key, kept separate from the pre-existing
786
+ (always-on) `audio_peaks` list, whose entries are an unrelated unitless RMS
787
+ figure used for `--highlights` scoring — a different unit needed a
788
+ different key so `audio_peaks`'s meaning does not change underneath a
789
+ caller reading it since 1.0.
790
+ - `--speech` reports a per-second zero-crossing-rate ratio,
791
+ `speech: [{time, speech_music_ratio}]` — speech's rapid consonant
792
+ transients raise the zero-crossing rate; sustained tones (music, room
793
+ tone) cross zero at a steadier rate. `1.0` means "typical for this file's
794
+ own median," not an absolute threshold. This is a measured number, not a
795
+ speech/music classification — nothing here decides which stretch is which.
796
+
754
797
  ### check.py — pre-delivery compliance
755
798
  ```
756
799
  check.py INPUT --platform youtube|shorts|reels|tiktok|x|linkedin|facebook|broadcast|podcast|custom [--no-loudness] [--json]
@@ -905,7 +948,8 @@ hand the path to the user together with the numbers.
905
948
 
906
949
  ### multicam.py — align several cameras and switch between them
907
950
  ```
908
- multicam.py REF CAM2 [CAM3 ...] [--switch "START-END:CAM,..."] | [--auto N] [--audio IDX] [--fix-drift]
951
+ multicam.py REF CAM2 [CAM3 ...] [--switch "START-END:CAM,..."|energy [--min-shot 1.5]] | [--auto N]
952
+ [--audio IDX] [--fix-drift] [--edl cuts.txt]
909
953
  [--offsets-only] [--width W --height H --fps N] [-o OUT]
910
954
  ```
911
955
  All inputs are aligned to the first one by audio (same engine as `sync.py`,
@@ -915,6 +959,23 @@ simply alternates every N seconds. Audio comes from the reference unless
915
959
  `--audio` picks another input, e.g. an external recorder that has no video.
916
960
  `--offsets-only` reports offsets and confidence without rendering.
917
961
 
962
+ **`--switch energy` (1.18)** auto-switches to whichever camera (of those with
963
+ a video stream) measures the loudest audio at each 0.25 s window on the
964
+ reference timeline, then folds any run shorter than `--min-shot` (default
965
+ 1.5 s) into its neighbour so the cut never lingers on a shot too short to
966
+ read. A measured loudest-camera pick, the same spirit as `scenes.py
967
+ --rank-by audio`: a proxy for who is talking, not a judgement — a loud crowd
968
+ or a hot mic wins over a quiet subject exactly like the scene ranking does.
969
+ `--edl` (1.18, any switch mode) writes the resulting cut list as `cut.py
970
+ --segments`-format `START-END` lines; the camera index for each cut is
971
+ already in the JSON `cuts` field (`[[start, end, camera], ...]`), which is
972
+ what `--edl` leaves out on purpose — one file for an NLE's cut list, one
973
+ field for the camera it came from. The multicam timeline needs no dedicated
974
+ project format: each cut is a `render.py` clip (`{"src": ..., "in": ...,
975
+ "out": ...}` on that camera's own timeline, shifted by its measured offset),
976
+ so a switch list can be re-rendered with different `--min-shot` values by
977
+ editing `project.json`'s `clips`, not by inventing a second timeline schema.
978
+
918
979
  ### verify.py — real-footage verification kit
919
980
  ```
920
981
  verify.py FILES_OR_FOLDERS [--quick] [--report verify.md] [--out DIR --keep] [--seconds 6] [--json]
@@ -1185,18 +1246,18 @@ draws whatever glyph the text font has; `--emoji none` strips them).
1185
1246
 
1186
1247
  ### sync.py — offset detection, alignment, drift correction
1187
1248
  ```
1188
- sync.py REFERENCE SECOND [--json] [--max-offset 30] [--analyze-seconds 120] [--fix-drift [--drift-window 60]]
1189
- [--replace-audio | --trim-second] [-o OUT]
1249
+ sync.py REFERENCE SOURCE [SOURCE ...] [--json] [--max-offset 30] [--analyze-seconds 120]
1250
+ [--fix-drift [--drift-window 60]] [--replace-audio | --trim-second] [-o OUT]
1190
1251
  ```
1191
1252
  Cross-correlates loudness envelopes: coarse FFT search (20 ms), then a direct
1192
1253
  1 ms refinement (pure Python, a 2-minute window takes ~1-3 s). Positive offset
1193
- = the second recording started later. `--replace-audio` writes the reference
1194
- video with the second file's audio aligned (video stream copied); the output
1195
- keeps the reference's full length -- a shorter or head-trimmed second file is
1254
+ = the source recording started later. `--replace-audio` writes the reference
1255
+ video with the source file's audio aligned (video stream copied); the output
1256
+ keeps the reference's full length -- a shorter or head-trimmed source file is
1196
1257
  padded with silence, never allowed to cut the picture.
1197
- `--trim-second` writes the second file shifted to the reference timeline.
1258
+ `--trim-second` writes the source file shifted to the reference timeline.
1198
1259
  `--fix-drift` measures the offset again near the end of the overlap, reports
1199
- the clock difference in ppm, and resamples the second file so a 60-minute
1260
+ the clock difference in ppm, and resamples the source file so a 60-minute
1200
1261
  take stays in sync (typical consumer devices drift 20-500 ppm = up to 1.8 s/h).
1201
1262
  Use it whenever the recording is longer than ~10 minutes. Check `confidence`
1202
1263
  (0–1, normalised correlation with a runner-up penalty); below 0.3 the match is
@@ -1206,6 +1267,20 @@ window 95 %, misses flagged below 0.3. Keep `--analyze-seconds` at least 4×
1206
1267
  `--max-offset` (default 120 s vs 30 s): lags with under 35 % overlap are
1207
1268
  ignored, so an offset larger than ~60 % of the window cannot be found.
1208
1269
 
1270
+ **One reference, 1+ SOURCE (1.18).** With exactly one SOURCE the CLI keeps its
1271
+ original shape byte for byte -- `second`, `offset_seconds`, `confidence`,
1272
+ `meaning`, `drift` at the top level, `--replace-audio`/`--trim-second`
1273
+ available -- with the same measurement additively mirrored under a new
1274
+ `sources: [{path, offset_s, confidence, drift_ppm}]` list. With 2+ SOURCEs the
1275
+ result is one offsets JSON, `{reference, sources: [...]}`, and no flat
1276
+ `second`/`offset_seconds` (there is no single pair to put there);
1277
+ `--replace-audio`/`--trim-second` refuse with `kind: input`, since each writes
1278
+ one synced output and there is more than one source to choose from. This is
1279
+ the same offset/drift measurement `multicam.py` already loops over per camera
1280
+ internally (it imports `sync.measure_offset`); `sync.py`'s own N-source CLI
1281
+ now exposes that loop directly, for aligning cameras without cutting between
1282
+ them.
1283
+
1209
1284
  ### color.py — HDR to SDR, LUTs, colour tags, Dolby Vision
1210
1285
  ```
1211
1286
  color.py INPUT --to-sdr [--tonemap hable|mobius|reinhard|bt2390] [--peak 1000] [--desat 0] [-o OUT]
@@ -7,8 +7,9 @@ error reporting, and provides a compact media probe used by every script.
7
7
  Since the refactor release after 1.15.0 the helpers live in one module per responsibility --
8
8
  runner (process execution and timeouts), probe (ffprobe and the measured facts), decision (the
9
9
  pure copy-vs-re-encode and capability choices), emit (result documents, die(), info()), color
10
- (colour tags and the HDR paths) and text (fonts, scripts, emoji, drawtext) -- and this file is a
11
- facade that re-exports every name they define. `import _common` and `from _common import x` mean
10
+ (colour tags and the HDR paths) and, since the refactor after 1.17.3, fonts, emoji, drawtext and wrap
11
+ (text.py is a re-export shim over those four) -- and this file is a facade that re-exports every name
12
+ they define. `import _common` and `from _common import x` mean
12
13
  exactly what they meant when this was one 3072-line module; nothing else about the package is
13
14
  part of the contract.
14
15
  """
@@ -67,12 +68,13 @@ from _common.emit import (
67
68
  )
68
69
  from _common.probe import (
69
70
  analyze_levels, _aspect_string, _bit_depth, decode_pcm_mono, detect_scenes, detect_silences, fingerprint,
70
- _fraction, keyframes_near, SCORE_RE, SIL_RE,
71
+ _fraction, keyframes_near, SCORE_RE, SIL_RE, decode_gray_frames,
71
72
  measured_level_dbfs, MEDIA_EXT, _output_failed, probe, rms_envelope, _to_float, _to_int, verify_output
72
73
  )
73
74
  from _common.decision import (
74
75
  aac_args, add_pad_fill_args, audio_codec_for, AUDIO_CODECS, brand_caption_style, BRAND_DEFAULTS,
75
76
  description_block, _evidence_rank, fmt_chapter_time, propose_chapters,
77
+ frame_flow, label_shot_flow, MOTION_GRID, MOTION_SEARCH, MOTION_STATIC_PX, MOTION_PAN_SPREAD,
76
78
  filler_spans, FILLER_WORDS, FILLER_AMBIGUOUS, FILLER_DISCOURSE_MARKERS, FILLER_MAX_WORD,
77
79
  FILLER_MIN_GAP, FILLER_PAD, normalise_filler_token,
78
80
  beat_grid, snap_points, BEAT_MIN_CONFIDENCE, BEAT_ONSET_K, BEAT_OCTAVE_MARGIN,
@@ -112,7 +114,7 @@ from _common.text import (
112
114
  SHAPING_SCRIPTS, text_width_em, _VS15, _VS16, WINDOWS_FONTS, _ZWJ
113
115
  )
114
116
 
115
- from _common import asr, color, decision, runner, text # noqa: F401,E402
117
+ from _common import asr, color, decision, drawtext, emoji, fonts, runner, text, wrap # noqa: F401,E402
116
118
 
117
119
  # `_common.emit` and `_common.probe` are the FUNCTIONS, as they have always been -- the
118
120
  # from-imports above rebound the package attribute the submodule import had set. The two modules
@@ -123,7 +125,7 @@ from _common import asr, color, decision, runner, text # noqa: F401,E402
123
125
  _emit_module = sys.modules["_common.emit"]
124
126
  _probe_module = sys.modules["_common.probe"]
125
127
 
126
- _MODULES = (runner, _emit_module, _probe_module, decision, color, text, asr)
128
+ _MODULES = (runner, _emit_module, _probe_module, decision, color, text, fonts, emoji, drawtext, wrap, asr)
127
129
 
128
130
 
129
131
  class _Facade(_types.ModuleType):
@@ -204,6 +206,7 @@ __all__ = [
204
206
  "place_output", "_plan_at_exit", "_plan_inputs", "_PLAN_STRIP", "PLAN_VERSION", "PREFERRED_FAMILIES",
205
207
  "print_json", "probe", "PROBE_TIMEOUT", "_progress_line", "read_text_or_die", "refuse_output_is_input",
206
208
  "_remember_output", "require_tool", "resolve_emoji_assets", "_result_v2", "rms_envelope", "run", "run_analysis",
209
+ "decode_gray_frames", "frame_flow", "label_shot_flow", "MOTION_GRID", "MOTION_SEARCH", "MOTION_STATIC_PX", "MOTION_PAN_SPREAD",
207
210
  "_run_captured", "run_keeping_subtitles", "run_tool", "_run_with_progress", "_SCRIPT_FONT_CACHE",
208
211
  "_script_font_entry", "script_font_for_text", "script_font_status", "_script_font_uncached", "_SCRIPT_RANGES",
209
212
  "SCRIPTS", "_sdr_bt709", "_set_current_ctx", "_SHAPING_BUILD_CACHE", "SHAPING_SCRIPTS", "shell_quote",
@@ -864,3 +864,89 @@ def filler_spans(words, wordlist, *, pad: float = FILLER_PAD, min_gap: float = F
864
864
  m["start"] = round(m["start"], 4)
865
865
  m["end"] = round(m["end"], 4)
866
866
  return merged
867
+
868
+
869
+ # --- 1.18.0: lightweight block-matching motion estimate (scenes.py --shots, cropdetect.py
870
+ # --motion-centre) ---------------------------------------------------------------------------
871
+
872
+ MOTION_GRID = 4 # NxN anchor blocks per frame
873
+ MOTION_SEARCH = 3 # +/- pixels searched per block, at the decoded (low) resolution
874
+ MOTION_STATIC_PX = 0.35 # average per-frame displacement below this, at decode resolution, is "static"
875
+ MOTION_PAN_SPREAD = 0.6 # block-to-block direction agreement above this (0..1) reads as a pan
876
+
877
+
878
+ def _block_match(prev: bytes, cur: bytes, w: int, h: int, cx: int, cy: int, half: int, search: int) -> "Tuple[float, float]":
879
+ """(dx, dy) that best aligns a `half*2` square centred at (cx, cy) in `prev` to `cur`,
880
+ searched over +/- `search` px by sum-of-absolute-differences. Coordinates and the returned
881
+ offset are in decoded-frame pixels (a handful of pixels a side at 1.18.0's sample size)."""
882
+ x0, y0 = max(half, min(w - half - 1, cx)), max(half, min(h - half - 1, cy))
883
+ ref = [prev[(y0 + dy) * w + (x0 + dx)] for dy in range(-half, half + 1) for dx in range(-half, half + 1)]
884
+
885
+ def sad_at(xx: int, yy: int) -> "Optional[int]":
886
+ if xx - half < 0 or xx + half >= w or yy - half < 0 or yy + half >= h:
887
+ return None
888
+ sad = 0
889
+ for dy in range(-half, half + 1):
890
+ row = (yy + dy) * w
891
+ for dx in range(-half, half + 1):
892
+ sad += abs(ref[(dy + half) * (2 * half + 1) + (dx + half)] - cur[row + xx + dx])
893
+ return sad
894
+
895
+ # Zero shift is the tie-break candidate, not the search order's first cell: on a textureless
896
+ # block (a flat colour, sky, an out-of-focus background) every offset scores the same SAD, and
897
+ # without an explicit tie towards "no motion" the scan used to report the search window's
898
+ # first corner as the measured displacement -- a still frame with nothing to match against
899
+ # read as steady motion in one direction, every time.
900
+ best_sad, best = sad_at(x0, y0), (0.0, 0.0)
901
+ if best_sad is None:
902
+ best_sad = float("inf")
903
+ for sy in range(-search, search + 1):
904
+ for sx in range(-search, search + 1):
905
+ if sx == 0 and sy == 0:
906
+ continue
907
+ sad = sad_at(x0 + sx, y0 + sy)
908
+ if sad is not None and sad < best_sad:
909
+ best_sad, best = sad, (float(sx), float(sy))
910
+ return best
911
+
912
+
913
+ def frame_flow(prev: bytes, cur: bytes, w: int, h: int, *, grid: int = MOTION_GRID,
914
+ search: int = MOTION_SEARCH) -> "Dict[str, Any]":
915
+ """One measurement between two consecutive decoded grayscale frames: the mean block
916
+ displacement vector, its magnitude, and how consistently the blocks agree on direction
917
+ (0 = every block moved a different way, 1 = every block agrees -- a pan or dolly moves the
918
+ whole frame one way, on-screen motion inside a mostly-static frame does not)."""
919
+ half = max(1, min(w, h) // (grid * 3))
920
+ vecs: "List[Tuple[float, float]]" = []
921
+ for gy in range(grid):
922
+ for gx in range(grid):
923
+ cx = int((gx + 0.5) * w / grid)
924
+ cy = int((gy + 0.5) * h / grid)
925
+ vecs.append(_block_match(prev, cur, w, h, cx, cy, half, search))
926
+ mdx = sum(v[0] for v in vecs) / len(vecs)
927
+ mdy = sum(v[1] for v in vecs) / len(vecs)
928
+ magnitude = math.hypot(mdx, mdy)
929
+ mean_len = sum(math.hypot(*v) for v in vecs) / len(vecs)
930
+ agreement = (magnitude / mean_len) if mean_len > 1e-6 else 1.0 # 1.0 = every block agrees
931
+ return {"dx": mdx, "dy": mdy, "magnitude": magnitude, "agreement": min(1.0, agreement)}
932
+
933
+
934
+ def label_shot_flow(flows: "Sequence[Dict[str, Any]]") -> "Dict[str, Any]":
935
+ """{label, flow_magnitude} for one shot from its per-frame-pair flow measurements.
936
+
937
+ static: mean displacement below MOTION_STATIC_PX. pan: above it, and blocks agree on
938
+ direction (a camera move shifts the whole frame). motion: above it, blocks disagree (motion
939
+ inside an otherwise still frame -- handheld jitter, or a subject moving across a static
940
+ background). This is a measured proxy, the same spirit as scenes.py --rank-by: it reports
941
+ what a coarse block match saw, not what is interesting about the shot."""
942
+ if not flows:
943
+ return {"label": "static", "flow_magnitude": 0.0}
944
+ magnitude = sum(f["magnitude"] for f in flows) / len(flows)
945
+ agreement = sum(f["agreement"] for f in flows) / len(flows)
946
+ if magnitude < MOTION_STATIC_PX:
947
+ label = "static"
948
+ elif agreement >= MOTION_PAN_SPREAD:
949
+ label = "pan"
950
+ else:
951
+ label = "motion"
952
+ return {"label": label, "flow_magnitude": round(magnitude, 3)}