ffmpeg-skill 0.1.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -17,9 +17,16 @@ npx ffmpeg-skill
17
17
  - **Probe first, verify last** — the skill forces the agent to read real duration/fps/resolution before editing and to check the result after, so you get "final.mp4: 59.98 s, 1080×1920, 30 fps" instead of guesses.
18
18
  - **Lossless when possible** — cuts and joins use stream copy by default; re-encoding only happens when it must (frame-accurate cuts, filters, format changes).
19
19
  - **Cut & join** segments with `mm:ss` / `hh:mm:ss.ms` times.
20
- - **Captions** — burn SRT/ASS with font, size, colour, outline and position control; generate SRT from a plain timed-text file.
21
- - **Fit** to an exact duration (pitch-preserving speed change or trim) and to 16:9 / 9:16 / 1:1 / 4:5 by padding or cropping.
22
- - **Multicam / external-audio sync** offset detection by cross-correlation implemented in pure Python (no numpy).
20
+ - **Silence removal / jump cuts** — detect dead air, keep a margin around speech, render frame-accurate in one pass; export the cut list for hand editing.
21
+ - **Join with transitions** crossfade, wipes, fade-to-black between mismatched clips (any size, fps, audio layout).
22
+ - **Agent eyes** — contact sheets, single frames and before/after comparisons as PNG so the agent verifies caption placement, crops and colour visually.
23
+ - **Plan before render** — every script has `--dry-run` (print the ffmpeg commands) and `--json` (structured result with a probe of the output).
24
+ - **Captions** — burn SRT/ASS with font, size, colour, outline and position control; generate SRT from a plain timed-text file; animated (fade/pop/slide) and word-by-word karaoke highlight styles for short-form video.
25
+ - **Fit** to an exact duration (pitch-preserving speed change or trim) and to 16:9 / 9:16 / 1:1 / 4:5 by padding or cropping; motion-interpolated or blended slow motion.
26
+ - **Real-world footage handling** — variable-frame-rate phone clips are conformed to constant fps automatically, rotation metadata is honoured, 10-bit HEVC and 5.1 sources are handled.
27
+ - **Multicam / external-audio sync** — offset detection by cross-correlation implemented in pure Python (no numpy), 1 ms resolution, plus clock-drift correction for long takes.
28
+ - **Colour management** — real HDR10/HLG → SDR BT.709 tone mapping, 3D LUT (.cube) for Log footage and looks, and metadata-only retagging.
29
+ - **Audio post** — voice clean-up chain (highpass, de-esser, FFT denoise, compressor), background music with sidechain ducking, fades, 5.1 → stereo downmix, track replacement.
23
30
  - **Loudness** — two-pass EBU R128 normalisation to −14 LUFS (or any target) with true-peak ceiling.
24
31
  - **Overlays** — logos, watermarks and titles with position, time range, opacity and fades.
25
32
  - **Export presets** — YouTube, Instagram Reels/Shorts/TikTok, X, ProRes 422 HQ master, H.265, GIF — all tagged BT.709.
@@ -59,14 +66,14 @@ Once installed, just talk to your agent. Five things you can say to Claude Code:
59
66
 
60
67
  1. **"Take `interview.mp4`, keep 0:45–3:10 and 5:00–6:30, and make it exactly 60 seconds for Reels."**
61
68
  → `probe.py` → `cut.py --segments 0:45-3:10,5:00-6:30` → `fit.py --duration 60 --aspect 9:16 --fit crop` → `export.py --preset reels` → `probe.py` to confirm 60.0 s at 1080×1920.
62
- 2. **"Burn these captions in, white text with a black outline at the top, in Japanese."**
63
- → `caption.py --text cues.txt --font "Noto Sans CJK JP" --position top --outline 3`.
64
- 3. **"The lav mic recording is out of sync with the camera — fix it and normalise to −14 LUFS."**
65
- → `sync.py camera.mp4 lav.wav --replace-audio` → `loudness.py` → report the detected offset and final LUFS.
69
+ 2. **"Burn these captions in TikTok style, words popping in with a yellow highlight, in Japanese."**
70
+ → `caption.py --text cues.txt --font "Noto Sans CJK JP" --animate pop --karaoke --highlight-color FFD200`.
71
+ 3. **"The lav mic recording is out of sync with the camera and drifts over the hour — fix it, clean up the hiss and normalise to −14 LUFS."**
72
+ → `sync.py camera.mp4 lav.wav --fix-drift --replace-audio` → `audio.py --voice` → `loudness.py` → report the detected offset, drift ppm and final LUFS.
66
73
  4. **"Put our logo in the top-right corner for the whole video at 80% opacity, and a title card for the first 4 seconds."**
67
74
  → `overlay.py --image logo.png --position top-right --scale 220 --opacity 0.8` → `overlay.py --text "…" --start 0 --end 4 --fade 0.4`.
68
- 5. **"Give me a ProRes master and an H.265 copy of the final cut."**
69
- → `export.py --preset prores` and `export.py --preset h265`.
75
+ 5. **"This iPhone HDR clip looks washed out on YouTube — fix it and give me a ProRes master too."**
76
+ → `probe.py` (shows `hdr: true`) → `color.py --to-sdr` → `export.py --preset youtube` and `export.py --preset prores`.
70
77
 
71
78
  The scripts also work on their own:
72
79
 
@@ -81,11 +88,16 @@ More examples: [examples/README.md](examples/README.md). To see everything run e
81
88
 
82
89
  | Script | What it does |
83
90
  |--------|--------------|
84
- | `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, colour space, audio channels as JSON |
91
+ | `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, bit depth, HDR format, colour space, rotation, audio channels as JSON |
85
92
  | `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact |
86
- | `caption.py` | Burn SRT/ASS (font, size, colour, outline, position); build SRT from timed plain text |
87
- | `fit.py` | Fit to a duration (speed or trim) and/or aspect ratio (pad or crop), force constant fps |
88
- | `sync.py` | Detect offset between two recordings by audio cross-correlation; output aligned video/audio |
93
+ | `silence.py` | Detect and remove silences (jump cuts), list or export the cut list |
94
+ | `join.py` | Concatenate clips with xfade transitions, normalising size, fps and audio |
95
+ | `look.py` | Contact sheet, single frames, side-by-side comparison as PNG for visual checks |
96
+ | `caption.py` | Burn SRT/ASS (font, size, colour, outline, position); build SRT from timed plain text; animated + karaoke ASS |
97
+ | `fit.py` | Fit to a duration (speed or trim, smooth slow-mo) and/or aspect ratio (pad or crop), force constant fps |
98
+ | `sync.py` | Detect offset between two recordings by audio cross-correlation (1 ms), correct clock drift; output aligned video/audio |
99
+ | `color.py` | HDR10/HLG → SDR BT.709 tone mapping, 3D LUT application, colour-tag rewriting |
100
+ | `audio.py` | Denoise / voice chain, music bed with auto-ducking, fades, downmix, replace track |
89
101
  | `loudness.py` | Two-pass EBU R128 `loudnorm` to −14 LUFS / −1 dBTP (or custom), video stream-copied |
90
102
  | `overlay.py` | Composite image/logo or drawtext title with position, time range, opacity, fade |
91
103
  | `export.py` | Presets: `youtube`, `youtube4k`, `reels`, `x`, `prores`, `h265`, `gif` |
@@ -94,7 +106,7 @@ All scripts: Python 3.9+, standard library only, `--help`, non-zero exit + stder
94
106
 
95
107
  ## Requirements
96
108
 
97
- - FFmpeg 5.0+ with `libx264`, `libx265`, `libass` and `prores_ks` (the default builds from Homebrew, apt and gyan.dev include all of them)
109
+ - FFmpeg 5.0+ with `libx264`, `libx265`, `libass`, `prores_ks` and `libzimg` (for `color.py --to-sdr`); the default builds from Homebrew, apt and gyan.dev include all of them
98
110
  - Python 3.9+
99
111
  - Node 16+ only for the `npx` installer
100
112
 
@@ -102,7 +114,7 @@ All scripts: Python 3.9+, standard library only, `--help`, non-zero exit + stder
102
114
 
103
115
  ```bash
104
116
  bash examples/make_demo.sh # generates footage, runs every script, rebuilds assets/demo.gif
105
- python3 tests/test_all.py # end-to-end tests (needs ffmpeg)
117
+ python3 tests/test_all.py # end-to-end tests incl. VFR, rotated, 5.1, 10-bit HDR10 and drifting sources (needs ffmpeg)
106
118
  node bin/install.js --dir /tmp/skills # try the installer without touching ~/.claude
107
119
  ```
108
120
 
package/SKILL.md CHANGED
@@ -1,6 +1,6 @@
1
1
  ---
2
2
  name: ffmpeg-skill
3
- description: Professional video editing with local FFmpeg — cut, caption, fit to duration/aspect, sync multicam audio, normalise loudness, overlay logos and export platform presets, all from Python stdlib scripts with no cloud or API keys.
3
+ description: Professional video editing with local FFmpeg — cut, remove silences, join with transitions, caption (animated/karaoke), fit to duration/aspect, sync multicam audio with drift correction, HDR-to-SDR and LUT colour, denoise/duck/mix audio, normalise loudness, overlay logos, export platform presets, and inspect frames to verify the result; Python stdlib scripts, no cloud or API keys.
4
4
  ---
5
5
 
6
6
  # ffmpeg-skill
@@ -23,15 +23,19 @@ path on stdout, and defaults the output name to `<input>_<operation>.<ext>`.
23
23
  (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode.
24
24
  `cut.py` and `loudness.py` stream-copy video by default; only pass
25
25
  `--accurate` to `cut.py` when the user needs frame-exact cuts.
26
- 3. **Chain operations in a sensible order.** Cut fit → caption/overlay →
27
- sync/loudness export. Do the destructive/aspect changes before burning
26
+ 3. **Plan with `--dry-run --json`, then execute.** Every script accepts
27
+ `--dry-run` (prints the ffmpeg commands, runs nothing) and `--json`
28
+ (structured result: output path, probe of the output, commands run). Use
29
+ them to confirm a plan before long encodes and to report exact facts.
30
+ 4. **Chain operations in a sensible order.** Colour (HDR→SDR / LUT) → cut →
31
+ fit → caption/overlay → sync → audio → loudness → export. Do the destructive/aspect changes before burning
28
32
  text so captions are sized for the final frame. Re-encode as few times as
29
33
  possible: if several re-encoding steps are needed, keep intermediates at
30
34
  CRF 18 (the default) and only use `export.py` for the last step.
31
- 4. **Verify the output.** Run `probe.py` on each result and confirm duration,
35
+ 5. **Verify the output.** Run `probe.py` on each result and confirm duration,
32
36
  resolution, fps and audio match what was requested. Report those numbers to
33
37
  the user (e.g. "final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo").
34
- 5. **Keep the user's originals.** Never overwrite the source file. Write new
38
+ 6. **Keep the user's originals.** Never overwrite the source file. Write new
35
39
  files next to the input or where the user asked.
36
40
 
37
41
  ## Request → script
@@ -51,6 +55,21 @@ path on stdout, and defaults the output name to `<input>_<operation>.<ext>`.
51
55
  | "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast) |
52
56
  | "export for YouTube / Reels / X", "give me a ProRes master", "make it HEVC" | `export.py input.mp4 --preset youtube|reels|x|prores|h265` |
53
57
  | "make a GIF preview" | `export.py input.mp4 --preset gif` |
58
+ | "cut out the pauses / dead air", "tighten it up", "jump cuts" | `silence.py input.mp4 [--threshold -40 --min-silence 0.8]` |
59
+ | "stitch these clips together", "add a crossfade between them" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` |
60
+ | "show me what it looks like", "check the captions are readable" | `look.py output.mp4` then view the PNG |
61
+ | "what would you run?", "don't render yet" | any script with `--dry-run` |
62
+ | "the colours look washed out / it's an iPhone HDR video" | `color.py input.mov --to-sdr` (probe shows `hdr: true`) |
63
+ | "apply this LUT", "convert the S-Log / V-Log footage" | `color.py input.mp4 --lut grade.cube [--lut-strength 0.7]` |
64
+ | "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (no re-encode) |
65
+ | "clean up the audio", "remove the hiss / room noise" | `audio.py input.mp4 --voice` (speech) or `--denoise` |
66
+ | "add background music under the talking" | `audio.py input.mp4 --music bed.mp3 --duck --fade-out 3` |
67
+ | "convert the 5.1 to stereo" | `audio.py input.mov --downmix` |
68
+ | "swap in the narration track" | `audio.py input.mp4 --replace narration.wav` |
69
+ | "the audio drifts out of sync over the hour" | `sync.py camera.mp4 recorder.wav --fix-drift --replace-audio` |
70
+ | "smooth slow motion", "half speed but fluid" | `fit.py input.mp4 --duration 2x --smooth interpolate` (slow) or `--smooth blend` |
71
+ | "TikTok-style captions with the words popping / highlighted" | `caption.py input.mp4 --text cues.txt --animate pop --karaoke` |
72
+ | "it's a phone video with variable frame rate" | nothing extra: every re-encoding script conforms VFR to constant fps automatically; `fit.py --fps 30` to pick the rate |
54
73
 
55
74
  ## Scripts
56
75
 
@@ -79,18 +98,57 @@ fit.py INPUT [--duration T --method speed|trim [--from-center] [--max-speed 4]]
79
98
  [--fps N] [-o OUT]
80
99
  ```
81
100
  `speed` retimes video and audio together (pitch-preserving `atempo`); it
82
- refuses factors beyond `--max-speed`. `trim` keeps the head (or the middle with
83
- `--from-center`). `--fps` forces constant frame rate use it on VFR sources.
101
+ refuses factors beyond `--max-speed`. For slow motion add `--smooth blend`
102
+ (frame blending, fast) or `--smooth interpolate` (motion-compensated
103
+ `minterpolate`, fluid but roughly 10-20x slower than realtime). `trim` keeps
104
+ the head (or the middle with `--from-center`). `--fps` forces a constant frame
105
+ rate; VFR sources are conformed automatically even without it.
84
106
 
85
- ### caption.py — subtitles
107
+ ### silence.py — remove dead air / jump cuts
108
+ ```
109
+ silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [-o OUT]
110
+ ```
111
+ Runs `silencedetect`, keeps `--margin` seconds of air around speech, drops
112
+ gaps shorter than `--min-silence`, and re-encodes once with `select`/`aselect`
113
+ (frame accurate). `--list` prints silences, kept ranges and seconds removed
114
+ without rendering; `--edl` saves the kept ranges in `cut.py --segments` format
115
+ so the user can edit the list by hand. Quiet rooms need `--threshold -40`
116
+ to `-45`; noisy ones `-30`. Always tell the user how many seconds were removed.
117
+
118
+ ### join.py — concatenate with transitions
119
+ ```
120
+ join.py CLIP1 CLIP2 [...] [--transition fade|dissolve|wipeleft|slideleft|fadeblack|fadewhite|circleopen|none]
121
+ [--duration 0.5] [--width W --height H] [--fps N] [--fit pad|crop] [-o OUT]
122
+ ```
123
+ Normalises every clip to one frame size, fps, `yuv420p` and 48 kHz stereo
124
+ (silent track generated for clips without audio), then chains `xfade` +
125
+ `acrossfade`. Output length = sum of clips − transition × (n−1). Clips must be
126
+ longer than 2 × the transition. Use `--transition none` for a plain cut.
127
+
128
+ ### look.py — see the result
129
+ ```
130
+ look.py INPUT [--tiles 4x3] [--width 1280] [-o sheet.png] # contact sheet with timecodes
131
+ look.py INPUT --at 2.5 [--at 7] [-o basename] # single frames -> basename_2.500s.png
132
+ look.py BEFORE --compare AFTER --at 4 [-o cmp.png] # side-by-side frame
133
+ ```
134
+ Outputs PNG. View it with the Read tool (or any image viewer) and judge the
135
+ frame like an editor would. Use `--compare` to show before/after to the user.
136
+
137
+ ### caption.py — subtitles (static, animated, karaoke)
86
138
  ```
87
139
  caption.py INPUT --srt FILE | --ass FILE | --text CUES.txt [--write-srt OUT.srt]
88
140
  [--font NAME] [--fonts-dir DIR] [--size N] [--color RRGGBB] [--outline N] [--outline-color RRGGBB]
89
- [--bold] [--box] [--position bottom|top|center|top-left|...] [--margin N] [-o OUT]
141
+ [--bold] [--box] [--position bottom|top|center|top-left|...] [--margin N]
142
+ [--animate none|fade|pop|slide] [--karaoke [--highlight-color RRGGBB]] [--write-ass OUT.ass] [-o OUT]
90
143
  caption.py --text CUES.txt --write-srt OUT.srt # generate the SRT only
91
144
  ```
92
145
  Text cue format, one per line: `0:00-0:03 Hello`, `00:00:03.500 --> 00:00:06 Two | lines`.
93
146
  Lines without a time run for `--auto-seconds` (3 s) after the previous cue. `|` is a line break.
147
+ `--animate`/`--karaoke` generate a styled ASS (PlayRes = video size) from the
148
+ SRT/text cues: `pop` is the short-form "bouncy" entrance, `--karaoke` fills each
149
+ word from `--color` to `--highlight-color` evenly across the cue (word timing
150
+ is distributed, not transcribed). The ASS is kept next to the output so the
151
+ user can hand-tune timings and re-run with `--ass`.
94
152
 
95
153
  ### overlay.py — logo, image, title
96
154
  ```
@@ -99,17 +157,47 @@ overlay.py INPUT --image PNG [--scale W | --scale-percent P] | --text "..." [--f
99
157
  ```
100
158
  Alpha in PNGs is respected. Fades apply to the overlay only; the video keeps playing.
101
159
 
102
- ### sync.py — offset detection and alignment
160
+ ### sync.py — offset detection, alignment, drift correction
103
161
  ```
104
- sync.py REFERENCE SECOND [--json] [--max-offset 30] [--analyze-seconds 120]
162
+ sync.py REFERENCE SECOND [--json] [--max-offset 30] [--analyze-seconds 120] [--fix-drift [--drift-window 60]]
105
163
  [--replace-audio | --trim-second] [-o OUT]
106
164
  ```
107
- Cross-correlates loudness envelopes (pure Python FFT, ~5 ms resolution).
108
- Positive offset = the second recording started later. `--replace-audio` writes
109
- the reference video with the second file's audio aligned (video stream copied).
165
+ Cross-correlates loudness envelopes: coarse FFT search (20 ms), then a direct
166
+ 1 ms refinement (pure Python, a 2-minute window takes ~1-3 s). Positive offset
167
+ = the second recording started later. `--replace-audio` writes the reference
168
+ video with the second file's audio aligned (video stream copied).
110
169
  `--trim-second` writes the second file shifted to the reference timeline.
111
- Check `confidence` (0–1); below ~0.3 the match is doubtful try a longer
112
- `--analyze-seconds` or a window that contains a clear event (clap).
170
+ `--fix-drift` measures the offset again near the end of the overlap, reports
171
+ the clock difference in ppm, and resamples the second file so a 60-minute
172
+ take stays in sync (typical consumer devices drift 20-500 ppm = up to 1.8 s/h).
173
+ Use it whenever the recording is longer than ~10 minutes. Check `confidence`
174
+ (0–1); below ~0.3 the match is doubtful — use a window with a clear event.
175
+
176
+ ### color.py — HDR to SDR, LUTs, colour tags
177
+ ```
178
+ color.py INPUT --to-sdr [--tonemap hable|mobius|reinhard|bt2390] [--peak 1000] [--desat 0] [-o OUT]
179
+ color.py INPUT --lut grade.cube [--lut-strength 0..1] [-o OUT]
180
+ color.py INPUT --retag bt709|bt2020-pq|bt2020-hlg|bt601 [-o OUT] # metadata only, stream copy
181
+ ```
182
+ `--to-sdr` does a real conversion: linearise (zscale, PQ or HLG), tone-map
183
+ (default `hable`, `mobius` keeps more highlight detail, `bt2390` is the
184
+ broadcast standard), then BT.709 gamma + matrix. Refuses when probe says the
185
+ input is not HDR unless `--force`. `--lut` applies a 3D .cube with
186
+ tetrahedral interpolation (Log→709 conversions, creative looks); blend with
187
+ `--lut-strength`. Everything else in the skill assumes SDR BT.709, so run this
188
+ first on HDR or Log sources.
189
+
190
+ ### audio.py — clean-up, music, ducking, layout
191
+ ```
192
+ audio.py INPUT [--voice | --denoise [--denoise-strength 25]] [--gain dB]
193
+ [--music FILE [--music-volume -14] [--duck [--duck-amount 12]] [--music-loop]]
194
+ [--fade-in S] [--fade-out S] [--stereo | --mono | --downmix] [--replace FILE] [-o OUT]
195
+ ```
196
+ `--voice` = highpass 80 Hz → de-esser → FFT denoise → gentle compressor, the
197
+ standard talking-head chain. `--duck` uses a sidechain compressor keyed by the
198
+ speech so music dips under dialogue and swells in pauses. `--downmix` uses the
199
+ ITU centre/LFE weights for 5.1/7.1 → stereo. Video is always stream-copied.
200
+ Run `loudness.py` after this for final levels.
113
201
 
114
202
  ### loudness.py — EBU R128 normalisation
115
203
  ```
@@ -131,21 +219,22 @@ trims to platform maximums (Reels 90 s, X 140 s) unless `--allow-long`.
131
219
 
132
220
  - **Variable frame rate (phone/screen recordings).** `probe.py` sets
133
221
  `variable_frame_rate_suspected` when `r_frame_rate` and `avg_frame_rate`
134
- disagree. VFR causes audio drift and bad concat joins. Fix it early with
135
- `fit.py --fps 30` (or 60) before cutting with `--accurate`, captioning or
136
- syncing; add `--accurate` to `cut.py` because copy-cuts on VFR are unreliable.
222
+ disagree. Every re-encoding script then adds `-fps_mode cfr` at the source's
223
+ average rate, and `cut.py` switches itself to `--accurate` (copy-cuts on VFR
224
+ are unreliable). Pick the rate explicitly with `fit.py --fps 30|60` when the
225
+ average is odd (e.g. 23.4 fps from dropped frames).
137
226
  - **Audio drift / sync.** Don't mix files with different frame rates or sample
138
227
  rates in one `cut.py --segments` join without re-encoding (`--accurate`).
139
- After `sync.py`, verify by running it again on the output: the offset should
140
- be ~0. If dialogue drifts over a long recording, the clocks differ; align the
141
- start with `sync.py`, then fix drift with `fit.py --duration` on the audio-only
142
- track to match the reference length.
143
- - **Colour.** All H.264/H.265 outputs are tagged BT.709 (`-colorspace`,
144
- `-color_primaries`, `-color_trc`) and `yuv420p`. If `probe.py` shows a
145
- BT.2020/HDR source (`color_transfer` smpte2084 or arib-std-b67), tell the
146
- user the output will be tagged as SDR BT.709 without tone mapping, and add
147
- `-vf zscale=t=linear:npl=100,tonemap=hable,zscale=p=bt709:t=bt709:m=bt709`
148
- manually with ffmpeg if they need a proper SDR conversion.
228
+ After `sync.py`, verify by running it again on the output: offset (and drift
229
+ ppm with `--fix-drift`) should be ~0. Recordings longer than ~10 minutes from
230
+ separate devices: always use `--fix-drift`.
231
+ - **Colour.** All H.264/H.265 outputs are tagged BT.709 and `yuv420p`. When
232
+ `probe.py` reports `hdr: true` (`hdr_format` HDR10/PQ, HLG or BT.2020), run
233
+ `color.py --to-sdr` **first**; other scripts would tag the HDR picture as
234
+ BT.709 and it would look flat and desaturated (`export.py` warns about this).
235
+ For Log footage (S-Log, V-Log, C-Log: looks grey and low-contrast but is
236
+ tagged SDR) apply the manufacturer's `.cube` with `color.py --lut`. Keep
237
+ ProRes masters at source colour: `export.py --preset prores` does not retag.
149
238
  - **CJK and other non-Latin text.** libass and drawtext need a font that has
150
239
  the glyphs. Check with `fc-list | grep -i cjk`. Then either name it
151
240
  (`caption.py --font "Noto Sans CJK JP"`) or point at the file
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "0.1.0",
4
- "description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg. No API keys, no cloud, no dependencies.",
3
+ "version": "0.3.0",
4
+ "description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg: cut, silence removal, transitions, captions, sync with drift correction, HDR to SDR, LUTs, audio clean-up and ducking, loudness, platform exports. No API keys, no cloud, no dependencies.",
5
5
  "keywords": ["ffmpeg", "video", "agent-skill", "claude-code", "cursor", "codex", "skill", "video-editing"],
6
6
  "license": "MIT",
7
7
  "author": "kajisho5",
@@ -55,10 +55,46 @@ def require_tool(name: str) -> str:
55
55
  return "" # unreachable
56
56
 
57
57
 
58
+ STATE: Dict[str, Any] = {"dry_run": False, "json": False, "commands": []}
59
+
60
+
61
+ def add_common(ap: "argparse.ArgumentParser") -> None:
62
+ """Add the flags every script shares."""
63
+ g = ap.add_argument_group("agent options")
64
+ g.add_argument("--dry-run", action="store_true", help="print the ffmpeg commands that would run, run nothing")
65
+ g.add_argument("--json", action="store_true", help="print a JSON result (output, probe, commands) on stdout instead of the path")
66
+
67
+
68
+ def apply_common(args: "argparse.Namespace") -> None:
69
+ STATE["dry_run"] = bool(getattr(args, "dry_run", False))
70
+ STATE["json"] = bool(getattr(args, "json", False))
71
+
72
+
73
+ def emit(output: Optional[str], **extra: Any) -> None:
74
+ """Final stdout line: the output path, or a JSON document with --json."""
75
+ if STATE["json"]:
76
+ doc: Dict[str, Any] = {"output": output, "dry_run": STATE["dry_run"], "commands": list(STATE["commands"])}
77
+ if output and not STATE["dry_run"] and os.path.exists(output):
78
+ doc["probe"] = probe(output)
79
+ doc.update(extra)
80
+ print_json(doc)
81
+ elif output:
82
+ print(output)
83
+
84
+
58
85
  def run(cmd: Sequence[str], *, quiet: bool = False, check: bool = True) -> subprocess.CompletedProcess:
59
- """Run a command, echoing it to stderr unless quiet. Exits on failure when check=True."""
86
+ """Run a command, echoing it to stderr unless quiet. Exits on failure when check=True.
87
+
88
+ With --dry-run, ffmpeg invocations are printed and skipped (ffprobe still runs so
89
+ scripts can plan); a fake successful CompletedProcess is returned.
90
+ """
91
+ is_ffmpeg = os.path.basename(cmd[0]).startswith("ffmpeg")
92
+ if is_ffmpeg:
93
+ STATE["commands"].append(" ".join(shell_quote(c) for c in cmd))
60
94
  if not quiet:
61
- info("$ " + " ".join(shell_quote(c) for c in cmd))
95
+ info(("[dry-run] $ " if STATE["dry_run"] and is_ffmpeg else "$ ") + " ".join(shell_quote(c) for c in cmd))
96
+ if STATE["dry_run"] and is_ffmpeg:
97
+ return subprocess.CompletedProcess(list(cmd), 0, "", "")
62
98
  proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
63
99
  if check and proc.returncode != 0:
64
100
  tail = "\n".join(proc.stderr.strip().splitlines()[-15:])
@@ -81,6 +117,11 @@ def ffmpeg_base(overwrite: bool = True) -> List[str]:
81
117
  def probe(path: str) -> Dict[str, Any]:
82
118
  """Return a compact, script-friendly description of a media file."""
83
119
  if not os.path.exists(path):
120
+ if STATE["dry_run"]:
121
+ return {"file": path, "dry_run": True, "format": None, "duration": 0.0, "size_bytes": 0, "bitrate": None,
122
+ "video": {"codec": None, "width": 0, "height": 0, "fps": None, "pix_fmt": None, "hdr": False,
123
+ "color_transfer": None, "color_primaries": None, "rotation": 0, "variable_frame_rate_suspected": False},
124
+ "audio": {"codec": None, "channels": 0, "sample_rate": 0}, "subtitle_streams": 0}
84
125
  die(f"input not found: {path}")
85
126
  ffprobe = require_tool("ffprobe")
86
127
  proc = run(
@@ -128,6 +169,10 @@ def probe(path: str) -> Dict[str, Any]:
128
169
  rotation = int(video["tags"]["rotate"])
129
170
  except ValueError:
130
171
  pass
172
+ pix = video.get("pix_fmt") or ""
173
+ trc = video.get("color_transfer") or ""
174
+ prim = video.get("color_primaries") or ""
175
+ hdr = trc in ("smpte2084", "arib-std-b67") or prim == "bt2020"
131
176
  out["video"] = {
132
177
  "codec": video.get("codec_name"),
133
178
  "profile": video.get("profile"),
@@ -139,6 +184,9 @@ def probe(path: str) -> Dict[str, Any]:
139
184
  "avg_frame_rate": video.get("avg_frame_rate"),
140
185
  "variable_frame_rate_suspected": vfr,
141
186
  "pix_fmt": video.get("pix_fmt"),
187
+ "bit_depth": 10 if "10" in pix else (12 if "12" in pix else 8),
188
+ "hdr": hdr,
189
+ "hdr_format": ("HDR10/PQ" if trc == "smpte2084" else "HLG" if trc == "arib-std-b67" else "BT.2020 SDR" if hdr else None),
142
190
  "color_space": video.get("color_space"),
143
191
  "color_primaries": video.get("color_primaries"),
144
192
  "color_transfer": video.get("color_transfer"),
@@ -208,6 +256,20 @@ def escape_drawtext(text: str) -> str:
208
256
  )
209
257
 
210
258
 
259
+ def cfr_args(meta: Optional[Dict[str, Any]], fps: Optional[float] = None) -> List[str]:
260
+ """Force a constant frame rate on output when the source looks VFR (or fps is given).
261
+
262
+ VFR sources (phone/screen recordings) drift against audio after cuts and joins,
263
+ so every re-encoding script passes this to conform them automatically.
264
+ """
265
+ v = (meta or {}).get("video") or {}
266
+ if fps is None and not v.get("variable_frame_rate_suspected"):
267
+ return []
268
+ rate = fps or v.get("fps") or 30.0
269
+ rate = round(rate) if abs(rate - round(rate)) < 0.02 else rate
270
+ return ["-fps_mode", "cfr", "-r", f"{rate:g}"]
271
+
272
+
211
273
  def x264_args(crf: int = 18, preset: str = "medium", keep_bt709: bool = True) -> List[str]:
212
274
  args = ["-c:v", "libx264", "-preset", preset, "-crf", str(crf), "-pix_fmt", "yuv420p", "-movflags", "+faststart"]
213
275
  if keep_bt709:
@@ -0,0 +1,130 @@
1
+ #!/usr/bin/env python3
2
+ """Audio post: denoise, voice clean-up, background music with auto-ducking,
3
+ fades and stereo/mono handling. Video is stream-copied.
4
+
5
+ Examples:
6
+ python3 audio.py interview.mp4 --denoise # FFT noise reduction
7
+ python3 audio.py interview.mp4 --voice # highpass + de-esser + compressor + denoise
8
+ python3 audio.py talk.mp4 --music bed.mp3 --duck # music under speech, auto-ducked
9
+ python3 audio.py talk.mp4 --music bed.mp3 --music-volume -18 --fade-out 3
10
+ python3 audio.py clip.mp4 --fade-in 0.5 --fade-out 1 --stereo
11
+ python3 audio.py surround.mov --downmix # 5.1 -> stereo with proper centre/LFE weights
12
+ python3 audio.py clip.mp4 --replace narration.wav # swap the audio track entirely
13
+ """
14
+ import argparse
15
+ import sys
16
+ from typing import List
17
+
18
+ from _common import add_common, apply_common, emit, audio_codec_for, default_output, die, ffmpeg_base, info, probe, run
19
+
20
+ VOICE_CHAIN = "highpass=f=80,deesser=i=0.4,afftdn=nf=-25:tn=1,acompressor=threshold=-18dB:ratio=3:attack=5:release=80:makeup=2"
21
+
22
+
23
+ def main() -> int:
24
+ ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
25
+ ap.add_argument("input")
26
+ ap.add_argument("-o", "--output", help="output file (default: <name>_audio.<ext>)")
27
+ clean = ap.add_argument_group("clean-up")
28
+ clean.add_argument("--denoise", action="store_true", help="FFT noise reduction (afftdn, adaptive)")
29
+ clean.add_argument("--denoise-strength", type=float, default=25.0, help="noise floor in dB to remove, 10..60 (default 25)")
30
+ clean.add_argument("--voice", action="store_true", help="speech preset: highpass 80 Hz, de-esser, denoise, gentle compression")
31
+ clean.add_argument("--gain", type=float, help="gain in dB applied to the main track")
32
+ music = ap.add_argument_group("music")
33
+ music.add_argument("--music", help="music file to mix underneath")
34
+ music.add_argument("--music-volume", type=float, default=-14.0, help="music level in dB relative to full scale (default -14)")
35
+ music.add_argument("--duck", action="store_true", help="auto-duck the music when the main track has speech (sidechain compressor)")
36
+ music.add_argument("--duck-amount", type=float, default=12.0, help="how many dB to duck (default 12)")
37
+ music.add_argument("--music-loop", action="store_true", help="loop the music if shorter than the video")
38
+ fades = ap.add_argument_group("fades / layout")
39
+ fades.add_argument("--fade-in", type=float, default=0.0, help="seconds")
40
+ fades.add_argument("--fade-out", type=float, default=0.0, help="seconds")
41
+ fades.add_argument("--stereo", action="store_true", help="force 2-channel output (mono is duplicated to both sides)")
42
+ fades.add_argument("--mono", action="store_true", help="force 1-channel output")
43
+ fades.add_argument("--downmix", action="store_true", help="downmix 5.1/7.1 to stereo using standard weights")
44
+ fades.add_argument("--replace", help="replace the audio with this file (trimmed/padded to the video)")
45
+ ap.add_argument("--bitrate", default="192k")
46
+ add_common(ap)
47
+ args = ap.parse_args()
48
+ apply_common(args)
49
+
50
+ meta = probe(args.input)
51
+ dur = meta.get("duration") or 0.0
52
+ has_video = bool(meta.get("video"))
53
+ if not meta.get("audio") and not args.replace:
54
+ die("input has no audio stream (use --replace to add one)")
55
+ output = args.output or default_output(args.input, "audio")
56
+
57
+ inputs: List[str] = ["-i", args.input]
58
+ main_src = "0:a:0"
59
+ idx = 1
60
+ if args.replace:
61
+ probe(args.replace)
62
+ inputs += ["-i", args.replace]
63
+ main_src = f"{idx}:a:0"
64
+ idx += 1
65
+
66
+ fx: List[str] = []
67
+ if args.downmix:
68
+ fx.append("pan=stereo|FL=0.707*FC+FL+0.5*BL+0.5*SL+0.5*LFE|FR=0.707*FC+FR+0.5*BR+0.5*SR+0.5*LFE")
69
+ if args.voice:
70
+ fx.append(VOICE_CHAIN)
71
+ elif args.denoise:
72
+ fx.append(f"afftdn=nf=-{args.denoise_strength:g}:tn=1")
73
+ if args.gain:
74
+ fx.append(f"volume={args.gain:g}dB")
75
+ if args.mono:
76
+ fx.append("pan=mono|c0=0.5*c0+0.5*c1")
77
+ elif args.stereo:
78
+ fx.append("aformat=channel_layouts=stereo")
79
+
80
+ graph: List[str] = []
81
+ graph.append(f"[{main_src}]{','.join(fx) if fx else 'anull'}[main]")
82
+ last = "main"
83
+
84
+ if args.music:
85
+ probe(args.music)
86
+ if args.music_loop:
87
+ inputs += ["-stream_loop", "-1", "-i", args.music]
88
+ else:
89
+ inputs += ["-i", args.music]
90
+ m = f"{idx}:a:0"
91
+ idx += 1
92
+ mfx = [f"volume={args.music_volume:g}dB", f"atrim=0:{dur:.3f}" if dur else "anull"]
93
+ if args.fade_out:
94
+ mfx.append(f"afade=t=out:st={max(0.0, dur - args.fade_out):.3f}:d={args.fade_out:g}")
95
+ graph.append(f"[{m}]{','.join(mfx)}[music]")
96
+ if args.duck:
97
+ graph.append("[main]asplit=2[mainA][sc]")
98
+ graph.append(
99
+ f"[music][sc]sidechaincompress=threshold=0.05:ratio={max(2.0, args.duck_amount / 3):.1f}:attack=20:release=400:makeup=1[ducked]"
100
+ )
101
+ graph.append("[mainA][ducked]amix=inputs=2:duration=first:dropout_transition=2:normalize=0[mix]")
102
+ else:
103
+ graph.append("[main][music]amix=inputs=2:duration=first:dropout_transition=2:normalize=0[mix]")
104
+ last = "mix"
105
+
106
+ post: List[str] = []
107
+ if args.fade_in:
108
+ post.append(f"afade=t=in:st=0:d={args.fade_in:g}")
109
+ if args.fade_out and dur:
110
+ post.append(f"afade=t=out:st={max(0.0, dur - args.fade_out):.3f}:d={args.fade_out:g}")
111
+ if args.replace and dur:
112
+ post.append(f"apad,atrim=0:{dur:.3f}")
113
+ if post:
114
+ graph.append(f"[{last}]{','.join(post)}[out]")
115
+ last = "out"
116
+
117
+ cmd = ffmpeg_base() + inputs + ["-filter_complex", ";".join(graph), "-map", f"[{last}]"]
118
+ if has_video:
119
+ cmd += ["-map", "0:v:0", "-c:v", "copy"]
120
+ cmd += audio_codec_for(output, args.bitrate) + ["-shortest", output]
121
+ run(cmd)
122
+ r = probe(output)
123
+ a = r["audio"]
124
+ info(f"wrote {output} ({r['duration']:.3f}s, audio {a['codec']} {a['channels']}ch {a['sample_rate']}Hz)")
125
+ emit(output)
126
+ return 0
127
+
128
+
129
+ if __name__ == "__main__":
130
+ sys.exit(main())