ffmpeg-skill 0.1.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +27 -15
- package/SKILL.md +118 -29
- package/package.json +2 -2
- package/scripts/__pycache__/_common.cpython-311.pyc +0 -0
- package/scripts/__pycache__/audio.cpython-311.pyc +0 -0
- package/scripts/__pycache__/caption.cpython-311.pyc +0 -0
- package/scripts/__pycache__/color.cpython-311.pyc +0 -0
- package/scripts/__pycache__/cut.cpython-311.pyc +0 -0
- package/scripts/__pycache__/export.cpython-311.pyc +0 -0
- package/scripts/__pycache__/fit.cpython-311.pyc +0 -0
- package/scripts/__pycache__/join.cpython-311.pyc +0 -0
- package/scripts/__pycache__/look.cpython-311.pyc +0 -0
- package/scripts/__pycache__/loudness.cpython-311.pyc +0 -0
- package/scripts/__pycache__/overlay.cpython-311.pyc +0 -0
- package/scripts/__pycache__/probe.cpython-311.pyc +0 -0
- package/scripts/__pycache__/silence.cpython-311.pyc +0 -0
- package/scripts/__pycache__/sync.cpython-311.pyc +0 -0
- package/scripts/_common.py +64 -2
- package/scripts/audio.py +130 -0
- package/scripts/caption.py +98 -5
- package/scripts/color.py +119 -0
- package/scripts/cut.py +16 -10
- package/scripts/export.py +9 -3
- package/scripts/fit.py +15 -2
- package/scripts/join.py +111 -0
- package/scripts/look.py +101 -0
- package/scripts/loudness.py +4 -2
- package/scripts/overlay.py +5 -3
- package/scripts/silence.py +123 -0
- package/scripts/sync.py +124 -28
package/README.md
CHANGED
|
@@ -17,9 +17,16 @@ npx ffmpeg-skill
|
|
|
17
17
|
- **Probe first, verify last** — the skill forces the agent to read real duration/fps/resolution before editing and to check the result after, so you get "final.mp4: 59.98 s, 1080×1920, 30 fps" instead of guesses.
|
|
18
18
|
- **Lossless when possible** — cuts and joins use stream copy by default; re-encoding only happens when it must (frame-accurate cuts, filters, format changes).
|
|
19
19
|
- **Cut & join** segments with `mm:ss` / `hh:mm:ss.ms` times.
|
|
20
|
-
- **
|
|
21
|
-
- **
|
|
22
|
-
- **
|
|
20
|
+
- **Silence removal / jump cuts** — detect dead air, keep a margin around speech, render frame-accurate in one pass; export the cut list for hand editing.
|
|
21
|
+
- **Join with transitions** — crossfade, wipes, fade-to-black between mismatched clips (any size, fps, audio layout).
|
|
22
|
+
- **Agent eyes** — contact sheets, single frames and before/after comparisons as PNG so the agent verifies caption placement, crops and colour visually.
|
|
23
|
+
- **Plan before render** — every script has `--dry-run` (print the ffmpeg commands) and `--json` (structured result with a probe of the output).
|
|
24
|
+
- **Captions** — burn SRT/ASS with font, size, colour, outline and position control; generate SRT from a plain timed-text file; animated (fade/pop/slide) and word-by-word karaoke highlight styles for short-form video.
|
|
25
|
+
- **Fit** to an exact duration (pitch-preserving speed change or trim) and to 16:9 / 9:16 / 1:1 / 4:5 by padding or cropping; motion-interpolated or blended slow motion.
|
|
26
|
+
- **Real-world footage handling** — variable-frame-rate phone clips are conformed to constant fps automatically, rotation metadata is honoured, 10-bit HEVC and 5.1 sources are handled.
|
|
27
|
+
- **Multicam / external-audio sync** — offset detection by cross-correlation implemented in pure Python (no numpy), 1 ms resolution, plus clock-drift correction for long takes.
|
|
28
|
+
- **Colour management** — real HDR10/HLG → SDR BT.709 tone mapping, 3D LUT (.cube) for Log footage and looks, and metadata-only retagging.
|
|
29
|
+
- **Audio post** — voice clean-up chain (highpass, de-esser, FFT denoise, compressor), background music with sidechain ducking, fades, 5.1 → stereo downmix, track replacement.
|
|
23
30
|
- **Loudness** — two-pass EBU R128 normalisation to −14 LUFS (or any target) with true-peak ceiling.
|
|
24
31
|
- **Overlays** — logos, watermarks and titles with position, time range, opacity and fades.
|
|
25
32
|
- **Export presets** — YouTube, Instagram Reels/Shorts/TikTok, X, ProRes 422 HQ master, H.265, GIF — all tagged BT.709.
|
|
@@ -59,14 +66,14 @@ Once installed, just talk to your agent. Five things you can say to Claude Code:
|
|
|
59
66
|
|
|
60
67
|
1. **"Take `interview.mp4`, keep 0:45–3:10 and 5:00–6:30, and make it exactly 60 seconds for Reels."**
|
|
61
68
|
→ `probe.py` → `cut.py --segments 0:45-3:10,5:00-6:30` → `fit.py --duration 60 --aspect 9:16 --fit crop` → `export.py --preset reels` → `probe.py` to confirm 60.0 s at 1080×1920.
|
|
62
|
-
2. **"Burn these captions in,
|
|
63
|
-
→ `caption.py --text cues.txt --font "Noto Sans CJK JP" --
|
|
64
|
-
3. **"The lav mic recording is out of sync with the camera — fix it and normalise to −14 LUFS."**
|
|
65
|
-
→ `sync.py camera.mp4 lav.wav --replace-audio` → `loudness.py` → report the detected offset and final LUFS.
|
|
69
|
+
2. **"Burn these captions in TikTok style, words popping in with a yellow highlight, in Japanese."**
|
|
70
|
+
→ `caption.py --text cues.txt --font "Noto Sans CJK JP" --animate pop --karaoke --highlight-color FFD200`.
|
|
71
|
+
3. **"The lav mic recording is out of sync with the camera and drifts over the hour — fix it, clean up the hiss and normalise to −14 LUFS."**
|
|
72
|
+
→ `sync.py camera.mp4 lav.wav --fix-drift --replace-audio` → `audio.py --voice` → `loudness.py` → report the detected offset, drift ppm and final LUFS.
|
|
66
73
|
4. **"Put our logo in the top-right corner for the whole video at 80% opacity, and a title card for the first 4 seconds."**
|
|
67
74
|
→ `overlay.py --image logo.png --position top-right --scale 220 --opacity 0.8` → `overlay.py --text "…" --start 0 --end 4 --fade 0.4`.
|
|
68
|
-
5. **"
|
|
69
|
-
→ `export.py --preset
|
|
75
|
+
5. **"This iPhone HDR clip looks washed out on YouTube — fix it and give me a ProRes master too."**
|
|
76
|
+
→ `probe.py` (shows `hdr: true`) → `color.py --to-sdr` → `export.py --preset youtube` and `export.py --preset prores`.
|
|
70
77
|
|
|
71
78
|
The scripts also work on their own:
|
|
72
79
|
|
|
@@ -81,11 +88,16 @@ More examples: [examples/README.md](examples/README.md). To see everything run e
|
|
|
81
88
|
|
|
82
89
|
| Script | What it does |
|
|
83
90
|
|--------|--------------|
|
|
84
|
-
| `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, colour space, audio channels as JSON |
|
|
91
|
+
| `probe.py` | Duration, fps (+ VFR detection), resolution, codecs, bit depth, HDR format, colour space, rotation, audio channels as JSON |
|
|
85
92
|
| `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact |
|
|
86
|
-
| `
|
|
87
|
-
| `
|
|
88
|
-
| `
|
|
93
|
+
| `silence.py` | Detect and remove silences (jump cuts), list or export the cut list |
|
|
94
|
+
| `join.py` | Concatenate clips with xfade transitions, normalising size, fps and audio |
|
|
95
|
+
| `look.py` | Contact sheet, single frames, side-by-side comparison as PNG for visual checks |
|
|
96
|
+
| `caption.py` | Burn SRT/ASS (font, size, colour, outline, position); build SRT from timed plain text; animated + karaoke ASS |
|
|
97
|
+
| `fit.py` | Fit to a duration (speed or trim, smooth slow-mo) and/or aspect ratio (pad or crop), force constant fps |
|
|
98
|
+
| `sync.py` | Detect offset between two recordings by audio cross-correlation (1 ms), correct clock drift; output aligned video/audio |
|
|
99
|
+
| `color.py` | HDR10/HLG → SDR BT.709 tone mapping, 3D LUT application, colour-tag rewriting |
|
|
100
|
+
| `audio.py` | Denoise / voice chain, music bed with auto-ducking, fades, downmix, replace track |
|
|
89
101
|
| `loudness.py` | Two-pass EBU R128 `loudnorm` to −14 LUFS / −1 dBTP (or custom), video stream-copied |
|
|
90
102
|
| `overlay.py` | Composite image/logo or drawtext title with position, time range, opacity, fade |
|
|
91
103
|
| `export.py` | Presets: `youtube`, `youtube4k`, `reels`, `x`, `prores`, `h265`, `gif` |
|
|
@@ -94,7 +106,7 @@ All scripts: Python 3.9+, standard library only, `--help`, non-zero exit + stder
|
|
|
94
106
|
|
|
95
107
|
## Requirements
|
|
96
108
|
|
|
97
|
-
- FFmpeg 5.0+ with `libx264`, `libx265`, `libass` and `
|
|
109
|
+
- FFmpeg 5.0+ with `libx264`, `libx265`, `libass`, `prores_ks` and `libzimg` (for `color.py --to-sdr`); the default builds from Homebrew, apt and gyan.dev include all of them
|
|
98
110
|
- Python 3.9+
|
|
99
111
|
- Node 16+ only for the `npx` installer
|
|
100
112
|
|
|
@@ -102,7 +114,7 @@ All scripts: Python 3.9+, standard library only, `--help`, non-zero exit + stder
|
|
|
102
114
|
|
|
103
115
|
```bash
|
|
104
116
|
bash examples/make_demo.sh # generates footage, runs every script, rebuilds assets/demo.gif
|
|
105
|
-
python3 tests/test_all.py # end-to-end tests (needs ffmpeg)
|
|
117
|
+
python3 tests/test_all.py # end-to-end tests incl. VFR, rotated, 5.1, 10-bit HDR10 and drifting sources (needs ffmpeg)
|
|
106
118
|
node bin/install.js --dir /tmp/skills # try the installer without touching ~/.claude
|
|
107
119
|
```
|
|
108
120
|
|
package/SKILL.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: ffmpeg-skill
|
|
3
|
-
description: Professional video editing with local FFmpeg — cut, caption, fit to duration/aspect, sync multicam audio, normalise loudness, overlay logos
|
|
3
|
+
description: Professional video editing with local FFmpeg — cut, remove silences, join with transitions, caption (animated/karaoke), fit to duration/aspect, sync multicam audio with drift correction, HDR-to-SDR and LUT colour, denoise/duck/mix audio, normalise loudness, overlay logos, export platform presets, and inspect frames to verify the result; Python stdlib scripts, no cloud or API keys.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# ffmpeg-skill
|
|
@@ -23,15 +23,19 @@ path on stdout, and defaults the output name to `<input>_<operation>.<ext>`.
|
|
|
23
23
|
(plain cuts on keyframes, remuxing, audio-only changes), do not re-encode.
|
|
24
24
|
`cut.py` and `loudness.py` stream-copy video by default; only pass
|
|
25
25
|
`--accurate` to `cut.py` when the user needs frame-exact cuts.
|
|
26
|
-
3. **
|
|
27
|
-
|
|
26
|
+
3. **Plan with `--dry-run --json`, then execute.** Every script accepts
|
|
27
|
+
`--dry-run` (prints the ffmpeg commands, runs nothing) and `--json`
|
|
28
|
+
(structured result: output path, probe of the output, commands run). Use
|
|
29
|
+
them to confirm a plan before long encodes and to report exact facts.
|
|
30
|
+
4. **Chain operations in a sensible order.** Colour (HDR→SDR / LUT) → cut →
|
|
31
|
+
fit → caption/overlay → sync → audio → loudness → export. Do the destructive/aspect changes before burning
|
|
28
32
|
text so captions are sized for the final frame. Re-encode as few times as
|
|
29
33
|
possible: if several re-encoding steps are needed, keep intermediates at
|
|
30
34
|
CRF 18 (the default) and only use `export.py` for the last step.
|
|
31
|
-
|
|
35
|
+
5. **Verify the output.** Run `probe.py` on each result and confirm duration,
|
|
32
36
|
resolution, fps and audio match what was requested. Report those numbers to
|
|
33
37
|
the user (e.g. "final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo").
|
|
34
|
-
|
|
38
|
+
6. **Keep the user's originals.** Never overwrite the source file. Write new
|
|
35
39
|
files next to the input or where the user asked.
|
|
36
40
|
|
|
37
41
|
## Request → script
|
|
@@ -51,6 +55,21 @@ path on stdout, and defaults the output name to `<input>_<operation>.<ext>`.
|
|
|
51
55
|
| "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast) |
|
|
52
56
|
| "export for YouTube / Reels / X", "give me a ProRes master", "make it HEVC" | `export.py input.mp4 --preset youtube|reels|x|prores|h265` |
|
|
53
57
|
| "make a GIF preview" | `export.py input.mp4 --preset gif` |
|
|
58
|
+
| "cut out the pauses / dead air", "tighten it up", "jump cuts" | `silence.py input.mp4 [--threshold -40 --min-silence 0.8]` |
|
|
59
|
+
| "stitch these clips together", "add a crossfade between them" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` |
|
|
60
|
+
| "show me what it looks like", "check the captions are readable" | `look.py output.mp4` then view the PNG |
|
|
61
|
+
| "what would you run?", "don't render yet" | any script with `--dry-run` |
|
|
62
|
+
| "the colours look washed out / it's an iPhone HDR video" | `color.py input.mov --to-sdr` (probe shows `hdr: true`) |
|
|
63
|
+
| "apply this LUT", "convert the S-Log / V-Log footage" | `color.py input.mp4 --lut grade.cube [--lut-strength 0.7]` |
|
|
64
|
+
| "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (no re-encode) |
|
|
65
|
+
| "clean up the audio", "remove the hiss / room noise" | `audio.py input.mp4 --voice` (speech) or `--denoise` |
|
|
66
|
+
| "add background music under the talking" | `audio.py input.mp4 --music bed.mp3 --duck --fade-out 3` |
|
|
67
|
+
| "convert the 5.1 to stereo" | `audio.py input.mov --downmix` |
|
|
68
|
+
| "swap in the narration track" | `audio.py input.mp4 --replace narration.wav` |
|
|
69
|
+
| "the audio drifts out of sync over the hour" | `sync.py camera.mp4 recorder.wav --fix-drift --replace-audio` |
|
|
70
|
+
| "smooth slow motion", "half speed but fluid" | `fit.py input.mp4 --duration 2x --smooth interpolate` (slow) or `--smooth blend` |
|
|
71
|
+
| "TikTok-style captions with the words popping / highlighted" | `caption.py input.mp4 --text cues.txt --animate pop --karaoke` |
|
|
72
|
+
| "it's a phone video with variable frame rate" | nothing extra: every re-encoding script conforms VFR to constant fps automatically; `fit.py --fps 30` to pick the rate |
|
|
54
73
|
|
|
55
74
|
## Scripts
|
|
56
75
|
|
|
@@ -79,18 +98,57 @@ fit.py INPUT [--duration T --method speed|trim [--from-center] [--max-speed 4]]
|
|
|
79
98
|
[--fps N] [-o OUT]
|
|
80
99
|
```
|
|
81
100
|
`speed` retimes video and audio together (pitch-preserving `atempo`); it
|
|
82
|
-
refuses factors beyond `--max-speed`.
|
|
83
|
-
|
|
101
|
+
refuses factors beyond `--max-speed`. For slow motion add `--smooth blend`
|
|
102
|
+
(frame blending, fast) or `--smooth interpolate` (motion-compensated
|
|
103
|
+
`minterpolate`, fluid but roughly 10-20x slower than realtime). `trim` keeps
|
|
104
|
+
the head (or the middle with `--from-center`). `--fps` forces a constant frame
|
|
105
|
+
rate; VFR sources are conformed automatically even without it.
|
|
84
106
|
|
|
85
|
-
###
|
|
107
|
+
### silence.py — remove dead air / jump cuts
|
|
108
|
+
```
|
|
109
|
+
silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [-o OUT]
|
|
110
|
+
```
|
|
111
|
+
Runs `silencedetect`, keeps `--margin` seconds of air around speech, drops
|
|
112
|
+
gaps shorter than `--min-silence`, and re-encodes once with `select`/`aselect`
|
|
113
|
+
(frame accurate). `--list` prints silences, kept ranges and seconds removed
|
|
114
|
+
without rendering; `--edl` saves the kept ranges in `cut.py --segments` format
|
|
115
|
+
so the user can edit the list by hand. Quiet rooms need `--threshold -40`
|
|
116
|
+
to `-45`; noisy ones `-30`. Always tell the user how many seconds were removed.
|
|
117
|
+
|
|
118
|
+
### join.py — concatenate with transitions
|
|
119
|
+
```
|
|
120
|
+
join.py CLIP1 CLIP2 [...] [--transition fade|dissolve|wipeleft|slideleft|fadeblack|fadewhite|circleopen|none]
|
|
121
|
+
[--duration 0.5] [--width W --height H] [--fps N] [--fit pad|crop] [-o OUT]
|
|
122
|
+
```
|
|
123
|
+
Normalises every clip to one frame size, fps, `yuv420p` and 48 kHz stereo
|
|
124
|
+
(silent track generated for clips without audio), then chains `xfade` +
|
|
125
|
+
`acrossfade`. Output length = sum of clips − transition × (n−1). Clips must be
|
|
126
|
+
longer than 2 × the transition. Use `--transition none` for a plain cut.
|
|
127
|
+
|
|
128
|
+
### look.py — see the result
|
|
129
|
+
```
|
|
130
|
+
look.py INPUT [--tiles 4x3] [--width 1280] [-o sheet.png] # contact sheet with timecodes
|
|
131
|
+
look.py INPUT --at 2.5 [--at 7] [-o basename] # single frames -> basename_2.500s.png
|
|
132
|
+
look.py BEFORE --compare AFTER --at 4 [-o cmp.png] # side-by-side frame
|
|
133
|
+
```
|
|
134
|
+
Outputs PNG. View it with the Read tool (or any image viewer) and judge the
|
|
135
|
+
frame like an editor would. Use `--compare` to show before/after to the user.
|
|
136
|
+
|
|
137
|
+
### caption.py — subtitles (static, animated, karaoke)
|
|
86
138
|
```
|
|
87
139
|
caption.py INPUT --srt FILE | --ass FILE | --text CUES.txt [--write-srt OUT.srt]
|
|
88
140
|
[--font NAME] [--fonts-dir DIR] [--size N] [--color RRGGBB] [--outline N] [--outline-color RRGGBB]
|
|
89
|
-
[--bold] [--box] [--position bottom|top|center|top-left|...] [--margin N]
|
|
141
|
+
[--bold] [--box] [--position bottom|top|center|top-left|...] [--margin N]
|
|
142
|
+
[--animate none|fade|pop|slide] [--karaoke [--highlight-color RRGGBB]] [--write-ass OUT.ass] [-o OUT]
|
|
90
143
|
caption.py --text CUES.txt --write-srt OUT.srt # generate the SRT only
|
|
91
144
|
```
|
|
92
145
|
Text cue format, one per line: `0:00-0:03 Hello`, `00:00:03.500 --> 00:00:06 Two | lines`.
|
|
93
146
|
Lines without a time run for `--auto-seconds` (3 s) after the previous cue. `|` is a line break.
|
|
147
|
+
`--animate`/`--karaoke` generate a styled ASS (PlayRes = video size) from the
|
|
148
|
+
SRT/text cues: `pop` is the short-form "bouncy" entrance, `--karaoke` fills each
|
|
149
|
+
word from `--color` to `--highlight-color` evenly across the cue (word timing
|
|
150
|
+
is distributed, not transcribed). The ASS is kept next to the output so the
|
|
151
|
+
user can hand-tune timings and re-run with `--ass`.
|
|
94
152
|
|
|
95
153
|
### overlay.py — logo, image, title
|
|
96
154
|
```
|
|
@@ -99,17 +157,47 @@ overlay.py INPUT --image PNG [--scale W | --scale-percent P] | --text "..." [--f
|
|
|
99
157
|
```
|
|
100
158
|
Alpha in PNGs is respected. Fades apply to the overlay only; the video keeps playing.
|
|
101
159
|
|
|
102
|
-
### sync.py — offset detection
|
|
160
|
+
### sync.py — offset detection, alignment, drift correction
|
|
103
161
|
```
|
|
104
|
-
sync.py REFERENCE SECOND [--json] [--max-offset 30] [--analyze-seconds 120]
|
|
162
|
+
sync.py REFERENCE SECOND [--json] [--max-offset 30] [--analyze-seconds 120] [--fix-drift [--drift-window 60]]
|
|
105
163
|
[--replace-audio | --trim-second] [-o OUT]
|
|
106
164
|
```
|
|
107
|
-
Cross-correlates loudness envelopes
|
|
108
|
-
|
|
109
|
-
the
|
|
165
|
+
Cross-correlates loudness envelopes: coarse FFT search (20 ms), then a direct
|
|
166
|
+
1 ms refinement (pure Python, a 2-minute window takes ~1-3 s). Positive offset
|
|
167
|
+
= the second recording started later. `--replace-audio` writes the reference
|
|
168
|
+
video with the second file's audio aligned (video stream copied).
|
|
110
169
|
`--trim-second` writes the second file shifted to the reference timeline.
|
|
111
|
-
|
|
112
|
-
|
|
170
|
+
`--fix-drift` measures the offset again near the end of the overlap, reports
|
|
171
|
+
the clock difference in ppm, and resamples the second file so a 60-minute
|
|
172
|
+
take stays in sync (typical consumer devices drift 20-500 ppm = up to 1.8 s/h).
|
|
173
|
+
Use it whenever the recording is longer than ~10 minutes. Check `confidence`
|
|
174
|
+
(0–1); below ~0.3 the match is doubtful — use a window with a clear event.
|
|
175
|
+
|
|
176
|
+
### color.py — HDR to SDR, LUTs, colour tags
|
|
177
|
+
```
|
|
178
|
+
color.py INPUT --to-sdr [--tonemap hable|mobius|reinhard|bt2390] [--peak 1000] [--desat 0] [-o OUT]
|
|
179
|
+
color.py INPUT --lut grade.cube [--lut-strength 0..1] [-o OUT]
|
|
180
|
+
color.py INPUT --retag bt709|bt2020-pq|bt2020-hlg|bt601 [-o OUT] # metadata only, stream copy
|
|
181
|
+
```
|
|
182
|
+
`--to-sdr` does a real conversion: linearise (zscale, PQ or HLG), tone-map
|
|
183
|
+
(default `hable`, `mobius` keeps more highlight detail, `bt2390` is the
|
|
184
|
+
broadcast standard), then BT.709 gamma + matrix. Refuses when probe says the
|
|
185
|
+
input is not HDR unless `--force`. `--lut` applies a 3D .cube with
|
|
186
|
+
tetrahedral interpolation (Log→709 conversions, creative looks); blend with
|
|
187
|
+
`--lut-strength`. Everything else in the skill assumes SDR BT.709, so run this
|
|
188
|
+
first on HDR or Log sources.
|
|
189
|
+
|
|
190
|
+
### audio.py — clean-up, music, ducking, layout
|
|
191
|
+
```
|
|
192
|
+
audio.py INPUT [--voice | --denoise [--denoise-strength 25]] [--gain dB]
|
|
193
|
+
[--music FILE [--music-volume -14] [--duck [--duck-amount 12]] [--music-loop]]
|
|
194
|
+
[--fade-in S] [--fade-out S] [--stereo | --mono | --downmix] [--replace FILE] [-o OUT]
|
|
195
|
+
```
|
|
196
|
+
`--voice` = highpass 80 Hz → de-esser → FFT denoise → gentle compressor, the
|
|
197
|
+
standard talking-head chain. `--duck` uses a sidechain compressor keyed by the
|
|
198
|
+
speech so music dips under dialogue and swells in pauses. `--downmix` uses the
|
|
199
|
+
ITU centre/LFE weights for 5.1/7.1 → stereo. Video is always stream-copied.
|
|
200
|
+
Run `loudness.py` after this for final levels.
|
|
113
201
|
|
|
114
202
|
### loudness.py — EBU R128 normalisation
|
|
115
203
|
```
|
|
@@ -131,21 +219,22 @@ trims to platform maximums (Reels 90 s, X 140 s) unless `--allow-long`.
|
|
|
131
219
|
|
|
132
220
|
- **Variable frame rate (phone/screen recordings).** `probe.py` sets
|
|
133
221
|
`variable_frame_rate_suspected` when `r_frame_rate` and `avg_frame_rate`
|
|
134
|
-
disagree.
|
|
135
|
-
`
|
|
136
|
-
|
|
222
|
+
disagree. Every re-encoding script then adds `-fps_mode cfr` at the source's
|
|
223
|
+
average rate, and `cut.py` switches itself to `--accurate` (copy-cuts on VFR
|
|
224
|
+
are unreliable). Pick the rate explicitly with `fit.py --fps 30|60` when the
|
|
225
|
+
average is odd (e.g. 23.4 fps from dropped frames).
|
|
137
226
|
- **Audio drift / sync.** Don't mix files with different frame rates or sample
|
|
138
227
|
rates in one `cut.py --segments` join without re-encoding (`--accurate`).
|
|
139
|
-
After `sync.py`, verify by running it again on the output:
|
|
140
|
-
be ~0.
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
BT.
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
228
|
+
After `sync.py`, verify by running it again on the output: offset (and drift
|
|
229
|
+
ppm with `--fix-drift`) should be ~0. Recordings longer than ~10 minutes from
|
|
230
|
+
separate devices: always use `--fix-drift`.
|
|
231
|
+
- **Colour.** All H.264/H.265 outputs are tagged BT.709 and `yuv420p`. When
|
|
232
|
+
`probe.py` reports `hdr: true` (`hdr_format` HDR10/PQ, HLG or BT.2020), run
|
|
233
|
+
`color.py --to-sdr` **first**; other scripts would tag the HDR picture as
|
|
234
|
+
BT.709 and it would look flat and desaturated (`export.py` warns about this).
|
|
235
|
+
For Log footage (S-Log, V-Log, C-Log: looks grey and low-contrast but is
|
|
236
|
+
tagged SDR) apply the manufacturer's `.cube` with `color.py --lut`. Keep
|
|
237
|
+
ProRes masters at source colour: `export.py --preset prores` does not retag.
|
|
149
238
|
- **CJK and other non-Latin text.** libass and drawtext need a font that has
|
|
150
239
|
the glyphs. Check with `fc-list | grep -i cjk`. Then either name it
|
|
151
240
|
(`caption.py --font "Noto Sans CJK JP"`) or point at the file
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg. No API keys, no cloud, no dependencies.",
|
|
3
|
+
"version": "0.3.0",
|
|
4
|
+
"description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg: cut, silence removal, transitions, captions, sync with drift correction, HDR to SDR, LUTs, audio clean-up and ducking, loudness, platform exports. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": ["ffmpeg", "video", "agent-skill", "claude-code", "cursor", "codex", "skill", "video-editing"],
|
|
6
6
|
"license": "MIT",
|
|
7
7
|
"author": "kajisho5",
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
package/scripts/_common.py
CHANGED
|
@@ -55,10 +55,46 @@ def require_tool(name: str) -> str:
|
|
|
55
55
|
return "" # unreachable
|
|
56
56
|
|
|
57
57
|
|
|
58
|
+
STATE: Dict[str, Any] = {"dry_run": False, "json": False, "commands": []}
|
|
59
|
+
|
|
60
|
+
|
|
61
|
+
def add_common(ap: "argparse.ArgumentParser") -> None:
|
|
62
|
+
"""Add the flags every script shares."""
|
|
63
|
+
g = ap.add_argument_group("agent options")
|
|
64
|
+
g.add_argument("--dry-run", action="store_true", help="print the ffmpeg commands that would run, run nothing")
|
|
65
|
+
g.add_argument("--json", action="store_true", help="print a JSON result (output, probe, commands) on stdout instead of the path")
|
|
66
|
+
|
|
67
|
+
|
|
68
|
+
def apply_common(args: "argparse.Namespace") -> None:
|
|
69
|
+
STATE["dry_run"] = bool(getattr(args, "dry_run", False))
|
|
70
|
+
STATE["json"] = bool(getattr(args, "json", False))
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
def emit(output: Optional[str], **extra: Any) -> None:
|
|
74
|
+
"""Final stdout line: the output path, or a JSON document with --json."""
|
|
75
|
+
if STATE["json"]:
|
|
76
|
+
doc: Dict[str, Any] = {"output": output, "dry_run": STATE["dry_run"], "commands": list(STATE["commands"])}
|
|
77
|
+
if output and not STATE["dry_run"] and os.path.exists(output):
|
|
78
|
+
doc["probe"] = probe(output)
|
|
79
|
+
doc.update(extra)
|
|
80
|
+
print_json(doc)
|
|
81
|
+
elif output:
|
|
82
|
+
print(output)
|
|
83
|
+
|
|
84
|
+
|
|
58
85
|
def run(cmd: Sequence[str], *, quiet: bool = False, check: bool = True) -> subprocess.CompletedProcess:
|
|
59
|
-
"""Run a command, echoing it to stderr unless quiet. Exits on failure when check=True.
|
|
86
|
+
"""Run a command, echoing it to stderr unless quiet. Exits on failure when check=True.
|
|
87
|
+
|
|
88
|
+
With --dry-run, ffmpeg invocations are printed and skipped (ffprobe still runs so
|
|
89
|
+
scripts can plan); a fake successful CompletedProcess is returned.
|
|
90
|
+
"""
|
|
91
|
+
is_ffmpeg = os.path.basename(cmd[0]).startswith("ffmpeg")
|
|
92
|
+
if is_ffmpeg:
|
|
93
|
+
STATE["commands"].append(" ".join(shell_quote(c) for c in cmd))
|
|
60
94
|
if not quiet:
|
|
61
|
-
info("$ " + " ".join(shell_quote(c) for c in cmd))
|
|
95
|
+
info(("[dry-run] $ " if STATE["dry_run"] and is_ffmpeg else "$ ") + " ".join(shell_quote(c) for c in cmd))
|
|
96
|
+
if STATE["dry_run"] and is_ffmpeg:
|
|
97
|
+
return subprocess.CompletedProcess(list(cmd), 0, "", "")
|
|
62
98
|
proc = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
|
|
63
99
|
if check and proc.returncode != 0:
|
|
64
100
|
tail = "\n".join(proc.stderr.strip().splitlines()[-15:])
|
|
@@ -81,6 +117,11 @@ def ffmpeg_base(overwrite: bool = True) -> List[str]:
|
|
|
81
117
|
def probe(path: str) -> Dict[str, Any]:
|
|
82
118
|
"""Return a compact, script-friendly description of a media file."""
|
|
83
119
|
if not os.path.exists(path):
|
|
120
|
+
if STATE["dry_run"]:
|
|
121
|
+
return {"file": path, "dry_run": True, "format": None, "duration": 0.0, "size_bytes": 0, "bitrate": None,
|
|
122
|
+
"video": {"codec": None, "width": 0, "height": 0, "fps": None, "pix_fmt": None, "hdr": False,
|
|
123
|
+
"color_transfer": None, "color_primaries": None, "rotation": 0, "variable_frame_rate_suspected": False},
|
|
124
|
+
"audio": {"codec": None, "channels": 0, "sample_rate": 0}, "subtitle_streams": 0}
|
|
84
125
|
die(f"input not found: {path}")
|
|
85
126
|
ffprobe = require_tool("ffprobe")
|
|
86
127
|
proc = run(
|
|
@@ -128,6 +169,10 @@ def probe(path: str) -> Dict[str, Any]:
|
|
|
128
169
|
rotation = int(video["tags"]["rotate"])
|
|
129
170
|
except ValueError:
|
|
130
171
|
pass
|
|
172
|
+
pix = video.get("pix_fmt") or ""
|
|
173
|
+
trc = video.get("color_transfer") or ""
|
|
174
|
+
prim = video.get("color_primaries") or ""
|
|
175
|
+
hdr = trc in ("smpte2084", "arib-std-b67") or prim == "bt2020"
|
|
131
176
|
out["video"] = {
|
|
132
177
|
"codec": video.get("codec_name"),
|
|
133
178
|
"profile": video.get("profile"),
|
|
@@ -139,6 +184,9 @@ def probe(path: str) -> Dict[str, Any]:
|
|
|
139
184
|
"avg_frame_rate": video.get("avg_frame_rate"),
|
|
140
185
|
"variable_frame_rate_suspected": vfr,
|
|
141
186
|
"pix_fmt": video.get("pix_fmt"),
|
|
187
|
+
"bit_depth": 10 if "10" in pix else (12 if "12" in pix else 8),
|
|
188
|
+
"hdr": hdr,
|
|
189
|
+
"hdr_format": ("HDR10/PQ" if trc == "smpte2084" else "HLG" if trc == "arib-std-b67" else "BT.2020 SDR" if hdr else None),
|
|
142
190
|
"color_space": video.get("color_space"),
|
|
143
191
|
"color_primaries": video.get("color_primaries"),
|
|
144
192
|
"color_transfer": video.get("color_transfer"),
|
|
@@ -208,6 +256,20 @@ def escape_drawtext(text: str) -> str:
|
|
|
208
256
|
)
|
|
209
257
|
|
|
210
258
|
|
|
259
|
+
def cfr_args(meta: Optional[Dict[str, Any]], fps: Optional[float] = None) -> List[str]:
|
|
260
|
+
"""Force a constant frame rate on output when the source looks VFR (or fps is given).
|
|
261
|
+
|
|
262
|
+
VFR sources (phone/screen recordings) drift against audio after cuts and joins,
|
|
263
|
+
so every re-encoding script passes this to conform them automatically.
|
|
264
|
+
"""
|
|
265
|
+
v = (meta or {}).get("video") or {}
|
|
266
|
+
if fps is None and not v.get("variable_frame_rate_suspected"):
|
|
267
|
+
return []
|
|
268
|
+
rate = fps or v.get("fps") or 30.0
|
|
269
|
+
rate = round(rate) if abs(rate - round(rate)) < 0.02 else rate
|
|
270
|
+
return ["-fps_mode", "cfr", "-r", f"{rate:g}"]
|
|
271
|
+
|
|
272
|
+
|
|
211
273
|
def x264_args(crf: int = 18, preset: str = "medium", keep_bt709: bool = True) -> List[str]:
|
|
212
274
|
args = ["-c:v", "libx264", "-preset", preset, "-crf", str(crf), "-pix_fmt", "yuv420p", "-movflags", "+faststart"]
|
|
213
275
|
if keep_bt709:
|
package/scripts/audio.py
ADDED
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Audio post: denoise, voice clean-up, background music with auto-ducking,
|
|
3
|
+
fades and stereo/mono handling. Video is stream-copied.
|
|
4
|
+
|
|
5
|
+
Examples:
|
|
6
|
+
python3 audio.py interview.mp4 --denoise # FFT noise reduction
|
|
7
|
+
python3 audio.py interview.mp4 --voice # highpass + de-esser + compressor + denoise
|
|
8
|
+
python3 audio.py talk.mp4 --music bed.mp3 --duck # music under speech, auto-ducked
|
|
9
|
+
python3 audio.py talk.mp4 --music bed.mp3 --music-volume -18 --fade-out 3
|
|
10
|
+
python3 audio.py clip.mp4 --fade-in 0.5 --fade-out 1 --stereo
|
|
11
|
+
python3 audio.py surround.mov --downmix # 5.1 -> stereo with proper centre/LFE weights
|
|
12
|
+
python3 audio.py clip.mp4 --replace narration.wav # swap the audio track entirely
|
|
13
|
+
"""
|
|
14
|
+
import argparse
|
|
15
|
+
import sys
|
|
16
|
+
from typing import List
|
|
17
|
+
|
|
18
|
+
from _common import add_common, apply_common, emit, audio_codec_for, default_output, die, ffmpeg_base, info, probe, run
|
|
19
|
+
|
|
20
|
+
VOICE_CHAIN = "highpass=f=80,deesser=i=0.4,afftdn=nf=-25:tn=1,acompressor=threshold=-18dB:ratio=3:attack=5:release=80:makeup=2"
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
def main() -> int:
|
|
24
|
+
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
|
25
|
+
ap.add_argument("input")
|
|
26
|
+
ap.add_argument("-o", "--output", help="output file (default: <name>_audio.<ext>)")
|
|
27
|
+
clean = ap.add_argument_group("clean-up")
|
|
28
|
+
clean.add_argument("--denoise", action="store_true", help="FFT noise reduction (afftdn, adaptive)")
|
|
29
|
+
clean.add_argument("--denoise-strength", type=float, default=25.0, help="noise floor in dB to remove, 10..60 (default 25)")
|
|
30
|
+
clean.add_argument("--voice", action="store_true", help="speech preset: highpass 80 Hz, de-esser, denoise, gentle compression")
|
|
31
|
+
clean.add_argument("--gain", type=float, help="gain in dB applied to the main track")
|
|
32
|
+
music = ap.add_argument_group("music")
|
|
33
|
+
music.add_argument("--music", help="music file to mix underneath")
|
|
34
|
+
music.add_argument("--music-volume", type=float, default=-14.0, help="music level in dB relative to full scale (default -14)")
|
|
35
|
+
music.add_argument("--duck", action="store_true", help="auto-duck the music when the main track has speech (sidechain compressor)")
|
|
36
|
+
music.add_argument("--duck-amount", type=float, default=12.0, help="how many dB to duck (default 12)")
|
|
37
|
+
music.add_argument("--music-loop", action="store_true", help="loop the music if shorter than the video")
|
|
38
|
+
fades = ap.add_argument_group("fades / layout")
|
|
39
|
+
fades.add_argument("--fade-in", type=float, default=0.0, help="seconds")
|
|
40
|
+
fades.add_argument("--fade-out", type=float, default=0.0, help="seconds")
|
|
41
|
+
fades.add_argument("--stereo", action="store_true", help="force 2-channel output (mono is duplicated to both sides)")
|
|
42
|
+
fades.add_argument("--mono", action="store_true", help="force 1-channel output")
|
|
43
|
+
fades.add_argument("--downmix", action="store_true", help="downmix 5.1/7.1 to stereo using standard weights")
|
|
44
|
+
fades.add_argument("--replace", help="replace the audio with this file (trimmed/padded to the video)")
|
|
45
|
+
ap.add_argument("--bitrate", default="192k")
|
|
46
|
+
add_common(ap)
|
|
47
|
+
args = ap.parse_args()
|
|
48
|
+
apply_common(args)
|
|
49
|
+
|
|
50
|
+
meta = probe(args.input)
|
|
51
|
+
dur = meta.get("duration") or 0.0
|
|
52
|
+
has_video = bool(meta.get("video"))
|
|
53
|
+
if not meta.get("audio") and not args.replace:
|
|
54
|
+
die("input has no audio stream (use --replace to add one)")
|
|
55
|
+
output = args.output or default_output(args.input, "audio")
|
|
56
|
+
|
|
57
|
+
inputs: List[str] = ["-i", args.input]
|
|
58
|
+
main_src = "0:a:0"
|
|
59
|
+
idx = 1
|
|
60
|
+
if args.replace:
|
|
61
|
+
probe(args.replace)
|
|
62
|
+
inputs += ["-i", args.replace]
|
|
63
|
+
main_src = f"{idx}:a:0"
|
|
64
|
+
idx += 1
|
|
65
|
+
|
|
66
|
+
fx: List[str] = []
|
|
67
|
+
if args.downmix:
|
|
68
|
+
fx.append("pan=stereo|FL=0.707*FC+FL+0.5*BL+0.5*SL+0.5*LFE|FR=0.707*FC+FR+0.5*BR+0.5*SR+0.5*LFE")
|
|
69
|
+
if args.voice:
|
|
70
|
+
fx.append(VOICE_CHAIN)
|
|
71
|
+
elif args.denoise:
|
|
72
|
+
fx.append(f"afftdn=nf=-{args.denoise_strength:g}:tn=1")
|
|
73
|
+
if args.gain:
|
|
74
|
+
fx.append(f"volume={args.gain:g}dB")
|
|
75
|
+
if args.mono:
|
|
76
|
+
fx.append("pan=mono|c0=0.5*c0+0.5*c1")
|
|
77
|
+
elif args.stereo:
|
|
78
|
+
fx.append("aformat=channel_layouts=stereo")
|
|
79
|
+
|
|
80
|
+
graph: List[str] = []
|
|
81
|
+
graph.append(f"[{main_src}]{','.join(fx) if fx else 'anull'}[main]")
|
|
82
|
+
last = "main"
|
|
83
|
+
|
|
84
|
+
if args.music:
|
|
85
|
+
probe(args.music)
|
|
86
|
+
if args.music_loop:
|
|
87
|
+
inputs += ["-stream_loop", "-1", "-i", args.music]
|
|
88
|
+
else:
|
|
89
|
+
inputs += ["-i", args.music]
|
|
90
|
+
m = f"{idx}:a:0"
|
|
91
|
+
idx += 1
|
|
92
|
+
mfx = [f"volume={args.music_volume:g}dB", f"atrim=0:{dur:.3f}" if dur else "anull"]
|
|
93
|
+
if args.fade_out:
|
|
94
|
+
mfx.append(f"afade=t=out:st={max(0.0, dur - args.fade_out):.3f}:d={args.fade_out:g}")
|
|
95
|
+
graph.append(f"[{m}]{','.join(mfx)}[music]")
|
|
96
|
+
if args.duck:
|
|
97
|
+
graph.append("[main]asplit=2[mainA][sc]")
|
|
98
|
+
graph.append(
|
|
99
|
+
f"[music][sc]sidechaincompress=threshold=0.05:ratio={max(2.0, args.duck_amount / 3):.1f}:attack=20:release=400:makeup=1[ducked]"
|
|
100
|
+
)
|
|
101
|
+
graph.append("[mainA][ducked]amix=inputs=2:duration=first:dropout_transition=2:normalize=0[mix]")
|
|
102
|
+
else:
|
|
103
|
+
graph.append("[main][music]amix=inputs=2:duration=first:dropout_transition=2:normalize=0[mix]")
|
|
104
|
+
last = "mix"
|
|
105
|
+
|
|
106
|
+
post: List[str] = []
|
|
107
|
+
if args.fade_in:
|
|
108
|
+
post.append(f"afade=t=in:st=0:d={args.fade_in:g}")
|
|
109
|
+
if args.fade_out and dur:
|
|
110
|
+
post.append(f"afade=t=out:st={max(0.0, dur - args.fade_out):.3f}:d={args.fade_out:g}")
|
|
111
|
+
if args.replace and dur:
|
|
112
|
+
post.append(f"apad,atrim=0:{dur:.3f}")
|
|
113
|
+
if post:
|
|
114
|
+
graph.append(f"[{last}]{','.join(post)}[out]")
|
|
115
|
+
last = "out"
|
|
116
|
+
|
|
117
|
+
cmd = ffmpeg_base() + inputs + ["-filter_complex", ";".join(graph), "-map", f"[{last}]"]
|
|
118
|
+
if has_video:
|
|
119
|
+
cmd += ["-map", "0:v:0", "-c:v", "copy"]
|
|
120
|
+
cmd += audio_codec_for(output, args.bitrate) + ["-shortest", output]
|
|
121
|
+
run(cmd)
|
|
122
|
+
r = probe(output)
|
|
123
|
+
a = r["audio"]
|
|
124
|
+
info(f"wrote {output} ({r['duration']:.3f}s, audio {a['codec']} {a['channels']}ch {a['sample_rate']}Hz)")
|
|
125
|
+
emit(output)
|
|
126
|
+
return 0
|
|
127
|
+
|
|
128
|
+
|
|
129
|
+
if __name__ == "__main__":
|
|
130
|
+
sys.exit(main())
|