ffmpeg-skill 0.9.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (57) hide show
  1. package/README.md +315 -122
  2. package/SKILL.md +115 -18
  3. package/bin/install.js +16 -2
  4. package/mcp/server.py +2 -0
  5. package/package.json +15 -3
  6. package/references/ci-platform-pitfalls.md +111 -0
  7. package/references/process-pitfalls.md +85 -0
  8. package/references/scripts.md +122 -11
  9. package/scripts/_common.py +247 -15
  10. package/scripts/_contract.py +420 -48
  11. package/scripts/audio.py +101 -8
  12. package/scripts/background.py +73 -0
  13. package/scripts/caption.py +97 -18
  14. package/scripts/check.py +21 -7
  15. package/scripts/color.py +104 -13
  16. package/scripts/crop.py +79 -0
  17. package/scripts/cut.py +85 -11
  18. package/scripts/export.py +16 -7
  19. package/scripts/fit.py +76 -12
  20. package/scripts/graphics.py +12 -3
  21. package/scripts/insert.py +128 -0
  22. package/scripts/join.py +88 -8
  23. package/scripts/loudness.py +3 -3
  24. package/scripts/multicam.py +11 -1
  25. package/scripts/overlay.py +64 -5
  26. package/scripts/proxy.py +82 -0
  27. package/scripts/render.py +13 -2
  28. package/scripts/reverse.py +56 -0
  29. package/scripts/scenes.py +15 -3
  30. package/scripts/sequence.py +124 -0
  31. package/scripts/silence.py +2 -2
  32. package/scripts/stabilize.py +83 -0
  33. package/scripts/sync.py +9 -1
  34. package/mcp/__pycache__/server.cpython-311.pyc +0 -0
  35. package/scripts/__pycache__/_common.cpython-311.pyc +0 -0
  36. package/scripts/__pycache__/_contract.cpython-311.pyc +0 -0
  37. package/scripts/__pycache__/audio.cpython-311.pyc +0 -0
  38. package/scripts/__pycache__/batch.cpython-311.pyc +0 -0
  39. package/scripts/__pycache__/caption.cpython-311.pyc +0 -0
  40. package/scripts/__pycache__/check.cpython-311.pyc +0 -0
  41. package/scripts/__pycache__/color.cpython-311.pyc +0 -0
  42. package/scripts/__pycache__/cut.cpython-311.pyc +0 -0
  43. package/scripts/__pycache__/export.cpython-311.pyc +0 -0
  44. package/scripts/__pycache__/fit.cpython-311.pyc +0 -0
  45. package/scripts/__pycache__/graphics.cpython-311.pyc +0 -0
  46. package/scripts/__pycache__/join.cpython-311.pyc +0 -0
  47. package/scripts/__pycache__/look.cpython-311.pyc +0 -0
  48. package/scripts/__pycache__/loudness.cpython-311.pyc +0 -0
  49. package/scripts/__pycache__/multicam.cpython-311.pyc +0 -0
  50. package/scripts/__pycache__/overlay.cpython-311.pyc +0 -0
  51. package/scripts/__pycache__/probe.cpython-311.pyc +0 -0
  52. package/scripts/__pycache__/render.cpython-311.pyc +0 -0
  53. package/scripts/__pycache__/report.cpython-311.pyc +0 -0
  54. package/scripts/__pycache__/scenes.cpython-311.pyc +0 -0
  55. package/scripts/__pycache__/silence.cpython-311.pyc +0 -0
  56. package/scripts/__pycache__/sync.cpython-311.pyc +0 -0
  57. package/scripts/__pycache__/verify.cpython-311.pyc +0 -0
package/SKILL.md CHANGED
@@ -5,10 +5,20 @@ description: Edit video and audio with local FFmpeg from natural-language reques
5
5
 
6
6
  # ffmpeg-skill
7
7
 
8
- Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. Every script has `--help`, and all of them accept `--dry-run` (print the ffmpeg commands, run nothing), `--json` (structured result with a probe of the output), `--fast` (preview quality) and `--progress`. Details for every flag: `references/scripts.md`. Device-specific behaviour (iPhone HDR, GoPro, DJI, screen recordings, Zoom): `references/devices.md`.
8
+ Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. Every script has `--help`, and all of them accept `--dry-run`, `--json` (structured result with a probe of the output), `--fast` (preview quality) and `--progress`. Writing tools run nothing under `--dry-run`; `probe`/`check`/`sync`/`multicam`/`scenes`/`report` may still run ffmpeg/ffprobe to measure or analyse — they just don't write their final artifact; `verify` accepts the flag but ignores it. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`). Details for every flag: `references/scripts.md`. Device-specific behaviour (iPhone HDR, GoPro, DJI, screen recordings, Zoom): `references/devices.md`.
9
9
 
10
10
  ## Workflow (always follow this order)
11
11
 
12
+ 0. **Check the environment once per session, if unfamiliar.** On a machine
13
+ you haven't confirmed capability on this session, run `doctor --json`
14
+ once: check `ok` and the target tool's `usable` before relying on it. If
15
+ `usable` isn't `yes`, don't run that tool — report the missing capability
16
+ instead of discovering it via a runtime failure (a missing `libass`,
17
+ `zscale`, or encoder is the common case, e.g. `caption.py`). Don't re-run
18
+ `doctor` per job — it queries `ffmpeg -filters`/`-encoders`, not free, and
19
+ once per session/unfamiliar machine is enough. `contract --json`'s full
20
+ tool schema is for a *planning* agent deciding which tool/params to use
21
+ from an abstract goal — not part of this per-job workflow.
12
22
  1. **Probe first.** Run `probe.py` on every input before touching it. Read the
13
23
  duration, fps, resolution, codecs, audio channels and the
14
24
  `variable_frame_rate_suspected` flag. Plan the edit from real numbers, never
@@ -18,8 +28,17 @@ Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>
18
28
  `cut.py` and `loudness.py` stream-copy video by default; only pass
19
29
  `--accurate` to `cut.py` when the user needs frame-exact cuts.
20
30
  3. **Plan with `--dry-run --json`, then execute.** Every script accepts
21
- `--dry-run` (prints the ffmpeg commands, runs nothing) and `--json`
22
- (structured result: output path, probe of the output, commands run). Use
31
+ `--dry-run` (prints the ffmpeg commands that would run) and `--json`
32
+ (structured result: output path, probe of the output, commands run). For
33
+ writing tools this means nothing is written; `probe`/`check` still run
34
+ ffprobe/loudness-measurement passes (they're read-only, so `--dry-run`
35
+ changes nothing for `probe`, and only skips the loudness pass for
36
+ `check`), `sync`/`multicam`/`scenes`/`report` still run ffmpeg/ffprobe to
37
+ measure or analyse, and `verify` accepts the flag but ignores it entirely
38
+ (its steps run regardless) — see `contract --json`'s `dry_run` field per
39
+ tool for exact semantics. Trust `--json`, not a dry-run's human-readable
40
+ summary line, for any number after the plan (dimensions in that line can
41
+ be a placeholder, not a computed preview — see `docs/contract.md`). Use
23
42
  them to confirm a plan before long encodes and to report exact facts.
24
43
  `--fast` gives a quick preview-quality render (x264 veryfast), `--progress`
25
44
  prints percent and ETA on stderr for long encodes.
@@ -40,17 +59,36 @@ Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>
40
59
  6. **Verify the output.** Run `probe.py` on each result and confirm duration,
41
60
  resolution, fps and audio match what was requested. Report those numbers to
42
61
  the user (e.g. "final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo").
62
+ A step is done only when the script exited 0 and the output probes as
63
+ expected. Writing the command is not doing the job; a non-zero exit, a
64
+ missing or empty file, or a probe that contradicts the request is a
65
+ failure, and the report says so with the script's error message.
43
66
  7. **Keep the user's originals.** Never overwrite the source file. Write new
44
67
  files next to the input or where the user asked.
45
68
  8. **Look at the picture.** Whenever the picture changed (captions, overlays,
46
69
  graphics, crop/pad, resize, colour, transitions) run `look.py OUTPUT`
47
- (contact sheet) or `look.py OUTPUT --at T`, view the PNG, and judge it like
48
- an editor: text inside the frame and not over faces, logos where asked,
49
- crops keeping the subject, colours not washed out, transitions landing
50
- where intended. The job is not finished until the report's `Look:` line
51
- names that PNG; a probe alone cannot see a caption sitting on someone's
52
- face. Audio-only jobs (sync, loudness, silence, or any job whose input is
53
- an audio file) write `Look: not needed`; there is no picture to inspect.
70
+ (contact sheet) or `look.py OUTPUT --at T`, view the PNG. The job is not
71
+ finished until the report's `Look:` line names that PNG; a probe alone
72
+ cannot see a caption sitting on someone's face. Audio-only jobs (sync,
73
+ loudness, silence, or any job whose input is an audio file) write
74
+ `Look: not needed`; there is no picture to inspect. What to look for
75
+ splits the same way `check.py`'s rows do in step 5:
76
+ - **Mechanical (verify and report as this skill's own job):** the
77
+ specified text/logo is present at the specified position, subtitles/text
78
+ appear at the specified timestamps, resolution has even dimensions.
79
+ Letterboxing/pillarboxing from `fit.py --fit pad` is the *correct*
80
+ result of that mode, not a defect — never flag it.
81
+ - **Judgement (report to the calling agent/user, don't silently pass or
82
+ fail):** whether a subject or face is cut off, whether text sits over a
83
+ face, whether colours look washed out, whether a transition "lands"
84
+ well or the edit feels cinematic. These require deciding what the
85
+ subject *is*, which belongs to the calling agent (see "What this skill
86
+ does and does not decide") — state what you see in one line and let the
87
+ calling agent or user judge it, don't decide it here.
88
+ If the execution environment cannot actually view images (no vision
89
+ capability), write `Look: PATH (pixels not inspected; agent has no image
90
+ view)` — never claim a picture was inspected when it wasn't, and don't
91
+ stall indefinitely waiting for a capability that isn't there.
54
92
 
55
93
 
56
94
  ## Before you run anything: what to ask, what to assume
@@ -62,10 +100,26 @@ Ask one short question only when the answer changes the output materially and th
62
100
  - **Captions** without a text source: use `--transcribe` if a local whisper exists, otherwise ask for the text or a timed file; never invent dialogue.
63
101
  - **Fonts and brand**: if the user mentions a brand, colours or "our font", ask for or create `brand.json` once and reuse it.
64
102
  - **CJK / non-Latin text**: check that a font exists before rendering (`fc-list :lang=ja file` / `:lang=ko` / `:lang=zh`); pass it with `--font "Name"` or `--font-file /path.ttf`. Tofu boxes are a failed job, not a style.
65
- - Anything else (crop position, transition type, caption style): pick the conventional default, say what you picked, and offer the alternative in one line.
103
+ - **Crop position** for `--fit crop`: default to centre, but if the request or the source names an off-centre subject ("keep the product on the right", "don't cut off my hands", a logo/person visibly off-centre in `look.py`'s sheet) use `--crop-x`/`--crop-y` (0=left/top, 1=right/bottom) instead of the silent centre guess. Ask which edge to keep when the sheet shows the subject near an edge and the request doesn't say.
104
+ - Anything else (transition type, caption style): pick the conventional default, say what you picked, and offer the alternative in one line.
66
105
 
67
106
  Do not ask for things `probe.py` can tell you.
68
107
 
108
+ ## What this skill does and does not decide
109
+
110
+ This skill cuts, joins, measures, syncs, exports and checks files — it executes an edit, it does not decide one. Some things that sound like part of the job but belong to the human, the calling agent, or another skill instead:
111
+
112
+ - **Which cut is the right one, or whether a deliverable is approvable for release** — this skill measures and reports (`check.py`'s PASS/WARN/FAIL, `cut.py`'s measured duration error); a production agent or the user decides whether that's good enough to ship.
113
+ - **What makes a highlight interesting** — `scenes.py --highlights` ranks by a measured proxy (audio energy or scene duration, see its own docs), never by understanding the content; treat its output as candidates, not a verdict.
114
+ - **Thumbnail or cover-image composition** — that's a design decision, not a measurement; a thumbnail-generation skill or the user makes it.
115
+ - **Understanding what a video is *about*** — this skill has no transcription or vision beyond `look.py`'s contact sheets, which exist for the calling agent's own eyes, not for this skill to interpret on its own.
116
+ - **Judging what looks good** — "apply this LUT" or "correct exposure by +0.3 stops" (`color.py`) is mechanical, parameter-determined execution and belongs here; "grade this scene to look cinematic" is a subjective judgement about what looks right and belongs in a colour-grading skill ([`color-grading-skill`](https://github.com/kajisho5/color-grading-skill), see README's "Standalone, and in an ecosystem") that decides the parameters and then calls `color.py` to apply them.
117
+ - **Picking a subject or region without being told one** — "crop to this exact box" or "crop to 9:16 keeping x=200,y=0" (`crop.py`/`fit.py --fit crop --crop-x/-y`) is mechanical once the box is known; "crop to keep the speaker in frame" requires deciding *what* the speaker is, which is a vision/composition judgement for the calling agent (from a `look.py` contact sheet) or a motion-graphics skill, not this one.
118
+
119
+ The line in general: if the same input and the same explicit parameters always produce the same, verifiable output, it belongs here. If the "right" answer depends on taste, content understanding, or what looks or sounds good, it belongs to whichever skill or agent makes that judgement — this skill only ever executes parameters it's given, never infers them from what something looks or sounds like.
120
+
121
+ If a request needs an FFmpeg feature none of the 28 scripts expose, say so and name the closest built-in option (`--dry-run` to show what would run, or a documented limitation) — never fall back to guessing a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards); it is exactly the failure mode this skill exists to prevent, so it is never the fallback when a script's flag doesn't cover something.
122
+
69
123
  ## Request → script
70
124
 
71
125
  | User says | Do |
@@ -75,14 +129,27 @@ Do not ask for things `probe.py` can tell you.
75
129
  | "keep only these parts", "remove the middle" | `cut.py input.mp4 --segments 0-1:00,1:30-2:00` |
76
130
  | "make it exactly 60 seconds", "fit it in 30s" | `fit.py input.mp4 --duration 60` (speed) or `--method trim` |
77
131
  | "make it vertical / for TikTok / 9:16", "square for Instagram" | `fit.py input.mp4 --aspect 9:16 --fit pad` (or `--fit crop`) |
132
+ | "resize to a specific height, width follows" | `fit.py input.mp4 --height 1080` (or `--width`, or both for an exact frame) |
133
+ | "crop to this exact box/rectangle" (known x/y/width/height, not an aspect ratio) | `crop.py input.mp4 --x 100 --y 0 --width 1080 --height 1920` |
134
+ | "turn this image into a N-second clip", "title card / end slate" | `insert.py title.png --duration 3` |
135
+ | "slow zoom on a photo", "Ken Burns effect" | `insert.py photo.jpg --duration 6 --zoom in --pan right --width 1920 --height 1080` |
136
+ | "rotate this 90 degrees", "mirror it horizontally" | `fit.py input.mp4 --rotate 90` / `fit.py input.mp4 --flip h` |
137
+ | "reverse this clip", "play it backwards" | `reverse.py input.mp4` |
138
+ | "stabilize this shaky footage" | `stabilize.py input.mp4` |
139
+ | "make a blank/colour background clip" | `background.py -o bg.mp4 --duration 3 --width 1920 --height 1080 --color 0x101010` |
140
+ | "turn these numbered frames into a video" | `sequence.py --dir frames --pattern "frame_%04d.png" --fps 24` |
78
141
  | "add subtitles from this SRT", "burn in captions" | `caption.py input.mp4 --srt subs.srt` |
79
142
  | "caption it with these lines" (plain text with times) | `caption.py input.mp4 --text cues.txt` |
143
+ | "add subtitles but keep them toggleable / editable", "mux in an SRT, don't burn it" | `caption.py input.mp4 --srt subs.srt --mode mux` |
80
144
  | "put our logo top-right", "add a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` |
81
145
  | "add a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` |
146
+ | "put this webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` |
147
+ | "remove the green screen", "chroma key this" | `overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00` |
82
148
  | "sync the lav mic to the camera", "line up the two cameras" | `sync.py camera.mp4 mic.wav --replace-audio` / `sync.py camA.mp4 camB.mp4 --trim-second` |
83
149
  | "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast) |
84
150
  | "export for YouTube / Reels / X", "give me a ProRes master", "make it HEVC" | `export.py input.mp4 --preset youtube|reels|x|prores|h265` |
85
151
  | "make a GIF preview" | `export.py input.mp4 --preset gif` |
152
+ | "make a small/low-res proxy for an analysis pass", "a cheap preview file" | `proxy.py input.mp4 [--width 640 --no-audio]` — not a delivery preset, see `export.py` for those |
86
153
  | "cut out the pauses / dead air", "tighten it up", "jump cuts" | `silence.py input.mp4 [--threshold -40 --min-silence 0.8]` |
87
154
  | "stitch these clips together", "add a crossfade between them" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` |
88
155
  | "show me what it looks like", "check the captions are readable" | `look.py output.mp4` then view the PNG |
@@ -102,11 +169,14 @@ Do not ask for things `probe.py` can tell you.
102
169
  | "show me progress", "quick preview first" | any encoding script with `--progress` and/or `--fast` |
103
170
  | "the colours look washed out / it's an iPhone HDR video" | `color.py input.mov --to-sdr` (probe shows `hdr: true`) |
104
171
  | "apply this LUT", "convert the S-Log / V-Log footage" | `color.py input.mp4 --lut grade.cube [--lut-strength 0.7]` |
105
- | "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (no re-encode) |
172
+ | "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (stream copy; re-encodes only if the copy can't carry the retagged colour info — check `reencoded` in `--json`) |
173
+ | "brighten it a touch / punch up the contrast and saturation / fix the white balance" | `color.py input.mp4 --correct --exposure 0.3 --contrast 1.1 --saturation 1.05 --temperature 5600 --tint -0.05` (typed, no filter string) |
106
174
  | "clean up the audio", "remove the hiss / room noise" | `audio.py input.mp4 --voice` (speech) or `--denoise` |
107
175
  | "add background music under the talking" | `audio.py input.mp4 --music bed.mp3 --duck --fade-out 3` |
108
176
  | "convert the 5.1 to stereo" | `audio.py input.mov --downmix` |
109
177
  | "swap in the narration track" | `audio.py input.mp4 --replace narration.wav` |
178
+ | "pull the audio out of this video", "give me the sound as WAV" | `audio.py input.mp4 -o input.wav` (any audio extension drops the picture; `--audio-stream 1` picks another track) |
179
+ | "compress the voice", "limit the peaks to -1 dB", "gate the room noise" | `audio.py input.mp4 --compress --comp-threshold -20 --comp-ratio 4` / `--limit --limit-ceiling -1` / `--gate --gate-threshold -45` (typed acompressor / alimiter / agate options, range-checked) |
110
180
  | "the audio drifts out of sync over the hour" | `sync.py camera.mp4 recorder.wav --fix-drift --replace-audio` |
111
181
  | "smooth slow motion", "half speed but fluid" | `fit.py input.mp4 --duration 2x --smooth interpolate` (slow) or `--smooth blend` |
112
182
  | "TikTok-style captions with the words popping / highlighted" | `caption.py input.mp4 --text cues.txt --animate pop --karaoke` |
@@ -124,12 +194,22 @@ commands work with `talk.wav` in place of `talk.mp4`. What changes:
124
194
  - The output extension picks the format: `-o out.mp3` converts, `-o out.wav`
125
195
  keeps PCM, `-o out.m4a` writes AAC. `audio.py in.wav -o out.mp3` with no
126
196
  other flag is a plain conversion.
127
- - `cut.py` stream-copies audio too, so trims are lossless unless the format
128
- cannot be cut on a packet boundary.
197
+ - `cut.py` stream-copies audio too, so trims land on a packet boundary
198
+ (`precision: packet`, a few ms; the JSON reports `duration_error_ms`). Pass
199
+ `--accurate` for a sample-exact trim: `precision: sample` when the output is
200
+ PCM or FLAC, `codec_frame` when a lossy codec (AAC, MP3, Opus) frames it
201
+ again. A `.wav` output is always PCM, never AAC packets inside a WAV.
202
+ - `join.py` joins audio-only clips as audio (`acrossfade` or a butt join) at
203
+ one sample rate and channel layout; the output must have an audio extension.
204
+ Video and audio clips cannot be mixed in one join.
205
+ - An audio extension on a video input (`audio.py talk.mp4 -o talk.wav`,
206
+ `cut.py talk.mp4 --start 1:00 --end 2:00 -o part.wav`) extracts the audio; the
207
+ output has no video stream. `audio.py --audio-stream N` picks a track when
208
+ `probe` lists several under `audio_streams`.
129
209
  - `Look: not needed` in the report; `Check:` still applies for loudness
130
210
  (`check.py file.wav --platform podcast` measures LUFS and true peak).
131
211
  - Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`,
132
- `color`, `export`, `join`, `scenes`, `look`) refuse an audio file with
212
+ `color`, `export`, `scenes`, `look`) refuse an audio file with
133
213
  "input has no video stream". Say so instead of forcing a video wrapper.
134
214
 
135
215
  | User says (audio file) | Do |
@@ -138,11 +218,16 @@ commands work with `talk.wav` in place of `talk.mp4`. What changes:
138
218
  | "remove the silence from this recording" | `silence.py talk.wav -o talk_tight.wav` |
139
219
  | "clean up the noise in this M4A" | `audio.py talk.m4a --voice -o talk_clean.m4a` (speech) or `--denoise` |
140
220
  | "convert this WAV to MP3" | `audio.py talk.wav -o talk.mp3` |
141
- | "trim this audio from 00:30 to 02:00" | `cut.py talk.wav --start 0:30 --end 2:00 -o talk_cut.wav` |
221
+ | "trim this audio from 00:30 to 02:00" | `cut.py talk.wav --start 0:30 --end 2:00 -o talk_cut.wav` (`--accurate` for sample-exact) |
222
+ | "join these recordings", "intro + episode + outro" | `join.py intro.wav episode.m4a outro.wav -o full.flac` (`--transition none` for a butt join) |
223
+ | "extract the audio from the video", "mp4 to wav" | `audio.py talk.mp4 -o talk.wav` (`--voice -o talk.m4a` to clean it on the way) |
224
+ | "compress / limit / gate the voice" | `audio.py talk.wav --compress --comp-threshold -20 --comp-ratio 4 --limit --limit-ceiling -1 -o talk_dyn.wav` |
142
225
  | "is this loud enough for Apple Podcasts?" | `check.py talk.m4a --platform podcast` |
143
226
 
144
227
  ## Report format
145
228
 
229
+ Reply in the language the user wrote their request in — a Japanese request gets a Japanese report, English gets English, Chinese gets Chinese, and so on for any other language. Keep the shape below and the field labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English (they read like log fields, not prose, and stay recognisable across languages); the sentences around them, any question asked, and any explanation of a judgement call are in the user's language. Never default to English because the tool names and flags happen to be English. A mid-conversation language switch follows the user's latest message, not the first one.
230
+
146
231
  Finish every job with this shape (numbers from `probe.py`/`check.py`, not memory):
147
232
 
148
233
  ```
@@ -155,20 +240,32 @@ Notes: source was VFR, conformed to 30 fps; audio was mono, made stereo
155
240
 
156
241
  Keep it to those five lines plus anything the user must decide. Attach the contact sheet when the edit touched the picture. Never report success without the probe of the output; never describe a fix you did not run.
157
242
 
243
+ When a step fails, replace `Done:` with `Failed:` and keep the rest honest:
244
+
245
+ ```
246
+ Failed: color.py --lut grade.cube exited 1 — ffmpeg: "Unable to parse LUT file" (the .cube is not a valid LUT)
247
+ Steps: probe -> color (failed); nothing written
248
+ Notes: send a valid .cube, or say if you want the clip left as is
249
+ ```
250
+
251
+ Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool, "message": ...}}` with `--json` and exits non-zero; quote the message, do not paraphrase it into a success.
252
+
158
253
  ## Things that look right but are wrong
159
254
 
160
255
  - Re-encoding an HDR (iPhone, HDR10) source through the SDR path: colours go flat. The scripts keep HDR; if you hand-write ffmpeg, do not tag BT.709 on BT.2020 pixels.
161
256
  - Lossless `-c copy` cuts on VFR or non-keyframe boundaries: the file "works" but starts on a frozen or wrong frame. `cut.py` re-encodes automatically when the snap exceeds 0.5 s; respect that.
162
- - A sync with `confidence` under 0.3, or an offset larger than 60 % of the analysis window: probably wrong; enlarge `--analyze-seconds` or find a clap.
257
+ - A sync or multicam alignment with `confidence` under 0.3, or an offset larger than 60 % of the analysis window: probably wrong; enlarge `--analyze-seconds` or find a clap. `multicam.py` reports one `confidence` per camera — check all of them, not just that the command succeeded, before trusting the cut.
258
+ - `sync.py`/`multicam.py` align audio tracks to each other, never lip sync (mouth movement vs. audio) — there is no face or mouth detection anywhere in this skill. A high confidence means the audio matched well, not that the picture looks right; if the user asks whether lip sync is correct, that needs a look at the actual video, not just the reported offset.
163
259
  - "Normalised" audio that still clips: check true peak, not just LUFS (`check.py` does both).
164
260
  - Normalising ambience or near-silence to a speech target: a clip measured at
165
261
  -40 LUFS or below is room tone, wind or nothing; raising it 25 dB raises the
166
262
  noise, not the content. Leave the level, say so, and offer music or narration.
167
263
  - Captions burned before a crop/resize: text lands off-frame. Frame changes first, then text.
168
264
  - Anything chained by hand through three re-encodes: use `render.py` so the plan is one file and the user can change one number.
169
- - `--fit crop` to reach 9:16 from 16:9 throws away 70 % of the width: a wide shot loses people at the edges. Check the sheet; pad (bars) or a reframe is often the honest answer.
265
+ - `--fit crop` to reach 9:16 from 16:9 throws away 70 % of the width: a wide shot loses people at the edges. Check the sheet; pad (bars), `--crop-x`/`--crop-y` toward the subject, or a reframe is often the honest answer — a silent centre crop is a guess, not a decision.
170
266
  - Conforming 60 fps to 30 halves the motion samples: fine for a talking head, visibly choppy for sports, gaming, drone pans. Keep 60 when the platform allows it.
171
267
  - "Make it 60 seconds" on a 3-minute talk by speed change is unwatchable (3×); by trim it drops two thirds of the words. Ask which, or propose a highlight cut with `scenes.py`.
268
+ - `scenes.py --highlights` defaults to the loudest scenes (`--rank-by audio`): a quiet but important moment (a confession, a punchline landing in silence) is skipped, and pure crowd noise or a mic bump can outrank it. `--rank-by duration` picks the longest unbroken scenes instead. Neither is "the best parts" — check the contact sheet (`--sheet`) before treating the picks as final.
172
269
 
173
270
  ## Gotchas
174
271
 
package/bin/install.js CHANGED
@@ -14,6 +14,9 @@
14
14
  * npx ffmpeg-skill --uninstall # remove from the selected targets
15
15
  * npx ffmpeg-skill contract --json # machine-readable execution contract (see docs/contract.md)
16
16
  * npx ffmpeg-skill doctor [--json] # which required ffmpeg capabilities this machine has
17
+ *
18
+ * Already installed? re-run `npx ffmpeg-skill` to refresh ~/.claude/skills/ffmpeg-skill
19
+ * Copies are not updated automatically.
17
20
  */
18
21
  'use strict';
19
22
 
@@ -40,7 +43,18 @@ if (has('--help') || has('-h')) {
40
43
 
41
44
  // `contract` / `doctor` are answered by scripts/_contract.py; everything else installs.
42
45
  if (args[0] === 'contract' || args[0] === 'doctor') {
43
- const py = spawnSync('python3', [path.join(ROOT, 'scripts', '_contract.py'), ...args], { stdio: 'inherit' });
46
+ // Windows Python installers commonly expose `python`/`py`, not `python3` (only the
47
+ // Microsoft Store package does); try python3 first (macOS/Linux convention), then fall
48
+ // back so `npx ffmpeg-skill doctor` doesn't silently fail with ENOENT on Windows.
49
+ const candidates = process.platform === 'win32' ? ['python3', 'python', 'py'] : ['python3'];
50
+ let py;
51
+ for (const cmd of candidates) {
52
+ py = spawnSync(cmd, [path.join(ROOT, 'scripts', '_contract.py'), ...args], { stdio: 'inherit' });
53
+ if (!py.error) break;
54
+ }
55
+ if (py.error) {
56
+ console.error(`error: could not find a Python interpreter (tried: ${candidates.join(', ')}). Install Python 3.9+ and ensure it is on PATH.`);
57
+ }
44
58
  process.exit(py.error ? 127 : py.status);
45
59
  }
46
60
 
@@ -77,7 +91,7 @@ function checkFfmpeg() {
77
91
  const r = spawnSync('ffmpeg', ['-version'], { encoding: 'utf8' });
78
92
  if (r.error || r.status !== 0) {
79
93
  console.warn('\n warning: ffmpeg was not found on PATH. The skill needs FFmpeg to run:');
80
- console.warn(' macOS: brew install ffmpeg');
94
+ console.warn(' macOS: brew install ffmpeg-full (the plain ffmpeg formula lacks subtitles/drawtext/zscale)');
81
95
  console.warn(' Ubuntu: sudo apt install ffmpeg');
82
96
  console.warn(' Windows: winget install Gyan.FFmpeg\n');
83
97
  return false;
package/mcp/server.py CHANGED
@@ -10,6 +10,8 @@ Run:
10
10
  python3 mcp/server.py # stdio transport
11
11
  Claude Desktop / Claude Code config example:
12
12
  {"mcpServers": {"ffmpeg-skill": {"command": "python3", "args": ["/path/to/ffmpeg-skill/mcp/server.py"]}}}
13
+ On Windows, use "python" instead of "python3" unless Python was installed from the Microsoft
14
+ Store (a python.org install exposes python/py, not python3) -- see README.md's MCP section.
13
15
  """
14
16
  import json
15
17
  import os
package/package.json CHANGED
@@ -1,8 +1,20 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "0.9.0",
4
- "description": "Agent Skill that lets coding agents (Claude Code, Cursor, Codex) do professional video editing with local FFmpeg: MCP server, batch processing, declarative project rendering, brand kits, motion-graphics templates, HTML delivery reports, scene detection, delivery checks, cut, silence removal, transitions, multicam, captions, sync with drift correction, HDR to SDR, LUTs, audio clean-up and ducking, loudness, platform exports. No API keys, no cloud, no dependencies.",
5
- "keywords": ["ffmpeg", "video", "agent-skill", "claude-code", "cursor", "codex", "skill", "video-editing"],
3
+ "version": "0.12.0",
4
+ "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 28 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
+ "keywords": [
6
+ "ffmpeg",
7
+ "video",
8
+ "video-editing",
9
+ "video-processing",
10
+ "audio",
11
+ "agent-skill",
12
+ "claude-code",
13
+ "cursor",
14
+ "codex",
15
+ "mcp",
16
+ "skill"
17
+ ],
6
18
  "license": "MIT",
7
19
  "author": "kajisho5",
8
20
  "repository": {
@@ -0,0 +1,111 @@
1
+ # Platform-specific CI pitfalls
2
+
3
+ Behaviour differences between macOS/Linux/Windows CI runners and their ffmpeg
4
+ builds that are not bugs in this repo's code — they were each independently
5
+ diagnosed once, at real cost (full CI cycles, log-reading, re-fixture
6
+ attempts). Written down so the next person (or the next session) does not
7
+ re-diagnose them from scratch. Add to this file whenever a fix in this repo
8
+ exists only because a platform's real, observed behaviour forced it — not
9
+ for hypothetical differences.
10
+
11
+ ## Windows
12
+
13
+ ### `-pattern_type glob` is unsupported on the Chocolatey ffmpeg build
14
+
15
+ The Windows GitHub Actions runner's `choco install ffmpeg` build fails with
16
+ `Pattern type 'glob' was selected but globbing is not supported by this
17
+ libavformat build` — glob support depends on how libavformat was compiled,
18
+ and this build lacks it entirely. There is no flag or workaround within
19
+ `-pattern_type glob` itself.
20
+
21
+ Fix used in `sequence.py`: resolve the frame list in Python (`glob.glob` or
22
+ walking consecutive numbered filenames) and feed ffmpeg an explicit
23
+ **concat-demuxer list file** instead of relying on `-pattern_type glob`.
24
+ This works identically on all three OSes since it never depends on
25
+ libavformat's own globbing.
26
+
27
+ ### The concat demuxer's "repeat last file" duration trick over-counts by one frame
28
+
29
+ The standard technique for giving the last file in a concat list a duration
30
+ (repeat its entry once with no explicit `duration` line, so ffmpeg holds it
31
+ until EOF) produced an extra frame's worth of output duration on some ffmpeg
32
+ builds — e.g. 1.2s of output for footage that should total 1.0s. Observed on
33
+ both macOS and Windows CI after switching `sequence.py` to the concat
34
+ demuxer (see above).
35
+
36
+ Fix: pass an explicit `-t <total_duration>` alongside the concat list so the
37
+ output is truncated to the intended length regardless of how the demuxer's
38
+ own end-marker behaves on a given build.
39
+
40
+ ### A `#!/bin/sh` fake-ffmpeg PATH shim is not portable to Windows
41
+
42
+ Several tests fake ffmpeg's behaviour (e.g. "exits 0 but writes nothing") by
43
+ dropping a `#!/bin/sh` script named `ffmpeg` earlier on `PATH`. This has no
44
+ Windows equivalent — `cmd.exe`/PowerShell do not execute a shebang script
45
+ named `ffmpeg` the way a POSIX shell resolves `ffmpeg` on `PATH`, so the
46
+ fake binary is silently never picked up and the test either fails for the
47
+ wrong reason or exercises the real ffmpeg instead.
48
+
49
+ Fix: `@unittest.skipIf(platform.system() == "Windows", "reason echoing this
50
+ note")` on every test that depends on this shim technique, rather than
51
+ trying to make the shim itself cross-platform. Established first on
52
+ `test_dry_run_never_runs_ffmpeg_and_writes_nothing` and the
53
+ `DoctorDetectionTests` class; the same pattern was later needed for
54
+ `test_output_verification_failures_are_loud` when it was added by a
55
+ different change that didn't carry the context forward. When adding a new
56
+ shim-based test, search the test files for this skip pattern and mirror it
57
+ rather than rediscovering the failure on a Windows CI run.
58
+
59
+ ### A real ffmpeg crash's OS-reported exit code does not match this repo's self-reported one, on Windows only
60
+
61
+ When ffmpeg genuinely crashes (e.g. parsing a corrupt `.cube` LUT, or being
62
+ told to write into a directory that does not exist), Windows reports the
63
+ subprocess's exit code as a large unsigned-32-bit value (`4294967295`,
64
+ `3199971767`, `4294967294`, ...) that does not exactly equal what this
65
+ repo's own `die()`-driven JSON `exit_code` field captured for the same
66
+ failure. Confirmed not a flake by re-running the identical commit's
67
+ identical job and getting byte-identical numbers both times; the mismatch
68
+ recurs on different failure sub-cases with different specific numbers each
69
+ time. This is inherent to how Windows reports a crashed child process's
70
+ exit status through Python's `subprocess` — not a bug in `verify_output()`
71
+ or `die()`.
72
+
73
+ Fix: tests that assert on `exit_code` for a `kind == "ffmpeg"` failure only
74
+ assert `!= 0` on Windows, and assert the exact expected value on
75
+ macOS/Linux. Do not chase exact-value parity on Windows for this class of
76
+ failure — it is not achievable without changing how the OS reports crashed
77
+ subprocesses, which is out of this repo's control.
78
+
79
+ ## macOS
80
+
81
+ ### `vidstabdetect`/`vidstabtransform` (libvidstab) behaves meaningfully differently across ffmpeg builds
82
+
83
+ A synthetic camera-shake test fixture that reliably gets *less* jittery
84
+ after `stabilize.py` on Linux CI can reliably get *more* jittery (a larger
85
+ measured frame-to-frame motion, not smaller) on macOS CI's ffmpeg build —
86
+ this was independently confirmed across three different fixture designs
87
+ (a single clean sine-wave jitter; a multi-frequency jitter including a fast
88
+ ~9.1Hz component; a retuned multi-frequency jitter in a more realistic
89
+ 1-2Hz hand-tremor range), all of which passed on Linux and all of which
90
+ failed differently on macOS. This points to a genuine behavioural
91
+ difference in libvidstab (or how it's built/linked) between the two
92
+ platforms' ffmpeg, not a fixable property of the test fixture — a fixture
93
+ cannot be tuned to satisfy two optical-flow implementations that disagree.
94
+
95
+ Fix: `test_stabilize_reduces_frame_to_frame_motion` only asserts the
96
+ quantitative "motion went down" claim on Linux
97
+ (`if platform.system() == "Linux":`); on every platform it still asserts
98
+ the tool ran, produced output, and the output has the expected duration —
99
+ so the test still catches a genuinely broken `stabilize.py`, just not a
100
+ libvidstab behavioural quirk that is outside this repo's control. Don't
101
+ spend another cycle retuning the fixture frequencies again — three attempts
102
+ already ruled that out.
103
+
104
+ ## General
105
+
106
+ When a test needs to special-case a platform, prefer gating with
107
+ `platform.system()` (already imported for this purpose in
108
+ `tests/test_all.py` and `tests/test_contract.py`) over inventing a new
109
+ mechanism, and write the skip/relaxation reason as a full sentence
110
+ explaining the underlying platform behaviour — not just "flaky on
111
+ Windows" — so a future reader doesn't have to re-derive it from the CI log.
@@ -0,0 +1,85 @@
1
+ # Process pitfalls
2
+
3
+ Mistakes made (or nearly made) while developing this repo that were not about FFmpeg
4
+ or a platform's behaviour — about the *process* of making a change safely. Written down
5
+ for the same reason `references/ci-platform-pitfalls.md` exists: a mistake that isn't
6
+ recorded gets repeated the next time a session starts fresh with no memory of it.
7
+
8
+ **This file is a living record.** Whenever a change here is made (or nearly made, then
9
+ caught before landing) because an existing guardrail — a pinned test, an environment
10
+ constraint, a platform's real behaviour under repeated attempts — wasn't checked first,
11
+ add an entry below. Don't wait to be asked.
12
+
13
+ ## Before narrowing a `required`/`optional` capability list, grep for the pinned test that checks it
14
+
15
+ `scripts/_contract.py`'s `TOOL_META[...]["required"]` drives `doctor`'s per-tool `usable`
16
+ answer (`_tool_usability()` in `_contract.py` only reads `required`, never `optional`).
17
+ Moving a capability from `required` to `optional` — even when it's honestly true that a
18
+ new flag makes it conditional — silently changes what `doctor` reports as `usable: no`
19
+ on a machine missing that capability, for the tool's *default* invocation too.
20
+
21
+ `tests/test_contract.py`'s `DoctorDetectionTests` pins specific `usable` outcomes against
22
+ real captured `ffmpeg -filters`/`-encoders` fixtures (e.g. a plain Homebrew macOS build
23
+ correctly reporting `caption.usable: "no"` because it lacks `filter:subtitles`). A change
24
+ to `required` that isn't checked against these first can pass a quick unit test and still
25
+ break this fixture-based guarantee.
26
+
27
+ Caught twice while adding capability metadata for new flags (`caption.py --mode mux` in
28
+ #51, `doctor`'s `gpu_encoders` in #52) — in both cases the fix was to grep
29
+ `tests/test_contract.py` for `usable` and `_doctor(` *before* editing `TOOL_META`, not
30
+ after a test failure revealed it. Do that grep first, every time `required`/`optional`
31
+ changes.
32
+
33
+ ## Git tag push and GitHub Release creation are not reachable from this environment
34
+
35
+ The git credentials available here can push to `refs/heads/*` (branches) but not
36
+ `refs/tags/*` — confirmed by a 403 straight from the git-receive-pack endpoint, not an
37
+ auth failure, meaning it's a deliberate scope restriction, not a bug to route around.
38
+ The GitHub MCP tool surface has no `create_release`/`create_tag` equivalent either
39
+ (`create_branch`, `create_pull_request`, `create_or_update_file` exist; nothing for
40
+ releases). A direct call to the GitHub REST API's `/releases` endpoint with a raw token
41
+ is also blocked by the outbound proxy itself (its own 403, pointing at Anthropic's docs,
42
+ not GitHub's).
43
+
44
+ Confirmed once (retried the tag push a second time "just in case" before accepting it).
45
+ Don't retry either path a second time — if `git push origin <tag>` 403s, or no
46
+ release-creation tool is found in one `ToolSearch` pass, say so once and hand the user
47
+ the two-minute browser-only path instead (open the repo's `/releases/new`, type the new
48
+ tag name in the tag field — GitHub creates it from the target branch on publish, no git
49
+ command needed).
50
+
51
+ ## A quantitative test failing three different ways across fixture redesigns means the platform, not the fixture, is the problem
52
+
53
+ `test_stabilize_reduces_frame_to_frame_motion` (macOS CI, `stabilize.py`) failed with
54
+ three independently redesigned shake fixtures in a row — each time the instinct was "the
55
+ fixture's frequencies must be wrong," each time the retuned fixture failed a *different*
56
+ way on the next CI run. The actual cause (libvidstab behaving differently across the
57
+ Linux and macOS ffmpeg builds) was diagnosable from the first failure: a synthetic
58
+ fixture that reliably improves under one implementation and reliably gets worse under
59
+ another is evidence the implementations disagree, not that the fixture is miscalibrated.
60
+
61
+ If a quantitative assertion fails on one platform, survives a redesign, and fails again
62
+ on the *same* platform in a different way: stop redesigning the fixture. Either restrict
63
+ the strict assertion to the platform where it's provably correct (keeping a weaker,
64
+ platform-general check — output exists, has the right duration — everywhere), or escalate
65
+ before spending a third CI cycle on it.
66
+
67
+ ## A fix merged after CHANGELOG.md's current-version section was drafted can silently miss it
68
+
69
+ `CHANGELOG.md`'s `## 0.12.0` section was written once, covering everything merged up to
70
+ that point. Two fixes that closed real issues after that point (#62's `--audio-stream`
71
+ extension via PR #72, #77's dry-run-dims fix via PR #88) landed with no further nudge to
72
+ go back and add a bullet — #62's fix actually got a bullet (its content is genuinely
73
+ described) but the `Closes #62` link was left off, and #77 was missed outright until a
74
+ direct question ("shouldn't this bump the version?") prompted a manual check. Neither was
75
+ caught by CI, because nothing checked CHANGELOG.md against what had actually been closed.
76
+
77
+ Caught by hand both times, then closed properly with `tests/test_contract.py`'s
78
+ `test_changelog_mentions_every_closed_issue_since_last_tag`, which walks `git log` back to
79
+ the latest release tag, extracts every `Closes #N.` from a commit body, and fails if that
80
+ issue number doesn't appear anywhere in `CHANGELOG.md`. This needs real history (`ci.yml`'s
81
+ `actions/checkout` step now passes `fetch-depth: 0` for exactly this reason — the default
82
+ shallow clone leaves no tag reachable to diff against, which would make the test silently
83
+ skip itself in CI, not fail). If this test ever needs to skip a genuinely changelog-less
84
+ closed issue (a pure process note, a duplicate, a revert of an unreleased change), name the
85
+ exemption in the test itself with a reason — don't just widen the regex or drop the check.