ffmpeg-skill 0.10.0 → 0.12.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +70 -10
- package/SKILL.md +89 -12
- package/bin/install.js +15 -1
- package/mcp/__pycache__/server.cpython-311.pyc +0 -0
- package/mcp/server.py +2 -0
- package/package.json +2 -2
- package/references/ci-platform-pitfalls.md +111 -0
- package/references/process-pitfalls.md +85 -0
- package/references/scripts.md +139 -12
- package/scripts/__pycache__/_common.cpython-311.pyc +0 -0
- package/scripts/__pycache__/_contract.cpython-311.pyc +0 -0
- package/scripts/__pycache__/audio.cpython-311.pyc +0 -0
- package/scripts/__pycache__/batch.cpython-311.pyc +0 -0
- package/scripts/__pycache__/caption.cpython-311.pyc +0 -0
- package/scripts/__pycache__/check.cpython-311.pyc +0 -0
- package/scripts/__pycache__/color.cpython-311.pyc +0 -0
- package/scripts/__pycache__/cut.cpython-311.pyc +0 -0
- package/scripts/__pycache__/export.cpython-311.pyc +0 -0
- package/scripts/__pycache__/fit.cpython-311.pyc +0 -0
- package/scripts/__pycache__/graphics.cpython-311.pyc +0 -0
- package/scripts/__pycache__/join.cpython-311.pyc +0 -0
- package/scripts/__pycache__/look.cpython-311.pyc +0 -0
- package/scripts/__pycache__/loudness.cpython-311.pyc +0 -0
- package/scripts/__pycache__/multicam.cpython-311.pyc +0 -0
- package/scripts/__pycache__/overlay.cpython-311.pyc +0 -0
- package/scripts/__pycache__/probe.cpython-311.pyc +0 -0
- package/scripts/__pycache__/render.cpython-311.pyc +0 -0
- package/scripts/__pycache__/report.cpython-311.pyc +0 -0
- package/scripts/__pycache__/scenes.cpython-311.pyc +0 -0
- package/scripts/__pycache__/silence.cpython-311.pyc +0 -0
- package/scripts/__pycache__/sync.cpython-311.pyc +0 -0
- package/scripts/__pycache__/verify.cpython-311.pyc +0 -0
- package/scripts/_common.py +255 -14
- package/scripts/_contract.py +276 -13
- package/scripts/audio.py +1 -1
- package/scripts/background.py +73 -0
- package/scripts/caption.py +106 -24
- package/scripts/color.py +129 -30
- package/scripts/crop.py +79 -0
- package/scripts/cut.py +2 -2
- package/scripts/export.py +1 -1
- package/scripts/fit.py +73 -14
- package/scripts/graphics.py +18 -7
- package/scripts/insert.py +128 -0
- package/scripts/join.py +14 -5
- package/scripts/look.py +13 -8
- package/scripts/loudness.py +3 -3
- package/scripts/multicam.py +1 -1
- package/scripts/overlay.py +78 -12
- package/scripts/proxy.py +82 -0
- package/scripts/reverse.py +56 -0
- package/scripts/scenes.py +8 -2
- package/scripts/sequence.py +124 -0
- package/scripts/silence.py +2 -2
- package/scripts/stabilize.py +101 -0
- package/scripts/sync.py +1 -1
package/README.md
CHANGED
|
@@ -25,10 +25,16 @@ npx ffmpeg-skill
|
|
|
25
25
|
|
|
26
26
|

|
|
27
27
|
|
|
28
|
-
`ffmpeg-skill` is an [Agent Skill](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills) for Claude Code, Cursor, Codex and any agent that reads `SKILL.md`. It teaches the agent a fixed workflow (probe → edit losslessly where possible → check → verify) and ships **
|
|
28
|
+
`ffmpeg-skill` is an [Agent Skill](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills) for Claude Code, Cursor, Codex and any agent that reads `SKILL.md`. It teaches the agent a fixed workflow (probe → edit losslessly where possible → check → verify) and ships **28 tools** that do the actual work with `ffmpeg` / `ffprobe`: cut, join, silence removal, fit to duration and aspect, captions and karaoke, overlays and motion graphics, HDR → SDR and LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, whole-edit project rendering, batch folders. Every tool is also an MCP tool, and the whole set is described by a machine-readable contract.
|
|
29
29
|
|
|
30
30
|
If `ffmpeg` and `python3` are on your PATH, it works: offline, on footage you would rather not upload.
|
|
31
31
|
|
|
32
|
+
> **SPEC** (Self-Producing Execution Contract), coined by this project's author
|
|
33
|
+
> [kajisho5](https://github.com/kajisho5): each tool's `input_schema` — the part of its contract
|
|
34
|
+
> and MCP tool definition that has to track the CLI flag-for-flag — is never hand-authored beside
|
|
35
|
+
> the code. It's derived, at run time, from the same `argparse` parser that already defines the
|
|
36
|
+
> CLI, and CI fails the build if any of it drifts. → [full explanation](#what-is-spec)
|
|
37
|
+
|
|
32
38
|
---
|
|
33
39
|
|
|
34
40
|
## Standalone, and in an ecosystem
|
|
@@ -75,6 +81,10 @@ npx ffmpeg-skill doctor
|
|
|
75
81
|
npx ffmpeg-skill contract --json | head -40
|
|
76
82
|
```
|
|
77
83
|
|
|
84
|
+
Already installed? re-run `npx ffmpeg-skill` to refresh `~/.claude/skills/ffmpeg-skill`. Copies are not updated automatically.
|
|
85
|
+
|
|
86
|
+
`doctor`'s overall `ok` and a single tool's `usable: no` are different signals: `ok` means nothing *required by every tool* is missing, but a plain Homebrew `ffmpeg` on macOS can still be `ok` while `caption.py` specifically can't run (no `subtitles` filter) — check `doctor --json`'s `tools` field for the per-tool answer, not just `ok`.
|
|
87
|
+
|
|
78
88
|
Then talk to your agent:
|
|
79
89
|
|
|
80
90
|
> "Take `interview.mp4`, keep 0:45–3:10 and 5:00–6:30, and make it exactly 60 seconds for Reels."
|
|
@@ -90,6 +100,8 @@ python3 $S/fit.py input.mp4 --duration 60 --aspect 9:16 --dry-run # print the
|
|
|
90
100
|
python3 $S/export.py input.mp4 --preset reels --json # structured result with a probe of the output
|
|
91
101
|
```
|
|
92
102
|
|
|
103
|
+
On Windows in Git Bash, `python3` is only on PATH if Python was installed from the Microsoft Store; a python.org install exposes `python` (or the `py` launcher) instead — replace `python3` with `python` above if you see a "command not found". `bin/install.js` and `doctor`/`contract` already handle this for you; only the raw script examples above need it spelled out manually.
|
|
104
|
+
|
|
93
105
|
More requests and the commands behind them: [examples/README.md](examples/README.md). To see everything run end-to-end on generated footage: `npm run demo`.
|
|
94
106
|
|
|
95
107
|
## How it works
|
|
@@ -127,7 +139,7 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
|
|
|
127
139
|
1. **Probe first.** No tool decides from the file name. `probe.py` measures duration, fps (with variable-frame-rate detection), resolution, rotation, bit depth, HDR format including Dolby Vision, colour tags and every audio stream before anything is cut.
|
|
128
140
|
2. **Lossless when possible.** `cut.py`, `join.py` and `loudness.py` stream-copy what they do not need to touch. Re-encoding happens only when it must: frame-accurate cuts, filters, format changes, or a keyframe farther than the tolerance.
|
|
129
141
|
3. **Plan before render.** Every tool takes `--dry-run` (print the ffmpeg command lines, write nothing), `--json` (structured result with a probe of the output), `--fast` (preview quality) and `--progress` (percent and ETA). A test runs every tool under `--dry-run` behind a fake ffmpeg and asserts that no ffmpeg call happened and no file appeared.
|
|
130
|
-
4. **Machine-readable contract.** `contract --json` describes all
|
|
142
|
+
4. **Machine-readable contract.** `contract --json` describes all 28 tools: input schema generated from the parser, output schema, role, required and conditional FFmpeg capabilities, dry-run support, the verification tools to run afterwards, whether a visual check is required, `mutates_input: false`. `provides` lists all 28 by a cross-repository Capability id (`ffmpeg-skill.cut`, `ffmpeg-skill.loudness`, ...) for [`kajisho5/AI-video-production-OS`](https://github.com/kajisho5/AI-video-production-OS)'s `CapabilityContract.provides` — see `docs/contract.md`.
|
|
131
143
|
5. **Contract-derived MCP.** `mcp/server.py` builds its `tools/list` from the contract. Tool names, order and `inputSchema` cannot drift from the scripts; a test keeps the two byte-identical.
|
|
132
144
|
6. **Capability detection.** `doctor` reads `ffmpeg -encoders / -filters / -bsfs` and reports which of the components the tools need are present on this build (libx264, libass, zscale, loudnorm, xfade, …), before a job fails inside ffmpeg.
|
|
133
145
|
7. **Unknown is not missing.** When a listing cannot be read (a layout the parser does not know, ffmpeg exiting non-zero) the affected capabilities are `unknown`: never `missing`, never silently `available`. An installed filter is not reported absent; a failed detection is not a pass.
|
|
@@ -136,7 +148,7 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
|
|
|
136
148
|
|
|
137
149
|
## Tools
|
|
138
150
|
|
|
139
|
-
|
|
151
|
+
28 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, non-zero exit and a reason on stderr on failure.
|
|
140
152
|
|
|
141
153
|
**Analysis and inspection**
|
|
142
154
|
|
|
@@ -153,7 +165,13 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
|
|
|
153
165
|
| `cut.py` | In/out or multi-segment cuts, lossless `-c copy` first, re-encode fallback, `--accurate` for frame-exact video and sample-exact audio; reports `precision` |
|
|
154
166
|
| `join.py` | Concatenate clips with xfade transitions, normalising size, fps and audio; audio-only inputs are joined as audio |
|
|
155
167
|
| `silence.py` | Detect and remove dead air (jump cuts) with a margin around speech; list or export the cut list |
|
|
156
|
-
| `fit.py` | Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad or crop, with `--crop-x`/`--crop-y` to keep an off-centre subject); force constant fps |
|
|
168
|
+
| `fit.py` | Fit to a duration (pitch-preserving speed change or trim, smooth slow-mo) and/or aspect ratio (pad or crop, with `--crop-x`/`--crop-y` to keep an off-centre subject) and/or exact `--width`/`--height`; rotate 90/180/270, flip h/v; force constant fps |
|
|
169
|
+
| `crop.py` | Crop to an exact pixel rectangle (`--x --y --width --height`) — distinct from `fit.py --fit crop`, which crops to an aspect ratio it computes itself |
|
|
170
|
+
| `insert.py` | Turn a still image into a silent, fixed-duration video clip (title card, end slate) at an exact frame size / fps, with an optional Ken Burns zoom/pan |
|
|
171
|
+
| `background.py` | Generate a solid-colour or two-colour gradient clip at an exact size/duration — no input file |
|
|
172
|
+
| `reverse.py` | Reverse playback (video and, unless `--no-audio`, audio) |
|
|
173
|
+
| `stabilize.py` | Two-pass motion stabilisation (`vidstabdetect`/`vidstabtransform`) |
|
|
174
|
+
| `sequence.py` | Numbered (`frame_%04d.png`) or glob-matched still images into a video |
|
|
157
175
|
|
|
158
176
|
**Audio**
|
|
159
177
|
|
|
@@ -168,15 +186,16 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
|
|
|
168
186
|
| Tool | What it does |
|
|
169
187
|
|---|---|
|
|
170
188
|
| `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; animated and word-by-word karaoke timed to the speech energy; optional local transcription |
|
|
171
|
-
| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades |
|
|
189
|
+
| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades; `--video` for picture-in-picture, `--chromakey` for green-screen compositing |
|
|
172
190
|
| `graphics.py` | Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs drawn by FFmpeg from a brand kit |
|
|
173
|
-
| `color.py` | HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting, typed primary correction (exposure/contrast/saturation/white balance) |
|
|
191
|
+
| `color.py` | HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting, typed primary correction (exposure/contrast/saturation/gamma/white balance/lift-gain/levels/curves) |
|
|
174
192
|
|
|
175
193
|
**Delivery**
|
|
176
194
|
|
|
177
195
|
| Tool | What it does |
|
|
178
196
|
|---|---|
|
|
179
197
|
| `export.py` | Presets `youtube`, `youtube4k`, `reels`, `x`, `prores`, `h265`, `gif`, all tagged BT.709 |
|
|
198
|
+
| `proxy.py` | Small, low-bitrate proxy for downstream AI analysis/preview/editing decisions — resize by `--width`/`--scale`, proxy-grade `--crf`, `--fps`, `--no-audio`; not a delivery preset |
|
|
180
199
|
| `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, broadcast and podcast specs, with the fix for each failure and a `format` / `judgement` kind per row |
|
|
181
200
|
| `report.py` | Single-file HTML delivery report: before/after sheets, media facts, loudness, compliance, the commands run |
|
|
182
201
|
|
|
@@ -205,6 +224,34 @@ Picture tools (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scen
|
|
|
205
224
|
|
|
206
225
|
## Built for agents
|
|
207
226
|
|
|
227
|
+
### What is SPEC?
|
|
228
|
+
|
|
229
|
+
This project's author, [kajisho5](https://github.com/kajisho5), coined **SPEC** (Self-Producing
|
|
230
|
+
Execution Contract) for the pattern this skill's tool layer is built on: each tool's `input_schema`
|
|
231
|
+
— the part of its contract that has to track the CLI exactly, flag for flag — is never
|
|
232
|
+
hand-authored side by side with the code. It is derived, at run time, from the one thing that
|
|
233
|
+
actually has to be correct for the CLI to work at all: the script's own `argparse` parser.
|
|
234
|
+
|
|
235
|
+
Concretely, `scripts/_contract.py`'s `_capture_parser()` imports every tool script and
|
|
236
|
+
intercepts its `parse_args()` call to get the live, fully-built parser object — flags, types,
|
|
237
|
+
choices, defaults, required/positional, mutually exclusive groups, all of it. `input_schema` is
|
|
238
|
+
built straight from that object. (The rest of a `ToolSpec` — `role`, `capabilities`, `inputs`,
|
|
239
|
+
`outputs`, `output_schema` — comes from a hand-authored table, `TOOL_META`, since those facts
|
|
240
|
+
aren't things a parser can express; only `input_schema` is parser-derived.)
|
|
241
|
+
|
|
242
|
+
- **The contract**'s `input_schema` for every tool is generated from the live parser directly.
|
|
243
|
+
- **The MCP server** (`mcp/server.py`) carries no schema of its own; `tools/list` is translated
|
|
244
|
+
straight from the contract, `input_schema` included.
|
|
245
|
+
- **The docs** (`docs/contract.md`'s field reference, this README's tool table) describe the same
|
|
246
|
+
shape. `tests/test_contract.py` runs on every CI run and fails the build if any of them drift
|
|
247
|
+
out of sync with what the code actually does — it catches drift, it doesn't fix it for you.
|
|
248
|
+
|
|
249
|
+
The result: add a flag to a script's `argparse` block, and `input_schema` and the MCP tool
|
|
250
|
+
definition follow with no second edit; if a docs page or a `TOOL_META` entry falls behind, CI
|
|
251
|
+
catches it rather than letting it drift silently. There is no separate `input_schema` file to
|
|
252
|
+
forget to update, and no version of "what CLI flags does this tool accept" that can quietly go
|
|
253
|
+
stale.
|
|
254
|
+
|
|
208
255
|
### Machine-readable contract
|
|
209
256
|
|
|
210
257
|
```bash
|
|
@@ -212,7 +259,7 @@ npx ffmpeg-skill contract --json # or: python3 scripts/_contract.py -
|
|
|
212
259
|
npx ffmpeg-skill contract --json --static # without environment detection
|
|
213
260
|
```
|
|
214
261
|
|
|
215
|
-
The contract is generated from the code that runs, not maintained beside it. For each of the
|
|
262
|
+
The contract is generated from the code that runs, not maintained beside it. For each of the 28 tools (`ffmpeg-skill/<name>`) it states:
|
|
216
263
|
|
|
217
264
|
| Field | Meaning |
|
|
218
265
|
|---|---|
|
|
@@ -235,17 +282,21 @@ The contract is generated from the code that runs, not maintained beside it. For
|
|
|
235
282
|
{"mcpServers": {"ffmpeg-skill": {"command": "python3", "args": ["/Users/you/.claude/skills/ffmpeg-skill/mcp/server.py"]}}}
|
|
236
283
|
```
|
|
237
284
|
|
|
238
|
-
`
|
|
285
|
+
On Windows, `python3` is only on PATH if Python was installed from the Microsoft Store; a python.org install exposes `python` (or the `py` launcher) instead — if your MCP client reports the server failed to start, change `"command"` above to `"python"` (or the full path from `where python`).
|
|
286
|
+
|
|
287
|
+
`mcp/server.py` is a stdio JSON-RPC transport with no tool table of its own. `tools/list` is derived from the contract at start-up: the same 28 names, the same order, and `inputSchema` translated from each tool's `input_schema`. `tools/call` maps structured arguments to argv and runs the named script; a raw `argv` form is accepted for compatibility and marked non-canonical. `python3 mcp/server.py --list` prints the tools; `--call probe '{"inputs": ["a.mp4"]}'` runs one from the shell.
|
|
239
288
|
|
|
240
289
|
### Capability detection
|
|
241
290
|
|
|
242
291
|
```bash
|
|
243
292
|
npx ffmpeg-skill doctor # human-readable
|
|
244
|
-
npx ffmpeg-skill doctor --json # available / missing / missing_optional / unknown / detection / errors / tools
|
|
293
|
+
npx ffmpeg-skill doctor --json # available / missing / missing_optional / unknown / detection / errors / tools / gpu_encoders
|
|
245
294
|
```
|
|
246
295
|
|
|
247
296
|
`doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`. `doctor --json`'s `tools` field folds that down to one answer per tool — `{"caption": {"usable": "no", "missing": ["filter:subtitles"], "fix": "..."}, ...}` — so "is `doctor` overall `ok`" and "can I run `caption.py` on this machine" are answered separately: a plain Homebrew `ffmpeg` is `ok` for tools that don't need `subtitles`/`drawtext`/`zscale`, while `caption`'s own `usable` is `"no"`.
|
|
248
297
|
|
|
298
|
+
`doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). No tool here uses one yet — every tool still assumes CPU x264/x265 — so this is purely informational and never affects `ok` or any tool's `usable`. GPU-accelerated encoding stays deliberately off the roadmap until there's a real-hardware-verified design for it (build-presence alone is not proof a job will succeed) — not a promised feature, just an honest "not yet, and not without proof it actually works."
|
|
299
|
+
|
|
249
300
|
## FFmpeg compatibility
|
|
250
301
|
|
|
251
302
|
The tools need FFmpeg 5.0 or later. The capability parser has been run against the listings of these builds:
|
|
@@ -315,6 +366,8 @@ FFmpeg itself:
|
|
|
315
366
|
- Python 3.9+, standard library only
|
|
316
367
|
- Node 16+ only for the `npx` installer
|
|
317
368
|
|
|
369
|
+
`doctor`'s own introspection calls (`ffmpeg -filters`/`-encoders`/`-bsfs`/`-version`) time out after 10s and report `failed` rather than hanging forever — those are meant to be fast. Every tool's actual media-processing `ffmpeg` invocation (cut, fit, caption, ...) has no timeout: a legitimate `--accurate` re-encode of a long file can genuinely take a long time, so bounding it would risk killing real work. `-nostdin` is always passed, so a hung ffmpeg process waiting on stdin cannot happen; a caller that needs a hard ceiling on a specific job should apply its own external timeout/kill around that one invocation.
|
|
370
|
+
|
|
318
371
|
## Development
|
|
319
372
|
|
|
320
373
|
```bash
|
|
@@ -327,15 +380,22 @@ node bin/install.js --dir /tmp/skills # try the installer without touching ~/.
|
|
|
327
380
|
|
|
328
381
|
CI (`.github/workflows/ci.yml`) runs on every pull request and on pushes to `main`, on Ubuntu (FFmpeg 6.1), macOS (Homebrew FFmpeg 8.x) and Windows (gyan.dev FFmpeg 9.x), and uploads each runner's FFmpeg listings as an artifact.
|
|
329
382
|
|
|
330
|
-
|
|
383
|
+
`tests/test_contract.py` runs on all three OSes, but a handful of its tests build a fake `ffmpeg` as a `#!/bin/sh` script on a PATH shim to force specific FFmpeg 6/7/8/9 fixture layouts through `doctor`'s parser — that technique isn't portable to Windows, so `test_dry_run_never_runs_ffmpeg_and_writes_nothing` and the whole `DoctorDetectionTests` class (fixture-driven layout parsing) are individually `skipIf`'d there and show as `skipped`, not silently absent, in the Windows job's log. Everything else — contract schema, `reencodes_*`, `doctor.tools`, MCP derivation, and every tool exercised through the contract, including `cut.py`'s provenance fields — runs against the real Windows `ffmpeg` on every PR. See [references/ci-platform-pitfalls.md](references/ci-platform-pitfalls.md) for this and other per-OS behaviour differences already diagnosed, before spending a CI cycle re-diagnosing a platform-only failure.
|
|
384
|
+
|
|
385
|
+
**Releasing**: bump `version` in `package.json`, merge to `main`, then tag that commit (`git tag vX.Y.Z && git push origin vX.Y.Z`). `.github/workflows/release.yml` picks up from there: it verifies the tag matches `package.json`'s version, extracts that version's `CHANGELOG.md` section, and publishes the GitHub Release automatically — tagging stays a deliberate, manual act; only the release-notes step is automated. A repo that depends on this one (an editing skill, an agent) should pin an `ffmpeg-skill` version by tag or npm version, not by tracking `main` — `main` can be ahead of the last published npm version.
|
|
386
|
+
|
|
387
|
+
Contributing a change: see [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
331
388
|
|
|
332
389
|
## Docs
|
|
333
390
|
|
|
334
391
|
| | |
|
|
335
392
|
|---|---|
|
|
393
|
+
| [CONTRIBUTING.md](CONTRIBUTING.md) | scope, dev setup, tests, PR expectations |
|
|
336
394
|
| [SKILL.md](SKILL.md) | what the agent reads: workflow, request → tool map, audio-only rules, report format, pitfalls |
|
|
337
395
|
| [references/scripts.md](references/scripts.md) | per-flag reference for every tool |
|
|
338
396
|
| [references/devices.md](references/devices.md) | real-device notes (iPhone HDR, GoPro, DJI, screen recordings) |
|
|
397
|
+
| [references/ci-platform-pitfalls.md](references/ci-platform-pitfalls.md) | per-OS ffmpeg/CI behaviour differences already diagnosed once — read before re-diagnosing a Windows/macOS-only test failure |
|
|
398
|
+
| [references/process-pitfalls.md](references/process-pitfalls.md) | process mistakes already made once (breaking a pinned test by narrowing a capability list, retrying a git/GitHub operation this environment can't do, re-designing a fixture instead of recognising a real platform difference) — a living record, add to it whenever one recurs |
|
|
339
399
|
| [docs/contract.md](docs/contract.md) | the execution contract field by field, MCP relationship, how a planner consumes it |
|
|
340
400
|
| [examples/README.md](examples/README.md) | natural-language requests and the commands behind them, `brand.json`, `project.json`, batch recipes |
|
|
341
401
|
| [tests/fixtures/README.md](tests/fixtures/README.md) | captured and constructed FFmpeg listings, which is which |
|
package/SKILL.md
CHANGED
|
@@ -5,10 +5,20 @@ description: Edit video and audio with local FFmpeg from natural-language reques
|
|
|
5
5
|
|
|
6
6
|
# ffmpeg-skill
|
|
7
7
|
|
|
8
|
-
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. Every script has `--help`, and all of them accept `--dry-run
|
|
8
|
+
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. Every script has `--help`, and all of them accept `--dry-run`, `--json` (structured result with a probe of the output), `--fast` (preview quality) and `--progress`. Writing tools run nothing under `--dry-run`; `probe`/`check`/`sync`/`multicam`/`scenes`/`report` may still run ffmpeg/ffprobe to measure or analyse — they just don't write their final artifact; `verify` accepts the flag but ignores it. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`). Details for every flag: `references/scripts.md`. Device-specific behaviour (iPhone HDR, GoPro, DJI, screen recordings, Zoom): `references/devices.md`.
|
|
9
9
|
|
|
10
10
|
## Workflow (always follow this order)
|
|
11
11
|
|
|
12
|
+
0. **Check the environment once per session, if unfamiliar.** On a machine
|
|
13
|
+
you haven't confirmed capability on this session, run `doctor --json`
|
|
14
|
+
once: check `ok` and the target tool's `usable` before relying on it. If
|
|
15
|
+
`usable` isn't `yes`, don't run that tool — report the missing capability
|
|
16
|
+
instead of discovering it via a runtime failure (a missing `libass`,
|
|
17
|
+
`zscale`, or encoder is the common case, e.g. `caption.py`). Don't re-run
|
|
18
|
+
`doctor` per job — it queries `ffmpeg -filters`/`-encoders`, not free, and
|
|
19
|
+
once per session/unfamiliar machine is enough. `contract --json`'s full
|
|
20
|
+
tool schema is for a *planning* agent deciding which tool/params to use
|
|
21
|
+
from an abstract goal — not part of this per-job workflow.
|
|
12
22
|
1. **Probe first.** Run `probe.py` on every input before touching it. Read the
|
|
13
23
|
duration, fps, resolution, codecs, audio channels and the
|
|
14
24
|
`variable_frame_rate_suspected` flag. Plan the edit from real numbers, never
|
|
@@ -18,8 +28,17 @@ Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>
|
|
|
18
28
|
`cut.py` and `loudness.py` stream-copy video by default; only pass
|
|
19
29
|
`--accurate` to `cut.py` when the user needs frame-exact cuts.
|
|
20
30
|
3. **Plan with `--dry-run --json`, then execute.** Every script accepts
|
|
21
|
-
`--dry-run` (prints the ffmpeg commands
|
|
22
|
-
(structured result: output path, probe of the output, commands run).
|
|
31
|
+
`--dry-run` (prints the ffmpeg commands that would run) and `--json`
|
|
32
|
+
(structured result: output path, probe of the output, commands run). For
|
|
33
|
+
writing tools this means nothing is written; `probe`/`check` still run
|
|
34
|
+
ffprobe/loudness-measurement passes (they're read-only, so `--dry-run`
|
|
35
|
+
changes nothing for `probe`, and only skips the loudness pass for
|
|
36
|
+
`check`), `sync`/`multicam`/`scenes`/`report` still run ffmpeg/ffprobe to
|
|
37
|
+
measure or analyse, and `verify` accepts the flag but ignores it entirely
|
|
38
|
+
(its steps run regardless) — see `contract --json`'s `dry_run` field per
|
|
39
|
+
tool for exact semantics. Trust `--json`, not a dry-run's human-readable
|
|
40
|
+
summary line, for any number after the plan (dimensions in that line can
|
|
41
|
+
be a placeholder, not a computed preview — see `docs/contract.md`). Use
|
|
23
42
|
them to confirm a plan before long encodes and to report exact facts.
|
|
24
43
|
`--fast` gives a quick preview-quality render (x264 veryfast), `--progress`
|
|
25
44
|
prints percent and ETA on stderr for long encodes.
|
|
@@ -40,17 +59,36 @@ Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>
|
|
|
40
59
|
6. **Verify the output.** Run `probe.py` on each result and confirm duration,
|
|
41
60
|
resolution, fps and audio match what was requested. Report those numbers to
|
|
42
61
|
the user (e.g. "final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo").
|
|
62
|
+
A step is done only when the script exited 0 and the output probes as
|
|
63
|
+
expected. Writing the command is not doing the job; a non-zero exit, a
|
|
64
|
+
missing or empty file, or a probe that contradicts the request is a
|
|
65
|
+
failure, and the report says so with the script's error message.
|
|
43
66
|
7. **Keep the user's originals.** Never overwrite the source file. Write new
|
|
44
67
|
files next to the input or where the user asked.
|
|
45
68
|
8. **Look at the picture.** Whenever the picture changed (captions, overlays,
|
|
46
69
|
graphics, crop/pad, resize, colour, transitions) run `look.py OUTPUT`
|
|
47
|
-
(contact sheet) or `look.py OUTPUT --at T`, view the PNG
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
70
|
+
(contact sheet) or `look.py OUTPUT --at T`, view the PNG. The job is not
|
|
71
|
+
finished until the report's `Look:` line names that PNG; a probe alone
|
|
72
|
+
cannot see a caption sitting on someone's face. Audio-only jobs (sync,
|
|
73
|
+
loudness, silence, or any job whose input is an audio file) write
|
|
74
|
+
`Look: not needed`; there is no picture to inspect. What to look for
|
|
75
|
+
splits the same way `check.py`'s rows do in step 5:
|
|
76
|
+
- **Mechanical (verify and report as this skill's own job):** the
|
|
77
|
+
specified text/logo is present at the specified position, subtitles/text
|
|
78
|
+
appear at the specified timestamps, resolution has even dimensions.
|
|
79
|
+
Letterboxing/pillarboxing from `fit.py --fit pad` is the *correct*
|
|
80
|
+
result of that mode, not a defect — never flag it.
|
|
81
|
+
- **Judgement (report to the calling agent/user, don't silently pass or
|
|
82
|
+
fail):** whether a subject or face is cut off, whether text sits over a
|
|
83
|
+
face, whether colours look washed out, whether a transition "lands"
|
|
84
|
+
well or the edit feels cinematic. These require deciding what the
|
|
85
|
+
subject *is*, which belongs to the calling agent (see "What this skill
|
|
86
|
+
does and does not decide") — state what you see in one line and let the
|
|
87
|
+
calling agent or user judge it, don't decide it here.
|
|
88
|
+
If the execution environment cannot actually view images (no vision
|
|
89
|
+
capability), write `Look: PATH (pixels not inspected; agent has no image
|
|
90
|
+
view)` — never claim a picture was inspected when it wasn't, and don't
|
|
91
|
+
stall indefinitely waiting for a capability that isn't there.
|
|
54
92
|
|
|
55
93
|
|
|
56
94
|
## Before you run anything: what to ask, what to assume
|
|
@@ -75,8 +113,12 @@ This skill cuts, joins, measures, syncs, exports and checks files — it execute
|
|
|
75
113
|
- **What makes a highlight interesting** — `scenes.py --highlights` ranks by a measured proxy (audio energy or scene duration, see its own docs), never by understanding the content; treat its output as candidates, not a verdict.
|
|
76
114
|
- **Thumbnail or cover-image composition** — that's a design decision, not a measurement; a thumbnail-generation skill or the user makes it.
|
|
77
115
|
- **Understanding what a video is *about*** — this skill has no transcription or vision beyond `look.py`'s contact sheets, which exist for the calling agent's own eyes, not for this skill to interpret on its own.
|
|
116
|
+
- **Judging what looks good** — "apply this LUT" or "correct exposure by +0.3 stops" (`color.py`) is mechanical, parameter-determined execution and belongs here; "grade this scene to look cinematic" is a subjective judgement about what looks right and belongs in a colour-grading skill ([`color-grading-skill`](https://github.com/kajisho5/color-grading-skill), see README's "Standalone, and in an ecosystem") that decides the parameters and then calls `color.py` to apply them.
|
|
117
|
+
- **Picking a subject or region without being told one** — "crop to this exact box" or "crop to 9:16 keeping x=200,y=0" (`crop.py`/`fit.py --fit crop --crop-x/-y`) is mechanical once the box is known; "crop to keep the speaker in frame" requires deciding *what* the speaker is, which is a vision/composition judgement for the calling agent (from a `look.py` contact sheet) or a motion-graphics skill, not this one.
|
|
78
118
|
|
|
79
|
-
|
|
119
|
+
The line in general: if the same input and the same explicit parameters always produce the same, verifiable output, it belongs here. If the "right" answer depends on taste, content understanding, or what looks or sounds good, it belongs to whichever skill or agent makes that judgement — this skill only ever executes parameters it's given, never infers them from what something looks or sounds like.
|
|
120
|
+
|
|
121
|
+
If a request needs an FFmpeg feature none of the 28 scripts expose, say so and name the closest built-in option (`--dry-run` to show what would run, or a documented limitation) — never fall back to guessing a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards); it is exactly the failure mode this skill exists to prevent, so it is never the fallback when a script's flag doesn't cover something.
|
|
80
122
|
|
|
81
123
|
## Request → script
|
|
82
124
|
|
|
@@ -87,14 +129,27 @@ If a request needs an FFmpeg feature none of the 21 scripts expose, say so and n
|
|
|
87
129
|
| "keep only these parts", "remove the middle" | `cut.py input.mp4 --segments 0-1:00,1:30-2:00` |
|
|
88
130
|
| "make it exactly 60 seconds", "fit it in 30s" | `fit.py input.mp4 --duration 60` (speed) or `--method trim` |
|
|
89
131
|
| "make it vertical / for TikTok / 9:16", "square for Instagram" | `fit.py input.mp4 --aspect 9:16 --fit pad` (or `--fit crop`) |
|
|
132
|
+
| "resize to a specific height, width follows" | `fit.py input.mp4 --height 1080` (or `--width`, or both for an exact frame) |
|
|
133
|
+
| "crop to this exact box/rectangle" (known x/y/width/height, not an aspect ratio) | `crop.py input.mp4 --x 100 --y 0 --width 1080 --height 1920` |
|
|
134
|
+
| "turn this image into a N-second clip", "title card / end slate" | `insert.py title.png --duration 3` |
|
|
135
|
+
| "slow zoom on a photo", "Ken Burns effect" | `insert.py photo.jpg --duration 6 --zoom in --pan right --width 1920 --height 1080` |
|
|
136
|
+
| "rotate this 90 degrees", "mirror it horizontally" | `fit.py input.mp4 --rotate 90` / `fit.py input.mp4 --flip h` |
|
|
137
|
+
| "reverse this clip", "play it backwards" | `reverse.py input.mp4` |
|
|
138
|
+
| "stabilize this shaky footage" | `stabilize.py input.mp4` |
|
|
139
|
+
| "make a blank/colour background clip" | `background.py -o bg.mp4 --duration 3 --width 1920 --height 1080 --color 0x101010` |
|
|
140
|
+
| "turn these numbered frames into a video" | `sequence.py --dir frames --pattern "frame_%04d.png" --fps 24` |
|
|
90
141
|
| "add subtitles from this SRT", "burn in captions" | `caption.py input.mp4 --srt subs.srt` |
|
|
91
142
|
| "caption it with these lines" (plain text with times) | `caption.py input.mp4 --text cues.txt` |
|
|
143
|
+
| "add subtitles but keep them toggleable / editable", "mux in an SRT, don't burn it" | `caption.py input.mp4 --srt subs.srt --mode mux` |
|
|
92
144
|
| "put our logo top-right", "add a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` |
|
|
93
145
|
| "add a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` |
|
|
146
|
+
| "put this webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` |
|
|
147
|
+
| "remove the green screen", "chroma key this" | `overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00` |
|
|
94
148
|
| "sync the lav mic to the camera", "line up the two cameras" | `sync.py camera.mp4 mic.wav --replace-audio` / `sync.py camA.mp4 camB.mp4 --trim-second` |
|
|
95
149
|
| "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast) |
|
|
96
150
|
| "export for YouTube / Reels / X", "give me a ProRes master", "make it HEVC" | `export.py input.mp4 --preset youtube|reels|x|prores|h265` |
|
|
97
151
|
| "make a GIF preview" | `export.py input.mp4 --preset gif` |
|
|
152
|
+
| "make a small/low-res proxy for an analysis pass", "a cheap preview file" | `proxy.py input.mp4 [--width 640 --no-audio]` — not a delivery preset, see `export.py` for those |
|
|
98
153
|
| "cut out the pauses / dead air", "tighten it up", "jump cuts" | `silence.py input.mp4 [--threshold -40 --min-silence 0.8]` |
|
|
99
154
|
| "stitch these clips together", "add a crossfade between them" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` |
|
|
100
155
|
| "show me what it looks like", "check the captions are readable" | `look.py output.mp4` then view the PNG |
|
|
@@ -114,7 +169,7 @@ If a request needs an FFmpeg feature none of the 21 scripts expose, say so and n
|
|
|
114
169
|
| "show me progress", "quick preview first" | any encoding script with `--progress` and/or `--fast` |
|
|
115
170
|
| "the colours look washed out / it's an iPhone HDR video" | `color.py input.mov --to-sdr` (probe shows `hdr: true`) |
|
|
116
171
|
| "apply this LUT", "convert the S-Log / V-Log footage" | `color.py input.mp4 --lut grade.cube [--lut-strength 0.7]` |
|
|
117
|
-
| "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (
|
|
172
|
+
| "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (stream copy; re-encodes only if the copy can't carry the retagged colour info — check `reencoded` in `--json`) |
|
|
118
173
|
| "brighten it a touch / punch up the contrast and saturation / fix the white balance" | `color.py input.mp4 --correct --exposure 0.3 --contrast 1.1 --saturation 1.05 --temperature 5600 --tint -0.05` (typed, no filter string) |
|
|
119
174
|
| "clean up the audio", "remove the hiss / room noise" | `audio.py input.mp4 --voice` (speech) or `--denoise` |
|
|
120
175
|
| "add background music under the talking" | `audio.py input.mp4 --music bed.mp3 --duck --fade-out 3` |
|
|
@@ -185,6 +240,16 @@ Notes: source was VFR, conformed to 30 fps; audio was mono, made stereo
|
|
|
185
240
|
|
|
186
241
|
Keep it to those five lines plus anything the user must decide. Attach the contact sheet when the edit touched the picture. Never report success without the probe of the output; never describe a fix you did not run.
|
|
187
242
|
|
|
243
|
+
When a step fails, replace `Done:` with `Failed:` and keep the rest honest:
|
|
244
|
+
|
|
245
|
+
```
|
|
246
|
+
Failed: color.py --lut grade.cube exited 1 — ffmpeg: "Unable to parse LUT file" (the .cube is not a valid LUT)
|
|
247
|
+
Steps: probe -> color (failed); nothing written
|
|
248
|
+
Notes: send a valid .cube, or say if you want the clip left as is
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool, "message": ...}}` with `--json` and exits non-zero; quote the message, do not paraphrase it into a success.
|
|
252
|
+
|
|
188
253
|
## Things that look right but are wrong
|
|
189
254
|
|
|
190
255
|
- Re-encoding an HDR (iPhone, HDR10) source through the SDR path: colours go flat. The scripts keep HDR; if you hand-write ffmpeg, do not tag BT.709 on BT.2020 pixels.
|
|
@@ -234,6 +299,18 @@ Keep it to those five lines plus anything the user must decide. Attach the conta
|
|
|
234
299
|
`caption.py --fonts-dir ./fonts --font "Noto Sans CJK JP"`). Without a
|
|
235
300
|
matching font you get boxes, not an error. Install: `apt install fonts-noto-cjk`,
|
|
236
301
|
`brew install --cask font-noto-sans-cjk`.
|
|
302
|
+
- **Windows drawtext crashes on some real builds.** On certain Windows ffmpeg
|
|
303
|
+
builds (e.g. winget's gyan.dev), `drawtext` crashes with an access violation
|
|
304
|
+
whenever it resolves a font by family name through fontconfig, even with a
|
|
305
|
+
valid `fonts.conf` (#100). `look.py`, `scenes.py --sheet`, `overlay.py --text`
|
|
306
|
+
and `graphics.py` all resolve a concrete `--font-file` by default when one is
|
|
307
|
+
available (`fontfile=` skips fontconfig entirely and is the form confirmed
|
|
308
|
+
not to crash), so this should already be handled automatically. If a
|
|
309
|
+
drawtext tool still crashes, pass `--font-file` explicitly rather than
|
|
310
|
+
relying on `--font`/`font=` resolution; `doctor` also runs a real one-frame
|
|
311
|
+
drawtext probe and reports `filter:drawtext` missing (with the crash detail
|
|
312
|
+
in `errors[]`) rather than a false "available" from the `-filters` listing
|
|
313
|
+
alone.
|
|
237
314
|
- **Keyframe cuts.** A lossless `cut.py` result may start up to one GOP (often
|
|
238
315
|
1–10 s) earlier than requested; the script re-encodes automatically when the
|
|
239
316
|
deviation exceeds 0.5 s. If the user insists on lossless output, pass
|
package/bin/install.js
CHANGED
|
@@ -14,6 +14,9 @@
|
|
|
14
14
|
* npx ffmpeg-skill --uninstall # remove from the selected targets
|
|
15
15
|
* npx ffmpeg-skill contract --json # machine-readable execution contract (see docs/contract.md)
|
|
16
16
|
* npx ffmpeg-skill doctor [--json] # which required ffmpeg capabilities this machine has
|
|
17
|
+
*
|
|
18
|
+
* Already installed? re-run `npx ffmpeg-skill` to refresh ~/.claude/skills/ffmpeg-skill
|
|
19
|
+
* Copies are not updated automatically.
|
|
17
20
|
*/
|
|
18
21
|
'use strict';
|
|
19
22
|
|
|
@@ -40,7 +43,18 @@ if (has('--help') || has('-h')) {
|
|
|
40
43
|
|
|
41
44
|
// `contract` / `doctor` are answered by scripts/_contract.py; everything else installs.
|
|
42
45
|
if (args[0] === 'contract' || args[0] === 'doctor') {
|
|
43
|
-
|
|
46
|
+
// Windows Python installers commonly expose `python`/`py`, not `python3` (only the
|
|
47
|
+
// Microsoft Store package does); try python3 first (macOS/Linux convention), then fall
|
|
48
|
+
// back so `npx ffmpeg-skill doctor` doesn't silently fail with ENOENT on Windows.
|
|
49
|
+
const candidates = process.platform === 'win32' ? ['python3', 'python', 'py'] : ['python3'];
|
|
50
|
+
let py;
|
|
51
|
+
for (const cmd of candidates) {
|
|
52
|
+
py = spawnSync(cmd, [path.join(ROOT, 'scripts', '_contract.py'), ...args], { stdio: 'inherit' });
|
|
53
|
+
if (!py.error) break;
|
|
54
|
+
}
|
|
55
|
+
if (py.error) {
|
|
56
|
+
console.error(`error: could not find a Python interpreter (tried: ${candidates.join(', ')}). Install Python 3.9+ and ensure it is on PATH.`);
|
|
57
|
+
}
|
|
44
58
|
process.exit(py.error ? 127 : py.status);
|
|
45
59
|
}
|
|
46
60
|
|
|
Binary file
|
package/mcp/server.py
CHANGED
|
@@ -10,6 +10,8 @@ Run:
|
|
|
10
10
|
python3 mcp/server.py # stdio transport
|
|
11
11
|
Claude Desktop / Claude Code config example:
|
|
12
12
|
{"mcpServers": {"ffmpeg-skill": {"command": "python3", "args": ["/path/to/ffmpeg-skill/mcp/server.py"]}}}
|
|
13
|
+
On Windows, use "python" instead of "python3" unless Python was installed from the Microsoft
|
|
14
|
+
Store (a python.org install exposes python/py, not python3) -- see README.md's MCP section.
|
|
13
15
|
"""
|
|
14
16
|
import json
|
|
15
17
|
import os
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "0.
|
|
4
|
-
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor:
|
|
3
|
+
"version": "0.12.5",
|
|
4
|
+
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 28 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ffmpeg",
|
|
7
7
|
"video",
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Platform-specific CI pitfalls
|
|
2
|
+
|
|
3
|
+
Behaviour differences between macOS/Linux/Windows CI runners and their ffmpeg
|
|
4
|
+
builds that are not bugs in this repo's code — they were each independently
|
|
5
|
+
diagnosed once, at real cost (full CI cycles, log-reading, re-fixture
|
|
6
|
+
attempts). Written down so the next person (or the next session) does not
|
|
7
|
+
re-diagnose them from scratch. Add to this file whenever a fix in this repo
|
|
8
|
+
exists only because a platform's real, observed behaviour forced it — not
|
|
9
|
+
for hypothetical differences.
|
|
10
|
+
|
|
11
|
+
## Windows
|
|
12
|
+
|
|
13
|
+
### `-pattern_type glob` is unsupported on the Chocolatey ffmpeg build
|
|
14
|
+
|
|
15
|
+
The Windows GitHub Actions runner's `choco install ffmpeg` build fails with
|
|
16
|
+
`Pattern type 'glob' was selected but globbing is not supported by this
|
|
17
|
+
libavformat build` — glob support depends on how libavformat was compiled,
|
|
18
|
+
and this build lacks it entirely. There is no flag or workaround within
|
|
19
|
+
`-pattern_type glob` itself.
|
|
20
|
+
|
|
21
|
+
Fix used in `sequence.py`: resolve the frame list in Python (`glob.glob` or
|
|
22
|
+
walking consecutive numbered filenames) and feed ffmpeg an explicit
|
|
23
|
+
**concat-demuxer list file** instead of relying on `-pattern_type glob`.
|
|
24
|
+
This works identically on all three OSes since it never depends on
|
|
25
|
+
libavformat's own globbing.
|
|
26
|
+
|
|
27
|
+
### The concat demuxer's "repeat last file" duration trick over-counts by one frame
|
|
28
|
+
|
|
29
|
+
The standard technique for giving the last file in a concat list a duration
|
|
30
|
+
(repeat its entry once with no explicit `duration` line, so ffmpeg holds it
|
|
31
|
+
until EOF) produced an extra frame's worth of output duration on some ffmpeg
|
|
32
|
+
builds — e.g. 1.2s of output for footage that should total 1.0s. Observed on
|
|
33
|
+
both macOS and Windows CI after switching `sequence.py` to the concat
|
|
34
|
+
demuxer (see above).
|
|
35
|
+
|
|
36
|
+
Fix: pass an explicit `-t <total_duration>` alongside the concat list so the
|
|
37
|
+
output is truncated to the intended length regardless of how the demuxer's
|
|
38
|
+
own end-marker behaves on a given build.
|
|
39
|
+
|
|
40
|
+
### A `#!/bin/sh` fake-ffmpeg PATH shim is not portable to Windows
|
|
41
|
+
|
|
42
|
+
Several tests fake ffmpeg's behaviour (e.g. "exits 0 but writes nothing") by
|
|
43
|
+
dropping a `#!/bin/sh` script named `ffmpeg` earlier on `PATH`. This has no
|
|
44
|
+
Windows equivalent — `cmd.exe`/PowerShell do not execute a shebang script
|
|
45
|
+
named `ffmpeg` the way a POSIX shell resolves `ffmpeg` on `PATH`, so the
|
|
46
|
+
fake binary is silently never picked up and the test either fails for the
|
|
47
|
+
wrong reason or exercises the real ffmpeg instead.
|
|
48
|
+
|
|
49
|
+
Fix: `@unittest.skipIf(platform.system() == "Windows", "reason echoing this
|
|
50
|
+
note")` on every test that depends on this shim technique, rather than
|
|
51
|
+
trying to make the shim itself cross-platform. Established first on
|
|
52
|
+
`test_dry_run_never_runs_ffmpeg_and_writes_nothing` and the
|
|
53
|
+
`DoctorDetectionTests` class; the same pattern was later needed for
|
|
54
|
+
`test_output_verification_failures_are_loud` when it was added by a
|
|
55
|
+
different change that didn't carry the context forward. When adding a new
|
|
56
|
+
shim-based test, search the test files for this skip pattern and mirror it
|
|
57
|
+
rather than rediscovering the failure on a Windows CI run.
|
|
58
|
+
|
|
59
|
+
### A real ffmpeg crash's OS-reported exit code does not match this repo's self-reported one, on Windows only
|
|
60
|
+
|
|
61
|
+
When ffmpeg genuinely crashes (e.g. parsing a corrupt `.cube` LUT, or being
|
|
62
|
+
told to write into a directory that does not exist), Windows reports the
|
|
63
|
+
subprocess's exit code as a large unsigned-32-bit value (`4294967295`,
|
|
64
|
+
`3199971767`, `4294967294`, ...) that does not exactly equal what this
|
|
65
|
+
repo's own `die()`-driven JSON `exit_code` field captured for the same
|
|
66
|
+
failure. Confirmed not a flake by re-running the identical commit's
|
|
67
|
+
identical job and getting byte-identical numbers both times; the mismatch
|
|
68
|
+
recurs on different failure sub-cases with different specific numbers each
|
|
69
|
+
time. This is inherent to how Windows reports a crashed child process's
|
|
70
|
+
exit status through Python's `subprocess` — not a bug in `verify_output()`
|
|
71
|
+
or `die()`.
|
|
72
|
+
|
|
73
|
+
Fix: tests that assert on `exit_code` for a `kind == "ffmpeg"` failure only
|
|
74
|
+
assert `!= 0` on Windows, and assert the exact expected value on
|
|
75
|
+
macOS/Linux. Do not chase exact-value parity on Windows for this class of
|
|
76
|
+
failure — it is not achievable without changing how the OS reports crashed
|
|
77
|
+
subprocesses, which is out of this repo's control.
|
|
78
|
+
|
|
79
|
+
## macOS
|
|
80
|
+
|
|
81
|
+
### `vidstabdetect`/`vidstabtransform` (libvidstab) behaves meaningfully differently across ffmpeg builds
|
|
82
|
+
|
|
83
|
+
A synthetic camera-shake test fixture that reliably gets *less* jittery
|
|
84
|
+
after `stabilize.py` on Linux CI can reliably get *more* jittery (a larger
|
|
85
|
+
measured frame-to-frame motion, not smaller) on macOS CI's ffmpeg build —
|
|
86
|
+
this was independently confirmed across three different fixture designs
|
|
87
|
+
(a single clean sine-wave jitter; a multi-frequency jitter including a fast
|
|
88
|
+
~9.1Hz component; a retuned multi-frequency jitter in a more realistic
|
|
89
|
+
1-2Hz hand-tremor range), all of which passed on Linux and all of which
|
|
90
|
+
failed differently on macOS. This points to a genuine behavioural
|
|
91
|
+
difference in libvidstab (or how it's built/linked) between the two
|
|
92
|
+
platforms' ffmpeg, not a fixable property of the test fixture — a fixture
|
|
93
|
+
cannot be tuned to satisfy two optical-flow implementations that disagree.
|
|
94
|
+
|
|
95
|
+
Fix: `test_stabilize_reduces_frame_to_frame_motion` only asserts the
|
|
96
|
+
quantitative "motion went down" claim on Linux
|
|
97
|
+
(`if platform.system() == "Linux":`); on every platform it still asserts
|
|
98
|
+
the tool ran, produced output, and the output has the expected duration —
|
|
99
|
+
so the test still catches a genuinely broken `stabilize.py`, just not a
|
|
100
|
+
libvidstab behavioural quirk that is outside this repo's control. Don't
|
|
101
|
+
spend another cycle retuning the fixture frequencies again — three attempts
|
|
102
|
+
already ruled that out.
|
|
103
|
+
|
|
104
|
+
## General
|
|
105
|
+
|
|
106
|
+
When a test needs to special-case a platform, prefer gating with
|
|
107
|
+
`platform.system()` (already imported for this purpose in
|
|
108
|
+
`tests/test_all.py` and `tests/test_contract.py`) over inventing a new
|
|
109
|
+
mechanism, and write the skip/relaxation reason as a full sentence
|
|
110
|
+
explaining the underlying platform behaviour — not just "flaky on
|
|
111
|
+
Windows" — so a future reader doesn't have to re-derive it from the CI log.
|