ffmpeg-skill 1.10.0 → 1.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +13 -6
- package/SKILL.md +111 -275
- package/docs/contract.md +12 -10
- package/package.json +2 -1
- package/references/gotchas.md +172 -0
- package/references/scripts.md +4 -2
- package/scripts/_common.py +119 -29
- package/scripts/_contract.py +20 -6
- package/scripts/batch.py +3 -0
- package/scripts/render.py +54 -0
package/SKILL.md
CHANGED
|
@@ -5,258 +5,146 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
|
|
|
5
5
|
|
|
6
6
|
# ffmpeg-skill
|
|
7
7
|
|
|
8
|
-
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. Every script has `--help
|
|
8
|
+
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. Every script has `--help`; every flag in detail: `references/scripts.md`. Device behaviour (iPhone HDR, GoPro, DJI, screen recordings, Zoom): `references/devices.md`. The long form of this file's one-line rules: `references/gotchas.md`.
|
|
9
9
|
|
|
10
|
-
|
|
10
|
+
Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (that document trimmed to status/output/verified, a compact `summary` and the command count — enough to confirm and report); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (consent to replace an existing output — without it the tool warns today, refuses from 2.0); `--plan FILE` (the dry run as a plan document `render.py FILE` executes later, refusing if an input changed: "plan → user confirms → execute" in one round trip). Every re-encoding tool also takes `--codec h264|hevc|av1|prores` and `--quality N` (CRF scale, replaces the deprecated `--crf`): unset, SDR is x264 and HDR is x265 Main10; `prores` needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (run `color.py --to-sdr` first).
|
|
11
|
+
|
|
12
|
+
Writing tools run nothing under `--dry-run`; `probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness` and `stabilize` may still run ffmpeg/ffprobe to measure or analyse — they just don't write their final artifact (nor side files such as `--edl`, `--sheet` or a generated `.ass`); `verify` accepts the flag but ignores it. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
11
13
|
|
|
12
|
-
|
|
13
|
-
you haven't confirmed capability on this session, run
|
|
14
|
-
`python3 <skill-dir>/scripts/_contract.py doctor --json` (the npm install
|
|
15
|
-
also exposes it as `npx ffmpeg-skill doctor --json`; there is no doctor.py)
|
|
16
|
-
once: check `ok` and the target tool's `usable` before relying on it. If
|
|
17
|
-
`usable` isn't `yes`, don't run that tool — report the missing capability
|
|
18
|
-
instead of discovering it via a runtime failure (a missing `libass`,
|
|
19
|
-
`zscale`, or encoder is the common case, e.g. `caption.py`). Don't re-run
|
|
20
|
-
`doctor` per job — it queries `ffmpeg -filters`/`-encoders`, not free, and
|
|
21
|
-
once per session/unfamiliar machine is enough. `contract --json`'s full
|
|
22
|
-
tool schema is for a *planning* agent deciding which tool/params to use
|
|
23
|
-
from an abstract goal — not part of this per-job workflow.
|
|
24
|
-
1. **Probe first.** Run `probe.py` on every input before touching it. Read the
|
|
25
|
-
duration, fps, resolution, codecs, audio channels and the
|
|
26
|
-
`variable_frame_rate_suspected` flag. Plan the edit from real numbers, never
|
|
27
|
-
from assumptions about the file.
|
|
28
|
-
2. **Prefer lossless.** If the request can be satisfied without re-encoding
|
|
29
|
-
(plain cuts on keyframes, remuxing, audio-only changes), do not re-encode.
|
|
30
|
-
`cut.py` and `loudness.py` stream-copy video by default; only pass
|
|
31
|
-
`--accurate` to `cut.py` when the user needs frame-exact cuts.
|
|
32
|
-
3. **Plan with `--dry-run --json`, then execute.** Every script accepts
|
|
33
|
-
`--dry-run` (prints the ffmpeg commands that would run; writing tools write
|
|
34
|
-
nothing, analysis tools still measure — the per-tool list is in the
|
|
35
|
-
opening paragraph above and in `contract --json`'s `dry_run` field) and
|
|
36
|
-
`--json` (structured result: output path, probe of the output, commands
|
|
37
|
-
run). Trust `--json`, not a dry-run's human-readable summary line, for any
|
|
38
|
-
number after the plan (dimensions in that line can be a placeholder, not a
|
|
39
|
-
computed preview — see `docs/contract.md`). Use them to confirm a plan
|
|
40
|
-
before long encodes and to report exact facts. `--fast` gives a quick
|
|
41
|
-
preview-quality render (x264 veryfast), `--progress` prints percent and
|
|
42
|
-
ETA on stderr for long encodes. Never point `-o` at a file
|
|
43
|
-
you did not create in this job unless the user asked for it to be replaced;
|
|
44
|
-
pass `--overwrite` only then.
|
|
45
|
-
4. **Chain operations in a sensible order.** Colour (HDR→SDR / LUT) → cut →
|
|
46
|
-
join → silence → fit → caption/overlay → sync → audio → loudness → export.
|
|
47
|
-
Do frame changes (fit/crop) before captions and overlays so text is sized
|
|
48
|
-
for the final frame. Re-encode as few times as possible: keep intermediates
|
|
49
|
-
at CRF 18 (the default) and only use `export.py` for the last step; for
|
|
50
|
-
anything with more than two steps use `render.py` with a project.json.
|
|
51
|
-
5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X`
|
|
52
|
-
for the destination the user named. Each row is marked `format` or
|
|
53
|
-
`judgement`. Format rows (codec, pixel format, size, true peak, colour
|
|
54
|
-
tags, VFR) are safe to fix mechanically. Judgement rows change the content:
|
|
55
|
-
duration (cut loses material, speed changes motion), aspect (crop loses
|
|
56
|
-
edges), fps (drops motion), loudness (ambience must not be boosted). Fix
|
|
57
|
-
those only when the user's request already implies the answer, otherwise
|
|
58
|
-
state the choice and its cost in one line. Mention WARNs; do not chase them.
|
|
59
|
-
6. **Verify the output.** Run `probe.py` on each result and confirm duration,
|
|
60
|
-
resolution, fps and audio match what was requested. Report those numbers to
|
|
61
|
-
the user (e.g. "final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo").
|
|
62
|
-
A step is done only when the script exited 0 and the output probes as
|
|
63
|
-
expected. Writing the command is not doing the job; a non-zero exit, a
|
|
64
|
-
missing or empty file, or a probe that contradicts the request is a
|
|
65
|
-
failure, and the report says so with the script's error message.
|
|
66
|
-
7. **Keep the user's originals.** Never overwrite the source file. Write new
|
|
67
|
-
files next to the input or where the user asked.
|
|
68
|
-
Set `FFMPEG_SKILL_NO_OVERWRITE=1` in the environment you run these scripts
|
|
69
|
-
in: an output path that already exists is then refused (`kind: input`)
|
|
70
|
-
instead of warned about, and `--overwrite` stays the one way to say "yes,
|
|
71
|
-
replace it". It is the recommended agent setting because an agent picking
|
|
72
|
-
output names cannot see which files the user already cares about — and it
|
|
73
|
-
is what 2.0 does by default.
|
|
74
|
-
8. **Look at the picture.** Whenever the picture changed (captions, overlays,
|
|
75
|
-
graphics, crop/pad, resize, colour, transitions, a `join.py` that scaled or
|
|
76
|
-
padded a clip to the first clip's frame, a `color.py --to-sdr` that tone-maps
|
|
77
|
-
an HDR source) run `look.py OUTPUT`
|
|
78
|
-
(contact sheet) or `look.py OUTPUT --at T`, view the PNG. The job is not
|
|
79
|
-
finished until the report's `Look:` line names that PNG; a probe alone
|
|
80
|
-
cannot see a caption sitting on someone's face. Audio-only jobs (sync,
|
|
81
|
-
loudness, silence, or any job whose input is an audio file) write
|
|
82
|
-
`Look: not needed`; there is no picture to inspect. What to look for
|
|
83
|
-
splits the same way `check.py`'s rows do in step 5:
|
|
84
|
-
- **Mechanical (verify and report as this skill's own job):** the
|
|
85
|
-
specified text/logo is present at the specified position, subtitles/text
|
|
86
|
-
appear at the specified timestamps, resolution has even dimensions.
|
|
87
|
-
Letterboxing/pillarboxing from `fit.py --fit pad` is the *correct*
|
|
88
|
-
result of that mode, not a defect — never flag it.
|
|
89
|
-
- **Judgement (report to the calling agent/user, don't silently pass or
|
|
90
|
-
fail):** whether a subject or face is cut off, whether text sits over a
|
|
91
|
-
face, whether colours look washed out, whether a transition "lands"
|
|
92
|
-
well or the edit feels cinematic. These require deciding what the
|
|
93
|
-
subject *is*, which belongs to the calling agent (see "What this skill
|
|
94
|
-
does and does not decide") — state what you see in one line and let the
|
|
95
|
-
calling agent or user judge it, don't decide it here.
|
|
96
|
-
If the execution environment cannot actually view images (no vision
|
|
97
|
-
capability), write `Look: PATH (pixels not inspected; agent has no image
|
|
98
|
-
view)` — never claim a picture was inspected when it wasn't, and don't
|
|
99
|
-
stall indefinitely waiting for a capability that isn't there.
|
|
14
|
+
## Workflow (always follow this order)
|
|
100
15
|
|
|
16
|
+
0. **Environment, not per job.** Run `python3 <skill-dir>/scripts/_contract.py doctor` (also `npx ffmpeg-skill doctor`; there is no doctor.py) in two cases only: before the first job on a machine you have not used this session, or after a tool failed with `kind: missing_tool`. It queries `ffmpeg -filters`/`-encoders`, so it is not a per-job step. Read `ok` and the tool's `usable`; if `usable` isn't `yes`, report the missing capability (a missing `libass`, `zscale` or encoder is the common case, e.g. `caption.py`) instead of discovering it through a runtime failure. `contract --json`'s full tool schema is for a *planning* agent choosing a tool from an abstract goal, not for this per-job workflow.
|
|
17
|
+
1. **Probe what you must plan from.** Run `probe.py` on each input you plan the edit from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks a question about a file. You do not need a separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output it wrote. Plan from real numbers, never assumptions.
|
|
18
|
+
2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; pass `--accurate` to `cut.py` only for frame-exact cuts.
|
|
19
|
+
3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's human-readable summary line, for any number after the plan (dimensions there can be a placeholder, not a computed preview — `docs/contract.md`). Use it to confirm a plan before long encodes and to report exact facts. `--fast` is preview quality (x264 veryfast), `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
|
|
20
|
+
4. **Chain in a sensible order.** Colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes (fit/crop) before captions and overlays, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18 (the default), `export.py` only for the last step. **Three or more steps: use `render.py` with a project.json** — one call, one JSON, one place for the user to change a number — rather than hand-chaining tools.
|
|
21
|
+
5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination the user named. Each row is `format` or `judgement`. Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix mechanically. Judgement rows change the content: duration (cut loses material, speed changes motion), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix those only when the request already implies the answer, otherwise state the choice and its cost in one line. Mention WARNs; do not chase them.
|
|
22
|
+
6. **Verify the output.** Confirm duration, resolution, fps and audio match the request — from the writing tool's own `--json`/`--json-brief` probe, or `probe.py` when you have none — and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected. Writing the command is not doing the job: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure, and the report says so with the script's error message.
|
|
23
|
+
7. **Keep the user's originals.** Never overwrite the source; write new files next to the input or where the user asked. Set `FFMPEG_SKILL_NO_OVERWRITE=1` in the environment you run these scripts in: an existing output path is then refused (`kind: input`) instead of warned about, and `--overwrite` stays the one way to say "yes, replace it". It is the recommended agent setting — an agent picking output names cannot see which files the user already cares about — and it is what 2.0 does by default.
|
|
24
|
+
8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` that scaled or padded a clip to the first clip's frame, a `color.py --to-sdr` that tone-maps an HDR source) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for a job about layout across the whole clip. The job is not finished until the report's `Look:` line names that PNG — a probe cannot see a caption sitting on someone's face. Audio-only jobs (sync, loudness, silence, or any job whose input is an audio file) write `Look: not needed`; there is no picture. What to look for splits like `check.py`'s rows in step 5:
|
|
25
|
+
- **Mechanical (this skill's own job to verify and report):** the specified text/logo is present at the specified position, subtitles/text appear at the specified timestamps, dimensions are even. Letterboxing/pillarboxing from `fit.py --fit pad` is the *correct* result of that mode, never a defect to flag.
|
|
26
|
+
- **Judgement (report it, don't silently pass or fail):** whether a subject or face is cut off, whether text sits over a face, whether colours look washed out, whether a transition lands. These need deciding what the subject *is*, which belongs to the calling agent (see "What this skill does and does not decide") — say what you see in one line and let them judge it.
|
|
27
|
+
With no vision capability, write `Look: PATH (pixels not inspected; agent has no image view)` — never claim a picture was inspected when it wasn't, and don't stall waiting for a capability that isn't there.
|
|
101
28
|
|
|
102
29
|
## Before you run anything: what to ask, what to assume
|
|
103
30
|
|
|
104
|
-
Ask one short question only when the answer changes the output materially and the request does not imply it. When several things are open at once (a vague "make it for social media" leaves destination, aspect method, length and captions unresolved),
|
|
31
|
+
Ask one short question only when the answer changes the output materially and the request does not imply it. When several things are open at once (a vague "make it for social media" leaves destination, aspect method, length and captions unresolved), don't ask them one per turn: propose one bundle with your defaults and let the user change any part ("Reels: 9:16 with padding, trimmed to 60 s, -14 LUFS, no captions — OK, or change something?"). One question, one answer, then the run. Never ask for what `probe.py` can tell you.
|
|
105
32
|
|
|
106
|
-
- **Destination** decides aspect, length limit, loudness and codec
|
|
107
|
-
- **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, and
|
|
108
|
-
- **Captions** without a text source:
|
|
33
|
+
- **Destination** decides aspect, length limit, loudness and codec; "for Reels" answers all four. No destination named and a plain cut/caption: keep the source format and say so. If the user says "export", "post" or "deliver", ask where.
|
|
34
|
+
- **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, and say which you chose. Ask when the content is a talk (trimming loses words) and the change is large.
|
|
35
|
+
- **Captions** without a text source: `--transcribe` if a local whisper exists, otherwise ask for the text or a timed file; never invent dialogue.
|
|
109
36
|
- **Fonts and brand**: if the user mentions a brand, colours or "our font", ask for or create `brand.json` once and reuse it.
|
|
110
|
-
- **CJK / non-Latin text**: check
|
|
111
|
-
- **Crop position** for `--fit crop`:
|
|
112
|
-
- Anything else (transition type, caption style): pick the conventional default, say what you picked,
|
|
113
|
-
|
|
114
|
-
Do not ask for things `probe.py` can tell you.
|
|
37
|
+
- **CJK / non-Latin text**: check a font exists before rendering (`fc-list :lang=ja file` / `:lang=ko` / `:lang=zh`) and pass it with `--font "Name"` or `--font-file /path.ttf`. Tofu boxes are a failed job, not a style.
|
|
38
|
+
- **Crop position** for `--fit crop`: centre by default, but when the request or the source names an off-centre subject ("keep the product on the right", "don't cut off my hands", someone visibly off-centre in the sheet) use `--crop-x`/`--crop-y` (0=left/top, 1=right/bottom) instead of a silent centre guess. Ask which edge to keep when the sheet shows the subject near an edge and the request doesn't say.
|
|
39
|
+
- Anything else (transition type, caption style): pick the conventional default, say what you picked, offer the alternative in one line.
|
|
115
40
|
|
|
116
41
|
## What this skill does and does not decide
|
|
117
42
|
|
|
118
|
-
This skill cuts, joins, measures, syncs, exports and checks files — it executes an edit, it does not decide one.
|
|
43
|
+
This skill cuts, joins, measures, syncs, exports and checks files — it executes an edit, it does not decide one. What belongs to the human, the calling agent or another skill:
|
|
119
44
|
|
|
120
|
-
- **Which cut is
|
|
121
|
-
- **What makes a highlight interesting** — `scenes.py --highlights` ranks by a measured proxy (audio energy
|
|
122
|
-
- **Thumbnail or cover
|
|
123
|
-
- **Understanding what a video is *about*** —
|
|
124
|
-
- **Judging what looks good** — "apply this LUT"
|
|
125
|
-
- **Picking a subject or region
|
|
45
|
+
- **Which cut is right, or whether a deliverable is approvable** — this skill measures and reports (`check.py`'s PASS/WARN/FAIL, `cut.py`'s measured duration error); the user or a production agent decides whether that ships.
|
|
46
|
+
- **What makes a highlight interesting** — `scenes.py --highlights` ranks by a measured proxy (audio energy, scene duration), never by understanding content: candidates, not a verdict.
|
|
47
|
+
- **Thumbnail or cover composition** — a design decision, not a measurement.
|
|
48
|
+
- **Understanding what a video is *about*** — there is no transcription or vision here beyond `look.py`'s contact sheets, which exist for the calling agent's eyes, not for this skill to interpret.
|
|
49
|
+
- **Judging what looks good** — "apply this LUT", "correct exposure by +0.3 stops" (`color.py`) is mechanical and belongs here; "grade this scene to look cinematic" belongs to a colour-grading skill ([`color-grading-skill`](https://github.com/kajisho5/color-grading-skill)) that decides the parameters and then calls `color.py`.
|
|
50
|
+
- **Picking a subject or region you were not given** — "crop to x=200,y=0" (`crop.py`/`fit.py --crop-x/-y`) is mechanical once the box is known; "crop to keep the speaker in frame" needs deciding *what* the speaker is — a judgement for the calling agent (from a `look.py` sheet) or a motion-graphics skill.
|
|
126
51
|
|
|
127
|
-
The line
|
|
52
|
+
The line: same input + same explicit parameters always producing the same verifiable output belongs here; anything that depends on taste, content understanding or what looks or sounds good belongs to whoever makes that judgement. This skill executes parameters it is given, never infers them from what something looks or sounds like.
|
|
128
53
|
|
|
129
54
|
If a request needs an FFmpeg feature none of the 42 scripts expose, say so and name the closest built-in option (`--dry-run` to show what would run, or a documented limitation) — never fall back to guessing a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards); it is exactly the failure mode this skill exists to prevent, so it is never the fallback when a script's flag doesn't cover something.
|
|
130
55
|
|
|
131
56
|
## Request → script
|
|
132
57
|
|
|
133
|
-
This table and `doctor`'s
|
|
58
|
+
This table and `doctor --json`'s `tools` list are the source of truth for what exists: name only a script you have seen in one of them, never a plausible-sounding one (there is no `doctor.py`, no `trim.py`, no `subtitle.py`).
|
|
134
59
|
|
|
135
|
-
|
|
60
|
+
Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`, and the times in cue and chapter files — take seconds, `mm:ss(.fff)`, `hh:mm:ss(.fff)` or four-part SMPTE `hh:mm:ss:ff`, with `@fps` naming the rate (`00:01:02:15@29.97`); tolerance-style flags that are a length rather than a point in time (`--min-silence`, `--margin`, `--min-keep`, `--fade`) are plain seconds. Use the timecode forms when the user pastes an editor's timecode list or an NLE cue sheet, so nothing is converted by hand on the way in.
|
|
136
61
|
|
|
137
62
|
| User says | Do |
|
|
138
63
|
|-----------|----|
|
|
139
|
-
| "what's in this file", "how long is it"
|
|
140
|
-
| "cut from 1:20 to 2:05", "trim the first 10
|
|
64
|
+
| "what's in this file", "how long is it" | `probe.py input.mp4` |
|
|
65
|
+
| "cut from 1:20 to 2:05", "trim the first 10 s" | `cut.py input.mp4 --start 1:20 --end 2:05` |
|
|
141
66
|
| "keep only these parts", "remove the middle" | `cut.py input.mp4 --segments 0-1:00,1:30-2:00` |
|
|
142
|
-
| "make it exactly 60 seconds"
|
|
143
|
-
| "make it vertical /
|
|
144
|
-
| "resize to a
|
|
145
|
-
| "crop to this exact box
|
|
146
|
-
| "are there black bars on this?"
|
|
147
|
-
| "this old footage is interlaced
|
|
148
|
-
| "
|
|
149
|
-
| "blur/pixelate this face/plate
|
|
150
|
-
| "
|
|
151
|
-
| "the horizon is tilted
|
|
152
|
-
| "turn this image into a
|
|
153
|
-
| "slow zoom on a photo", "Ken Burns
|
|
154
|
-
| "rotate
|
|
155
|
-
| "reverse this clip"
|
|
67
|
+
| "make it exactly 60 seconds" | `fit.py input.mp4 --duration 60` (speed) or `--method trim` |
|
|
68
|
+
| "make it vertical / 9:16 / square" | `fit.py input.mp4 --aspect 9:16 --fit pad` (or `--fit crop`; `--pad-fill blur` for blurred bars) |
|
|
69
|
+
| "resize to a height/width" | `fit.py input.mp4 --height 1080` (or `--width`, or both for an exact frame) |
|
|
70
|
+
| "crop to this exact box" (known x/y/w/h) | `crop.py input.mp4 --x 100 --y 0 --width 1080 --height 1920` |
|
|
71
|
+
| "are there black bars on this?" | `cropdetect.py input.mp4` |
|
|
72
|
+
| "this old footage is interlaced" | `deinterlace.py input.mp4` |
|
|
73
|
+
| "it's grainy/noisy, clean it up" | `denoise.py input.mp4 --strength medium` |
|
|
74
|
+
| "blur/pixelate this face/plate" (known box) | `redact.py input.mp4 --x 820 --y 140 --width 240 --height 240 --mode pixelate` |
|
|
75
|
+
| "flat view out of this 360 video" (known yaw/pitch/fov) | `sphere.py insta360.mp4 --yaw 90 --pitch 0 --h-fov 100 --v-fov 70` |
|
|
76
|
+
| "the horizon is tilted" (known degrees) | `straighten.py tilted.mp4 --degrees -2.5` |
|
|
77
|
+
| "turn this image into a clip", "title card" | `insert.py title.png --duration 3` |
|
|
78
|
+
| "slow zoom on a photo", "Ken Burns" | `insert.py photo.jpg --duration 6 --zoom in --pan right --width 1920 --height 1080` |
|
|
79
|
+
| "rotate 90 degrees", "mirror it" | `fit.py input.mp4 --rotate 90` / `fit.py input.mp4 --flip h` |
|
|
80
|
+
| "reverse this clip" | `reverse.py input.mp4` |
|
|
156
81
|
| "stabilize this shaky footage" | `stabilize.py input.mp4` |
|
|
157
82
|
| "make a blank/colour background clip" | `background.py -o bg.mp4 --duration 3 --width 1920 --height 1080 --color 0x101010` |
|
|
158
83
|
| "turn these numbered frames into a video" | `sequence.py --dir frames --pattern "frame_%04d.png" --fps 24` |
|
|
159
|
-
| "
|
|
160
|
-
| "hold on this frame
|
|
161
|
-
| "add
|
|
162
|
-
| "speed up here,
|
|
163
|
-
| "loop this
|
|
164
|
-
| "cut to the product shot
|
|
165
|
-
| "add chapters
|
|
166
|
-
| "set the title / artist / comment
|
|
167
|
-
| "put these videos in a 4x2 grid
|
|
84
|
+
| "waveform/spectrum video for this track" | `waveform.py podcast.wav -o waveform.mp4` |
|
|
85
|
+
| "hold on this frame", "freeze the last frame" | `freeze.py clip.mp4 --hold 2` |
|
|
86
|
+
| "add black at the start" | `pad.py clip.mp4 --start 1.5` |
|
|
87
|
+
| "speed up here, slow-mo there" (known segments) | `speedramp.py action.mp4 --segment 0-3:1.0 --segment 3-4:0.25 --segment 4-8:2.0` |
|
|
88
|
+
| "loop this clip to fill 30 seconds" | `loop.py bg_loop.mp4 --duration 30` |
|
|
89
|
+
| "cut to the product shot 0:12-0:16", "B-roll over this bit" | `broll.py talk.mp4 --insert product.mp4 --at 12 --end 16` (repeat `--insert/--at`; `--audio b\|mix`) |
|
|
90
|
+
| "add chapters", "chapter markers for YouTube" | `metadata.py episode.mp4 --chapters chapters.txt` (`TIME TITLE` per line; streams copied) |
|
|
91
|
+
| "set the title / artist / comment" | `metadata.py episode.mp4 --title "Episode 12" --artist "Studio"` |
|
|
92
|
+
| "put these videos in a 4x2 grid" | `grid.py t1.mp4 ... t8.mp4 --cols 4 --rows 2` |
|
|
168
93
|
| "add subtitles from this SRT", "burn in captions" | `caption.py input.mp4 --srt subs.srt` |
|
|
169
|
-
| "caption it with these lines" (
|
|
170
|
-
| "
|
|
171
|
-
| "
|
|
172
|
-
| "
|
|
173
|
-
| "
|
|
174
|
-
| "remove the green screen"
|
|
175
|
-
| "sync the lav mic
|
|
176
|
-
| "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5`
|
|
177
|
-
| "cut this and make it HEVC / AV1
|
|
178
|
-
| "export for YouTube / Reels / X", "
|
|
94
|
+
| "caption it with these lines" (text with times) | `caption.py input.mp4 --text cues.txt` |
|
|
95
|
+
| "keep the subtitles toggleable", "mux in an SRT" | `caption.py input.mp4 --srt subs.srt --mode mux` |
|
|
96
|
+
| "our logo top-right", "a watermark" | `overlay.py input.mp4 --image logo.png --position top-right --scale 200` |
|
|
97
|
+
| "a title for the first 4 seconds" | `overlay.py input.mp4 --text "Title" --position top --start 0 --end 4 --fade 0.4` |
|
|
98
|
+
| "webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` |
|
|
99
|
+
| "remove the green screen" | `overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00` |
|
|
100
|
+
| "sync the lav mic", "line up two cameras" | `sync.py camera.mp4 mic.wav --replace-audio` / `sync.py camA.mp4 camB.mp4 --trim-second` |
|
|
101
|
+
| "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` podcast, `-I -23` broadcast) |
|
|
102
|
+
| "cut this and make it HEVC / AV1 / ProRes" (output codec named) | `cut.py input.mp4 --start 0:10 --end 0:40 --codec hevc` (`--codec`/`--quality` on any re-encoding tool; ProRes needs `-o NAME.mov`) |
|
|
103
|
+
| "export for YouTube / Reels / X", "a ProRes master" | `export.py input.mp4 --preset youtube\|reels\|x\|prores\|h265` (`--normalize` hits the loudness spec in the same call) |
|
|
179
104
|
| "make a GIF preview" | `export.py input.mp4 --preset gif` |
|
|
180
|
-
| "
|
|
181
|
-
| "cut out the pauses
|
|
182
|
-
| "stitch these clips
|
|
183
|
-
| "show me what it looks like", "
|
|
105
|
+
| "a small proxy / cheap preview file" | `proxy.py input.mp4 [--width 640 --no-audio]` — not a delivery preset (those are `export.py`) |
|
|
106
|
+
| "cut out the pauses", "jump cuts" | `silence.py input.mp4 [--threshold -40 --min-silence 0.8]` |
|
|
107
|
+
| "stitch these clips", "add a crossfade" | `join.py a.mp4 b.mp4 c.mp4 --transition fade --duration 0.5` |
|
|
108
|
+
| "show me what it looks like", "are the captions readable" | `look.py output.mp4 --tiles 3x2` then view the PNG |
|
|
184
109
|
| "what would you run?", "don't render yet" | any script with `--dry-run` |
|
|
185
|
-
| "
|
|
186
|
-
| "is this OK to upload?"
|
|
187
|
-
| "
|
|
188
|
-
| "
|
|
189
|
-
| "use our brand fonts/colours/logo" |
|
|
110
|
+
| "a 60 s highlight from this hour" | `scenes.py long.mp4 --highlights 6 --target 60 --edl picks.txt` → `cut.py --segments` |
|
|
111
|
+
| "is this OK to upload?" | `check.py final.mp4 --platform reels` |
|
|
112
|
+
| "several changes to the same edit", 3+ steps | `render.py --init project.json`, edit, `render.py project.json` |
|
|
113
|
+
| "a lower third with my name", "countdown intro", "progress bar" | `graphics.py input.mp4 --template lower-third --name "..." --title "..." --start 2 --end 8` |
|
|
114
|
+
| "use our brand fonts/colours/logo" | `--brand brand.json` on caption/overlay/graphics, or `"brand"` in project.json |
|
|
190
115
|
| "send me a summary of what you did" | `report.py --before raw.mov --after final.mp4 --platform youtube -o report.html` |
|
|
191
|
-
| "do this to every file in the folder"
|
|
192
|
-
| "transcribe it and caption it" | `caption.py input.mp4 --transcribe --animate pop --karaoke` (needs a local whisper;
|
|
116
|
+
| "do this to every file in the folder" | `batch.py FOLDER --recipe batch.json` (steps or a render project; cached) |
|
|
117
|
+
| "transcribe it and caption it" | `caption.py input.mp4 --transcribe --animate pop --karaoke` (needs a local whisper; else `--text`) |
|
|
193
118
|
| "three cameras, cut between them" | `multicam.py camA.mp4 camB.mp4 camC.mp4 --switch "0-20:0,20-40:1,40-60:2"` |
|
|
194
|
-
| "
|
|
195
|
-
| "does it look like Log / S-Log
|
|
119
|
+
| "iPhone Dolby Vision clip looks wrong" | `color.py clip.mov --to-sdr` or `--strip-dovi` (keep HDR, drop the DV layer) |
|
|
120
|
+
| "does it look like Log / S-Log?" | `probe.py clip.mp4 --analyze` (`looks_like_log`) then `color.py --lut` |
|
|
196
121
|
| "test the tool on my real files" | `verify.py ~/Footage --report verify.md` |
|
|
197
122
|
| "show me progress", "quick preview first" | any encoding script with `--progress` and/or `--fast` |
|
|
198
|
-
| "the colours look washed out /
|
|
199
|
-
| "apply this LUT", "convert the S-Log
|
|
200
|
-
| "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (stream copy; re-encodes only if the copy can't carry
|
|
201
|
-
| "brighten it
|
|
202
|
-
| "clean up the audio", "remove the hiss
|
|
123
|
+
| "the colours look washed out / iPhone HDR" | `color.py input.mov --to-sdr` (probe shows `hdr: true`) |
|
|
124
|
+
| "apply this LUT", "convert the S-Log footage" | `color.py input.mp4 --lut grade.cube [--lut-strength 0.7]` |
|
|
125
|
+
| "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (stream copy; re-encodes only if the copy can't carry it — see `reencoded`) |
|
|
126
|
+
| "brighten it / punch up contrast / fix white balance" | `color.py input.mp4 --correct --exposure 0.3 --contrast 1.1 --saturation 1.05 --temperature 5600 --tint -0.05` |
|
|
127
|
+
| "clean up the audio", "remove the hiss" | `audio.py input.mp4 --voice` (speech) or `--denoise` |
|
|
203
128
|
| "add background music under the talking" | `audio.py input.mp4 --music bed.mp3 --duck --fade-out 3` |
|
|
204
129
|
| "convert the 5.1 to stereo" | `audio.py input.mov --downmix` |
|
|
205
130
|
| "swap in the narration track" | `audio.py input.mp4 --replace narration.wav` |
|
|
206
|
-
| "pull the audio out
|
|
207
|
-
| "compress the voice", "limit the peaks
|
|
131
|
+
| "pull the audio out", "give me the sound as WAV" | `audio.py input.mp4 -o input.wav` (an audio extension drops the picture; `--audio-stream 1` picks a track) |
|
|
132
|
+
| "compress the voice", "limit the peaks", "gate the room noise" | `audio.py input.mp4 --compress --comp-threshold -20 --comp-ratio 4` / `--limit --limit-ceiling -1` / `--gate --gate-threshold -45` |
|
|
208
133
|
| "the audio drifts out of sync over the hour" | `sync.py camera.mp4 recorder.wav --fix-drift --replace-audio` |
|
|
209
134
|
| "smooth slow motion", "half speed but fluid" | `fit.py input.mp4 --duration 2x --smooth interpolate` (slow) or `--smooth blend` |
|
|
210
|
-
| "TikTok-style captions with the words popping
|
|
211
|
-
| "it's a phone video with variable frame rate" | nothing extra:
|
|
212
|
-
|
|
135
|
+
| "TikTok-style captions with the words popping" | `caption.py input.mp4 --text cues.txt --animate pop --karaoke` |
|
|
136
|
+
| "it's a phone video with variable frame rate" | nothing extra: re-encodes conform VFR to constant fps; `fit.py --fps 30` picks the rate |
|
|
213
137
|
|
|
214
138
|
## Audio-only files
|
|
215
139
|
|
|
216
|
-
Audio
|
|
217
|
-
|
|
218
|
-
podcast` all accept WAV, FLAC, MP3, M4A/AAC, OGG and Opus (any container ffmpeg
|
|
219
|
-
can read) and write the codec that fits the output extension, so the same
|
|
220
|
-
commands work with `talk.wav` in place of `talk.mp4`. What changes:
|
|
221
|
-
|
|
222
|
-
- The output extension picks the format: `-o out.mp3` converts, `-o out.wav`
|
|
223
|
-
keeps PCM, `-o out.m4a` writes AAC. `audio.py in.wav -o out.mp3` with no
|
|
224
|
-
other flag is a plain conversion.
|
|
225
|
-
- `cut.py` stream-copies audio too, so trims land on a packet boundary
|
|
226
|
-
(`precision: packet`, a few ms; the JSON reports `duration_error_ms`). Pass
|
|
227
|
-
`--accurate` for a sample-exact trim: `precision: sample` when the output is
|
|
228
|
-
PCM or FLAC, `codec_frame` when a lossy codec (AAC, MP3, Opus) frames it
|
|
229
|
-
again. A `.wav` output is always PCM, never AAC packets inside a WAV.
|
|
230
|
-
- `join.py` joins audio-only clips as audio (`acrossfade` or a butt join) at
|
|
231
|
-
one sample rate and channel layout; the output must have an audio extension.
|
|
232
|
-
Video and audio clips cannot be mixed in one join.
|
|
233
|
-
- An audio extension on a video input (`audio.py talk.mp4 -o talk.wav`,
|
|
234
|
-
`cut.py talk.mp4 --start 1:00 --end 2:00 -o part.wav`) extracts the audio; the
|
|
235
|
-
output has no video stream. `audio.py --audio-stream N` picks a track when
|
|
236
|
-
`probe` lists several under `audio_streams`.
|
|
237
|
-
- `Look: not needed` in the report; `Check:` still applies for loudness
|
|
238
|
-
(`check.py file.wav --platform podcast` measures LUFS and true peak).
|
|
239
|
-
- Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`,
|
|
240
|
-
`color`, `export`, `scenes`, `look`) refuse an audio file with
|
|
241
|
-
"input has no video stream". Say so instead of forcing a video wrapper.
|
|
242
|
-
|
|
243
|
-
| User says (audio file) | Do |
|
|
244
|
-
|-----------|----|
|
|
245
|
-
| "normalise this WAV to -14 LUFS", "podcast levels" | `loudness.py talk.wav -I -14 --tp -1 -o talk_norm.wav` (`-I -16 --tp -1.5` for podcasts) |
|
|
246
|
-
| "remove the silence from this recording" | `silence.py talk.wav -o talk_tight.wav` |
|
|
247
|
-
| "clean up the noise in this M4A" | `audio.py talk.m4a --voice -o talk_clean.m4a` (speech) or `--denoise` |
|
|
248
|
-
| "convert this WAV to MP3" | `audio.py talk.wav -o talk.mp3` |
|
|
249
|
-
| "trim this audio from 00:30 to 02:00" | `cut.py talk.wav --start 0:30 --end 2:00 -o talk_cut.wav` (`--accurate` for sample-exact) |
|
|
250
|
-
| "join these recordings", "intro + episode + outro" | `join.py intro.wav episode.m4a outro.wav -o full.flac` (`--transition none` for a butt join) |
|
|
251
|
-
| "extract the audio from the video", "mp4 to wav" | `audio.py talk.mp4 -o talk.wav` (`--voice -o talk.m4a` to clean it on the way) |
|
|
252
|
-
| "compress / limit / gate the voice" | `audio.py talk.wav --compress --comp-threshold -20 --comp-ratio 4 --limit --limit-ceiling -1 -o talk_dyn.wav` |
|
|
253
|
-
| "is this loud enough for Apple Podcasts?" | `check.py talk.m4a --platform podcast` |
|
|
140
|
+
Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`, `audio.py`, `sync.py` and `check.py --platform podcast` take WAV, FLAC, MP3, M4A/AAC, OGG and Opus, and the output extension picks the format. `Look: not needed` in the report; `Check:` still applies (`check.py file.wav --platform podcast`). Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" — say so instead of forcing a video wrapper. The same commands work with `talk.wav` in place of `talk.mp4` (see the table above); the audio-specific recipes, packet vs sample precision, joining and extracting one track: `references/gotchas.md#audio-only-files`.
|
|
141
|
+
|
|
254
142
|
|
|
255
143
|
## Report format
|
|
256
144
|
|
|
257
|
-
Reply in the language the request itself is written in
|
|
145
|
+
Reply in the language the request itself is written in — the language of the user's own sentences, not a language the request talks about (a request for subtitles in another language is still answered in the language it was written in) and not the language of a tool's error text or file names. Keep the field labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English: they read like log fields and stay recognisable across languages. Everything around them — the sentences, any question, any explanation of a judgement call — is in the user's language. Never default to English because the tool names and flags are English, and never drift into another language because the job was short or the report is a failure: a one-line "file does not exist" is written in the request's language too. A mid-conversation switch follows the user's latest message, not the first one. This holds for a one-command job: a three-second audio trim answered with English labels, numbers and one Japanese word in `Notes:` is an English report — the `Done:` description (what was cut, from where) and `Steps:` are in the user's language even when the values are technical.
|
|
258
146
|
|
|
259
|
-
Finish every job with this shape (numbers from `probe.py`/`check.py`, not memory):
|
|
147
|
+
Finish every job with this shape (numbers from a tool's `--json` or `probe.py`/`check.py`, not memory):
|
|
260
148
|
|
|
261
149
|
```
|
|
262
150
|
Done: final.mp4 — 59.98 s, 1080x1920, 30 fps, H.264, AAC stereo, -14.1 LUFS
|
|
@@ -266,9 +154,7 @@ Look: final_sheet.png (captions inside the safe area, logo top-right)
|
|
|
266
154
|
Notes: source was VFR, conformed to 30 fps; audio was mono, made stereo
|
|
267
155
|
```
|
|
268
156
|
|
|
269
|
-
The same five lines for a Japanese request, prose in Japanese around the English labels
|
|
270
|
-
(this is the shape a short job keeps too; English `Done:`/`Steps:` sentences with one Japanese
|
|
271
|
-
word in `Notes:` is not a Japanese report):
|
|
157
|
+
The same five lines for a Japanese request, prose in Japanese around the English labels (the shape a short job keeps too; English `Done:`/`Steps:` sentences with one Japanese word in `Notes:` is not a Japanese report):
|
|
272
158
|
|
|
273
159
|
```
|
|
274
160
|
Done: final.mp4 — 59.98 秒、1080x1920、30 fps、H.264、AAC ステレオ、-14.1 LUFS
|
|
@@ -290,73 +176,23 @@ Look: not needed (nothing written)
|
|
|
290
176
|
Notes: send a valid .cube, or say if you want the clip left as is
|
|
291
177
|
```
|
|
292
178
|
|
|
293
|
-
A refusal (
|
|
179
|
+
A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run (usually only probe), `Look: not needed`. Both keep the five labels so a failed report scans like a successful one — including the shortest failure: a missing input or an invalid LUT still gets all five lines, never prose headings in their place. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change that would make the retry meaningful.
|
|
294
180
|
|
|
295
181
|
Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message, do not paraphrase it into a success.
|
|
296
182
|
|
|
297
183
|
## Things that look right but are wrong
|
|
298
184
|
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
-
|
|
302
|
-
-
|
|
303
|
-
-
|
|
304
|
-
-
|
|
305
|
-
|
|
306
|
-
|
|
307
|
-
- Captions burned before a crop/resize
|
|
308
|
-
-
|
|
309
|
-
-
|
|
310
|
-
-
|
|
311
|
-
-
|
|
312
|
-
-
|
|
313
|
-
- `scenes.py --highlights` defaults to the loudest scenes (`--rank-by audio`): a quiet but important moment (a confession, a punchline landing in silence) is skipped, and pure crowd noise or a mic bump can outrank it. `--rank-by duration` picks the longest unbroken scenes instead. Neither is "the best parts" — check the contact sheet (`--sheet`) before treating the picks as final.
|
|
314
|
-
|
|
315
|
-
## Gotchas
|
|
316
|
-
|
|
317
|
-
- **Variable frame rate (phone/screen recordings).** `probe.py` sets
|
|
318
|
-
`variable_frame_rate_suspected` when `r_frame_rate` and `avg_frame_rate`
|
|
319
|
-
disagree. Every re-encoding script then adds `-fps_mode cfr` at the source's
|
|
320
|
-
average rate, and `cut.py` switches itself to `--accurate` (copy-cuts on VFR
|
|
321
|
-
are unreliable). Pick the rate explicitly with `fit.py --fps 30|60` when the
|
|
322
|
-
average is odd (e.g. 23.4 fps from dropped frames).
|
|
323
|
-
- **Audio drift / sync.** Don't mix files with different frame rates or sample
|
|
324
|
-
rates in one `cut.py --segments` join without re-encoding (`--accurate`).
|
|
325
|
-
After `sync.py`, verify by running it again on the output: offset (and drift
|
|
326
|
-
ppm with `--fix-drift`) should be ~0. Recordings longer than ~10 minutes from
|
|
327
|
-
separate devices: always use `--fix-drift`.
|
|
328
|
-
- **Colour.** SDR outputs are H.264 tagged BT.709 `yuv420p`. When `probe.py`
|
|
329
|
-
reports `hdr: true` (HDR10/PQ, HLG, Dolby Vision, BT.2020), every editing
|
|
330
|
-
script keeps the output HDR (HEVC Main10, source colour tags) so nothing is
|
|
331
|
-
silently flattened. Decide with the user: keep HDR (fine for YouTube/phones)
|
|
332
|
-
or run `color.py --to-sdr` first for SDR-only destinations, LUT work or
|
|
333
|
-
H.264 deliverables. `hdr: true` counts BT.2020 primaries too, so it is also
|
|
334
|
-
true for a wide-gamut SDR file; `hdr_signal: true` is the narrower fact —
|
|
335
|
-
a real PQ / HLG / Dolby Vision transfer — and `hdr_format` names the
|
|
336
|
-
in-between case (`BT.2020 SDR`). `export.py` platform presets are SDR and warn on HDR
|
|
337
|
-
input. iPhone `.mov` files also carry timecode/metadata tracks; scripts map
|
|
338
|
-
only the first audio track, so extra tracks are dropped on re-encode.
|
|
339
|
-
For Log footage (S-Log, V-Log, C-Log: looks grey and low-contrast but is
|
|
340
|
-
tagged SDR) run `probe.py --analyze`; `looks_like_log: true` means apply the
|
|
341
|
-
manufacturer's `.cube` with `color.py --lut` before anything else. Keep
|
|
342
|
-
ProRes masters at source colour: `export.py --preset prores` does not retag.
|
|
343
|
-
- **CJK and other non-Latin text.** libass and drawtext need a font that has
|
|
344
|
-
the glyphs. Check with `fc-list | grep -i cjk`. Then either name it
|
|
345
|
-
(`caption.py --font "Noto Sans CJK JP"`) or point at the file
|
|
346
|
-
(`overlay.py --font-file /usr/share/fonts/opentype/noto/NotoSansCJK-Regular.ttc`,
|
|
347
|
-
`caption.py --fonts-dir ./fonts --font "Noto Sans CJK JP"`). Without a
|
|
348
|
-
matching font you get boxes, not an error. Install: `apt install fonts-noto-cjk`,
|
|
349
|
-
`brew install --cask font-noto-sans-cjk`.
|
|
350
|
-
- **Windows drawtext crashes on some real builds** (#100): the drawtext tools
|
|
351
|
-
resolve a concrete `--font-file` by default, which avoids it; if one still
|
|
352
|
-
crashes, pass `--font-file` explicitly. Details: `references/ci-platform-pitfalls.md`.
|
|
353
|
-
- **Keyframe cuts.** A lossless `cut.py` result may start up to one GOP (often
|
|
354
|
-
1–10 s) earlier than requested; the script re-encodes automatically when the
|
|
355
|
-
deviation exceeds 0.5 s. If the user insists on lossless output, pass
|
|
356
|
-
`--tolerance -1` and tell them the cut lands on the nearest earlier keyframe.
|
|
357
|
-
- **Rotation metadata.** Phone footage often has a `rotation` tag; `probe.py`
|
|
358
|
-
reports it and `fit.py` accounts for it when computing the output frame.
|
|
359
|
-
- **Odd dimensions.** `yuv420p` needs even width/height; `fit.py` and
|
|
360
|
-
`export.py` round to even values automatically.
|
|
361
|
-
- **Speed.** Re-encodes use x264 `medium`. For long files add `--preset veryfast`
|
|
362
|
-
to intermediates and keep the default for the final export.
|
|
185
|
+
One line each; the full version is `references/gotchas.md`, worth reading before a job in that area.
|
|
186
|
+
|
|
187
|
+
- HDR (iPhone, HDR10) re-encoded through an SDR path goes flat; the scripts keep HDR, and `hdr: true` is wider than `hdr_signal: true` (a real PQ/HLG/Dolby Vision transfer). Details: [#hdr-and-colour](references/gotchas.md#hdr-and-colour)
|
|
188
|
+
- Log footage (S-Log/V-Log/C-Log) is tagged SDR and looks grey: `probe.py --analyze`, then `color.py --lut` before anything else. Details: [#log-footage](references/gotchas.md#log-footage)
|
|
189
|
+
- A `-c copy` cut can start on a wrong or frozen frame; `cut.py` re-encodes past a 0.5 s snap — respect it. Details: [#keyframe-cuts](references/gotchas.md#keyframe-cuts)
|
|
190
|
+
- VFR phone/screen recordings: re-encodes conform to CFR, `cut.py` switches to `--accurate`; pick the rate with `fit.py --fps` when the average is odd. Details: [#variable-frame-rate](references/gotchas.md#variable-frame-rate)
|
|
191
|
+
- Sync/multicam `confidence` under 0.3 (or a huge offset) is probably wrong — check every camera, and remember these align audio, never lip sync. Details: [#sync-multicam-and-drift](references/gotchas.md#sync-multicam-and-drift)
|
|
192
|
+
- "Normalised" audio can still clip (check true peak), and ambience at -40 LUFS or below must never be raised to a speech target. Details: [#loudness-and-ambience](references/gotchas.md#loudness-and-ambience)
|
|
193
|
+
- Captions burned before a crop/resize land off-frame; burned small then upscaled by `export.py` they come out soft — fit to the delivery size first. Non-Latin text without a real font renders boxes, not an error. Details: [#captions-fonts-and-text-order](references/gotchas.md#captions-fonts-and-text-order)
|
|
194
|
+
- `--fit crop` 16:9 → 9:16 throws away 70 % of the width, 60→30 fps halves the motion, and "60 seconds" by speed or by trim are different answers — say which and why. Details: [#reframing-fps-and-duration](references/gotchas.md#reframing-fps-and-duration)
|
|
195
|
+
- `yuv420p` needs even dimensions and phone rotation tags are honoured, both automatically. Details: [#dimensions-and-rotation](references/gotchas.md#dimensions-and-rotation)
|
|
196
|
+
- `scenes.py --highlights` ranks by loudness (or duration), never by meaning: check the sheet before treating picks as final. Details: [#highlights](references/gotchas.md#highlights)
|
|
197
|
+
- Three hand-chained re-encodes should be one `render.py` project; re-encodes use x264 `medium`. Details: [#chaining-and-speed](references/gotchas.md#chaining-and-speed)
|
|
198
|
+
- Windows drawtext crashes on some builds (#100): pass `--font-file` explicitly if one does. Details: `references/ci-platform-pitfalls.md`
|
package/docs/contract.md
CHANGED
|
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
|
|
|
21
21
|
| Field | Meaning | Changes when |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
|
|
24
|
-
| `skill.version` | the npm / package.json version (`1.
|
|
24
|
+
| `skill.version` | the npm / package.json version (`1.11.0`) | any release |
|
|
25
25
|
|
|
26
26
|
A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
|
|
27
27
|
ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
|
|
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
|
|
|
88
88
|
|
|
89
89
|
| What 2.0 removes | Since | Replacement | To be ready today |
|
|
90
90
|
|---|---|---|---|
|
|
91
|
-
| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.
|
|
92
|
-
| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.
|
|
93
|
-
| `json` and `progress` in the MCP `inputSchema` | 1.
|
|
94
|
-
| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.
|
|
95
|
-
| Overwriting an existing output with only a warning | 1.
|
|
91
|
+
| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.11.0 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
|
|
92
|
+
| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.11.0 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
|
|
93
|
+
| `json` and `progress` in the MCP `inputSchema` | 1.11.0 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
|
|
94
|
+
| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.11.0 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
|
|
95
|
+
| Overwriting an existing output with only a warning | 1.11.0 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
|
|
96
96
|
|
|
97
97
|
## Skill
|
|
98
98
|
|
|
99
99
|
```json
|
|
100
100
|
{
|
|
101
101
|
"contract_version": "1.0",
|
|
102
|
-
"deprecated": [{"what": "...", "since": "1.
|
|
103
|
-
"skill": {"id": "ffmpeg-skill", "version": "1.
|
|
102
|
+
"deprecated": [{"what": "...", "since": "1.11.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
|
|
103
|
+
"skill": {"id": "ffmpeg-skill", "version": "1.11.0", "execution_mode": "local", "kind": "execution",
|
|
104
104
|
"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
|
|
105
105
|
"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
|
|
106
106
|
"requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
|
|
@@ -128,6 +128,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
|
|
|
128
128
|
| `output_schema` | what `--json` prints on stdout |
|
|
129
129
|
| `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
|
|
130
130
|
| `supports_json` | whether `--json` exists |
|
|
131
|
+
| `supports_json_brief` | whether `--json-brief` exists (1.10.2): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
|
|
131
132
|
| `mutates_input` | always `false`: no tool overwrites its input |
|
|
132
133
|
| `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
|
|
133
134
|
| `verification` | `{required, tools}`: which tools to run on the output afterwards |
|
|
@@ -426,8 +427,9 @@ identical to the translation of `contract --json`), which the tests check.
|
|
|
426
427
|
them required). They are transport flags `mcp/server.py` sets itself -- it appends `--json` for
|
|
427
428
|
every tool but `look` and `probe` -- rather than arguments a caller chooses, and 2.0 drops them
|
|
428
429
|
for good (see "What 2.0 changes"). The flag is opt-in and changes nothing else: without it
|
|
429
|
-
`tools/list`
|
|
430
|
-
pins
|
|
430
|
+
`tools/list` carries the tool names, argument names and `required` lists the frozen 1.x snapshot
|
|
431
|
+
pins -- descriptions may change between releases (the `--crf` deprecation mark did) -- so a lean
|
|
432
|
+
client and a default client see the same tools with the same names.
|
|
431
433
|
|
|
432
434
|
## Consuming the contract from an agent
|
|
433
435
|
|