ffmpeg-skill 1.11.0 → 1.12.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -6
- package/SKILL.md +15 -12
- package/docs/contract.md +28 -9
- package/package.json +1 -1
- package/references/gotchas.md +49 -7
- package/references/scripts.md +57 -5
- package/scripts/_common.py +378 -0
- package/scripts/_contract.py +53 -6
- package/scripts/caption.py +449 -16
- package/scripts/graphics.py +18 -3
- package/scripts/overlay.py +9 -1
- package/scripts/render.py +5 -4
package/README.md
CHANGED
|
@@ -153,7 +153,7 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
|
|
|
153
153
|
|
|
154
154
|
## Tools
|
|
155
155
|
|
|
156
|
-
42 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, `--plan FILE` (a dry run written as a plan `render.py` executes later), non-zero exit and a reason on stderr on failure. `--json-brief` (1.
|
|
156
|
+
42 public tools, all Python 3.9 standard library, all with `--help`, `--dry-run`, `--json`, `--plan FILE` (a dry run written as a plan `render.py` executes later), non-zero exit and a reason on stderr on failure. `--json-brief` (1.11.0) prints the same result document trimmed to what a caller acts on — status, output, `verified`, a compact `summary` of the output probe, the tool's own keys and the command count instead of the command lines — for roughly a third of the bytes; `--json` itself is unchanged. Every re-encoding tool takes `--codec h264|hevc|av1|prores` and `--quality N` (1.8), and every time flag takes seconds, `mm:ss`, `hh:mm:ss.fff` or SMPTE `hh:mm:ss:ff` with an optional `@fps` suffix (1.9).
|
|
157
157
|
|
|
158
158
|
**Analysis and inspection**
|
|
159
159
|
|
|
@@ -204,7 +204,7 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
|
|
|
204
204
|
|
|
205
205
|
| Tool | What it does |
|
|
206
206
|
|---|---|
|
|
207
|
-
| `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; animated and word-by-word karaoke timed to the speech energy; optional local transcription |
|
|
207
|
+
| `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; wraps to the safe area by measured width with `--max-lines`/`--min-duration`/`--offset`; picks a font by script for non-Latin text (`--lang`); animated and word-by-word karaoke timed to the speech energy or real word timings; optional local transcription |
|
|
208
208
|
| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades; `--video` for picture-in-picture, `--chromakey` for green-screen compositing |
|
|
209
209
|
| `graphics.py` | Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs drawn by FFmpeg from a brand kit |
|
|
210
210
|
| `color.py` | HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting, typed primary correction (exposure/contrast/saturation/gamma/white balance/lift-gain/levels/curves) |
|
|
@@ -259,7 +259,7 @@ built straight from that object. (The rest of a `ToolSpec` — `role`, `capabili
|
|
|
259
259
|
aren't things a parser can express; only `input_schema` is parser-derived.)
|
|
260
260
|
|
|
261
261
|
- **The contract**'s `input_schema` for every tool is generated from the live parser directly.
|
|
262
|
-
- **SKILL.md is two-tier** (1.
|
|
262
|
+
- **SKILL.md is two-tier** (1.11.0): the file the agent loads every session keeps the workflow, the
|
|
263
263
|
request→script table and one line per gotcha; the long-form detail lives in `references/gotchas.md`
|
|
264
264
|
and the other `references/` files, read only when a job needs it.
|
|
265
265
|
- **The MCP server** (`mcp/server.py`) carries no schema of its own; `tools/list` is translated
|
|
@@ -319,7 +319,7 @@ npx ffmpeg-skill doctor # human-readable
|
|
|
319
319
|
npx ffmpeg-skill doctor --json # available / missing / missing_optional / unknown / detection / errors / tools / gpu_encoders
|
|
320
320
|
```
|
|
321
321
|
|
|
322
|
-
`doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`. `doctor --json`'s `tools` field folds that down to one answer per tool — `{"caption": {"usable": "no", "missing": ["filter:subtitles"], "fix": "..."}, ...}` — so "is `doctor` overall `ok`" and "can I run `caption.py` on this machine" are answered separately: a plain Homebrew `ffmpeg` is `ok` for tools that don't need `subtitles`/`drawtext`/`zscale`, while `caption`'s own `usable` is `"no"`.
|
|
322
|
+
`doctor` reads `ffmpeg -encoders`, `-filters` and `-bsfs` and resolves every capability the contract declares against this machine's build. Three states per capability: `available`, `missing`, `unknown`. Exit 0 when everything required is available, 1 when something required is missing, 2 when nothing is proven missing but a required capability is unknown. With detection on (the default), `contract --json` carries the same lists under `capabilities`. `doctor` also reports fonts: the default drawtext family, and `fonts.scripts` — one `available`/`missing`/`unknown` per writing system (ja, zh, ko, ar, he, hi, th, ru, el) with the file it would use — so "can this machine render Korean captions" is answered before the job, not after. `doctor --json`'s `tools` field folds that down to one answer per tool — `{"caption": {"usable": "no", "missing": ["filter:subtitles"], "fix": "..."}, ...}` — so "is `doctor` overall `ok`" and "can I run `caption.py` on this machine" are answered separately: a plain Homebrew `ffmpeg` is `ok` for tools that don't need `subtitles`/`drawtext`/`zscale`, while `caption`'s own `usable` is `"no"`.
|
|
323
323
|
|
|
324
324
|
`doctor --json`'s `gpu_encoders` reports which GPU-backed encoders (`nvenc`, `videotoolbox`, `qsv`, `vaapi`, `amf`) this ffmpeg *build* was compiled with — read from `-encoders` alone, so it proves the capability shipped, not that the GPU/driver on this machine will actually accept a job (that needs a real encode, which `doctor`'s introspection never runs). No tool here uses one yet — every tool still assumes CPU x264/x265 — so this is purely informational and never affects `ok` or any tool's `usable`. GPU-accelerated encoding stays deliberately off the roadmap until there's a real-hardware-verified design for it (build-presence alone is not proof a job will succeed) — not a promised feature, just an honest "not yet, and not without proof it actually works."
|
|
325
325
|
|
|
@@ -333,7 +333,7 @@ The short list for humans. The agent-facing version, with the reasoning, is the
|
|
|
333
333
|
- **Loudness targets.** −14 LUFS / −1 dBTP for YouTube and social platforms (the `loudness.py` default), `-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast. A clip measured at −40 LUFS or below is room tone, not content; raising it raises the noise. Check true peak as well as LUFS: `check.py file --platform podcast` measures both.
|
|
334
334
|
- **Frame changes first, text second.** Captions and overlays burned before a crop or resize end up off-frame. Reframe, then caption.
|
|
335
335
|
- **Cropping 16:9 to 9:16 discards 70 % of the width.** `fit.py --fit crop` centres by default; pass `--crop-x`/`--crop-y` toward the subject, or pad with `--fit pad --pad-fill blur`. Look at the contact sheet before deciding.
|
|
336
|
-
- **Non-Latin
|
|
336
|
+
- **Non-Latin text picks a font by script (1.12).** Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari, Thai, Cyrillic and Greek cues, titles and overlays resolve a font file that covers them automatically, and a machine with no such font fails the job (`kind: input`) instead of rendering boxes. `doctor --json`'s `fonts.scripts` says which languages this machine can render; `--lang ja|ko` disambiguates Han-only text; an explicit `--font`/`--font-file` is always kept.
|
|
337
337
|
- **Silence detection finds nothing?** The default threshold is −35 dBFS. The tool prints a hint with the track's measured level; raise the threshold (`silence.py --threshold -25`) or shorten `--min-silence`.
|
|
338
338
|
- **Sync results carry a confidence.** Below 0.3, or an offset near the edge of the analysis window, is probably wrong: enlarge `--analyze-seconds` or find a clap. Recordings over ten minutes from separate devices need `sync.py --fix-drift`.
|
|
339
339
|
- **Outputs are never overwritten silently.** An existing output path is warned about today and refused from 2.0; set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) to get the refusal now and pass `--overwrite` where a replacement is intended.
|
|
@@ -363,6 +363,8 @@ FFmpeg 8 shortened the flag column of `ffmpeg -filters`. A parser anchored on th
|
|
|
363
363
|
| **F1 0.97** | `scenes.py`, 53 hard cuts between single takes, precision 0.95, recall 1.00 at the default threshold |
|
|
364
364
|
| **exact to the sample** | `cut.py --accurate` on WAV, FLAC (44.1 kHz) and AAC → WAV; WAV stream copy within 2 ms; AAC output +21 ms of encoder priming, reported as `codec_frame` (0.9.1) |
|
|
365
365
|
| **72 / 72** | agent runs of 24 prompts (12 English edits, 8 Japanese, 4 that must be declined), three repeats, graded by an independent model: routing, honest refusals and user's language 72/72, report format 71/72, visual check whenever the picture changed 24/24 (0.8.4) |
|
|
366
|
+
| **36 / 36** | 1.11.1 re-run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader): routing 36/36, honest refusals and failures 36/36 with 0 false successes and 0 raw ffmpeg calls, report format 36/36, user's language 36/36, visual check 8/8, trigger set 22/22, Opus quality mean 4.75. The 1.11.1 wording did what it said (`doctor` before a job 23 of 36 runs → 0, `--json-brief` 4 → 23) and tokens per run stayed flat at 71.8k, because about 64k of every run is the host's own context; the token-diet theme closes here. Details in `evals/results/iteration-12.json` |
|
|
367
|
+
| **36 / 36** | 1.11.0 re-run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader): routing 36/36, honest refusals and failures 36/36 with 0 false successes and 0 raw ffmpeg calls, report format 36/36, user's language 36/36, visual check 8/8, trigger set 22/22. First iteration to measure tokens per run: mean 72.2k against 68.7k at 1.10.0, mostly a fixed per-run floor the skill does not control (a refusal run that only reads SKILL.md costs about 64k), plus `doctor` on 23 of 36 runs; 1.11.1 rewords step 0 and iteration 12 re-measures. Details in `evals/results/iteration-11.json` |
|
|
366
368
|
| **108 / 108** | 1.10.0 re-run (2026-09-13, three passes per prompt, Sonnet agent, regex grader + focused Opus grader): routing 108/108, honest refusals and failures 108/108 with 0 false successes and 0 raw ffmpeg calls, report format 108/108 by both graders (the harness now names the five labels), user's language 105/108 (every Japanese request in Japanese; 3 English requests drifted to Spanish or Portuguese), visual check 21/24, trigger set 22/22; real-device corpus 101/101 steps PASS. Details in `evals/results/iteration-10.json` |
|
|
367
369
|
| **108 / 108** | 1.9.0 re-run (2026-09-13, three passes per prompt, Sonnet agent, regex grader + independent Opus grader): routing 108/108, honest refusals and failures 108/108 with 0 false successes and 0 raw ffmpeg calls, visual check 23/26, user's language 105/108 (every Japanese request answered in Japanese; 3 English requests drifted to Spanish), report format 108/108 by regex (65/108 by the stricter grader, which now counts any missing label), trigger set 22/22; 12 of 15 platform jobs were one encode and `render.py` rendered once in 3/3 (was 1/3). The 1.9.0 time grammar was not used by any agent. Details in `evals/results/iteration-9.json` |
|
|
368
370
|
| **108 / 108** | 1.8.0 re-run (2026-09-12, three passes per prompt, Sonnet agent, regex grader + independent Opus grader that re-probed 22 outputs): routing 108/108, honest refusals and failures 108/108 with 0 false successes and 0 raw ffmpeg calls, visual check 22/24, report format 108/108 by regex (91/108 by the stricter grader: 'What/how:' in place of Steps:), user's language 98/108 by the stricter grader (Japanese labels-only reports counted), trigger set 22/22; 13 of 14 platform exports used `--normalize` and platform jobs went from three encodes to one; r04/f01 now answered in the request's language 5/6 (was 0/6). Details in `evals/results/iteration-8.json` |
|
|
@@ -452,7 +454,7 @@ Contributing a change: see [CONTRIBUTING.md](CONTRIBUTING.md).
|
|
|
452
454
|
| | |
|
|
453
455
|
|---|---|
|
|
454
456
|
| [CONTRIBUTING.md](CONTRIBUTING.md) | scope, dev setup, tests, PR expectations |
|
|
455
|
-
| [docs/roadmap.md](docs/roadmap.md) | 1.8.0 to 1.
|
|
457
|
+
| [docs/roadmap.md](docs/roadmap.md) | 1.8.0 to 1.21.0 one theme per minor (1.8 to 1.10 pre-ship 2.0, 1.11 the token diet), and what 2.0.0 then removes |
|
|
456
458
|
| [docs/design-decisions.md](docs/design-decisions.md) | behaviours that look like bugs but are decisions, with rationale and the pinning test; read before filing a bug |
|
|
457
459
|
| [CODE_OF_CONDUCT.md](CODE_OF_CONDUCT.md) | Contributor Covenant 2.1; reports go through the SECURITY.md channel |
|
|
458
460
|
| [SECURITY.md](SECURITY.md) | how to report a vulnerability privately |
|
package/SKILL.md
CHANGED
|
@@ -5,15 +5,15 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
|
|
|
5
5
|
|
|
6
6
|
# ffmpeg-skill
|
|
7
7
|
|
|
8
|
-
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`.
|
|
8
|
+
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. This file is enough to do a job: the table below routes the request, the recipes give the flags, and `--help` on the one script you are about to run is the cheapest full flag list. The reference files cost as much to read as this file does, so open one only when it answers a question you actually have: `references/scripts.md` (every flag of all 42 scripts, for comparing tools), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end of this file).
|
|
9
9
|
|
|
10
|
-
Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (that document trimmed to status/output/verified, a compact `summary` and the command count —
|
|
10
|
+
Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (that document trimmed to status/output/verified, a compact `summary` and the command count — prefer it on every writing step, `--json` when you need the full probe); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (consent to replace an existing output — without it the tool warns today, refuses from 2.0); `--plan FILE` (the dry run as a plan document `render.py FILE` executes later, refusing if an input changed: "plan → confirm → execute" in one round trip). Every re-encoding tool also takes `--codec h264|hevc|av1|prores` and `--quality N` (CRF scale, replaces the deprecated `--crf`): unset, SDR is x264 and HDR is x265 Main10; `prores` needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (run `color.py --to-sdr` first).
|
|
11
11
|
|
|
12
12
|
Writing tools run nothing under `--dry-run`; `probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness` and `stabilize` may still run ffmpeg/ffprobe to measure or analyse — they just don't write their final artifact (nor side files such as `--edl`, `--sheet` or a generated `.ass`); `verify` accepts the flag but ignores it. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
13
13
|
|
|
14
14
|
## Workflow (always follow this order)
|
|
15
15
|
|
|
16
|
-
0. **Environment, not
|
|
16
|
+
0. **Environment, only on failure.** Do not start a job with `doctor`: on a working machine it is a call that tells you nothing the job needs, and on a broken one the script fails on its own with `kind: missing_tool` (no ffmpeg) or an ffmpeg error naming the filter or encoder (`No such filter: 'subtitles'`). Run `python3 <skill-dir>/scripts/_contract.py doctor` (also `npx ffmpeg-skill doctor`; there is no doctor.py) after such a failure, or when the user asks what the machine can do. Read `ok` and the tool's `usable`; if `usable` isn't `yes`, report the missing capability (a missing `libass`, `zscale` or encoder is the common case, e.g. `caption.py`) instead of discovering it through a runtime failure. `contract --json`'s full tool schema is for a *planning* agent choosing a tool from an abstract goal, not for this per-job workflow.
|
|
17
17
|
1. **Probe what you must plan from.** Run `probe.py` on each input you plan the edit from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks a question about a file. You do not need a separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output it wrote. Plan from real numbers, never assumptions.
|
|
18
18
|
2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; pass `--accurate` to `cut.py` only for frame-exact cuts.
|
|
19
19
|
3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's human-readable summary line, for any number after the plan (dimensions there can be a placeholder, not a computed preview — `docs/contract.md`). Use it to confirm a plan before long encodes and to report exact facts. `--fast` is preview quality (x264 veryfast), `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
|
|
@@ -34,7 +34,7 @@ Ask one short question only when the answer changes the output materially and th
|
|
|
34
34
|
- **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, and say which you chose. Ask when the content is a talk (trimming loses words) and the change is large.
|
|
35
35
|
- **Captions** without a text source: `--transcribe` if a local whisper exists, otherwise ask for the text or a timed file; never invent dialogue.
|
|
36
36
|
- **Fonts and brand**: if the user mentions a brand, colours or "our font", ask for or create `brand.json` once and reuse it.
|
|
37
|
-
- **CJK / non-Latin text**:
|
|
37
|
+
- **CJK / non-Latin text**: let the tool pick the font by script (`--font` turns that off); `--lang ja|ko` for Han-only text. `doctor --json` `.fonts.scripts` says what renders here. Tofu is a failed job, not a style.
|
|
38
38
|
- **Crop position** for `--fit crop`: centre by default, but when the request or the source names an off-centre subject ("keep the product on the right", "don't cut off my hands", someone visibly off-centre in the sheet) use `--crop-x`/`--crop-y` (0=left/top, 1=right/bottom) instead of a silent centre guess. Ask which edge to keep when the sheet shows the subject near an edge and the request doesn't say.
|
|
39
39
|
- Anything else (transition type, caption style): pick the conventional default, say what you picked, offer the alternative in one line.
|
|
40
40
|
|
|
@@ -137,12 +137,12 @@ Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offse
|
|
|
137
137
|
|
|
138
138
|
## Audio-only files
|
|
139
139
|
|
|
140
|
-
Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`, `audio.py`, `sync.py` and `check.py --platform podcast` take WAV, FLAC, MP3, M4A/AAC, OGG and Opus, and the output extension picks the format. `Look: not needed` in the report; `Check:` still applies (`check.py file.wav --platform podcast`). Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" — say so instead of forcing a video wrapper. The same commands work with `talk.wav` in place of `talk.mp4
|
|
140
|
+
Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`, `audio.py`, `sync.py` and `check.py --platform podcast` take WAV, FLAC, MP3, M4A/AAC, OGG and Opus, and the output extension picks the format. `Look: not needed` in the report; `Check:` still applies (`check.py file.wav --platform podcast`). Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" — say so instead of forcing a video wrapper. The same commands work with `talk.wav` in place of `talk.mp4`; audio-specific recipes, packet vs sample precision, joining and extracting one track: `references/gotchas.md#audio-only-files`.
|
|
141
141
|
|
|
142
142
|
|
|
143
143
|
## Report format
|
|
144
144
|
|
|
145
|
-
Reply in the language the request itself is written in — the
|
|
145
|
+
Reply in the language the request itself is written in — the user's own sentences, not a language the request talks about (a request for subtitles in another language is still answered in the language it was written in) and not the language of a tool's error text or file names. Chinese, Korean, Spanish, Portuguese, French, German, Arabic or any other language works exactly the same way. Keep the field labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English: they read like log fields and stay recognisable across languages. Everything around them — the sentences, any question, any explanation of a judgement call — is in the user's language. Never default to English because the tool names and flags are English, and never drift into another language because the job was short or the report is a failure: a one-line "file does not exist" is written in the request's language too. A mid-conversation switch follows the user's latest message, not the first one. This holds for a one-command job: a three-second audio trim with English `Done:`/`Steps:` sentences and one word of the user's language in `Notes:` is an English report — the descriptions (what was cut, from where) are in the user's language even when the values are technical.
|
|
146
146
|
|
|
147
147
|
Finish every job with this shape (numbers from a tool's `--json` or `probe.py`/`check.py`, not memory):
|
|
148
148
|
|
|
@@ -154,7 +154,7 @@ Look: final_sheet.png (captions inside the safe area, logo top-right)
|
|
|
154
154
|
Notes: source was VFR, conformed to 30 fps; audio was mono, made stereo
|
|
155
155
|
```
|
|
156
156
|
|
|
157
|
-
The same five lines for a Japanese request
|
|
157
|
+
The same five lines for a Japanese request:
|
|
158
158
|
|
|
159
159
|
```
|
|
160
160
|
Done: final.mp4 — 59.98 秒、1080x1920、30 fps、H.264、AAC ステレオ、-14.1 LUFS
|
|
@@ -164,7 +164,9 @@ Look: final_sheet.png(字幕はセーフエリア内、ロゴは右上)
|
|
|
164
164
|
Notes: 元は VFR だったので 30 fps に揃えた。音声はモノラルだったのでステレオにした
|
|
165
165
|
```
|
|
166
166
|
|
|
167
|
-
|
|
167
|
+
Same shape in every other language, labels still English — zh: `Done: final.mp4 — 59.98 秒、1080x1920、30 fps、H.264` / `Steps: 0:12-1:12 剪切 -> 9:16 裁剪 -> 字幕 -> Reels 导出`; ko: `Done: final.mp4 — 59.98초, 1080x1920, 30 fps, H.264` / `Steps: 0:12-1:12 컷 -> 9:16 크롭 -> 자막 -> Reels 내보내기`.
|
|
168
|
+
|
|
169
|
+
Keep it to those five lines plus anything the user must decide. Attach the contact sheet when the edit touched the picture. Never report success without the output probe; never describe a fix you did not run.
|
|
168
170
|
|
|
169
171
|
When a step fails, replace `Done:` with `Failed:` and keep the rest honest:
|
|
170
172
|
|
|
@@ -176,13 +178,13 @@ Look: not needed (nothing written)
|
|
|
176
178
|
Notes: send a valid .cube, or say if you want the clip left as is
|
|
177
179
|
```
|
|
178
180
|
|
|
179
|
-
A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run (usually only probe), `Look: not needed`. Both keep the five labels so a failed report scans like a successful one —
|
|
181
|
+
A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run (usually only probe), `Look: not needed`. Both keep the five labels so a failed report scans like a successful one — the shortest failure (a missing input, an invalid LUT) still gets all five lines, never prose headings in their place. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change that makes a retry meaningful.
|
|
180
182
|
|
|
181
|
-
Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message,
|
|
183
|
+
Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message, never paraphrase it into a success.
|
|
182
184
|
|
|
183
185
|
## Things that look right but are wrong
|
|
184
186
|
|
|
185
|
-
One line each
|
|
187
|
+
One line each, and each line is enough to act on; open the linked `references/gotchas.md` section only when the job is in that area and the line leaves you with a question.
|
|
186
188
|
|
|
187
189
|
- HDR (iPhone, HDR10) re-encoded through an SDR path goes flat; the scripts keep HDR, and `hdr: true` is wider than `hdr_signal: true` (a real PQ/HLG/Dolby Vision transfer). Details: [#hdr-and-colour](references/gotchas.md#hdr-and-colour)
|
|
188
190
|
- Log footage (S-Log/V-Log/C-Log) is tagged SDR and looks grey: `probe.py --analyze`, then `color.py --lut` before anything else. Details: [#log-footage](references/gotchas.md#log-footage)
|
|
@@ -190,7 +192,8 @@ One line each; the full version is `references/gotchas.md`, worth reading before
|
|
|
190
192
|
- VFR phone/screen recordings: re-encodes conform to CFR, `cut.py` switches to `--accurate`; pick the rate with `fit.py --fps` when the average is odd. Details: [#variable-frame-rate](references/gotchas.md#variable-frame-rate)
|
|
191
193
|
- Sync/multicam `confidence` under 0.3 (or a huge offset) is probably wrong — check every camera, and remember these align audio, never lip sync. Details: [#sync-multicam-and-drift](references/gotchas.md#sync-multicam-and-drift)
|
|
192
194
|
- "Normalised" audio can still clip (check true peak), and ambience at -40 LUFS or below must never be raised to a speech target. Details: [#loudness-and-ambience](references/gotchas.md#loudness-and-ambience)
|
|
193
|
-
- Captions burned before a crop/resize land off-frame; burned small then upscaled by `export.py` they come out soft — fit to the delivery size first.
|
|
195
|
+
- Captions burned before a crop/resize land off-frame; burned small then upscaled by `export.py` they come out soft — fit to the delivery size first. Details: [#captions-fonts-and-text-order](references/gotchas.md#captions-fonts-and-text-order)
|
|
196
|
+
- Non-Latin text picks a font by script since 1.12; `doctor --json` `fonts.scripts` says which languages this machine renders; no font = failed job, not tofu. Details: [#fonts-by-script](references/gotchas.md#fonts-by-script)
|
|
194
197
|
- `--fit crop` 16:9 → 9:16 throws away 70 % of the width, 60→30 fps halves the motion, and "60 seconds" by speed or by trim are different answers — say which and why. Details: [#reframing-fps-and-duration](references/gotchas.md#reframing-fps-and-duration)
|
|
195
198
|
- `yuv420p` needs even dimensions and phone rotation tags are honoured, both automatically. Details: [#dimensions-and-rotation](references/gotchas.md#dimensions-and-rotation)
|
|
196
199
|
- `scenes.py --highlights` ranks by loudness (or duration), never by meaning: check the sheet before treating picks as final. Details: [#highlights](references/gotchas.md#highlights)
|
package/docs/contract.md
CHANGED
|
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
|
|
|
21
21
|
| Field | Meaning | Changes when |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
|
|
24
|
-
| `skill.version` | the npm / package.json version (`1.
|
|
24
|
+
| `skill.version` | the npm / package.json version (`1.12.0`) | any release |
|
|
25
25
|
|
|
26
26
|
A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
|
|
27
27
|
ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
|
|
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
|
|
|
88
88
|
|
|
89
89
|
| What 2.0 removes | Since | Replacement | To be ready today |
|
|
90
90
|
|---|---|---|---|
|
|
91
|
-
| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.
|
|
92
|
-
| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.
|
|
93
|
-
| `json` and `progress` in the MCP `inputSchema` | 1.
|
|
94
|
-
| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.
|
|
95
|
-
| Overwriting an existing output with only a warning | 1.
|
|
91
|
+
| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.12.0 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
|
|
92
|
+
| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.12.0 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
|
|
93
|
+
| `json` and `progress` in the MCP `inputSchema` | 1.12.0 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
|
|
94
|
+
| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.12.0 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
|
|
95
|
+
| Overwriting an existing output with only a warning | 1.12.0 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
|
|
96
96
|
|
|
97
97
|
## Skill
|
|
98
98
|
|
|
99
99
|
```json
|
|
100
100
|
{
|
|
101
101
|
"contract_version": "1.0",
|
|
102
|
-
"deprecated": [{"what": "...", "since": "1.
|
|
103
|
-
"skill": {"id": "ffmpeg-skill", "version": "1.
|
|
102
|
+
"deprecated": [{"what": "...", "since": "1.12.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
|
|
103
|
+
"skill": {"id": "ffmpeg-skill", "version": "1.12.0", "execution_mode": "local", "kind": "execution",
|
|
104
104
|
"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
|
|
105
105
|
"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
|
|
106
106
|
"requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
|
|
@@ -128,7 +128,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
|
|
|
128
128
|
| `output_schema` | what `--json` prints on stdout |
|
|
129
129
|
| `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
|
|
130
130
|
| `supports_json` | whether `--json` exists |
|
|
131
|
-
| `supports_json_brief` | whether `--json-brief` exists (1.
|
|
131
|
+
| `supports_json_brief` | whether `--json-brief` exists (1.12.0): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
|
|
132
132
|
| `mutates_input` | always `false`: no tool overwrites its input |
|
|
133
133
|
| `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
|
|
134
134
|
| `verification` | `{required, tools}`: which tools to run on the output afterwards |
|
|
@@ -319,6 +319,25 @@ when `fc-match` itself is not on PATH or fails. Like `gpu_encoders`, this is pur
|
|
|
319
319
|
and never affects `ok` or any tool's `usable` — a substituted font is not a broken tool, just a
|
|
320
320
|
typeface the caller didn't ask for.
|
|
321
321
|
|
|
322
|
+
Since 1.12 the same field also carries `scripts`: one entry per writing system the tools detect,
|
|
323
|
+
`{"ja": {"status": "available"|"missing"|"unknown", "file": "/path/to/font.ttc"|null}, "zh": ...,
|
|
324
|
+
"ko": ..., "ar": ..., "he": ..., "hi": ..., "th": ..., "ru": ..., "el": ...}`. It answers "which
|
|
325
|
+
languages can this machine actually render", which no filter or encoder capability asks:
|
|
326
|
+
`available` means `fc-list :lang=<code>` (Linux/macOS) or a known system font file (Windows) covers
|
|
327
|
+
the script, `missing` means fontconfig knows none, `unknown` means there is no working fontconfig to
|
|
328
|
+
ask (no `fc-list` on PATH, or it failed). `caption.py`, `graphics.py` and `overlay.py --text` resolve
|
|
329
|
+
a font by script automatically and fail with `kind: input` rather than render boxes **only for
|
|
330
|
+
`missing`**: `unknown` is not `missing` here any more than anywhere else in this document — the job
|
|
331
|
+
runs with the font as given and one info line says the coverage could not be verified. So a
|
|
332
|
+
`missing` script here is a job that will not run until a font is installed — but, like the default
|
|
333
|
+
font, it never affects `ok` or any tool's
|
|
334
|
+
`usable` (the tool works, this machine just has no glyphs for that language). The plain-text
|
|
335
|
+
`doctor` summarises the whole map on one `fonts:` line, which also carries the default font's
|
|
336
|
+
`detail` in brackets when its status is not `available` and that detail is short enough to keep the
|
|
337
|
+
line to one screen width; a longer explanation, and the per-script files, are `--json` only.
|
|
338
|
+
`--lang`/`--language` (caption, graphics) is the hint that says whether Han-only text is Chinese,
|
|
339
|
+
Japanese or Korean.
|
|
340
|
+
|
|
322
341
|
## Invocation
|
|
323
342
|
|
|
324
343
|
Structured arguments are the canonical way to call a tool, on the CLI or through MCP.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.12.0",
|
|
4
4
|
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ffmpeg",
|
package/references/gotchas.md
CHANGED
|
@@ -83,18 +83,60 @@ come out soft (a 1280x720 source fit to 9:16 is 406x720 until export scales it t
|
|
|
83
83
|
1080x1920) — fit to the delivery size first
|
|
84
84
|
(`fit.py --width 1080 --height 1920`), then caption, then export.
|
|
85
85
|
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
(
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
86
|
+
Cue length and timing are handled for you since 1.12: `caption.py` wraps every
|
|
87
|
+
cue to the safe area by measured width at the chosen `--size`, splits a cue past
|
|
88
|
+
`--max-lines` (default 2) into consecutive cues, holds a cue shorter than
|
|
89
|
+
`--min-duration` (default 1.0 s) — never past the next cue — and shifts
|
|
90
|
+
everything by `--offset SECONDS`. It reports what it changed on one `cues:` line
|
|
91
|
+
and writes the adjusted copy next to the output, never over the file you passed
|
|
92
|
+
in. Wrapping needs the input video (the line width comes from its real frame
|
|
93
|
+
size); with `--write-srt` alone only the timing flags apply.
|
|
93
94
|
|
|
94
95
|
Windows drawtext crashes on some real builds (#100): the drawtext tools resolve a
|
|
95
96
|
concrete `--font-file` by default, which avoids it; if one still crashes, pass
|
|
96
97
|
`--font-file` explicitly. Details: `references/ci-platform-pitfalls.md`.
|
|
97
98
|
|
|
99
|
+
### Fonts by script
|
|
100
|
+
libass and drawtext draw an empty box per character they have no glyph for, and
|
|
101
|
+
ffmpeg still exits 0 — a video full of tofu is the classic "it worked" failure.
|
|
102
|
+
Since 1.12 `caption.py`, `graphics.py` and `overlay.py --text` detect the script
|
|
103
|
+
of the text they are about to draw (Japanese, Chinese, Korean, Arabic, Hebrew,
|
|
104
|
+
Devanagari, Thai, Cyrillic, Greek) and resolve a font file that covers it,
|
|
105
|
+
printing one line — `font: /usr/share/fonts/.../wqy-zenhei.ttc (covers ko)`.
|
|
106
|
+
**No font for the script is a failed job** (`kind: input`), not a warning.
|
|
107
|
+
|
|
108
|
+
- What this machine can render: `python3 scripts/_contract.py doctor --json`,
|
|
109
|
+
field `fonts.scripts` (`available` / `missing` / `unknown` per language, with
|
|
110
|
+
the file it would use). The plain-text `doctor` says the same in one line.
|
|
111
|
+
- What fontconfig has: `fc-list ":lang=ja" file family` (`ja`, `zh-cn`, `ko`,
|
|
112
|
+
`ar`, `he`, `hi`, `th`, `ru`, `el`).
|
|
113
|
+
- Install: `apt install fonts-noto-cjk fonts-noto-core`, or
|
|
114
|
+
`brew install --cask font-noto-sans-cjk font-noto-sans-arabic`, or point at a
|
|
115
|
+
file with `--font-file` (`overlay.py`, `graphics.py`) / `--fonts-dir`
|
|
116
|
+
(`caption.py`).
|
|
117
|
+
- Han characters alone (no kana, no hangul) are read as Chinese. Japanese or
|
|
118
|
+
Korean hanja text with no kana needs `--lang ja` / `--lang ko`
|
|
119
|
+
(`caption.py --language` is the same flag), or `"lang"` in brand.json.
|
|
120
|
+
- An explicit `--font`, an explicit `--font-file`, or a font your brand file
|
|
121
|
+
itself names is always kept, even when fontconfig says it does not cover the script: you get one info
|
|
122
|
+
line saying so, not a silent substitution. A brand file that never names a
|
|
123
|
+
font is not a choice — the script still picks one.
|
|
124
|
+
- `--fonts-dir` (`caption.py`) adds faces to the search, it does not switch the
|
|
125
|
+
check off: if nothing in the directory covers the script, one line says so and
|
|
126
|
+
a covering font is resolved as usual.
|
|
127
|
+
- **No fontconfig is `unknown`, not `missing`.** With no `fc-list` on PATH (or
|
|
128
|
+
one that fails) coverage cannot be verified: the job runs with the font as
|
|
129
|
+
given behind one info line, because libass and drawtext have font backends of
|
|
130
|
+
their own. Only fontconfig answering "nothing covers this" fails the job.
|
|
131
|
+
- **RTL:** libass shapes and reorders Arabic and Hebrew correctly, so
|
|
132
|
+
`caption.py` — which renders every subtitle through libass, `subtitles=` and
|
|
133
|
+
`ass=` alike — is right for them by construction. `drawtext` (`overlay.py
|
|
134
|
+
--text`, `graphics.py`) depends on the build: `ffmpeg -version` showing
|
|
135
|
+
`--enable-libfribidi` (and `--enable-libharfbuzz`) shapes and reorders RTL
|
|
136
|
+
correctly too; a build without them draws logical order with unjoined
|
|
137
|
+
letterforms. Nothing in the tools checks this, so on an unknown machine a
|
|
138
|
+
caption is the safe place for Arabic/Hebrew.
|
|
139
|
+
|
|
98
140
|
### Reframing, fps and duration
|
|
99
141
|
`--fit crop` to reach 9:16 from 16:9 throws away 70 % of the width: a wide shot
|
|
100
142
|
loses people at the edges. Check the sheet; pad (bars), `--crop-x`/`--crop-y`
|
package/references/scripts.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Script reference
|
|
2
2
|
|
|
3
|
-
Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.
|
|
3
|
+
Every script prints the same information with `--help`; this file exists so the agent can read several at once. All scripts accept `--dry-run`, `--json`, `--json-brief` (since 1.11.0: the same JSON result trimmed to `status`, `output`, `dry_run`, `verified`, a compact `summary` of the output probe -- duration_s, width, height, fps, vcodec, acodec, channels, and lufs when measured -- the tool's own keys, and the count of commands instead of the command lines; it implies `--json`, leaves `--json`'s own output untouched, and failures print the usual full failure document), `--fast`, `--progress`, `--timeout SECONDS`, `--overwrite`, `--plan FILE` (the dry run written as a plan document that `render.py FILE` executes later; see render.py), `-o OUT`; every editing tool that re-encodes (not `export.py`, whose preset decides the codec) also takes `--codec h264|hevc|av1|prores` (the encoder for the re-encode; default x264 for SDR, x265 Main10 for HDR, unchanged) and `--quality N` (CRF scale, overrides `--crf`; up to 63 for av1; ignored by prores). `--crf` is deprecated since 1.10.0 (it warns on stderr and is removed in 2.0): use `--quality`, except on `export.py`, whose `--crf` is not an alias and stays. With `FFMPEG_SKILL_NO_OVERWRITE=1` in the environment, any tool refuses (`kind: input`) to replace an existing output unless `--overwrite` is given. `--codec hevc` on SDR writes 8-bit BT.709 HEVC (`hvc1`), `av1` uses SVT-AV1 (libaom fallback), `prores` is 422 HQ and needs an explicit `-o NAME.mov` (or `.mkv`), `h264` refuses an HDR source (`kind: input`, run `color.py --to-sdr` first). `export.py` keeps choosing the codec from its preset and has neither flag; a `render.py` project cannot choose a codec either -- but `--dry-run` only guarantees nothing is written for writing tools: `probe` (read-only, `--dry-run` changes nothing) still runs ffprobe, `check`/`sync`/`multicam`/`scenes`/`cropdetect`/`report`/`silence`/`loudness`/`stabilize` still run their ffmpeg/ffprobe measurements (a dry-run plan rests on real numbers; they just don't write the final artifact), and `verify` accepts the flag but ignores it entirely. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
4
4
|
|
|
5
5
|
## Time grammar (every time-taking flag, 1.9)
|
|
6
6
|
|
|
@@ -465,21 +465,35 @@ Inside this skill, call the scripts directly; the server is for other hosts.
|
|
|
465
465
|
### graphics.py — motion-graphics templates
|
|
466
466
|
```
|
|
467
467
|
graphics.py INPUT --template lower-third|title|chapter|progress|countdown|bug [--name] [--title] [--subtitle]
|
|
468
|
-
[--from N] [--start S] [--end E] [--position CORNER] [--brand brand.json] [--primary RRGGBB] [--scale 1.0] [-o OUT]
|
|
468
|
+
[--from N] [--start S] [--end E] [--position CORNER] [--brand brand.json] [--primary RRGGBB] [--scale 1.0] [--lang XX] [-o OUT]
|
|
469
469
|
```
|
|
470
470
|
Drawn with drawbox/drawtext/overlay — no PNG assets needed. Sizes scale with
|
|
471
471
|
the frame's short side; colours, font and safe margin come from `--brand`.
|
|
472
472
|
Lower-third slides in over 0.4 s and out over 0.3 s; title/chapter/bug fade.
|
|
473
|
+
Non-Latin `--name`/`--title`/`--subtitle` text picks a font file by script the
|
|
474
|
+
same way `caption.py` does (`--lang XX` disambiguates Han-only text; no font for
|
|
475
|
+
the script fails the job). RTL shaping in drawtext depends on the ffmpeg build
|
|
476
|
+
(`--enable-libfribidi`/`--enable-libharfbuzz` shape it correctly, a build without
|
|
477
|
+
them does not); `caption.py` always shapes, because it renders through libass:
|
|
478
|
+
`references/gotchas.md#fonts-by-script`.
|
|
473
479
|
|
|
474
480
|
### brand.json — one file for fonts, colours, logo, margins
|
|
475
481
|
```json
|
|
476
482
|
{"font": "Noto Sans CJK JP", "font_file": "fonts/NotoSansCJK-Bold.ttc",
|
|
477
483
|
"colors": {"primary": "FF6A00", "text": "FFFFFF", "outline": "000000", "background": "0B1D2A"},
|
|
478
484
|
"logo": "logo.png", "logo_position": "top-right", "logo_scale": 160, "logo_opacity": 0.9,
|
|
479
|
-
"safe_margin": 48, "
|
|
485
|
+
"safe_margin": 48, "lang": "ja",
|
|
486
|
+
"styles": {"caption": {"font": "Noto Sans CJK JP", "size": 28, "colour": "FFFFFF", "box": false, "position": "bottom"}},
|
|
487
|
+
"caption": {"size": 28, "position": "bottom", "animate": "pop", "karaoke": true, "bold": true}}
|
|
480
488
|
```
|
|
481
489
|
`caption.py --brand`, `overlay.py --brand --logo`, `graphics.py --brand`, and
|
|
482
|
-
`"brand": "brand.json"` in a render project. Explicit flags still win.
|
|
490
|
+
`"brand": "brand.json"` in a render project. Explicit flags still win.
|
|
491
|
+
`styles.caption` (1.12) is the one caption look every project shares —
|
|
492
|
+
`{font, size, colour, box, position}`, British or American spelling of colour —
|
|
493
|
+
read by `caption.py` and, for `font` and `colour`, by `graphics.py`; it wins
|
|
494
|
+
over the older top-level `caption` block where both set the same key, and that
|
|
495
|
+
block still carries the burn-in-only defaults (`animate`, `karaoke`, `bold`,
|
|
496
|
+
`outline`). `"lang"` is the script hint `--lang` would give. When a
|
|
483
497
|
user mentions brand guidelines, colours, "our font" or a logo, ask for or
|
|
484
498
|
write a brand.json once and reuse it across every output.
|
|
485
499
|
|
|
@@ -526,7 +540,8 @@ frame like an editor would. Use `--compare` to show before/after to the user.
|
|
|
526
540
|
### caption.py — subtitles (static, animated, karaoke)
|
|
527
541
|
```
|
|
528
542
|
caption.py INPUT --srt FILE | --ass FILE | --text CUES.txt [--write-srt OUT.srt]
|
|
529
|
-
[--mode burn|mux] [--audio-stream N] [--fps N]
|
|
543
|
+
[--mode burn|mux] [--audio-stream N] [--fps N] [--lang XX] [--offset TIME]
|
|
544
|
+
[--max-lines N] [--min-duration S]
|
|
530
545
|
[--font NAME] [--fonts-dir DIR] [--size N] [--color RRGGBB] [--outline N] [--outline-color RRGGBB]
|
|
531
546
|
[--bold] [--box] [--position bottom|top|center|top-left|...] [--margin N]
|
|
532
547
|
[--animate none|fade|pop|slide] [--karaoke [--highlight-color RRGGBB]] [--write-ass OUT.ass] [-o OUT]
|
|
@@ -543,6 +558,43 @@ word from `--color` to `--highlight-color` across the cue; `--karaoke-timing
|
|
|
543
558
|
energy` (default) follows the speech loudness in the audio, `even` splits the
|
|
544
559
|
cue equally (word timing is derived, not transcribed). The ASS is kept next to the
|
|
545
560
|
user can hand-tune timings and re-run with `--ass`.
|
|
561
|
+
Readable by default (1.12): every cue is wrapped to the safe area (90 % of the
|
|
562
|
+
frame width) at the chosen `--size`, measured per script — CJK and Thai count a
|
|
563
|
+
full em per character, Latin per character from a table read off DejaVu Sans (so
|
|
564
|
+
an all-caps line measures as wide as it draws), Cyrillic/Greek about 0.55,
|
|
565
|
+
Arabic/Hebrew 0.6, Devanagari 0.7, and a combining mark nothing at all —
|
|
566
|
+
breaking between characters for CJK/Thai and at spaces otherwise, but never
|
|
567
|
+
between a character and the combining marks that belong to it (Thai tone marks
|
|
568
|
+
and vowel signs, Devanagari matras, Arabic and Hebrew points). A cue that would need more than `--max-lines` (default 2) is split
|
|
569
|
+
into consecutive cues sharing its time; a cue shorter than `--min-duration`
|
|
570
|
+
(default 1.0 s) is held longer, never past the next cue's start; `--offset
|
|
571
|
+
TIME` shifts every cue (seconds, `mm:ss`, `hh:mm:ss.ms` or `hh:mm:ss:ff`, a
|
|
572
|
+
leading `-` for earlier) for `--text`, `--srt` and `--ass`. One `cues:` info line reports what changed. A file you passed in is
|
|
573
|
+
never edited: the adjusted copy is written next to the output
|
|
574
|
+
(`<out>_adjusted.srt`, `<out>_offset.ass`) and burned instead — under `--dry-run`
|
|
575
|
+
/`--plan` the planned command names that same copy and the plan says where it
|
|
576
|
+
comes from, but nothing is written until the real run. `--min-duration` and
|
|
577
|
+
`--offset` also work with `--write-srt` alone; `--max-lines` does not, because
|
|
578
|
+
wrapping needs the input video's real frame size.
|
|
579
|
+
|
|
580
|
+
Fonts by script (1.12): with no `--font` and no font named in your brand file, the
|
|
581
|
+
script of the cue text (Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari,
|
|
582
|
+
Thai, Cyrillic, Greek) picks a font file that covers it, logged as `font: <file>
|
|
583
|
+
(covers ko)`. Han-only text is read as Chinese unless `--lang ja|ko` (or
|
|
584
|
+
brand.json `"lang"`) says otherwise; `--language` is the same flag, and still
|
|
585
|
+
tags the subtitle stream under `--mode mux` and sets `--transcribe`'s language.
|
|
586
|
+
No font for the script fails the job (`kind: input`) instead of rendering boxes
|
|
587
|
+
— but only when fontconfig answered: with no working `fc-list` the coverage is
|
|
588
|
+
`unknown`, and the job runs with the font as given behind one info line. An
|
|
589
|
+
explicit font is always kept, with an info line when it does not cover the text;
|
|
590
|
+
`--fonts-dir` is searched first and checked with `fc-scan`, and a directory that
|
|
591
|
+
does not cover the script gets one line and a font resolved by script anyway. `doctor --json` `fonts.scripts` lists what this machine can render. See
|
|
592
|
+
`references/gotchas.md#fonts-by-script` (RTL: use captions, not drawtext).
|
|
593
|
+
|
|
594
|
+
`--karaoke` uses real per-word timings when the transcript has them (a whisper
|
|
595
|
+
`<stem>.json` or `<stem>.words.json` next to the SRT, `{"segments": [{"words":
|
|
596
|
+
[{"word", "start", "end"}]}]}`); otherwise `--karaoke-timing` decides.
|
|
597
|
+
|
|
546
598
|
`--mode burn` (default) renders subtitles into the picture and always
|
|
547
599
|
re-encodes both streams. `--mode mux` copies video and audio untouched and
|
|
548
600
|
adds the SRT as a separate, player-toggleable subtitle stream instead —
|