ffmpeg-skill 1.14.0 → 1.15.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +12 -4
- package/SKILL.md +4 -4
- package/docs/contract.md +32 -11
- package/package.json +1 -1
- package/references/ci-platform-pitfalls.md +6 -1
- package/references/gotchas.md +70 -1
- package/references/scripts.md +49 -7
- package/scripts/_ass_overlay.py +155 -0
- package/scripts/_common/__init__.py +187 -0
- package/scripts/_common/color.py +69 -0
- package/scripts/_common/decision.py +415 -0
- package/scripts/_common/emit.py +287 -0
- package/scripts/_common/probe.py +382 -0
- package/scripts/_common/runner.py +1056 -0
- package/scripts/_common/text.py +980 -0
- package/scripts/_contract.py +19 -4
- package/scripts/caption.py +326 -92
- package/scripts/graphics.py +295 -19
- package/scripts/overlay.py +35 -2
- package/scripts/_common.py +0 -2516
package/README.md
CHANGED
|
@@ -39,7 +39,7 @@ npx ffmpeg-skill
|
|
|
39
39
|
</tr>
|
|
40
40
|
</table>
|
|
41
41
|
|
|
42
|
-
Left half is the input, right half is what the command produced. **[All
|
|
42
|
+
Left half is the input, right half is what the command produced. **[All 53 before/after demos, with the exact command under each one →](docs/demos.md)** — all of it generated from synthetic footage by `python3 demos/build.py`, so you can rebuild every frame of it yourself.
|
|
43
43
|
|
|
44
44
|
`ffmpeg-skill` is an [Agent Skill](https://docs.anthropic.com/en/docs/agents-and-tools/agent-skills) for Claude Code, Cursor, Codex and any agent that reads `SKILL.md`. It teaches the agent a fixed workflow (probe → edit losslessly where possible → check → verify) and ships **42 tools** that do the actual work with `ffmpeg` / `ffprobe`: cut, join, silence removal, fit to duration and aspect, captions and karaoke, overlays and motion graphics, HDR → SDR and LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, whole-edit project rendering, batch folders. Every tool is also an MCP tool, and the whole set is described by a machine-readable contract.
|
|
45
45
|
|
|
@@ -236,7 +236,7 @@ These are the rules the skill file gives the agent and the code enforces.
|
|
|
236
236
|
|---|---|
|
|
237
237
|
| `caption.py` | Burn SRT/ASS with font, size, colour, outline, position; build SRT from timed plain text; wraps to the safe area by measured width with `--max-lines`/`--min-duration`/`--offset`; picks a font by script for non-Latin text (`--lang`); animated and word-by-word karaoke timed to the speech energy or real word timings; optional local transcription |
|
|
238
238
|
| `overlay.py` | Logos, watermarks and titles with position, time range, opacity, fades; `--platform NAME` keeps them clear of that destination's UI; `--video` for picture-in-picture, `--chromakey` for green-screen compositing |
|
|
239
|
-
| `graphics.py` | Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs, social stickers, opening hook cards and meme captions drawn by FFmpeg from a brand kit; `--platform NAME` keeps them inside that destination's safe zone |
|
|
239
|
+
| `graphics.py` | Lower-thirds, title cards, chapter chips, progress bars, countdowns, corner bugs, social stickers, opening hook cards and meme captions drawn by FFmpeg from a brand kit; `--platform NAME` keeps them inside that destination's safe zone; `--text-render` routes shaping scripts through libass and `--emoji-assets` composites colour emoji |
|
|
240
240
|
| `color.py` | HDR10 / HLG / Dolby Vision → SDR BT.709 tone mapping, DV layer stripping, 3D LUT (.cube), colour-tag rewriting, typed primary correction (exposure/contrast/saturation/gamma/white balance/lift-gain/levels/curves) |
|
|
241
241
|
|
|
242
242
|
**Delivery**
|
|
@@ -361,6 +361,8 @@ The short list for humans. The agent-facing version, with the reasoning, is the
|
|
|
361
361
|
- **Loudness targets.** −14 LUFS / −1 dBTP for YouTube and social platforms (the `loudness.py` default), `-I -16 --tp -1.5` for podcasts, `-I -23` for broadcast. A clip measured at −40 LUFS or below is room tone, not content; raising it raises the noise. Check true peak as well as LUFS: `check.py file --platform podcast` measures both.
|
|
362
362
|
- **Frame changes first, text second.** Captions and overlays burned before a crop or resize end up off-frame. Reframe, then caption.
|
|
363
363
|
- **Cropping 16:9 to 9:16 discards 70 % of the width.** `fit.py --fit crop` centres by default; pass `--crop-x`/`--crop-y` toward the subject, or pad with `--fit pad --pad-fill blur`. Look at the contact sheet before deciding.
|
|
364
|
+
- **Emoji in captions and titles (1.15).** `caption.py`/`graphics.py --emoji-assets DIR` composites a PNG per emoji (Twemoji/Noto naming, `1f389.png`) on top of the text, because drawtext cannot load a colour emoji font at all and an installed one does not prove libass will draw it in colour — `doctor --json`'s `fonts.emoji` answers that from a render probe. Without assets the run still succeeds and reports `mode: mono`. Nothing is ever downloaded.
|
|
365
|
+
- **Indic and Thai text shaped correctly in titles and lower-thirds (1.15).** `graphics.py` renders Devanagari, Bengali, Tamil, Thai and Lao through libass automatically (`text_renderer: "ass"`), because drawtext never reorders matras or re-clusters marks; Arabic and Hebrew were already correct on a fribidi build. `--text-render drawtext` with such a script is refused, never rendered wrongly.
|
|
364
366
|
- **Non-Latin text picks a font by script (1.12).** Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari, Thai, Cyrillic and Greek cues, titles and overlays resolve a font file that covers them automatically, and a machine with no such font fails the job (`kind: input`) instead of rendering boxes. `doctor --json`'s `fonts.scripts` says which languages this machine can render; `--lang ja|ko` disambiguates Han-only text; an explicit `--font`/`--font-file` is always kept.
|
|
365
367
|
- **Silence detection finds nothing?** The default threshold is −35 dBFS. The tool prints a hint with the track's measured level; raise the threshold (`silence.py --threshold -25`) or shorten `--min-silence`.
|
|
366
368
|
- **Sync results carry a confidence.** Below 0.3, or an offset near the edge of the analysis window, is probably wrong: enlarge `--analyze-seconds` or find a clap. Recordings over ten minutes from separate devices need `sync.py --fix-drift`.
|
|
@@ -384,8 +386,11 @@ FFmpeg 8 shortened the flag column of `ffmpeg -filters`. A parser anchored on th
|
|
|
384
386
|
## Tested on real footage
|
|
385
387
|
|
|
386
388
|
**What is tested where.** The contract and the test suite (`tests/test_contract.py`,
|
|
387
|
-
`tests/test_all.py
|
|
388
|
-
|
|
389
|
+
`tests/test_all.py`, which aggregates one module per tool group — `test_analysis.py`,
|
|
390
|
+
`test_editing.py`, `test_audio.py`, `test_picture.py`, `test_delivery.py`,
|
|
391
|
+
`test_orchestration.py` — over the shared footage in `tests/_fixtures.py`) run on Linux, macOS
|
|
392
|
+
and Windows on every pull request, minus the handful of POSIX-shim tests listed under
|
|
393
|
+
[Development](#development). The real-device media corpus
|
|
389
394
|
(`tests/corpus.py`) has been run on Linux and macOS; the full corpus has **not** been run on
|
|
390
395
|
Windows yet, and neither has an install by someone other than the maintainer been reproduced
|
|
391
396
|
there — [issue #143](https://github.com/kajisho5/ffmpeg-skill/issues/143) tracks both. Treat the
|
|
@@ -400,6 +405,8 @@ type on every OS.
|
|
|
400
405
|
| **F1 0.97** | `scenes.py`, 53 hard cuts between single takes, precision 0.95, recall 1.00 at the default threshold |
|
|
401
406
|
| **exact to the sample** | `cut.py --accurate` on WAV, FLAC (44.1 kHz) and AAC → WAV; WAV stream copy within 2 ms; AAC output +21 ms of encoder priming, reported as `codec_frame` (0.9.1) |
|
|
402
407
|
| **72 / 72** | agent runs of 24 prompts (12 English edits, 8 Japanese, 4 that must be declined), three repeats, graded by an independent model: routing, honest refusals and user's language 72/72, report format 71/72, visual check whenever the picture changed 24/24 (0.8.4) |
|
|
408
|
+
| **82 / 82** | 1.15.0 run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader on 28 runs, stills extracted inside the emoji window, check.py re-run on every delivery output) on the set grown to 82 prompts (emoji captions and title cards, a Hindi and a Thai lower-third): routing 82/82, honest refusals and failures 82/82 with 0 false successes and 0 raw ffmpeg calls, report format 82/82 (both iteration-15 label defects closed: `dl8` and `he2` now carry one `Failed:`), user's language 82/82 by regex (81/82 by Opus: one Spanish report with three English labels), non-Latin glyphs 11/11 (Devanagari through `graphics.py` is fixed; Thai lower-third and captions correct), emoji visible in colour in 3/3 runs given PNG assets and reported monochrome in the one that was not, visual check 23/24, delivery 12/13 one encode and 13/13 platform pass, trigger set 40/40, Opus quality mean 4.68. Still open: the caption breaker splits phrases (`dl1`, `dl4` unchanged) — queued for 1.16.0. Tokens per run flat at 73.3k on the same 76. Details in `evals/results/iteration-16.json` |
|
|
409
|
+
| **76 / 76** | 1.14.0 run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader on 26 runs, check.py re-run on every delivery output): routing 76/76, honest refusals and failures 76/76 with 0 false successes and 0 raw ffmpeg calls, report format 76/76, user's language 76/76 by regex (75/76 by Opus: one Spanish report with three English labels), visual check 18/18, trigger set 38/38, Opus quality mean 4.58. The delivery templates did their job: 12 of 13 delivery requests went through `render.py --template`, finished in one encode (was 3 of 7) and all 13 pass their platform check (was 7 of 8). Tokens per run flat at 73.4k. Details in `evals/results/iteration-15.json` |
|
|
403
410
|
| **76 / 76** | 1.13.0 run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader) on the set grown to 76 prompts: 18 in Thai, Hindi, Hebrew, Russian, Greek, Vietnamese, Indonesian, Turkish and Italian, and 8 delivery requests (TikTok, Reels, Shorts, LinkedIn, Douyin, podcast): routing 76/76, honest refusals and failures 76/76 with 0 false successes and 0 raw ffmpeg calls, report format 76/76, user's language 76/76 across seventeen languages, visual check 18/18, trigger set 38/38, Opus quality mean 4.65 over the 26 new runs. One real defect found: Hindi through `graphics.py` (drawtext) comes out wrong-shaped even though the font covers Devanagari; captions through libass are fine (queued for 1.15.0). Four delivery runs spent a second encode for loudness, which 1.14.0's templates address. Tokens per run flat at 72.3k. Details in `evals/results/iteration-14.json` |
|
|
404
411
|
| **50 / 50** | 1.12.0 run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader) on the set grown to 50 prompts with two each in Chinese, Korean, Spanish, Portuguese, French, German and Arabic: routing 50/50, honest refusals and failures 50/50 with 0 false successes and 0 raw ffmpeg calls, report format 50/50, user's language 50/50 across nine languages, visual check 13/13, trigger set 29/29, Opus quality mean 4.83. Every non-Latin caption and lower-third picked a covering font by itself and rendered real glyphs (Arabic shaped and right-to-left); tokens per run unchanged at 72.3k. Details in `evals/results/iteration-13.json` |
|
|
405
412
|
| **36 / 36** | 1.11.1 re-run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader): routing 36/36, honest refusals and failures 36/36 with 0 false successes and 0 raw ffmpeg calls, report format 36/36, user's language 36/36, visual check 8/8, trigger set 22/22, Opus quality mean 4.75. The 1.11.1 wording did what it said (`doctor` before a job 23 of 36 runs → 0, `--json-brief` 4 → 23) and tokens per run stayed flat at 71.8k, because about 64k of every run is the host's own context; the token-diet theme closes here. Details in `evals/results/iteration-12.json` |
|
|
@@ -474,6 +481,7 @@ FFmpeg itself:
|
|
|
474
481
|
|
|
475
482
|
```bash
|
|
476
483
|
npm test # tests/test_all.py (end-to-end incl. VFR, rotated, 5.1, HDR10, drifting sources) + tests/test_contract.py
|
|
484
|
+
python3 tests/test_picture.py # one tool group on its own (analysis, editing, audio, picture, delivery, orchestration)
|
|
477
485
|
npm run release-check # pack, install, contract from the installed copy, MCP == contract, doctor, tests, contract evals
|
|
478
486
|
npm run demo # python3 demos/build.py: synthetic footage -> every before/after demo + docs/demos/*.gif
|
|
479
487
|
npm run demo:pipeline # examples/make_demo.sh: the older single end-to-end run of every script
|
package/SKILL.md
CHANGED
|
@@ -149,7 +149,7 @@ Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`,
|
|
|
149
149
|
|
|
150
150
|
## Report format
|
|
151
151
|
|
|
152
|
-
Reply in the language the request itself is written in — the user's own sentences, not a language the request talks about (a request for subtitles in another language is still answered in the language it was written in) and not the language of a tool's error text or file names.
|
|
152
|
+
Reply in the language the request itself is written in — the user's own sentences, not a language the request talks about (a request for subtitles in another language is still answered in the language it was written in) and not the language of a tool's error text or file names. Keep the field labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English: they read like log fields across languages. Everything around them — the sentences, any question, any explanation of a judgement call — is in the user's language. Never default to English because the tool names are English, and never drift because the job was short or the report is a failure: a one-line "file does not exist" is in the request's language too. A mid-conversation switch follows the user's latest message. English `Done:`/`Steps:` sentences with one word of the user's language in `Notes:` is an English report — the descriptions are in the user's language even when the values are technical.
|
|
153
153
|
|
|
154
154
|
Finish every job with this shape (numbers from `--json` or `probe.py`/`check.py`, not memory):
|
|
155
155
|
|
|
@@ -171,8 +171,6 @@ Look: final_sheet.png(字幕はセーフエリア内、ロゴは右上)
|
|
|
171
171
|
Notes: 元は VFR だったので 30 fps に揃えた。音声はモノラルだったのでステレオにした
|
|
172
172
|
```
|
|
173
173
|
|
|
174
|
-
Same shape in every other language, labels still English — zh: `Done: final.mp4 — 59.98 秒、1080x1920、30 fps、H.264` / `Steps: 0:12-1:12 剪切 -> 9:16 裁剪 -> 字幕 -> Reels 导出`; ko: `Done: final.mp4 — 59.98초, 1080x1920, 30 fps, H.264` / `Steps: 0:12-1:12 컷 -> 9:16 크롭 -> 자막 -> Reels 내보내기`.
|
|
175
|
-
|
|
176
174
|
Keep it to those five lines plus anything the user must decide. Never report success without the output probe; never describe a fix you did not run.
|
|
177
175
|
|
|
178
176
|
When a step fails, replace `Done:` with `Failed:` and keep the rest honest:
|
|
@@ -185,7 +183,7 @@ Look: not needed (nothing written)
|
|
|
185
183
|
Notes: send a valid .cube, or say if you want the clip left as is
|
|
186
184
|
```
|
|
187
185
|
|
|
188
|
-
A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure still gets all five labels, never prose headings. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change that makes a retry meaningful.
|
|
186
|
+
A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. The shortest failure still gets all five labels, never prose headings. A refusal that still delivers something is `Failed:` — the label answers the request as asked; the alternative goes in `Notes:`. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change that makes a retry meaningful.
|
|
189
187
|
|
|
190
188
|
Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message, never paraphrase it.
|
|
191
189
|
|
|
@@ -200,6 +198,8 @@ One line each, each enough to act on; open the linked `references/gotchas.md` se
|
|
|
200
198
|
- Sync/multicam `confidence` under 0.3 (or a huge offset) is probably wrong — check every camera; these align audio, never lip sync. Details: [#sync-multicam-and-drift](references/gotchas.md#sync-multicam-and-drift)
|
|
201
199
|
- "Normalised" audio can still clip (check true peak), and ambience at -40 LUFS or below must never be raised to a speech target. Details: [#loudness-and-ambience](references/gotchas.md#loudness-and-ambience)
|
|
202
200
|
- Captions burned before a crop/resize land off-frame; burned small then upscaled by `export.py` they come out soft — fit to the delivery size first. Details: [#captions-fonts-and-text-order](references/gotchas.md#captions-fonts-and-text-order)
|
|
201
|
+
- Emoji need `--emoji-assets DIR` (a PNG per glyph) to render in colour; without it they come out monochrome and the run says so. Details: [#emoji](references/gotchas.md#emoji)
|
|
202
|
+
- `graphics.py` renders Devanagari, Bengali, Tamil and Thai through libass automatically — drawtext cannot shape them.
|
|
203
203
|
- Non-Latin text picks a font by script since 1.12; `doctor --json` `fonts.scripts` says which languages this machine renders; no font = failed job. Details: [#fonts-by-script](references/gotchas.md#fonts-by-script)
|
|
204
204
|
- `--fit crop` 16:9 → 9:16 throws away 70 % of the width, 60→30 fps halves the motion, and "60 seconds" by speed or by trim are different answers — say which and why. Details: [#reframing-fps-and-duration](references/gotchas.md#reframing-fps-and-duration)
|
|
205
205
|
- TikTok/Reels cover the bottom fifth and the right column with their own UI — templates keep text out of those zones; `look.py --safe tiktok` shows them. Details: [#platform-safe-zones](references/gotchas.md#platform-safe-zones)
|
package/docs/contract.md
CHANGED
|
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
|
|
|
21
21
|
| Field | Meaning | Changes when |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
|
|
24
|
-
| `skill.version` | the npm / package.json version (`1.
|
|
24
|
+
| `skill.version` | the npm / package.json version (`1.15.1`) | any release |
|
|
25
25
|
|
|
26
26
|
A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
|
|
27
27
|
ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
|
|
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
|
|
|
88
88
|
|
|
89
89
|
| What 2.0 removes | Since | Replacement | To be ready today |
|
|
90
90
|
|---|---|---|---|
|
|
91
|
-
| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.
|
|
92
|
-
| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.
|
|
93
|
-
| `json` and `progress` in the MCP `inputSchema` | 1.
|
|
94
|
-
| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.
|
|
95
|
-
| Overwriting an existing output with only a warning | 1.
|
|
91
|
+
| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.15.1 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
|
|
92
|
+
| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.15.1 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
|
|
93
|
+
| `json` and `progress` in the MCP `inputSchema` | 1.15.1 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
|
|
94
|
+
| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.15.1 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
|
|
95
|
+
| Overwriting an existing output with only a warning | 1.15.1 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
|
|
96
96
|
|
|
97
97
|
## Skill
|
|
98
98
|
|
|
99
99
|
```json
|
|
100
100
|
{
|
|
101
101
|
"contract_version": "1.0",
|
|
102
|
-
"deprecated": [{"what": "...", "since": "1.
|
|
103
|
-
"skill": {"id": "ffmpeg-skill", "version": "1.
|
|
102
|
+
"deprecated": [{"what": "...", "since": "1.15.1", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
|
|
103
|
+
"skill": {"id": "ffmpeg-skill", "version": "1.15.1", "execution_mode": "local", "kind": "execution",
|
|
104
104
|
"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
|
|
105
105
|
"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
|
|
106
106
|
"requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
|
|
@@ -128,7 +128,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
|
|
|
128
128
|
| `output_schema` | what `--json` prints on stdout |
|
|
129
129
|
| `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
|
|
130
130
|
| `supports_json` | whether `--json` exists |
|
|
131
|
-
| `supports_json_brief` | whether `--json-brief` exists (1.
|
|
131
|
+
| `supports_json_brief` | whether `--json-brief` exists (1.15.1): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
|
|
132
132
|
| `mutates_input` | always `false`: no tool overwrites its input |
|
|
133
133
|
| `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
|
|
134
134
|
| `verification` | `{required, tools}`: which tools to run on the output afterwards |
|
|
@@ -361,7 +361,8 @@ typeface the caller didn't ask for.
|
|
|
361
361
|
|
|
362
362
|
Since 1.12 the same field also carries `scripts`: one entry per writing system the tools detect,
|
|
363
363
|
`{"ja": {"status": "available"|"missing"|"unknown", "file": "/path/to/font.ttc"|null}, "zh": ...,
|
|
364
|
-
"ko": ..., "ar": ..., "he": ..., "hi": ..., "
|
|
364
|
+
"ko": ..., "ar": ..., "he": ..., "hi": ..., "bn": ..., "ta": ..., "th": ..., "lo": ..., "ru": ...,
|
|
365
|
+
"el": ...}` (`bn`, `ta` and `lo` were added in 1.15). It answers "which
|
|
365
366
|
languages can this machine actually render", which no filter or encoder capability asks:
|
|
366
367
|
`available` means `fc-list :lang=<code>` (Linux/macOS) or a known system font file (Windows) covers
|
|
367
368
|
the script, `missing` means fontconfig knows none, `unknown` means there is no working fontconfig to
|
|
@@ -378,6 +379,25 @@ line to one screen width; a longer explanation, and the per-script files, are `-
|
|
|
378
379
|
`--lang`/`--language` (caption, graphics) is the hint that says whether Han-only text is Chinese,
|
|
379
380
|
Japanese or Korean.
|
|
380
381
|
|
|
382
|
+
Since 1.15 `fonts` also carries `emoji`: `{"mode": "color"|"png"|"mono"|"none", "color_font": "Noto
|
|
383
|
+
Color Emoji"|null, "color_font_file": "..."|null, "libass_color": true|false|null, "assets":
|
|
384
|
+
"/path"|null, "detail": "...", "fix": "..."}`. `libass_color` comes from a **render probe** — one
|
|
385
|
+
64x64 frame with an emoji cue through `subtitles=`, chroma-tested — because an installed colour
|
|
386
|
+
emoji family proves nothing: Noto Color Emoji installs cleanly on builds whose libass still draws a
|
|
387
|
+
monochrome outline. `null` means the probe was not run: `contract --json --static` (and every other
|
|
388
|
+
static/JSON-only path) skips it, exactly as it skips the rest of the environment detection. `mode`
|
|
389
|
+
is `color` when the probe says colour, else `png` when an emoji assets directory resolves, else
|
|
390
|
+
`mono` when some installed face has a glyph, else `none`. Informational like the rest of `fonts`:
|
|
391
|
+
it never moves `ok` or any tool's `usable`.
|
|
392
|
+
|
|
393
|
+
`caption.py`, `graphics.py` and `overlay.py` gained `--emoji auto|color|png|mono|none`,
|
|
394
|
+
`--emoji-assets DIR`, `--emoji-scale FLOAT` and `--emoji-max N` in 1.15; `graphics.py` also gained
|
|
395
|
+
`--text-render auto|ass|drawtext` and `--write-ass PATH`. New success keys: `emoji`
|
|
396
|
+
(`{"mode", "count", "clusters", "assets", "missing", "overlays"}`) on `caption.py` and
|
|
397
|
+
`graphics.py`, and `text_renderer` (`"ass"`|`"drawtext"`), `script` and `ass` (the generated file,
|
|
398
|
+
when one was written) on `graphics.py`. `--json-brief` carries `emoji.mode` and `emoji.count` only.
|
|
399
|
+
All additive: `contract_version` stays 1.0.
|
|
400
|
+
|
|
381
401
|
## Invocation
|
|
382
402
|
|
|
383
403
|
Structured arguments are the canonical way to call a tool, on the CLI or through MCP.
|
|
@@ -430,7 +450,8 @@ measurement dicts flattened, plus any numeric top-level key such as `expected_du
|
|
|
430
450
|
1.x keys are not moved; the environment variable only adds the key, and its absence is the
|
|
431
451
|
default until 2.0.
|
|
432
452
|
|
|
433
|
-
Success is decided by `verify_output` in `_common.py
|
|
453
|
+
Success is decided by `verify_output` in `_common` (`scripts/_common/probe.py`), not by the
|
|
454
|
+
ffmpeg exit code alone:
|
|
434
455
|
the file must exist, be non-empty and give ffprobe at least one stream. A tool that ran
|
|
435
456
|
ffmpeg successfully but has no usable artifact fails with `kind: output` (a 0-byte file is
|
|
436
457
|
removed so a later step cannot mistake it for a result).
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.15.1",
|
|
4
4
|
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ffmpeg",
|
|
@@ -124,7 +124,12 @@ these had ever shown up before.
|
|
|
124
124
|
- **`drawtext` `boxborderw=v|h` (and the four-value form) is 6.1+.** 5.x and 6.0 fail the whole
|
|
125
125
|
filter with "Error setting option boxborderw to value 9|16". `_common.drawtext_boxborderw()`
|
|
126
126
|
emits the larger single value on older builds (`_common.ffmpeg_version()` parses
|
|
127
|
-
`ffmpeg -version` once; it is the only place the tools branch on a version string).
|
|
127
|
+
`ffmpeg -version` once; it is the only place the tools branch on a version string). Both names
|
|
128
|
+
are still spelled exactly that way: `_common` is a package since the refactor release after
|
|
129
|
+
1.15.0, and its `__init__.py` re-exports every name the single module defined, so
|
|
130
|
+
`_common.drawtext_boxborderw()`, `_common.ffmpeg_version()` and `_common.bt709_tag_args()`
|
|
131
|
+
below resolve through the facade unchanged. The definitions are in `_common/text.py`,
|
|
132
|
+
`_common/runner.py` and `_common/color.py` respectively.
|
|
128
133
|
- **`showwaves` keeps emitting frames after the audio ends, `-shortest` notwithstanding.** A
|
|
129
134
|
12 s source came out 14.08 s on 5.1.1. `waveform.py` now also passes `-t <source duration>`.
|
|
130
135
|
- **`-display_rotation` is 6.0+.** Only the test fixture builder used it (to make a rotated
|
package/references/gotchas.md
CHANGED
|
@@ -109,7 +109,7 @@ libass and drawtext draw an empty box per character they have no glyph for, and
|
|
|
109
109
|
ffmpeg still exits 0 — a video full of tofu is the classic "it worked" failure.
|
|
110
110
|
Since 1.12 `caption.py`, `graphics.py` and `overlay.py --text` detect the script
|
|
111
111
|
of the text they are about to draw (Japanese, Chinese, Korean, Arabic, Hebrew,
|
|
112
|
-
Devanagari, Thai, Cyrillic, Greek) and resolve a font file that covers it,
|
|
112
|
+
Devanagari, Bengali, Tamil, Thai, Lao, Cyrillic, Greek) and resolve a font file that covers it,
|
|
113
113
|
printing one line — `font: /usr/share/fonts/.../wqy-zenhei.ttc (covers ko)`.
|
|
114
114
|
**No font for the script is a failed job** (`kind: input`), not a warning.
|
|
115
115
|
|
|
@@ -145,6 +145,75 @@ printing one line — `font: /usr/share/fonts/.../wqy-zenhei.ttc (covers ko)`.
|
|
|
145
145
|
letterforms. Nothing in the tools checks this, so on an unknown machine a
|
|
146
146
|
caption is the safe place for Arabic/Hebrew.
|
|
147
147
|
|
|
148
|
+
**Shaping is a second problem, and it is not the same one.** A font that covers
|
|
149
|
+
the script still has to be *shaped*: Devanagari matras are reordered, Thai and
|
|
150
|
+
Lao marks re-cluster. drawtext does bidi and Arabic joining correctly on a build
|
|
151
|
+
compiled with fribidi (so Arabic and Hebrew are already right today), but it
|
|
152
|
+
never reorders or re-clusters, because it does not use harfbuzz on any build.
|
|
153
|
+
libass does. Since 1.15 `graphics.py` therefore renders a shaping script
|
|
154
|
+
(`hi bn ta te kn ml gu pa si th lo km my`) through libass automatically — one
|
|
155
|
+
generated `<output>_gfx.ass` next to the file, `text_renderer: "ass"` in the
|
|
156
|
+
JSON — and `--text-render auto|ass|drawtext` overrides it; `--text-render
|
|
157
|
+
drawtext` with such a script is a refusal, not a wrong frame. Latin, CJK and
|
|
158
|
+
Arabic output is unchanged. `overlay.py --text` has no ASS route yet and refuses
|
|
159
|
+
a shaping script, naming `caption.py`/`graphics.py`.
|
|
160
|
+
|
|
161
|
+
### Emoji
|
|
162
|
+
Colour emoji need **either** a libass that can draw colour glyphs **or** a
|
|
163
|
+
directory of PNGs — and on most builds only the second one exists.
|
|
164
|
+
|
|
165
|
+
- **An installed colour emoji font proves nothing.** Noto Color Emoji installs
|
|
166
|
+
cleanly on Ubuntu and libass on the distro ffmpeg still renders a monochrome
|
|
167
|
+
outline (`Glyph 0x1F389 not found, broken font? Trying all charmaps`). The
|
|
168
|
+
only honest test is a render, which is what `doctor` runs:
|
|
169
|
+
`doctor --json` → `.fonts.emoji` (`mode`: `color` / `png` / `mono` / `none`,
|
|
170
|
+
plus `libass_color` from that render). `contract --json --static` is the way
|
|
171
|
+
to get the contract without running it; `doctor` itself always probes.
|
|
172
|
+
- **drawtext cannot load an emoji font at all.** Not a degraded render: a
|
|
173
|
+
CBDT/sbix face fails filter initialisation outright (`Could not set font size
|
|
174
|
+
to 48 pixels: invalid library handle`, and at the font's own strike
|
|
175
|
+
`Monocromatic (1bpp) fonts are not supported.`) and no file is written. That
|
|
176
|
+
is why the colour route is a PNG overlay, never a drawtext font.
|
|
177
|
+
- **PNGs**: `--emoji-assets DIR`, a directory of files named by code point in
|
|
178
|
+
the Twemoji/Noto convention — lowercase hex joined by `-`: `1f389.png`,
|
|
179
|
+
`1f1ef-1f1f5.png`, `1f469-200d-1f4bb.png`. Twemoji's `assets/72x72`
|
|
180
|
+
(CC-BY 4.0) and Noto Emoji's `png/128` (OFL/Apache-2.0) are the two people
|
|
181
|
+
already have. **The skill never downloads anything**: a missing directory is
|
|
182
|
+
`kind: input`, never a silent fetch. The same directory can come from
|
|
183
|
+
`brand.json` (`styles.caption.emoji_assets`) or `FFMPEG_SKILL_EMOJI_ASSETS`.
|
|
184
|
+
- **Mixed lines are the normal case.** The text stays in the ASS with an
|
|
185
|
+
invisible placeholder reserving exactly the emoji's box, and the PNG is
|
|
186
|
+
composited on top after the `ass=` filter — libass keeps the shaping and the
|
|
187
|
+
karaoke, including inside a `\kf` run (the placeholder is its own
|
|
188
|
+
zero-duration segment).
|
|
189
|
+
- **What counts as an emoji.** A cluster only starts at an emoji *base*.
|
|
190
|
+
U+200D ZWJ and U+200C ZWNJ are ordinary Indic/Persian orthography (`क्ष` is
|
|
191
|
+
ka + virama + ZWJ + ssa) and are glue only *between* two emoji; a VS16 or a
|
|
192
|
+
skin-tone modifier after a plain letter is not an emoji either. U+FE0E
|
|
193
|
+
(VS15) asks for the character, so `❤︎` is text and `❤️` is an emoji.
|
|
194
|
+
- **Placement tolerance.** The position comes from the same averaged em table
|
|
195
|
+
the wrap uses, so an emoji at the start or end of a line is exact and one in
|
|
196
|
+
the middle of a Latin line drifts by the accumulated rounding of the
|
|
197
|
+
characters before it — **measured at 17 px, 0.28 em, about 3 % of the line
|
|
198
|
+
width** on a 24 px caption over a 1280-wide frame, always landing inside the
|
|
199
|
+
reserved gap rather than on a glyph. It never leaves the safe area. An RTL
|
|
200
|
+
line (Arabic, Hebrew) is laid out right-to-left, so the position is measured
|
|
201
|
+
from the rendered end of the line, not the logical prefix.
|
|
202
|
+
- **Degraded paths are honest, not silent.** `mode: mono` means libass draws
|
|
203
|
+
whatever glyph the text font has (it has a fallback chain), exits 0, and says
|
|
204
|
+
so in a `warning:` line and in `notes` — so a `graphics.py` run that would
|
|
205
|
+
otherwise use drawtext, which loads one font file and has **no** fallback
|
|
206
|
+
chain, is routed to libass rather than reporting a monochrome glyph it would
|
|
207
|
+
actually draw as an empty box; a run pinned to `--text-render drawtext`
|
|
208
|
+
degrades to `none` and says so. `mode: none` strips the cluster from the drawn
|
|
209
|
+
text (a missing decoration must not fail a delivery) but refuses a
|
|
210
|
+
`graphics.py` template whose text is *only* emoji — that frame would be blank.
|
|
211
|
+
`--emoji-max` (default 60) caps the overlay count; `--emoji-max 0` means none.
|
|
212
|
+
- **`--animate` moves the PNG too.** The overlay gets a matching alpha fade, so
|
|
213
|
+
the emoji arrives and leaves with the line instead of popping in.
|
|
214
|
+
- `overlay.py --text` has no PNG route: `--emoji png|color` there names
|
|
215
|
+
`caption.py`/`graphics.py` instead.
|
|
216
|
+
|
|
148
217
|
### Reframing, fps and duration
|
|
149
218
|
`--fit crop` to reach 9:16 from 16:9 throws away 70 % of the width: a wide shot
|
|
150
219
|
loses people at the edges. Check the sheet; pad (bars), `--crop-x`/`--crop-y`
|
package/references/scripts.md
CHANGED
|
@@ -550,17 +550,38 @@ Inside this skill, call the scripts directly; the server is for other hosts.
|
|
|
550
550
|
graphics.py INPUT --template lower-third|title|chapter|progress|countdown|bug|sticker|hook|meme
|
|
551
551
|
[--name] [--title] [--subtitle] [--text] [--top] [--bottom] [--duration 3]
|
|
552
552
|
[--from N] [--start S] [--end E] [--position CORNER] [--margin PX] [--platform NAME]
|
|
553
|
-
[--brand brand.json] [--primary RRGGBB] [--scale 1.0] [--lang XX]
|
|
553
|
+
[--brand brand.json] [--primary RRGGBB] [--scale 1.0] [--lang XX]
|
|
554
|
+
[--text-render auto|ass|drawtext] [--write-ass OUT.ass]
|
|
555
|
+
[--emoji auto|color|png|mono|none] [--emoji-assets DIR] [--emoji-scale 1.0] [--emoji-max 60] [-o OUT]
|
|
554
556
|
```
|
|
555
557
|
Drawn with drawbox/drawtext/overlay — no PNG assets needed. Sizes scale with
|
|
556
558
|
the frame's short side; colours, font and safe margin come from `--brand`.
|
|
557
559
|
Lower-third slides in over 0.4 s and out over 0.3 s; title/chapter/bug fade.
|
|
558
560
|
Non-Latin `--name`/`--title`/`--subtitle` text picks a font file by script the
|
|
559
561
|
same way `caption.py` does (`--lang XX` disambiguates Han-only text; no font for
|
|
560
|
-
the script fails the job).
|
|
561
|
-
|
|
562
|
-
|
|
563
|
-
|
|
562
|
+
the script fails the job). Arabic and Hebrew are already correct through drawtext
|
|
563
|
+
on a build with `--enable-libfribidi` (bidi + joining). What drawtext cannot do on
|
|
564
|
+
any build is reorder and re-cluster — Devanagari matras, Thai/Lao mark stacking —
|
|
565
|
+
because it does not use harfbuzz. Since 1.15 `--text-render auto` (the default)
|
|
566
|
+
therefore routes those scripts through libass: the template's geometry is written
|
|
567
|
+
as a generated `<output>_gfx.ass` (`--write-ass PATH` names it) and burned with
|
|
568
|
+
`ass=`, reported as `text_renderer: "ass"` with `script` and `ass` in the JSON.
|
|
569
|
+
Latin/CJK/Arabic frames are pixel-identical to 1.14 (the drawtext command
|
|
570
|
+
line is not: since 1.15 every drawn label is passed as `textfile=<tmp>:expansion=none`
|
|
571
|
+
rather than `text=`, so a `--dry-run` compared against 1.14 differs by design). `--text-render ass` forces the
|
|
572
|
+
route; `--text-render drawtext` with a shaping script is refused by name rather
|
|
573
|
+
than rendering a wrong frame. See `references/gotchas.md#fonts-by-script`.
|
|
574
|
+
`--emoji*` works as on `caption.py` below; a template whose text is *only* emoji
|
|
575
|
+
and that this machine can draw none of is `kind: input`, because that frame would
|
|
576
|
+
be blank. `--emoji none` strips the clusters from the drawn text, and a job whose
|
|
577
|
+
emoji would fall to `mode: mono` is routed through libass (which has a font
|
|
578
|
+
fallback chain) instead of drawtext (which does not, and would draw an empty box).
|
|
579
|
+
|
|
580
|
+
Every drawn label goes to drawtext as `textfile=<path>:expansion=none`. The file
|
|
581
|
+
is UTF-8, mode 0600, in a private per-run temp directory created with
|
|
582
|
+
`tempfile.mkdtemp()`, written only when the command that names it actually runs
|
|
583
|
+
(so `--dry-run` and the ASS route write nothing) and removed when the process
|
|
584
|
+
ends. A plan printed by `--dry-run` therefore names a path that does not exist.
|
|
564
585
|
|
|
565
586
|
All three are usable from a `render.py` project too: a `graphics[]` entry takes `text`, `top`,
|
|
566
587
|
`bottom`, `duration`, `margin` and `platform` alongside the older keys.
|
|
@@ -642,7 +663,8 @@ caption.py INPUT --srt FILE | --ass FILE | --text CUES.txt [--write-srt OUT.srt]
|
|
|
642
663
|
[--max-lines N] [--min-duration S]
|
|
643
664
|
[--font NAME] [--fonts-dir DIR] [--size N] [--color RRGGBB] [--outline N] [--outline-color RRGGBB]
|
|
644
665
|
[--bold] [--box] [--position bottom|top|center|top-left|...] [--margin N]
|
|
645
|
-
[--animate none|fade|pop|slide] [--karaoke [--highlight-color RRGGBB]] [--write-ass OUT.ass]
|
|
666
|
+
[--animate none|fade|pop|slide] [--karaoke [--highlight-color RRGGBB]] [--write-ass OUT.ass]
|
|
667
|
+
[--emoji auto|color|png|mono|none] [--emoji-assets DIR] [--emoji-scale 1.0] [--emoji-max 60] [-o OUT]
|
|
646
668
|
caption.py --text CUES.txt --write-srt OUT.srt # generate the SRT only
|
|
647
669
|
```
|
|
648
670
|
Text cue format, one per line: `0:00-0:03 Hello`, `00:00:03.500 --> 00:00:06 Two | lines`,
|
|
@@ -656,7 +678,18 @@ word from `--color` to `--highlight-color` across the cue; `--karaoke-timing
|
|
|
656
678
|
energy` (default) follows the speech loudness in the audio, `even` splits the
|
|
657
679
|
cue equally (word timing is derived, not transcribed). The ASS is kept next to the
|
|
658
680
|
user can hand-tune timings and re-run with `--ass`.
|
|
659
|
-
|
|
681
|
+
Emoji (1.15): `--emoji-assets DIR` is a directory of PNGs named by code point
|
|
682
|
+
(`1f389.png`, `1f1ef-1f1f5.png`, `1f469-200d-1f4bb.png` — the Twemoji/Noto
|
|
683
|
+
convention), also read from `brand.json` `styles.caption.emoji_assets` and
|
|
684
|
+
`FFMPEG_SKILL_EMOJI_ASSETS`. With one, the cue text keeps its place in the ASS
|
|
685
|
+
with an invisible placeholder reserving the emoji's box and each PNG is
|
|
686
|
+
composited on top (`--emoji-scale` sizes the box, `--emoji-max` caps the count).
|
|
687
|
+
Without one the run still succeeds and says `emoji: {"mode": "mono"}` plus a
|
|
688
|
+
warning; `--emoji none` strips them; `--emoji color` insists on a colour-capable
|
|
689
|
+
libass and refuses otherwise. Nothing is ever downloaded. What this machine can
|
|
690
|
+
do: `doctor --json` → `.fonts.emoji`. Details: `references/gotchas.md#emoji`.
|
|
691
|
+
|
|
692
|
+
Readable by default (1.12, rebalanced in 1.15): every cue is wrapped to the safe area (90 % of the
|
|
660
693
|
frame width) at the chosen `--size`, measured per script — CJK and Thai count a
|
|
661
694
|
full em per character, Latin per character from a table read off DejaVu Sans (so
|
|
662
695
|
an all-caps line measures as wide as it draws), Cyrillic/Greek about 0.55,
|
|
@@ -710,6 +743,7 @@ stream selection would have picked.
|
|
|
710
743
|
### overlay.py — logo, image, title, video picture-in-picture, chroma key
|
|
711
744
|
```
|
|
712
745
|
overlay.py INPUT --image PNG [--scale W | --scale-percent P] | --text "..." [--font-file F.ttf] [--font-size N] [--box]
|
|
746
|
+
[--emoji auto|mono|none] [--emoji-assets DIR]
|
|
713
747
|
| --video CLIP [--chromakey COLOR [--chromakey-similarity 0-1] [--chromakey-blend 0-1]]
|
|
714
748
|
[--position top-right|bottom-left|center|X,Y] [--margin N] [--platform NAME] [--start T] [--end T] [--fade S] [--opacity 0-1] [-o OUT]
|
|
715
749
|
```
|
|
@@ -725,6 +759,14 @@ input's audio is kept, the PiP layer's own audio is dropped. `--chromakey`
|
|
|
725
759
|
`--fade S` fades the overlay in at `--start` (or 0); the fade-out happens
|
|
726
760
|
only at `--end`, so a logo with no `--end` stays to the last frame.
|
|
727
761
|
|
|
762
|
+
Since 1.15 the drawn text goes to drawtext in a **file** (`textfile=`,
|
|
763
|
+
`expansion=none`), so `'` and `%` survive verbatim — `--text "it's 100% done"`
|
|
764
|
+
used to lose both. `overlay.py` still draws through drawtext, which cannot shape
|
|
765
|
+
Devanagari/Thai-class scripts and cannot load a colour emoji font: a shaping
|
|
766
|
+
script is refused by name pointing at `caption.py`/`graphics.py`, and
|
|
767
|
+
`--emoji png|color` is refused the same way (`--emoji mono`, the default here,
|
|
768
|
+
draws whatever glyph the text font has; `--emoji none` strips them).
|
|
769
|
+
|
|
728
770
|
### sync.py — offset detection, alignment, drift correction
|
|
729
771
|
```
|
|
730
772
|
sync.py REFERENCE SECOND [--json] [--max-offset 30] [--analyze-seconds 120] [--fix-drift [--drift-window 60]]
|
|
@@ -0,0 +1,155 @@
|
|
|
1
|
+
#!/usr/bin/env python3
|
|
2
|
+
"""Render a set of positioned text elements as an ASS file, so libass draws them instead of
|
|
3
|
+
drawtext.
|
|
4
|
+
|
|
5
|
+
Private helper (leading underscore): not a tool, no TOOL_META entry, no contract or MCP surface.
|
|
6
|
+
|
|
7
|
+
Why it exists (1.15): drawtext cannot SHAPE. On a build with fribidi it gets bidi and Arabic
|
|
8
|
+
joining right, but it never reorders or re-clusters -- Devanagari matras come out in logical
|
|
9
|
+
order and Thai/Lao marks stack wrongly -- because drawtext does not use harfbuzz even in an
|
|
10
|
+
--enable-libharfbuzz build. libass does. `graphics.py` keeps computing exactly the geometry it
|
|
11
|
+
computed before; only the renderer changes, and only for text that needs it.
|
|
12
|
+
|
|
13
|
+
Each element is a dict:
|
|
14
|
+
|
|
15
|
+
{"text", "x", "y", "size", "color", "font", "bold", "outline", "outline_color",
|
|
16
|
+
"shadow", "align", "start", "end", "fade", "move", "scale_t", "box", "box_color"}
|
|
17
|
+
|
|
18
|
+
`align` is the ASS numpad alignment of (x, y) -- 7 is top-left, 5 centre, 2 bottom-centre -- so
|
|
19
|
+
a centred title needs no text-width measurement at all. `move` is (x1, y1, x2, y2, t1, t2) in
|
|
20
|
+
milliseconds relative to the line's own start; `fade` is (in_ms, out_ms); `scale_t` is
|
|
21
|
+
(t1_ms, t2_ms, percent) for a pop.
|
|
22
|
+
"""
|
|
23
|
+
from typing import Any, Dict, List, Optional, Sequence
|
|
24
|
+
|
|
25
|
+
|
|
26
|
+
# One sentinel character stands in drawn text where an emoji was, so the placeholder is inserted
|
|
27
|
+
# AFTER the brace-stripping that keeps caller-supplied text from injecting override commands.
|
|
28
|
+
EMOJI_SENTINEL = "\ue000"
|
|
29
|
+
|
|
30
|
+
|
|
31
|
+
def emoji_placeholder(box_px: float) -> str:
|
|
32
|
+
"""The ASS override that reserves exactly `box_px` of advance and draws nothing.
|
|
33
|
+
|
|
34
|
+
Measured, not assumed (1.15 spec, open question 2). U+2588 FULL BLOCK is NOT 1.0 em: its
|
|
35
|
+
advance measured 0.83 em in FreeSans, 0.79 in WenQuanYi Zen Hei and 0.66 in DejaVu Sans,
|
|
36
|
+
IPAPGothic and Loma, so a block reserves the wrong gap in every face this repo resolves.
|
|
37
|
+
The spec's fallback, alpha-hidden figure spaces (U+2007), measured 0.46-0.55 em and only
|
|
38
|
+
quantises the gap to half an em. What IS exact in all five faces is that same alpha-hidden
|
|
39
|
+
whitespace carried by `\fsp` (letter spacing, in script pixels) on a zero-width space: the
|
|
40
|
+
next glyph starts exactly `box_px` later, with no font dependence at all (verified by render,
|
|
41
|
+
including inside a karaoke run, where the placeholder is its own zero-duration \kf segment).
|
|
42
|
+
`\r` restores the style for the rest of the line.
|
|
43
|
+
"""
|
|
44
|
+
return "{\\alpha&HFF&\\fsp%.1f}\u200b{\\r}" % box_px
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
def ass_time(sec: float) -> str:
|
|
48
|
+
cs = int(round(max(0.0, sec) * 100))
|
|
49
|
+
h, rem = divmod(cs, 360000)
|
|
50
|
+
m, rem = divmod(rem, 6000)
|
|
51
|
+
s, cs = divmod(rem, 100)
|
|
52
|
+
return f"{h}:{m:02d}:{s:02d}.{cs:02d}"
|
|
53
|
+
|
|
54
|
+
|
|
55
|
+
def ass_color(hex_rgb: str, alpha: int = 0) -> str:
|
|
56
|
+
h = str(hex_rgb).lstrip("#")
|
|
57
|
+
if len(h) != 6:
|
|
58
|
+
raise ValueError(f"colour must be RRGGBB hex, got '{hex_rgb}'")
|
|
59
|
+
return f"&H{alpha:02X}{h[4:6]}{h[2:4]}{h[0:2]}".upper()
|
|
60
|
+
|
|
61
|
+
|
|
62
|
+
def ass_field(name: str) -> str:
|
|
63
|
+
"""A font name or style name for a comma-delimited ASS field: no escape mechanism exists, so
|
|
64
|
+
the delimiters are dropped (the same call caption.py's ass_font_name() makes)."""
|
|
65
|
+
out = "".join(ch for ch in str(name or "") if ord(ch) >= 0x20 and ch not in ",:\\'")
|
|
66
|
+
return out or "Sans"
|
|
67
|
+
|
|
68
|
+
|
|
69
|
+
# After a literal backslash, these characters would start an ASS sequence libass acts on
|
|
70
|
+
# (\N, \n, \h) or an override block (\{ is an escape, so \\{ is ambiguous). A zero-width space
|
|
71
|
+
# between the two breaks the sequence without changing what the reader sees.
|
|
72
|
+
_ASS_AFTER_BACKSLASH = frozenset("Nnh{}")
|
|
73
|
+
|
|
74
|
+
|
|
75
|
+
def ass_escape(text: str) -> str:
|
|
76
|
+
"""Element text for a Dialogue, with every character the user typed still in it.
|
|
77
|
+
|
|
78
|
+
`{` and `}` would open and close an override block -- real style and animation commands
|
|
79
|
+
(\\pos, \\t, \\fscx), so caller-supplied text containing them could reposition, rescale or
|
|
80
|
+
recolour itself and everything after it. libass has escapes for exactly this (`\\{`, `\\}`),
|
|
81
|
+
so 1.15.0 escapes them instead of deleting them: `A {b} c \\ d` now reaches the picture
|
|
82
|
+
verbatim through the ASS route, the way it already did through drawtext. A literal backslash
|
|
83
|
+
needs no escape of its own in libass (`\\\\` renders as TWO backslashes, it is not an escape);
|
|
84
|
+
only a backslash immediately before one of _ASS_AFTER_BACKSLASH is ambiguous, and a
|
|
85
|
+
zero-width space parts them.
|
|
86
|
+
"""
|
|
87
|
+
s = str(text or "")
|
|
88
|
+
out = []
|
|
89
|
+
for i, ch in enumerate(s):
|
|
90
|
+
if ch == "{":
|
|
91
|
+
out.append("\\{")
|
|
92
|
+
elif ch == "}":
|
|
93
|
+
out.append("\\}")
|
|
94
|
+
elif ch == "\n":
|
|
95
|
+
out.append("\\N")
|
|
96
|
+
elif ch == "\\":
|
|
97
|
+
out.append("\\\u200b" if (i + 1 < len(s) and s[i + 1] in _ASS_AFTER_BACKSLASH) else "\\")
|
|
98
|
+
else:
|
|
99
|
+
out.append(ch)
|
|
100
|
+
return "".join(out)
|
|
101
|
+
|
|
102
|
+
|
|
103
|
+
def ass_text(text: str) -> str:
|
|
104
|
+
"""Back-compatible name for ass_escape()."""
|
|
105
|
+
return ass_escape(text)
|
|
106
|
+
|
|
107
|
+
|
|
108
|
+
def text_overlay_ass(elements: Sequence[Dict[str, Any]], *, play_w: int, play_h: int,
|
|
109
|
+
path: str, fonts_dir: Optional[str] = None) -> str:
|
|
110
|
+
"""Write `elements` as an ASS file at `path` and return the path."""
|
|
111
|
+
header = [
|
|
112
|
+
"[Script Info]", "ScriptType: v4.00+", f"PlayResX: {play_w}", f"PlayResY: {play_h}",
|
|
113
|
+
"WrapStyle: 2", "ScaledBorderAndShadow: yes", "",
|
|
114
|
+
"[V4+ Styles]",
|
|
115
|
+
"Format: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, "
|
|
116
|
+
"BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, "
|
|
117
|
+
"BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding",
|
|
118
|
+
]
|
|
119
|
+
events: List[str] = []
|
|
120
|
+
for i, el in enumerate(elements):
|
|
121
|
+
name = f"E{i}"
|
|
122
|
+
colour = ass_color(el.get("color", "FFFFFF"))
|
|
123
|
+
outline_colour = ass_color(el.get("outline_color", "000000"))
|
|
124
|
+
box = bool(el.get("box"))
|
|
125
|
+
back = ass_color(el.get("box_color", el.get("outline_color", "000000")),
|
|
126
|
+
int(el.get("box_alpha", 0)))
|
|
127
|
+
header.append(
|
|
128
|
+
f"Style: {name},{ass_field(el.get('font') or 'Sans')},{int(round(el['size']))},"
|
|
129
|
+
f"{colour},{colour},{outline_colour},{back},{-1 if el.get('bold') else 0},0,0,0,"
|
|
130
|
+
f"100,100,0,0,{3 if box else 1},{float(el.get('outline', 0)):.1f},"
|
|
131
|
+
f"{float(el.get('shadow', 0)):.1f},{int(el.get('align', 7))},0,0,0,1")
|
|
132
|
+
tags = [f"\\an{int(el.get('align', 7))}"]
|
|
133
|
+
move = el.get("move")
|
|
134
|
+
if move:
|
|
135
|
+
x1, y1, x2, y2, t1, t2 = move
|
|
136
|
+
tags.append(f"\\move({x1:.0f},{y1:.0f},{x2:.0f},{y2:.0f},{t1:.0f},{t2:.0f})")
|
|
137
|
+
else:
|
|
138
|
+
tags.append(f"\\pos({el['x']:.0f},{el['y']:.0f})")
|
|
139
|
+
fade = el.get("fade")
|
|
140
|
+
if fade:
|
|
141
|
+
tags.append(f"\\fad({fade[0]:.0f},{fade[1]:.0f})")
|
|
142
|
+
scale_t = el.get("scale_t")
|
|
143
|
+
if scale_t:
|
|
144
|
+
t1, t2, pct = scale_t
|
|
145
|
+
tags.append(f"\\fscx{pct:.0f}\\fscy{pct:.0f}\\t({t1:.0f},{t2:.0f},\\fscx100\\fscy100)")
|
|
146
|
+
events.append(
|
|
147
|
+
f"Dialogue: 0,{ass_time(el['start'])},{ass_time(el['end'])},{name},,0,0,0,,"
|
|
148
|
+
"{" + "".join(tags) + "}" + ass_text(el["text"]).replace(
|
|
149
|
+
EMOJI_SENTINEL, emoji_placeholder(el.get("box_px") or el["size"])))
|
|
150
|
+
body = "\n".join(header + ["", "[Events]",
|
|
151
|
+
"Format: Layer, Start, End, Style, Name, MarginL, MarginR, "
|
|
152
|
+
"MarginV, Effect, Text"] + events) + "\n"
|
|
153
|
+
with open(path, "w", encoding="utf-8-sig") as fh:
|
|
154
|
+
fh.write(body)
|
|
155
|
+
return path
|