ffmpeg-skill 1.12.0 → 1.13.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -5
- package/SKILL.md +19 -16
- package/docs/contract.md +24 -9
- package/package.json +1 -1
- package/references/gotchas.md +8 -0
- package/references/scripts.md +63 -6
- package/scripts/_contract.py +9 -4
- package/scripts/audio.py +99 -5
- package/scripts/check.py +15 -0
- package/scripts/loudness.py +3 -0
- package/scripts/render.py +64 -11
package/README.md
CHANGED
|
@@ -196,9 +196,9 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
|
|
|
196
196
|
|
|
197
197
|
| Tool | What it does |
|
|
198
198
|
|---|---|
|
|
199
|
-
| `audio.py` | Voice clean-up chain, FFT denoise, typed compressor / limiter / gate, music bed with sidechain ducking, fades, 5.1 → stereo, track replacement, extraction (`-o out.wav`), `--audio-stream N` |
|
|
199
|
+
| `audio.py` | Voice clean-up chain at three strengths (`--voice light\|medium\|strong`), FFT denoise, typed compressor / limiter / gate, music bed with sidechain ducking (`--duck-amount/-threshold/-attack/-release`), a never-ducked effects bed (`--effects`), `--stereo-widen`, fades, 5.1 → stereo, track replacement, extraction (`-o out.wav`), `--audio-stream N` |
|
|
200
200
|
| `sync.py` | Offset between two recordings by audio cross-correlation (1 ms, pure Python), clock-drift correction; aligned video or audio out (audio-to-audio only — no lip-sync/face detection) |
|
|
201
|
-
| `loudness.py` | Two-pass EBU R128 `loudnorm` to −14 LUFS / −1 dBTP or any target, video stream-copied; the written file is measured again and re-encoded until it meets `--tp` (lossy encoders overshoot); `--measure-only` |
|
|
201
|
+
| `loudness.py` | Two-pass EBU R128 `loudnorm` to −14 LUFS / −1 dBTP or any target (`--lra` for the range), video stream-copied; the written file is measured again and re-encoded until it meets `--tp` (lossy encoders overshoot); `--measure-only` |
|
|
202
202
|
|
|
203
203
|
**Picture**
|
|
204
204
|
|
|
@@ -215,14 +215,14 @@ These are the rules the skill file gives the agent and the code enforces. Togeth
|
|
|
215
215
|
|---|---|
|
|
216
216
|
| `export.py` | Presets `youtube`, `youtube4k`, `reels`, `x`, `prores`, `h265`, `gif`, all tagged BT.709; `--normalize` meets the platform's loudness in the same call (`render.py` turns it on by default for platform presets) |
|
|
217
217
|
| `proxy.py` | Small, low-bitrate proxy for downstream AI analysis/preview/editing decisions — resize by `--width`/`--scale`, proxy-grade `--crf` (deprecated alias of `--quality`), `--fps`, `--no-audio`; not a delivery preset |
|
|
218
|
-
| `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, broadcast and podcast specs, with the fix for each failure and a `format` / `judgement` kind per row |
|
|
218
|
+
| `check.py` | PASS / WARN / FAIL against YouTube, Shorts, Reels, TikTok, X, LinkedIn, broadcast and podcast specs (podcast also reports chapter markers and channel count), with the fix for each failure and a `format` / `judgement` kind per row |
|
|
219
219
|
| `report.py` | Single-file HTML delivery report: before/after sheets, media facts, loudness, compliance, the commands run |
|
|
220
220
|
|
|
221
221
|
**Orchestration**
|
|
222
222
|
|
|
223
223
|
| Tool | What it does |
|
|
224
224
|
|---|---|
|
|
225
|
-
| `render.py` | Render a whole edit from a declarative `project.json` (clips, transitions, captions, overlays, music, loudness, export, check); `--init`, `--dry-run`, `--stop-after` |
|
|
225
|
+
| `render.py` | Render a whole edit from a declarative `project.json` (clips, transitions, captions, overlays, music and stem levels, loudness, export, chapter markers, check); `--init`, `--dry-run`, `--stop-after` |
|
|
226
226
|
| `batch.py` | Apply a step recipe or a project to a folder with a content-hash cache; `--watch` |
|
|
227
227
|
| `multicam.py` | Align any number of cameras and recorders by audio (with drift correction) and cut between them from a switch list |
|
|
228
228
|
| `verify.py` | Run the toolchain on real device files and report PASS / FAIL per step |
|
|
@@ -237,7 +237,7 @@ WAV, FLAC, MP3, M4A/AAC, OGG and Opus go through `probe`, `cut`, `join`, `silenc
|
|
|
237
237
|
- **Join.** `join.py intro.wav episode.m4a outro.wav -o full.flac` resamples every clip to one rate and channel layout and crossfades them (`--transition none` for a butt join). Audio and video inputs cannot be mixed in one join.
|
|
238
238
|
- **Sample-accurate trims.** `cut.py talk.wav --start 1.2345 --end 2.3456 --accurate` trims at the sample; the JSON reports `precision` (`packet` for a stream copy, `sample` for PCM / FLAC, `codec_frame` when a lossy encoder frames the audio again, `frame` for video) and the measured `duration_error_ms`. A `.wav` never receives compressed packets.
|
|
239
239
|
- **Typed dynamics.** `audio.py --compress --comp-threshold -20 --comp-ratio 4`, `--limit --limit-ceiling -1`, `--gate --gate-threshold -45`. Each flag is one documented option of FFmpeg's `acompressor`, `alimiter` or `agate`, range-checked before ffmpeg runs; no filter string is accepted from the caller.
|
|
240
|
-
- **Loudness.** `loudness.py talk.wav -I -16 --tp -1.5 -o talk.m4a` for podcast levels; `check.py talk.m4a --platform podcast` measures LUFS and true peak.
|
|
240
|
+
- **Loudness.** `loudness.py talk.wav -I -16 --tp -1.5 -o talk.m4a` for podcast levels; `check.py talk.m4a --platform podcast` measures LUFS and true peak and reports the chapter markers and channel count.
|
|
241
241
|
|
|
242
242
|
Picture tools (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" instead of inventing a picture.
|
|
243
243
|
|
|
@@ -363,6 +363,7 @@ FFmpeg 8 shortened the flag column of `ffmpeg -filters`. A parser anchored on th
|
|
|
363
363
|
| **F1 0.97** | `scenes.py`, 53 hard cuts between single takes, precision 0.95, recall 1.00 at the default threshold |
|
|
364
364
|
| **exact to the sample** | `cut.py --accurate` on WAV, FLAC (44.1 kHz) and AAC → WAV; WAV stream copy within 2 ms; AAC output +21 ms of encoder priming, reported as `codec_frame` (0.9.1) |
|
|
365
365
|
| **72 / 72** | agent runs of 24 prompts (12 English edits, 8 Japanese, 4 that must be declined), three repeats, graded by an independent model: routing, honest refusals and user's language 72/72, report format 71/72, visual check whenever the picture changed 24/24 (0.8.4) |
|
|
366
|
+
| **50 / 50** | 1.12.0 run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader) on the set grown to 50 prompts with two each in Chinese, Korean, Spanish, Portuguese, French, German and Arabic: routing 50/50, honest refusals and failures 50/50 with 0 false successes and 0 raw ffmpeg calls, report format 50/50, user's language 50/50 across nine languages, visual check 13/13, trigger set 29/29, Opus quality mean 4.83. Every non-Latin caption and lower-third picked a covering font by itself and rendered real glyphs (Arabic shaped and right-to-left); tokens per run unchanged at 72.3k. Details in `evals/results/iteration-13.json` |
|
|
366
367
|
| **36 / 36** | 1.11.1 re-run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader): routing 36/36, honest refusals and failures 36/36 with 0 false successes and 0 raw ffmpeg calls, report format 36/36, user's language 36/36, visual check 8/8, trigger set 22/22, Opus quality mean 4.75. The 1.11.1 wording did what it said (`doctor` before a job 23 of 36 runs → 0, `--json-brief` 4 → 23) and tokens per run stayed flat at 71.8k, because about 64k of every run is the host's own context; the token-diet theme closes here. Details in `evals/results/iteration-12.json` |
|
|
367
368
|
| **36 / 36** | 1.11.0 re-run (2026-09-13, one pass per prompt, Sonnet agent, regex grader + focused Opus grader): routing 36/36, honest refusals and failures 36/36 with 0 false successes and 0 raw ffmpeg calls, report format 36/36, user's language 36/36, visual check 8/8, trigger set 22/22. First iteration to measure tokens per run: mean 72.2k against 68.7k at 1.10.0, mostly a fixed per-run floor the skill does not control (a refusal run that only reads SKILL.md costs about 64k), plus `doctor` on 23 of 36 runs; 1.11.1 rewords step 0 and iteration 12 re-measures. Details in `evals/results/iteration-11.json` |
|
|
368
369
|
| **108 / 108** | 1.10.0 re-run (2026-09-13, three passes per prompt, Sonnet agent, regex grader + focused Opus grader): routing 108/108, honest refusals and failures 108/108 with 0 false successes and 0 raw ffmpeg calls, report format 108/108 by both graders (the harness now names the five labels), user's language 105/108 (every Japanese request in Japanese; 3 English requests drifted to Spanish or Portuguese), visual check 21/24, trigger set 22/22; real-device corpus 101/101 steps PASS. Details in `evals/results/iteration-10.json` |
|
package/SKILL.md
CHANGED
|
@@ -5,30 +5,30 @@ description: 'Edit video and audio with local FFmpeg from natural-language reque
|
|
|
5
5
|
|
|
6
6
|
# ffmpeg-skill
|
|
7
7
|
|
|
8
|
-
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. This file is enough to do a job: the table below routes the request
|
|
8
|
+
Scripts live in `scripts/` next to this file; run them with `python3 <skill-dir>/scripts/<name>.py`. This file is enough to do a job: the table below routes the request and `--help` on the one script you are about to run is the cheapest full flag list. The reference files cost as much to read as this file does, so open one only when it answers a question you actually have: `references/scripts.md` (every flag of all 42 scripts, for comparing tools), `references/devices.md` (iPhone HDR, GoPro, DJI, screen recordings, Zoom), `references/gotchas.md` (the long form of the one-line rules at the end of this file).
|
|
9
9
|
|
|
10
|
-
Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (that document trimmed to status/output/verified, a compact `summary` and the command count — prefer it on every writing step
|
|
10
|
+
Shared flags, on every script: `--dry-run`; `--json` (output path, a probe of the output, the commands run); `--json-brief` (that document trimmed to status/output/verified, a compact `summary` and the command count — prefer it on every writing step); `--fast` (preview quality); `--progress`; `--timeout SECONDS` (`kind: timeout`, default 1800); `--overwrite` (consent to replace an existing output — without it the tool warns today, refuses from 2.0); `--plan FILE` (the dry run as a plan document `render.py FILE` executes later, refusing if an input changed: "plan → confirm → execute" in one round trip). Every re-encoding tool also takes `--codec h264|hevc|av1|prores` and `--quality N` (CRF scale, replaces the deprecated `--crf`): unset, SDR is x264 and HDR is x265 Main10; `prores` needs an explicit `-o NAME.mov`, `h264` refuses an HDR source (`color.py --to-sdr` first).
|
|
11
11
|
|
|
12
12
|
Writing tools run nothing under `--dry-run`; `probe`, `check`, `sync`, `multicam`, `scenes`, `cropdetect`, `report`, `silence`, `loudness` and `stabilize` may still run ffmpeg/ffprobe to measure or analyse — they just don't write their final artifact (nor side files such as `--edl`, `--sheet` or a generated `.ass`); `verify` accepts the flag but ignores it. Exact per-tool semantics: `contract --json`'s `dry_run` field (or `docs/contract.md`).
|
|
13
13
|
|
|
14
14
|
## Workflow (always follow this order)
|
|
15
15
|
|
|
16
|
-
0. **Environment, only on failure.** Do not start a job with `doctor`: on a working machine it
|
|
16
|
+
0. **Environment, only on failure.** Do not start a job with `doctor`: on a working machine it tells you nothing the job needs, and on a broken one the script fails on its own with `kind: missing_tool` (no ffmpeg) or an ffmpeg error naming the filter or encoder (`No such filter: 'subtitles'`). Run `python3 <skill-dir>/scripts/_contract.py doctor` (also `npx ffmpeg-skill doctor`; there is no doctor.py) after such a failure, or when the user asks what the machine can do. Read `ok` and the tool's `usable`; if `usable` isn't `yes`, report the missing capability (usually `libass`, `zscale` or an encoder) instead of discovering it through a runtime failure. `contract --json`'s full tool schema is for a *planning* agent choosing a tool from an abstract goal, not for this workflow.
|
|
17
17
|
1. **Probe what you must plan from.** Run `probe.py` on each input you plan the edit from — duration, fps, resolution, codecs, channels, `variable_frame_rate_suspected` — and whenever the user asks a question about a file. You do not need a separate probe before every edit: every writing tool's `--json` already carries its input and a probe of the output it wrote. Plan from real numbers, never assumptions.
|
|
18
18
|
2. **Prefer lossless.** If the request can be met without re-encoding (plain cuts on keyframes, remuxing, audio-only changes), do not re-encode. `cut.py` and `loudness.py` stream-copy video by default; pass `--accurate` to `cut.py` only for frame-exact cuts.
|
|
19
19
|
3. **Plan with `--dry-run --json`, then execute.** Trust `--json`, not a dry run's human-readable summary line, for any number after the plan (dimensions there can be a placeholder, not a computed preview — `docs/contract.md`). Use it to confirm a plan before long encodes and to report exact facts. `--fast` is preview quality (x264 veryfast), `--progress` prints percent/ETA on stderr. Never point `-o` at a file you did not create in this job unless the user asked for it to be replaced; pass `--overwrite` only then.
|
|
20
20
|
4. **Chain in a sensible order.** Colour (HDR→SDR / LUT) → cut → join → silence → fit → caption/overlay → sync → audio → loudness → export. Frame changes (fit/crop) before captions and overlays, so text is sized for the final frame. Re-encode as few times as possible: intermediates at CRF 18 (the default), `export.py` only for the last step. **Three or more steps: use `render.py` with a project.json** — one call, one JSON, one place for the user to change a number — rather than hand-chaining tools.
|
|
21
|
-
5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination the user named. Each row is `format` or `judgement`. Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix mechanically. Judgement rows change the content: duration (cut loses material
|
|
22
|
-
6. **Verify the output.** Confirm duration, resolution, fps and audio match the request — from the writing tool's own `--json`/`--json-brief` probe, or `probe.py`
|
|
21
|
+
5. **Check the deliverable.** Before reporting, run `check.py OUTPUT --platform X` for the destination the user named. Each row is `format` or `judgement`. Format rows (codec, pixel format, size, true peak, colour tags, VFR) are safe to fix mechanically. Judgement rows change the content: duration (cut loses material), aspect (crop loses edges), fps (drops motion), loudness (ambience must not be boosted) — fix those only when the request already implies the answer, otherwise state the choice and its cost in one line. Mention WARNs; do not chase them.
|
|
22
|
+
6. **Verify the output.** Confirm duration, resolution, fps and audio match the request — from the writing tool's own `--json`/`--json-brief` probe, or `probe.py` — and report those numbers ("final.mp4: 59.98 s, 1080x1920, 30 fps, AAC stereo"). A step is done only when the script exited 0 and the output probes as expected: a non-zero exit, a missing or empty file, or a probe that contradicts the request is a failure, and the report says so with the script's error message.
|
|
23
23
|
7. **Keep the user's originals.** Never overwrite the source; write new files next to the input or where the user asked. Set `FFMPEG_SKILL_NO_OVERWRITE=1` in the environment you run these scripts in: an existing output path is then refused (`kind: input`) instead of warned about, and `--overwrite` stays the one way to say "yes, replace it". It is the recommended agent setting — an agent picking output names cannot see which files the user already cares about — and it is what 2.0 does by default.
|
|
24
|
-
8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` that scaled or padded a clip to the first clip's frame, a `color.py --to-sdr` that tone-maps an HDR source) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for a job about layout across the whole clip. The job is not finished until the report's `Look:` line names that PNG — a probe cannot see a caption sitting on someone's face. Audio-only jobs
|
|
24
|
+
8. **Look at the picture.** Whenever the picture changed (captions, overlays, graphics, crop/pad, resize, colour, transitions, a `join.py` that scaled or padded a clip to the first clip's frame, a `color.py --to-sdr` that tone-maps an HDR source) run `look.py OUTPUT --tiles 3x2` (or `--at T` for one frame) and view the PNG; the full 4x3 sheet is for a job about layout across the whole clip. The job is not finished until the report's `Look:` line names that PNG — a probe cannot see a caption sitting on someone's face. Audio-only jobs write `Look: not needed`; there is no picture. What to look for splits like `check.py`'s rows in step 5:
|
|
25
25
|
- **Mechanical (this skill's own job to verify and report):** the specified text/logo is present at the specified position, subtitles/text appear at the specified timestamps, dimensions are even. Letterboxing/pillarboxing from `fit.py --fit pad` is the *correct* result of that mode, never a defect to flag.
|
|
26
26
|
- **Judgement (report it, don't silently pass or fail):** whether a subject or face is cut off, whether text sits over a face, whether colours look washed out, whether a transition lands. These need deciding what the subject *is*, which belongs to the calling agent (see "What this skill does and does not decide") — say what you see in one line and let them judge it.
|
|
27
27
|
With no vision capability, write `Look: PATH (pixels not inspected; agent has no image view)` — never claim a picture was inspected when it wasn't, and don't stall waiting for a capability that isn't there.
|
|
28
28
|
|
|
29
29
|
## Before you run anything: what to ask, what to assume
|
|
30
30
|
|
|
31
|
-
Ask one short question only when the answer changes the output materially and the request does not imply it. When several things are open at once (a vague "make it for social media" leaves destination, aspect
|
|
31
|
+
Ask one short question only when the answer changes the output materially and the request does not imply it. When several things are open at once (a vague "make it for social media" leaves destination, aspect, length and captions open), don't ask them one per turn: propose one bundle with your defaults and let the user change any part ("Reels: 9:16 with padding, trimmed to 60 s, -14 LUFS, no captions — OK, or change something?"). One question, one answer, then the run. Never ask for what `probe.py` can tell you.
|
|
32
32
|
|
|
33
33
|
- **Destination** decides aspect, length limit, loudness and codec; "for Reels" answers all four. No destination named and a plain cut/caption: keep the source format and say so. If the user says "export", "post" or "deliver", ask where.
|
|
34
34
|
- **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, and say which you chose. Ask when the content is a talk (trimming loses words) and the change is large.
|
|
@@ -51,13 +51,13 @@ This skill cuts, joins, measures, syncs, exports and checks files — it execute
|
|
|
51
51
|
|
|
52
52
|
The line: same input + same explicit parameters always producing the same verifiable output belongs here; anything that depends on taste, content understanding or what looks or sounds good belongs to whoever makes that judgement. This skill executes parameters it is given, never infers them from what something looks or sounds like.
|
|
53
53
|
|
|
54
|
-
If a request needs an FFmpeg feature none of the 42 scripts expose, say so and name the closest built-in option (`--dry-run` to show what would run, or a documented limitation) — never fall back to guessing a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards)
|
|
54
|
+
If a request needs an FFmpeg feature none of the 42 scripts expose, say so and name the closest built-in option (`--dry-run` to show what would run, or a documented limitation) — never fall back to guessing a raw `ffmpeg`/`ffprobe` invocation or a hand-built filter graph outside `scripts/*.py`. A raw command bypasses every guarantee this skill makes (no shell, typed arguments, verification afterwards), so it is never the fallback when a script's flag doesn't cover something.
|
|
55
55
|
|
|
56
56
|
## Request → script
|
|
57
57
|
|
|
58
58
|
This table and `doctor --json`'s `tools` list are the source of truth for what exists: name only a script you have seen in one of them, never a plausible-sounding one (there is no `doctor.py`, no `trim.py`, no `subtitle.py`).
|
|
59
59
|
|
|
60
|
-
Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`, and the times in cue and chapter files — take seconds, `mm:ss(.fff)`, `hh:mm:ss(.fff)` or four-part SMPTE `hh:mm:ss:ff`, with `@fps` naming the rate (`00:01:02:15@29.97`); tolerance-style flags that are a length rather than a point in time (`--min-silence`, `--margin`, `--min-keep`, `--fade`) are plain seconds. Use the timecode forms when the user pastes an editor's
|
|
60
|
+
Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`, and the times in cue and chapter files — take seconds, `mm:ss(.fff)`, `hh:mm:ss(.fff)` or four-part SMPTE `hh:mm:ss:ff`, with `@fps` naming the rate (`00:01:02:15@29.97`); tolerance-style flags that are a length rather than a point in time (`--min-silence`, `--margin`, `--min-keep`, `--fade`) are plain seconds. Use the timecode forms when the user pastes an editor's or NLE cue sheet, so nothing is converted by hand.
|
|
61
61
|
|
|
62
62
|
| User says | Do |
|
|
63
63
|
|-----------|----|
|
|
@@ -87,7 +87,7 @@ Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offse
|
|
|
87
87
|
| "speed up here, slow-mo there" (known segments) | `speedramp.py action.mp4 --segment 0-3:1.0 --segment 3-4:0.25 --segment 4-8:2.0` |
|
|
88
88
|
| "loop this clip to fill 30 seconds" | `loop.py bg_loop.mp4 --duration 30` |
|
|
89
89
|
| "cut to the product shot 0:12-0:16", "B-roll over this bit" | `broll.py talk.mp4 --insert product.mp4 --at 12 --end 16` (repeat `--insert/--at`; `--audio b\|mix`) |
|
|
90
|
-
| "add chapters", "chapter markers for YouTube" | `metadata.py episode.mp4 --chapters chapters.txt` (`TIME TITLE` per line; streams copied) |
|
|
90
|
+
| "add chapters", "chapter markers for YouTube" | `metadata.py episode.mp4 --chapters chapters.txt` (`TIME TITLE` per line; streams copied; a `render.py` project spells it `"chapters"`) |
|
|
91
91
|
| "set the title / artist / comment" | `metadata.py episode.mp4 --title "Episode 12" --artist "Studio"` |
|
|
92
92
|
| "put these videos in a 4x2 grid" | `grid.py t1.mp4 ... t8.mp4 --cols 4 --rows 2` |
|
|
93
93
|
| "add subtitles from this SRT", "burn in captions" | `caption.py input.mp4 --srt subs.srt` |
|
|
@@ -98,7 +98,7 @@ Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offse
|
|
|
98
98
|
| "webcam clip in the corner", "picture-in-picture" | `overlay.py input.mp4 --video webcam.mp4 --position bottom-right --scale 480` |
|
|
99
99
|
| "remove the green screen" | `overlay.py bg.mp4 --video greenscreen.mp4 --chromakey 0x00ff00` |
|
|
100
100
|
| "sync the lav mic", "line up two cameras" | `sync.py camera.mp4 mic.wav --replace-audio` / `sync.py camA.mp4 camB.mp4 --trim-second` |
|
|
101
|
-
| "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` podcast, `-I -23` broadcast) |
|
|
101
|
+
| "fix the audio levels", "normalise to -14 LUFS" | `loudness.py input.mp4` (`-I -16 --tp -1.5` podcast, `-I -23` broadcast; `--lra N` for the range) |
|
|
102
102
|
| "cut this and make it HEVC / AV1 / ProRes" (output codec named) | `cut.py input.mp4 --start 0:10 --end 0:40 --codec hevc` (`--codec`/`--quality` on any re-encoding tool; ProRes needs `-o NAME.mov`) |
|
|
103
103
|
| "export for YouTube / Reels / X", "a ProRes master" | `export.py input.mp4 --preset youtube\|reels\|x\|prores\|h265` (`--normalize` hits the loudness spec in the same call) |
|
|
104
104
|
| "make a GIF preview" | `export.py input.mp4 --preset gif` |
|
|
@@ -109,6 +109,7 @@ Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offse
|
|
|
109
109
|
| "what would you run?", "don't render yet" | any script with `--dry-run` |
|
|
110
110
|
| "a 60 s highlight from this hour" | `scenes.py long.mp4 --highlights 6 --target 60 --edl picks.txt` → `cut.py --segments` |
|
|
111
111
|
| "is this OK to upload?" | `check.py final.mp4 --platform reels` |
|
|
112
|
+
| "a podcast episode with chapters" | `loudness.py ep.wav -I -16 --tp -1.5` → `metadata.py ep.m4a --chapters chapters.txt` → `check.py ep.m4a --platform podcast` (chapters and channels rows) |
|
|
112
113
|
| "several changes to the same edit", 3+ steps | `render.py --init project.json`, edit, `render.py project.json` |
|
|
113
114
|
| "a lower third with my name", "countdown intro", "progress bar" | `graphics.py input.mp4 --template lower-third --name "..." --title "..." --start 2 --end 8` |
|
|
114
115
|
| "use our brand fonts/colours/logo" | `--brand brand.json` on caption/overlay/graphics, or `"brand"` in project.json |
|
|
@@ -124,8 +125,10 @@ Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offse
|
|
|
124
125
|
| "apply this LUT", "convert the S-Log footage" | `color.py input.mp4 --lut grade.cube [--lut-strength 0.7]` |
|
|
125
126
|
| "the colours are tagged wrong" | `color.py input.mp4 --retag bt709` (stream copy; re-encodes only if the copy can't carry it — see `reencoded`) |
|
|
126
127
|
| "brighten it / punch up contrast / fix white balance" | `color.py input.mp4 --correct --exposure 0.3 --contrast 1.1 --saturation 1.05 --temperature 5600 --tint -0.05` |
|
|
127
|
-
| "clean up the audio", "remove the hiss" | `audio.py input.mp4 --voice` (speech) or `--denoise` |
|
|
128
|
-
| "add background music under the talking" | `audio.py input.mp4 --music bed.mp3 --duck --fade-out 3` |
|
|
128
|
+
| "clean up the audio", "remove the hiss" | `audio.py input.mp4 --voice` (speech; `--voice light\|medium\|strong`) or `--denoise` |
|
|
129
|
+
| "add background music under the talking" | `audio.py input.mp4 --music bed.mp3 --duck --fade-out 3` (`--effects sfx.wav` adds a third bed, never ducked; project levels: `audio.stems`) |
|
|
130
|
+
| "make the music duck harder / come back faster" | add `--duck-amount 18 --duck-threshold -30 --duck-release 250` (`--duck-attack` too) |
|
|
131
|
+
| "the mix sounds narrow", "wider stereo" | `audio.py band.wav --stereo-widen 0.5` (needs a real stereo source; 5.1 needs `--downmix`) |
|
|
129
132
|
| "convert the 5.1 to stereo" | `audio.py input.mov --downmix` |
|
|
130
133
|
| "swap in the narration track" | `audio.py input.mp4 --replace narration.wav` |
|
|
131
134
|
| "pull the audio out", "give me the sound as WAV" | `audio.py input.mp4 -o input.wav` (an audio extension drops the picture; `--audio-stream 1` picks a track) |
|
|
@@ -137,12 +140,12 @@ Timestamp flags — `--start`, `--end`, `--at`, `--from`, `--duration`, `--offse
|
|
|
137
140
|
|
|
138
141
|
## Audio-only files
|
|
139
142
|
|
|
140
|
-
Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`, `audio.py`, `sync.py` and `check.py --platform podcast` take WAV, FLAC, MP3, M4A/AAC, OGG and Opus, and the output extension picks the format. `Look: not needed` in the report; `Check:` still applies
|
|
143
|
+
Audio is a first-class input: `probe.py`, `cut.py`, `silence.py`, `loudness.py`, `audio.py`, `sync.py` and `check.py --platform podcast` take WAV, FLAC, MP3, M4A/AAC, OGG and Opus, and the output extension picks the format. `Look: not needed` in the report; `Check:` still applies. Scripts that need a picture (`fit`, `caption`, `overlay`, `graphics`, `color`, `export`, `scenes`, `look`) refuse an audio file with "input has no video stream" — say so instead of forcing a video wrapper. The same commands work with `talk.wav` in place of `talk.mp4`; audio recipes, packet vs sample precision, joining and extracting one track: `references/gotchas.md#audio-only-files`.
|
|
141
144
|
|
|
142
145
|
|
|
143
146
|
## Report format
|
|
144
147
|
|
|
145
|
-
Reply in the language the request itself is written in — the user's own sentences, not a language the request talks about (a request for subtitles in another language is still answered in the language it was written in) and not the language of a tool's error text or file names.
|
|
148
|
+
Reply in the language the request itself is written in — the user's own sentences, not a language the request talks about (a request for subtitles in another language is still answered in the language it was written in) and not the language of a tool's error text or file names. Any language works the same way. Keep the field labels (`Done:`, `Steps:`, `Check:`, `Look:`, `Notes:`) in English: they read like log fields and stay recognisable across languages. Everything around them — the sentences, any question, any explanation of a judgement call — is in the user's language. Never default to English because the tool names and flags are English, and never drift because the job was short or the report is a failure: a one-line "file does not exist" is written in the request's language too. A mid-conversation switch follows the user's latest message. This holds for a one-command job: English `Done:`/`Steps:` sentences with one word of the user's language in `Notes:` is an English report — the descriptions are in the user's language even when the values are technical.
|
|
146
149
|
|
|
147
150
|
Finish every job with this shape (numbers from a tool's `--json` or `probe.py`/`check.py`, not memory):
|
|
148
151
|
|
|
@@ -178,7 +181,7 @@ Look: not needed (nothing written)
|
|
|
178
181
|
Notes: send a valid .cube, or say if you want the clip left as is
|
|
179
182
|
```
|
|
180
183
|
|
|
181
|
-
A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run (usually only probe), `Look: not needed`. Both keep the five labels so a failed report scans like a successful one — the shortest failure
|
|
184
|
+
A refusal (a judgement this skill does not make, or something outside its scope) uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run (usually only probe), `Look: not needed`. Both keep the five labels so a failed report scans like a successful one — the shortest failure still gets all five lines, never prose headings. When a failure JSON carries `error.hint`, quote it in `Notes:`: it is the flag change that makes a retry meaningful.
|
|
182
185
|
|
|
183
186
|
Every script prints `{"status": "failed", "error": {"kind": input | ffmpeg | output | missing_tool | timeout | verification | interrupted, "message": ...}}` with `--json` and exits non-zero; quote the message, never paraphrase it into a success.
|
|
184
187
|
|
package/docs/contract.md
CHANGED
|
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
|
|
|
21
21
|
| Field | Meaning | Changes when |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
|
|
24
|
-
| `skill.version` | the npm / package.json version (`1.
|
|
24
|
+
| `skill.version` | the npm / package.json version (`1.13.0`) | any release |
|
|
25
25
|
|
|
26
26
|
A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
|
|
27
27
|
ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
|
|
@@ -88,19 +88,19 @@ spelling keeps working until 2.0.
|
|
|
88
88
|
|
|
89
89
|
| What 2.0 removes | Since | Replacement | To be ready today |
|
|
90
90
|
|---|---|---|---|
|
|
91
|
-
| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.
|
|
92
|
-
| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.
|
|
93
|
-
| `json` and `progress` in the MCP `inputSchema` | 1.
|
|
94
|
-
| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.
|
|
95
|
-
| Overwriting an existing output with only a warning | 1.
|
|
91
|
+
| The per-tool v1 success keys next to `result_v2` (`output`, `probe`, `commands`, `verified`, `verification` and each tool's own keys at the top level) | 1.13.0 | `result_v2`, promoted to the top level in 2.0 | Run with `FFMPEG_SKILL_RESULT_V2=1` and read `result_v2` (`metrics`, `notes`, `details`) instead of the top-level keys |
|
|
92
|
+
| `--crf` as an alias of `--quality` on every re-encoding tool that takes `--quality` (`export.py` keeps `--crf`: its preset chooses the encoder) | 1.13.0 | `--quality N` (the same CRF scale, codec-neutral) | Pass `--quality`; `--crf` warns on stderr and is marked in `--help` |
|
|
93
|
+
| `json` and `progress` in the MCP `inputSchema` | 1.13.0 | nothing: the transport sets them itself | Stop sending them from an MCP client; run the server with `FFMPEG_SKILL_MCP_LEAN=1` to see the 2.0 schema |
|
|
94
|
+
| `hdr` meaning "BT.2020 primaries *or* a PQ/HLG transfer" in `probe` | 1.13.0 | `hdr_signal` (true only for PQ / HLG / Dolby Vision); in 2.0 `hdr` takes that meaning | Key on `hdr_signal` for "is this a real HDR signal" and on `hdr_format` for the `BT.2020 SDR` case |
|
|
95
|
+
| Overwriting an existing output with only a warning | 1.13.0 | `--overwrite` as explicit consent (refused without it from 2.0) | Set `FFMPEG_SKILL_NO_OVERWRITE=1` (the recommended agent setting) and pass `--overwrite` where a replacement is intended |
|
|
96
96
|
|
|
97
97
|
## Skill
|
|
98
98
|
|
|
99
99
|
```json
|
|
100
100
|
{
|
|
101
101
|
"contract_version": "1.0",
|
|
102
|
-
"deprecated": [{"what": "...", "since": "1.
|
|
103
|
-
"skill": {"id": "ffmpeg-skill", "version": "1.
|
|
102
|
+
"deprecated": [{"what": "...", "since": "1.13.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
|
|
103
|
+
"skill": {"id": "ffmpeg-skill", "version": "1.13.0", "execution_mode": "local", "kind": "execution",
|
|
104
104
|
"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
|
|
105
105
|
"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
|
|
106
106
|
"requirements": {"python": ">=3.9 (standard library only)", "ffmpeg": ">=5.0", "ffprobe": ">=5.0"},
|
|
@@ -128,7 +128,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
|
|
|
128
128
|
| `output_schema` | what `--json` prints on stdout |
|
|
129
129
|
| `supports_dry_run`, `dry_run` | whether `--dry-run` plans without running ffmpeg or writing files |
|
|
130
130
|
| `supports_json` | whether `--json` exists |
|
|
131
|
-
| `supports_json_brief` | whether `--json-brief` exists (1.
|
|
131
|
+
| `supports_json_brief` | whether `--json-brief` exists (1.13.0): the same success document with `probe` replaced by a compact `summary` (`duration_s`, `width`, `height`, `fps`, `vcodec`, `acodec`, `channels`, and `lufs` when the tool measured one), `commands` replaced by the number of commands run, and the per-step `verification` list dropped (its verdict stays in `verified`). Tool-specific keys are unchanged, `--json`'s own output is unchanged, and a failure prints the same failure document either way |
|
|
132
132
|
| `mutates_input` | always `false`: no tool overwrites its input |
|
|
133
133
|
| `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
|
|
134
134
|
| `verification` | `{required, tools}`: which tools to run on the output afterwards |
|
|
@@ -354,6 +354,21 @@ runs a shell, evaluates strings, or executes anything other than the named scrip
|
|
|
354
354
|
|
|
355
355
|
## JSON output
|
|
356
356
|
|
|
357
|
+
Per-tool keys added in 1.13: `audio` (`audio.py`) reports the mix it built — the
|
|
358
|
+
`--voice` level, `stereo_widen`, whether an `--effects` bed was mixed, and with
|
|
359
|
+
`--music` the `music_volume` plus a `duck` object naming the threshold (dB and
|
|
360
|
+
linear), ratio, attack and release actually used, or `null` when `--duck` was not
|
|
361
|
+
given. `loudness.py` reports `measured` (the input's loudnorm measurement,
|
|
362
|
+
including `input_lra`) and `targets` (the requested lufs / tp / lra).
|
|
363
|
+
`check.py --platform podcast` adds two informational rows to `checks`,
|
|
364
|
+
`channels` and `chapters`.
|
|
365
|
+
|
|
366
|
+
The MCP `audio` tool publishes `voice` as `{"type": "string", "enum": ["light",
|
|
367
|
+
"medium", "strong"]}`. A client that still sends the 1.12 boolean `{"voice":
|
|
368
|
+
true}` keeps working: `true` emits the bare `--voice`, which is `medium` — the
|
|
369
|
+
chain the flag has always produced. Send the string when you can; the boolean
|
|
370
|
+
is accepted at runtime and means `medium`.
|
|
371
|
+
|
|
357
372
|
Success (`exit 0`): one document matching `output_schema`, always with
|
|
358
373
|
`status: "completed"`, `output`, `dry_run`, `commands`, and `probe` of the output when a
|
|
359
374
|
file was written. `probe` prints its measurement document directly.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.13.0",
|
|
4
4
|
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ffmpeg",
|
package/references/gotchas.md
CHANGED
|
@@ -74,6 +74,14 @@ clip measured at -40 LUFS or below is room tone, wind or nothing; raising it
|
|
|
74
74
|
25 dB raises the noise, not the content. Leave the level, say so, and offer music
|
|
75
75
|
or narration.
|
|
76
76
|
|
|
77
|
+
Do not add a speech gate in front of the measurement either: `loudnorm`'s EBU
|
|
78
|
+
R128 integrated loudness already applies the −70 LUFS absolute and −10 LU
|
|
79
|
+
relative gates, which drop the same quiet blocks a `silencedetect` pass would.
|
|
80
|
+
A speech-span gate was measured against the whole-file measurement on every
|
|
81
|
+
fixture in the repo, including one that is half digital silence, and moved the
|
|
82
|
+
result by at most 0.6 LU — inside `check.py`'s own ±1 LU tolerance — for the
|
|
83
|
+
cost of a second full decode. That is why `loudness.py` has no speech-gate flag.
|
|
84
|
+
|
|
77
85
|
## Text and framing
|
|
78
86
|
|
|
79
87
|
### Captions, fonts and text order
|
package/references/scripts.md
CHANGED
|
@@ -404,8 +404,25 @@ so the rendered file meets the platform's loudness without a separate pass. Sinc
|
|
|
404
404
|
1.9.0 it is on by default when the preset is a platform (`youtube|youtube4k|reels|x`)
|
|
405
405
|
and the project has no `loudness` stage; `"normalize": false` opts out.
|
|
406
406
|
|
|
407
|
+
`"audio": {"stems": {"dialogue": -2, "music": -18, "effects": -24}}` names one
|
|
408
|
+
level per element of the mix: `dialogue` is the main track's gain, `music` the
|
|
409
|
+
bed's level, `effects` the level of the third file `"audio": {"effects":
|
|
410
|
+
"sfx.wav"}` adds (never ducked). Each maps to the flag of the same meaning
|
|
411
|
+
(`--gain`, `--music-volume`, `--effects-volume`); an explicit flag next to a
|
|
412
|
+
stem wins, and a stems level with no file to apply it to (`effects` without
|
|
413
|
+
`"effects"`, `music` without `"music"`) is refused, `kind: input`.
|
|
414
|
+
`"audio": {"voice": "light"|"medium"|"strong"}` picks the voice strength
|
|
415
|
+
(`true` is `medium`).
|
|
416
|
+
|
|
417
|
+
`"chapters"` is a chapters file path, or an inline list of `{"at": TIME,
|
|
418
|
+
"title": STR}`; it runs `metadata.py` on the delivered file as the last stage
|
|
419
|
+
before `check`, so the markers are in the file that ships (streams copied). The
|
|
420
|
+
entries and the file path are validated before the first stage runs, and the
|
|
421
|
+
stage plans its `metadata.py` command under `--dry-run`/`--plan` like every
|
|
422
|
+
other stage, so the plan lists `chapters` and the run does the same work.
|
|
423
|
+
|
|
407
424
|
Stages: clips (cut, optional speed) → join (transition) → silence → fit →
|
|
408
|
-
captions → graphics → overlays → audio → loudness → export → check. Keys mirror the
|
|
425
|
+
captions → graphics → overlays → audio → loudness → export → chapters → check. Keys mirror the
|
|
409
426
|
CLI flags of each script (see the docstring); a key `render.py` does not read -- at
|
|
410
427
|
the top level or in any stage/clip object -- is refused (`kind: input`) naming the
|
|
411
428
|
key and the nearest valid one, never silently ignored. Use it whenever an edit has
|
|
@@ -437,6 +454,11 @@ step before reporting a deliverable; fix FAILs, mention WARNs. Without
|
|
|
437
454
|
`--platform` the youtube spec is assumed and the judgement rows (duration,
|
|
438
455
|
aspect, fps, resolution, loudness, true peak) come back as WARN with a `notes`
|
|
439
456
|
line, not FAIL: name the platform when the file is a delivery for it.
|
|
457
|
+
`--platform podcast` adds two informational rows: `channels` (PASS for mono or
|
|
458
|
+
stereo, WARN above — podcast players downmix 5.1 unpredictably) and `chapters`
|
|
459
|
+
(PASS when the container carries at least one marker, WARN `none` otherwise —
|
|
460
|
+
write them with `metadata.py --chapters`). Neither can FAIL a delivery, and
|
|
461
|
+
neither appears for another platform.
|
|
440
462
|
|
|
441
463
|
### batch.py — same recipe over a folder, cached
|
|
442
464
|
```
|
|
@@ -684,13 +706,42 @@ first on HDR or Log sources.
|
|
|
684
706
|
|
|
685
707
|
### audio.py — clean-up, music, ducking, layout
|
|
686
708
|
```
|
|
687
|
-
audio.py INPUT [--voice | --denoise [--denoise-strength 25]] [--gain dB]
|
|
688
|
-
[--music FILE [--music-volume -14] [--duck [--duck-amount 12]
|
|
709
|
+
audio.py INPUT [--voice [light|medium|strong] | --denoise [--denoise-strength 25]] [--gain dB]
|
|
710
|
+
[--music FILE [--music-volume -14] [--duck [--duck-amount 12] [--duck-threshold -26.02]
|
|
711
|
+
[--duck-attack 20] [--duck-release 400]] [--music-loop]]
|
|
712
|
+
[--effects FILE [--effects-volume -14]] [--stereo-widen 0..1]
|
|
689
713
|
[--fade-in S] [--fade-out S] [--stereo | --mono | --downmix] [--replace FILE] [-o OUT]
|
|
690
714
|
```
|
|
691
|
-
`--voice`
|
|
692
|
-
|
|
693
|
-
|
|
715
|
+
`--voice` takes a strength; a bare `--voice` is `medium`, the chain it has always
|
|
716
|
+
produced. The exact filter chains:
|
|
717
|
+
|
|
718
|
+
| level | chain |
|
|
719
|
+
| --- | --- |
|
|
720
|
+
| `light` | `highpass=f=80,acompressor=threshold=-18dB:ratio=2:attack=5:release=80:makeup=1` |
|
|
721
|
+
| `medium` | `highpass=f=80,deesser=i=0.4,afftdn=nf=-25:tn=1,acompressor=threshold=-18dB:ratio=3:attack=5:release=80:makeup=2` |
|
|
722
|
+
| `strong` | the `medium` chain, then `deesser=i=0.6,acompressor=threshold=-24dB:ratio=4:attack=5:release=120:makeup=3,alimiter=limit=0.891251:level=disabled` |
|
|
723
|
+
|
|
724
|
+
`light` for a good room (rumble and level only, noise floor and sibilance left
|
|
725
|
+
alone), `medium` for a normal talking head, `strong` for phone/laptop audio.
|
|
726
|
+
`strong` is the only level with a limiter, so it is the only one whose peaks
|
|
727
|
+
stop at −1 dBFS: `medium` (the default, and the chain a bare `--voice` gets) can
|
|
728
|
+
clip a hot source, since its make-up gain has nothing above it — normalise
|
|
729
|
+
afterwards with `loudness.py`, or use `strong`, which measures quieter and safer.
|
|
730
|
+
`--duck` uses a sidechain compressor keyed by the speech so music dips under
|
|
731
|
+
dialogue and swells in pauses:
|
|
732
|
+
`sidechaincompress=threshold=0.05:ratio=<amount/3, min 2>:attack=20:release=400:makeup=1`
|
|
733
|
+
by default. `--duck-threshold DB` (default −26.02 dBFS, i.e. the 0.05 linear),
|
|
734
|
+
`--duck-attack MS` (20) and `--duck-release MS` (400) move each one; a lower
|
|
735
|
+
threshold ducks on quieter speech, a shorter release brings the bed back faster.
|
|
736
|
+
`--json`'s `audio` block reports the settings the run actually used.
|
|
737
|
+
`--effects FILE` mixes a third track (sound effects, atmos) at
|
|
738
|
+
`--effects-volume` and is never ducked — effects are cut to the picture.
|
|
739
|
+
`--stereo-widen 0..1` widens the stereo image (`extrastereo=m=1+2*amount`) and
|
|
740
|
+
needs a real stereo source: it scales the side signal (L−R), so a mono track
|
|
741
|
+
duplicated to two channels has nothing to scale. A 1-channel input is refused
|
|
742
|
+
(`kind: input`) — `--stereo` duplicates it but does not widen it — and more than
|
|
743
|
+
two channels are refused unless `--downmix` is given too, in which case the
|
|
744
|
+
widening runs on the stereo fold-down. `--downmix` uses the
|
|
694
745
|
ITU centre/LFE weights for 5.1/7.1 → stereo. `--mono` averages a stereo pair,
|
|
695
746
|
leaves a 1-channel input untouched and downmixes >2 channels through
|
|
696
747
|
swresample. Video is always stream-copied, and so is a subtitle/data track
|
|
@@ -701,6 +752,12 @@ Run `loudness.py` after this for final levels.
|
|
|
701
752
|
```
|
|
702
753
|
loudness.py INPUT [-I -14] [--tp -1] [--lra 11] [--measure-only] [-o OUT]
|
|
703
754
|
```
|
|
755
|
+
`--lra N` is the loudness-range target in LU (default 11): lower it to squeeze a
|
|
756
|
+
wide-dynamic mix into a phone speaker, raise it to leave a film mix alone.
|
|
757
|
+
`--json` reports the measured range on both sides — `measured.input_lra` for the
|
|
758
|
+
input, `result.input_lra` for the written file, with `targets` echoing the
|
|
759
|
+
requested lufs / tp / lra.
|
|
760
|
+
|
|
704
761
|
Two-pass `loudnorm`: measure, then apply with measured values (linear mode when
|
|
705
762
|
the true-peak ceiling allows). Video and any subtitle/data track are
|
|
706
763
|
stream-copied (`dropped_non_av_streams` reports a track the container refused); audio becomes AAC in
|
package/scripts/_contract.py
CHANGED
|
@@ -157,9 +157,11 @@ TOOL_META: Dict[str, Dict[str, Any]] = {
|
|
|
157
157
|
"multicam": dict(role="execution", inputs=["reference camera", "other cameras / recorders"], outputs=["switched multicam video artifact"],
|
|
158
158
|
required=FF + [X264, AAC], optional=[HDR_X265],
|
|
159
159
|
video_required=True, audio_only=False, visual=True, verify=["probe", "look"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
|
|
160
|
-
"audio": dict(role="execution", inputs=["video or audio asset", "music bed (--music) or replacement track (--replace)"], outputs=["artifact with the processed audio (video stream-copied, or dropped when -o has an audio extension)"],
|
|
160
|
+
"audio": dict(role="execution", inputs=["video or audio asset", "music bed (--music) or replacement track (--replace)", "effects/atmos track (--effects)"], outputs=["artifact with the processed audio (video stream-copied, or dropped when -o has an audio extension)"],
|
|
161
161
|
required=FF, optional=[{"capability": "filter:afftdn", "when": "--denoise / --voice"}, {"capability": "filter:sidechaincompress", "when": "--duck"},
|
|
162
|
-
{"capability": "filter:acompressor", "when": "--compress / --voice"}, {"capability": "filter:alimiter", "when": "--limit"}, {"capability": "filter:agate", "when": "--gate"},
|
|
162
|
+
{"capability": "filter:acompressor", "when": "--compress / --voice"}, {"capability": "filter:alimiter", "when": "--limit / --voice strong"}, {"capability": "filter:agate", "when": "--gate"},
|
|
163
|
+
{"capability": "filter:deesser", "when": "--voice medium (the default) / --voice strong"},
|
|
164
|
+
{"capability": "filter:extrastereo", "when": "--stereo-widen"},
|
|
163
165
|
{"capability": AAC, "when": "output extension isn't .mp3/.opus/.ogg/.flac (audio_codec_for()'s default)"}] + AUDIO_OUT,
|
|
164
166
|
video_required=False, audio_only=True, visual=False, verify=["probe"], produces_artifact=True, idempotency="content_equivalent", deterministic=True),
|
|
165
167
|
"loudness": dict(role="analysis_and_execution", inputs=["video or audio asset"], outputs=["loudness measurement JSON (--measure-only)", "normalised artifact (video stream-copied)"],
|
|
@@ -411,7 +413,9 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
|
|
|
411
413
|
extra = {"loudness": {"type": "object", "description": "platform presets with audio: the written file's lufs/tp against the platform's target_lufs/target_tp, ok true when inside the spec; normalized true when --normalize ran loudness.py on the file"},
|
|
412
414
|
"notes": {"type": "array", "items": {"type": "string"}}}
|
|
413
415
|
elif name == "loudness":
|
|
414
|
-
extra = {"measured": {"type": "object", "description": "
|
|
416
|
+
extra = {"measured": {"type": "object", "description": "the loudnorm measurement of the input (input_i, input_tp, input_lra, input_thresh, target_offset); with --measure-only it is the whole result"},
|
|
417
|
+
"targets": {"type": "object", "description": "the requested lufs / tp / lra"},
|
|
418
|
+
"result": {"type": "object", "description": "the written file measured again (input_i, input_tp, input_lra, ...), plus tp_ceiling_used, audio_bitrate_used and encodes"}}
|
|
415
419
|
elif name == "cut":
|
|
416
420
|
extra = {"expected_duration": {"type": "number", "description": "seconds requested"},
|
|
417
421
|
"duration_error_ms": {"type": ["number", "null"], "description": "written minus requested, measured by ffprobe (null under --dry-run)"},
|
|
@@ -426,7 +430,8 @@ def output_schema(name: str, meta: Dict[str, Any]) -> Dict[str, Any]:
|
|
|
426
430
|
elif name == "audio":
|
|
427
431
|
extra = {"video": {"type": "boolean", "description": "true when the input's video stream was copied; false for an audio output extension (extraction)"},
|
|
428
432
|
"audio_stream": {"type": "integer", "description": "which input audio stream was processed (--audio-stream)"},
|
|
429
|
-
"dynamics": {"type": "array", "items": {"enum": ["agate", "acompressor", "alimiter"]}, "description": "typed dynamics filters applied, in graph order"}
|
|
433
|
+
"dynamics": {"type": "array", "items": {"enum": ["agate", "acompressor", "alimiter"]}, "description": "typed dynamics filters applied, in graph order"},
|
|
434
|
+
"audio": {"type": "object", "description": "what the mix was built from: voice (null | light | medium | strong), stereo_widen, effects/effects_volume, and with --music the music_volume plus duck (null when --duck was not given, else the threshold in dB and linear, ratio, attack_ms, release_ms, amount_db actually used)"}}
|
|
430
435
|
props = dict(base)
|
|
431
436
|
props.update(extra)
|
|
432
437
|
required = ["status", "output", "dry_run", "commands"]
|
package/scripts/audio.py
CHANGED
|
@@ -7,9 +7,12 @@ drops the picture, so `audio.py talk.mp4 -o talk.wav` is an extraction.
|
|
|
7
7
|
Examples:
|
|
8
8
|
python3 audio.py interview.mp4 --denoise # FFT noise reduction
|
|
9
9
|
python3 audio.py interview.mp4 --voice # highpass + de-esser + compressor + denoise
|
|
10
|
+
python3 audio.py interview.mp4 --voice light # highpass + gentle compression only (light|medium|strong)
|
|
10
11
|
python3 audio.py talk.mp4 --music bed.mp3 --duck # music under speech, auto-ducked
|
|
11
12
|
python3 audio.py talk.mp4 --music bed.mp3 --music-volume -18 --music-fade-out 3 # bed fades, voice does not
|
|
12
13
|
python3 audio.py clip.mp4 --fade-in 0.5 --fade-out 1 --stereo
|
|
14
|
+
python3 audio.py talk.mp4 --music bed.mp3 --duck --duck-threshold -30 --duck-release 250 # ducks earlier and recovers faster
|
|
15
|
+
python3 audio.py band.wav --stereo-widen 0.5 -o wide.wav # wider stereo image (a real stereo source; mono is refused)
|
|
13
16
|
python3 audio.py surround.mov --downmix # 5.1 -> stereo with proper centre/LFE weights
|
|
14
17
|
python3 audio.py clip.mp4 --replace narration.wav # swap the audio track entirely
|
|
15
18
|
python3 audio.py interview.mp4 -o interview.wav # extract the audio (no video in the output)
|
|
@@ -18,12 +21,29 @@ Examples:
|
|
|
18
21
|
"""
|
|
19
22
|
import argparse
|
|
20
23
|
import sys
|
|
21
|
-
from typing import List
|
|
24
|
+
from typing import Any, Dict, List
|
|
22
25
|
|
|
23
26
|
from _common import STATE, add_common, apply_common, audio_codec_for, db_to_linear, default_output, die, emit, ffmpeg_base, info, is_audio_output, probe, run, run_keeping_subtitles, fmt_secs
|
|
24
27
|
|
|
25
28
|
VOICE_CHAIN = "highpass=f=80,deesser=i=0.4,afftdn=nf=-25:tn=1,acompressor=threshold=-18dB:ratio=3:attack=5:release=80:makeup=2"
|
|
26
29
|
|
|
30
|
+
# --voice [light|medium|strong] (1.13). "medium" is the chain --voice has always produced, so a
|
|
31
|
+
# bare --voice (and every existing call and MCP request) is byte-identical to before. "light"
|
|
32
|
+
# leaves the noise floor and the sibilance alone -- it only removes rumble and evens the level,
|
|
33
|
+
# which is what a good room recording needs; "strong" is for phone/laptop audio: a harder
|
|
34
|
+
# de-esser, a second compression stage and a soft limiter at -1 dBFS so the peaks stop there
|
|
35
|
+
# instead of at whatever the make-up gain produced.
|
|
36
|
+
VOICE_LEVELS = {
|
|
37
|
+
"light": "highpass=f=80,acompressor=threshold=-18dB:ratio=2:attack=5:release=80:makeup=1",
|
|
38
|
+
"medium": VOICE_CHAIN,
|
|
39
|
+
"strong": VOICE_CHAIN + ",deesser=i=0.6,acompressor=threshold=-24dB:ratio=4:attack=5:release=120:makeup=3,alimiter=limit=0.891251:level=disabled",
|
|
40
|
+
}
|
|
41
|
+
|
|
42
|
+
# The sidechain threshold the music bed has used since 1.4 is the linear 0.05 that ffmpeg's
|
|
43
|
+
# sidechaincompress takes; -26.0206 dBFS is that same number in the unit the flag speaks, so the
|
|
44
|
+
# default command line is unchanged to the byte while the value is now sayable.
|
|
45
|
+
DUCK_THRESHOLD_DB = -26.0206
|
|
46
|
+
|
|
27
47
|
# Typed dynamics: every flag maps to one real option of one ffmpeg filter, validated against the
|
|
28
48
|
# range that filter documents (ffmpeg -h filter=acompressor / alimiter / agate). dB flags are
|
|
29
49
|
# converted to the linear value the filter takes, so no string reaches the graph unchecked.
|
|
@@ -79,13 +99,23 @@ def main() -> int:
|
|
|
79
99
|
clean = ap.add_argument_group("clean-up")
|
|
80
100
|
clean.add_argument("--denoise", action="store_true", help="FFT noise reduction (afftdn, adaptive)")
|
|
81
101
|
clean.add_argument("--denoise-strength", type=float, default=25.0, help="noise floor in dB to remove, 10..60 (default 25)")
|
|
82
|
-
clean.add_argument("--voice",
|
|
102
|
+
clean.add_argument("--voice", nargs="?", const="medium", choices=["light", "medium", "strong"], default=None,
|
|
103
|
+
help="speech preset (default medium when the flag is given bare): light = highpass 80 Hz + gentle compression; "
|
|
104
|
+
"medium = highpass, de-esser, denoise, gentle compression; strong = medium plus a harder de-esser, a second "
|
|
105
|
+
"compressor and a soft limiter at -1 dBFS. MCP/JSON callers may still send the 1.12 boolean true, "
|
|
106
|
+
"which is the bare flag and so means medium")
|
|
83
107
|
clean.add_argument("--gain", type=float, help="gain in dB applied to the main track")
|
|
84
108
|
music = ap.add_argument_group("music")
|
|
85
109
|
music.add_argument("--music", help="music file to mix underneath")
|
|
86
110
|
music.add_argument("--music-volume", type=float, default=-14.0, help="music level in dB relative to full scale (default -14)")
|
|
87
111
|
music.add_argument("--duck", action="store_true", help="auto-duck the music when the main track has speech (sidechain compressor)")
|
|
88
112
|
music.add_argument("--duck-amount", type=float, default=12.0, help="how many dB to duck (default 12)")
|
|
113
|
+
music.add_argument("--duck-threshold", type=float, default=DUCK_THRESHOLD_DB,
|
|
114
|
+
help="sidechain threshold in dBFS: the main track is heard as speech above this (default -26.02, the 0.05 linear used since 1.4)")
|
|
115
|
+
music.add_argument("--duck-attack", type=float, default=20.0, help="ms the bed takes to duck once speech starts (default 20)")
|
|
116
|
+
music.add_argument("--duck-release", type=float, default=400.0, help="ms the bed takes to come back up after speech (default 400)")
|
|
117
|
+
music.add_argument("--effects", help="a third track (sound effects/atmos) mixed in at --effects-volume; never ducked")
|
|
118
|
+
music.add_argument("--effects-volume", type=float, default=-14.0, help="effects level in dB relative to full scale (default -14)")
|
|
89
119
|
music.add_argument("--music-loop", action="store_true", help="loop the music if shorter than the video")
|
|
90
120
|
fades = ap.add_argument_group("fades / layout")
|
|
91
121
|
fades.add_argument("--fade-in", type=float, default=0.0, help="seconds")
|
|
@@ -94,6 +124,10 @@ def main() -> int:
|
|
|
94
124
|
channels = fades.add_mutually_exclusive_group()
|
|
95
125
|
channels.add_argument("--stereo", action="store_true", help="force 2-channel output (mono is duplicated to both sides)")
|
|
96
126
|
channels.add_argument("--mono", action="store_true", help="force 1-channel output")
|
|
127
|
+
fades.add_argument("--stereo-widen", type=float, default=None, metavar="AMOUNT",
|
|
128
|
+
help="widen the stereo image, 0..1 (0 = untouched, 1 = maximum); needs a real stereo source: a mono input is "
|
|
129
|
+
"refused (duplicating it leaves both channels identical, so there is nothing to widen) and more than two "
|
|
130
|
+
"channels are refused unless --downmix folds them to stereo first")
|
|
97
131
|
fades.add_argument("--downmix", action="store_true", help="downmix 5.1/7.1 to stereo using standard weights")
|
|
98
132
|
fades.add_argument("--replace", help="replace the audio with this file (trimmed/padded to the video)")
|
|
99
133
|
dyn = ap.add_argument_group("dynamics (typed; each flag is one option of ffmpeg's acompressor / alimiter / agate)")
|
|
@@ -124,6 +158,26 @@ def main() -> int:
|
|
|
124
158
|
if not getattr(args, switch) and any(getattr(args, f) is not None for f in DYNAMICS[flag_group]):
|
|
125
159
|
die(f"--{switch} is off but one of its parameters was given; add --{switch}")
|
|
126
160
|
|
|
161
|
+
# Same rule as the typed dynamics above, for the ducking knobs: a parameter for a switch that
|
|
162
|
+
# is off does nothing, and a caller who says --duck-release 250 and gets the default 400 ms has
|
|
163
|
+
# no way to notice. --duck itself needs a bed to duck.
|
|
164
|
+
duck_params = [f"--duck-{name}" for name in ("threshold", "attack", "release")
|
|
165
|
+
if getattr(args, f"duck_{name}") != ap.get_default(f"duck_{name}")] + \
|
|
166
|
+
(["--duck-amount"] if args.duck_amount != ap.get_default("duck_amount") else [])
|
|
167
|
+
if not args.duck and duck_params:
|
|
168
|
+
die(f"--duck is off but {duck_params[0]} was given; add --duck")
|
|
169
|
+
if args.duck and not args.music:
|
|
170
|
+
die("--duck ducks the music bed under the main track, but no --music was given; add --music FILE")
|
|
171
|
+
|
|
172
|
+
for flag, value, lo, hi in (("--duck-amount", args.duck_amount, 0.0, 60.0),
|
|
173
|
+
("--duck-threshold", args.duck_threshold, -60.0, 0.0),
|
|
174
|
+
("--duck-attack", args.duck_attack, 0.01, 2000.0),
|
|
175
|
+
("--duck-release", args.duck_release, 0.01, 9000.0)):
|
|
176
|
+
if not (lo <= value <= hi):
|
|
177
|
+
die(f"{flag} {value:g} is outside {lo:g}..{hi:g} (the range ffmpeg's sidechaincompress accepts)")
|
|
178
|
+
if args.stereo_widen is not None and not (0.0 <= args.stereo_widen <= 1.0):
|
|
179
|
+
die(f"--stereo-widen must be 0..1 (0 = untouched, 1 = maximum), got {args.stereo_widen:g}")
|
|
180
|
+
|
|
127
181
|
meta = probe(args.input)
|
|
128
182
|
dur = meta.get("duration") or 0.0
|
|
129
183
|
has_video = bool(meta.get("video"))
|
|
@@ -136,6 +190,19 @@ def main() -> int:
|
|
|
136
190
|
die(f"--audio-stream {args.audio_stream}: input has {len(streams)} audio stream(s), 0..{len(streams) - 1}")
|
|
137
191
|
if args.audio_stream and not streams and not STATE.dry_run:
|
|
138
192
|
die("--audio-stream needs an input with audio streams")
|
|
193
|
+
in_channels = (meta.get("audio") or {}).get("channels") or 0
|
|
194
|
+
if args.stereo_widen is not None:
|
|
195
|
+
if args.mono:
|
|
196
|
+
die("--stereo-widen and --mono contradict each other: there is no stereo image in a 1-channel output")
|
|
197
|
+
# Widening scales the side signal (L-R). Duplicating a mono track to two channels leaves
|
|
198
|
+
# L == R, so the side signal is exactly zero and scaling it changes nothing: --stereo is
|
|
199
|
+
# not a way in, it is a way to a file that measures mono no matter the amount asked for.
|
|
200
|
+
if in_channels == 1:
|
|
201
|
+
die("--stereo-widen needs a real stereo source: mono has no stereo image to widen; keep it mono or "
|
|
202
|
+
"use --stereo to duplicate it, but widening needs a real stereo source")
|
|
203
|
+
if in_channels > 2 and not args.downmix:
|
|
204
|
+
die(f"--stereo-widen needs a stereo track; this input has {in_channels} channels. Add --downmix to fold it "
|
|
205
|
+
"to stereo first (the widening then happens after the downmix), or leave the channels alone.")
|
|
139
206
|
|
|
140
207
|
inputs: List[str] = ["-i", args.input]
|
|
141
208
|
main_src = f"0:a:{args.audio_stream}"
|
|
@@ -150,7 +217,7 @@ def main() -> int:
|
|
|
150
217
|
if args.downmix:
|
|
151
218
|
fx.append("pan=stereo|FL=0.707*FC+FL+0.5*BL+0.5*SL+0.5*LFE|FR=0.707*FC+FR+0.5*BR+0.5*SR+0.5*LFE")
|
|
152
219
|
if args.voice:
|
|
153
|
-
fx.append(
|
|
220
|
+
fx.append(VOICE_LEVELS[args.voice])
|
|
154
221
|
elif args.denoise:
|
|
155
222
|
if not 10 <= args.denoise_strength <= 60:
|
|
156
223
|
die(f"--denoise-strength must be 10..60 (dB of noise floor to remove), got {args.denoise_strength:g}")
|
|
@@ -172,6 +239,11 @@ def main() -> int:
|
|
|
172
239
|
# 1 channel: already mono; the stereo pan used to halve it (-6 dB) because c1 was silence
|
|
173
240
|
elif args.stereo:
|
|
174
241
|
fx.append("aformat=channel_layouts=stereo")
|
|
242
|
+
if args.stereo_widen is not None:
|
|
243
|
+
# extrastereo widens by scaling the side (L-R) signal: m=1 is the input, m=3 is as wide
|
|
244
|
+
# as it goes before the centre collapses. It runs after the channel layout is settled, so
|
|
245
|
+
# a --downmix 5.1 source is widened on the stereo fold-down rather than on six channels.
|
|
246
|
+
fx.append(f"extrastereo=m={1 + 2 * args.stereo_widen:g}")
|
|
175
247
|
|
|
176
248
|
graph: List[str] = []
|
|
177
249
|
graph.append(f"[{main_src}]{','.join(fx) if fx else 'anull'}[main]")
|
|
@@ -192,13 +264,26 @@ def main() -> int:
|
|
|
192
264
|
if args.duck:
|
|
193
265
|
graph.append("[main]asplit=2[mainA][sc]")
|
|
194
266
|
graph.append(
|
|
195
|
-
f"[music][sc]sidechaincompress=threshold=
|
|
267
|
+
f"[music][sc]sidechaincompress=threshold={db_to_linear(args.duck_threshold):.6g}"
|
|
268
|
+
f":ratio={max(2.0, args.duck_amount / 3):.1f}:attack={args.duck_attack:g}:release={args.duck_release:g}:makeup=1[ducked]"
|
|
196
269
|
)
|
|
197
270
|
graph.append("[mainA][ducked]amix=inputs=2:duration=first:dropout_transition=2:normalize=0[mix]")
|
|
198
271
|
else:
|
|
199
272
|
graph.append("[main][music]amix=inputs=2:duration=first:dropout_transition=2:normalize=0[mix]")
|
|
200
273
|
last = "mix"
|
|
201
274
|
|
|
275
|
+
if args.effects:
|
|
276
|
+
# A third bed, mixed in at its own level and deliberately never ducked: effects are cut
|
|
277
|
+
# to the picture, so dipping them under speech would move them off their own frames.
|
|
278
|
+
probe(args.effects)
|
|
279
|
+
inputs += ["-i", args.effects]
|
|
280
|
+
e = f"{idx}:a:0"
|
|
281
|
+
idx += 1
|
|
282
|
+
efx = [f"volume={args.effects_volume:g}dB", f"atrim=0:{dur:.3f}" if dur else "anull"]
|
|
283
|
+
graph.append(f"[{e}]{','.join(efx)}[effects]")
|
|
284
|
+
graph.append(f"[{last}][effects]amix=inputs=2:duration=first:dropout_transition=2:normalize=0[mixfx]")
|
|
285
|
+
last = "mixfx"
|
|
286
|
+
|
|
202
287
|
post: List[str] = []
|
|
203
288
|
if args.fade_in:
|
|
204
289
|
post.append(f"afade=t=in:st=0:d={args.fade_in:g}")
|
|
@@ -237,7 +322,16 @@ def main() -> int:
|
|
|
237
322
|
die(f"{output} unexpectedly contains a video stream")
|
|
238
323
|
info(f"wrote {output} ({fmt_secs(r['duration'])}, audio {a['codec']} {a['channels']}ch {a['sample_rate']}Hz"
|
|
239
324
|
+ (", video stream-copied" if has_video and not audio_out else ", video dropped" if has_video else "") + ")")
|
|
240
|
-
|
|
325
|
+
audio_block: Dict[str, Any] = {"voice": args.voice, "stereo_widen": args.stereo_widen,
|
|
326
|
+
"effects": bool(args.effects), "effects_volume": args.effects_volume if args.effects else None}
|
|
327
|
+
if args.music:
|
|
328
|
+
audio_block["music_volume"] = args.music_volume
|
|
329
|
+
audio_block["duck"] = ({"amount_db": args.duck_amount, "threshold_db": round(args.duck_threshold, 4),
|
|
330
|
+
"threshold_linear": float(f"{db_to_linear(args.duck_threshold):.6g}"),
|
|
331
|
+
"ratio": round(max(2.0, args.duck_amount / 3), 1),
|
|
332
|
+
"attack_ms": args.duck_attack, "release_ms": args.duck_release}
|
|
333
|
+
if args.duck else None)
|
|
334
|
+
emit(output, audio=audio_block, video=bool(has_video and not audio_out), audio_stream=args.audio_stream,
|
|
241
335
|
dynamics=[f for f in (args.gate and "agate", args.compress and "acompressor", args.limit and "alimiter") if f],
|
|
242
336
|
dropped_non_av_streams=dropped_streams)
|
|
243
337
|
return 0
|
package/scripts/check.py
CHANGED
|
@@ -162,6 +162,21 @@ def main() -> int:
|
|
|
162
162
|
|
|
163
163
|
if a:
|
|
164
164
|
row("audio", "PASS", f"{a.get('codec')} {a.get('channels')}ch {a.get('sample_rate')}Hz", "present")
|
|
165
|
+
if args.platform == "podcast":
|
|
166
|
+
# Podcast rows, informational: neither can fail a delivery, both are things a
|
|
167
|
+
# publisher notices after the fact. A 5.1 podcast master is the common one -- every
|
|
168
|
+
# player downmixes it, none of them the same way, and the centre-heavy dialogue
|
|
169
|
+
# comes back at a level nobody checked.
|
|
170
|
+
ch = a.get("channels") or 0
|
|
171
|
+
row("channels", "PASS" if ch in (1, 2) else "WARN", f"{ch}ch", "1 (mono) or 2 (stereo)",
|
|
172
|
+
"audio.py --downmix (5.1/7.1 to stereo with the standard weights) or audio.py --mono",
|
|
173
|
+
reason="podcast players downmix 5.1 unpredictably")
|
|
174
|
+
if args.platform == "podcast":
|
|
175
|
+
chapters = meta.get("chapters") or []
|
|
176
|
+
row("chapters", "PASS" if chapters else "WARN", f"{len(chapters)}" if chapters else "none", ">= 1 chapter marker",
|
|
177
|
+
"metadata.py episode.m4a --chapters chapters.txt (`TIME TITLE` per line; streams copied)",
|
|
178
|
+
reason="chapter markers are optional, but a podcast app shows them as the episode's seekable table of contents")
|
|
179
|
+
if a:
|
|
165
180
|
if a.get("sample_rate") and a["sample_rate"] not in (44100, 48000):
|
|
166
181
|
row("sample rate", "WARN", a["sample_rate"], "44100 or 48000", "loudness.py --sample-rate 48000")
|
|
167
182
|
if not args.no_loudness and spec["lufs"] is not None:
|
package/scripts/loudness.py
CHANGED
|
@@ -11,6 +11,7 @@ Examples:
|
|
|
11
11
|
python3 loudness.py input.mp4 # -14 LUFS, -1 dBTP
|
|
12
12
|
python3 loudness.py podcast.wav -I -16 --tp -1.5 -o podcast_norm.wav
|
|
13
13
|
python3 loudness.py input.mp4 --measure-only
|
|
14
|
+
python3 loudness.py music.wav --lra 7 # tighter loudness range target
|
|
14
15
|
"""
|
|
15
16
|
import argparse
|
|
16
17
|
import json
|
|
@@ -155,6 +156,8 @@ def main() -> int:
|
|
|
155
156
|
kind="verification", output=output, result=result,
|
|
156
157
|
hint="raise --audio-bitrate (e.g. 256k) or deliver a lossless format (wav/flac) and let the platform encode")
|
|
157
158
|
emit(output, result=result, dropped_non_av_streams=dropped_streams,
|
|
159
|
+
measured={k: stats[k] for k in ("input_i", "input_tp", "input_lra", "input_thresh", "target_offset", "silent")},
|
|
160
|
+
targets={"lufs": args.lufs, "tp": args.tp, "lra": args.lra},
|
|
158
161
|
verification=[{"step": "loudness",
|
|
159
162
|
"ok": bool(after.get("silent")) or (abs(float(after["input_i"]) - args.lufs) <= 1.0 and float(after["input_tp"]) <= args.tp + 0.1),
|
|
160
163
|
"lufs": float(after["input_i"]), "tp": float(after["input_tp"]), "target_lufs": args.lufs, "target_tp": args.tp}])
|
package/scripts/render.py
CHANGED
|
@@ -24,15 +24,17 @@ Project format (all keys optional except clips):
|
|
|
24
24
|
{"logo": true},
|
|
25
25
|
{"text": "Episode 12", "position": "bottom", "start": 1, "end": 5, "fade": 0.3, "box": true}
|
|
26
26
|
],
|
|
27
|
-
"audio": {"voice":
|
|
27
|
+
"audio": {"voice": "medium", "music": "bed.mp3", "music_volume": -16, "duck": true, "music_fade_out": 2,
|
|
28
|
+
"effects": "sfx.wav", "stems": {"dialogue": 0, "music": -18, "effects": -22}},
|
|
28
29
|
"loudness": {"lufs": -14, "tp": -1},
|
|
29
30
|
"fit": {"duration": 60},
|
|
30
31
|
"export": {"preset": "reels", "normalize": true}, (default for platform presets; false opts out)
|
|
31
|
-
"check": {"platform": "reels"}
|
|
32
|
+
"check": {"platform": "reels"},
|
|
33
|
+
"chapters": "chapters.txt" (or [{"at": "0:00", "title": "Intro"}, ...])
|
|
32
34
|
}
|
|
33
35
|
|
|
34
36
|
Stages run in this order: clips (cut) → join → silence → fit → captions →
|
|
35
|
-
graphics → overlays → audio → loudness → export → check. Missing stages are
|
|
37
|
+
graphics → overlays → audio → loudness → export → chapters → check. Missing stages are
|
|
36
38
|
skipped. "brand" points caption/graphics/overlay at a brand.json (fonts,
|
|
37
39
|
colours, logo, safe margin); {"logo": true} in overlays places the brand logo.
|
|
38
40
|
|
|
@@ -76,6 +78,7 @@ TEMPLATE = {
|
|
|
76
78
|
"loudness": {"lufs": -14, "tp": -1},
|
|
77
79
|
"fit": None,
|
|
78
80
|
"export": {"preset": "youtube", "normalize": True},
|
|
81
|
+
"chapters": None,
|
|
79
82
|
"check": {"platform": "youtube"},
|
|
80
83
|
}
|
|
81
84
|
|
|
@@ -85,7 +88,7 @@ TEMPLATE = {
|
|
|
85
88
|
# untrimmed, and a mistyped stage name dropped the stage -- both reported as a success (review 9).
|
|
86
89
|
OBJECT_KEYS: Dict[str, frozenset] = {
|
|
87
90
|
"project": frozenset({"output", "frame", "clips", "transition", "silence", "brand", "captions",
|
|
88
|
-
"graphics", "overlays", "audio", "loudness", "fit", "export", "check"}),
|
|
91
|
+
"graphics", "overlays", "audio", "loudness", "fit", "export", "check", "chapters"}),
|
|
89
92
|
"clips[]": frozenset({"src", "in", "out", "speed"}),
|
|
90
93
|
"frame": frozenset({"aspect", "width", "height", "fps"}),
|
|
91
94
|
"transition": frozenset({"type", "duration"}),
|
|
@@ -98,15 +101,20 @@ OBJECT_KEYS: Dict[str, frozenset] = {
|
|
|
98
101
|
"overlays[]": frozenset({"logo", "image", "text", "position", "start", "end", "fade", "opacity",
|
|
99
102
|
"scale", "font_size", "font", "font_file", "margin", "box"}),
|
|
100
103
|
"audio": frozenset({"music", "replace", "music_volume", "fade_in", "fade_out", "music_fade_out",
|
|
101
|
-
"gain", "duck_amount", "
|
|
102
|
-
"mono", "downmix"
|
|
104
|
+
"gain", "duck_amount", "duck_threshold", "duck_attack", "duck_release",
|
|
105
|
+
"voice", "denoise", "duck", "music_loop", "stereo", "mono", "downmix",
|
|
106
|
+
"stereo_widen", "effects", "effects_volume", "stems"}),
|
|
107
|
+
"audio.stems": frozenset({"dialogue", "music", "effects"}),
|
|
108
|
+
"chapters[]": frozenset({"at", "title"}),
|
|
103
109
|
"loudness": frozenset({"lufs", "tp"}),
|
|
104
110
|
"fit": frozenset({"duration", "method", "aspect", "fit", "width", "height", "fps", "smooth"}),
|
|
105
111
|
"export": frozenset({"preset", "fit", "crf", "normalize"}),
|
|
106
112
|
"check": frozenset({"platform"}),
|
|
107
113
|
}
|
|
108
114
|
# Typos difflib cannot see: a clip is trimmed with in/out, not the start/end that time a title.
|
|
109
|
-
NEAR_KEYS: Dict[str, Dict[str, str]] = {"clips[]": {"start": "in", "end": "out", "from": "in", "to": "out"}
|
|
115
|
+
NEAR_KEYS: Dict[str, Dict[str, str]] = {"clips[]": {"start": "in", "end": "out", "from": "in", "to": "out"},
|
|
116
|
+
"audio.stems": {"voice": "dialogue", "speech": "dialogue", "sfx": "effects", "bed": "music"},
|
|
117
|
+
"chapters[]": {"start": "at", "time": "at", "name": "title"}}
|
|
110
118
|
|
|
111
119
|
|
|
112
120
|
def check_keys(obj: Any, schema: str, label: str) -> None:
|
|
@@ -126,6 +134,14 @@ def validate_project(proj: Dict[str, Any]) -> None:
|
|
|
126
134
|
check_keys(proj, "project", "project")
|
|
127
135
|
for name in ("frame", "transition", "silence", "captions", "audio", "loudness", "fit", "export", "check"):
|
|
128
136
|
check_keys(proj.get(name), name, name)
|
|
137
|
+
check_keys((proj.get("audio") or {}).get("stems"), "audio.stems", "audio.stems")
|
|
138
|
+
if isinstance(proj.get("chapters"), list):
|
|
139
|
+
# Every other project error is raised here, before the first ffmpeg call; a chapter typo
|
|
140
|
+
# found inside the last stage costs a whole render and leaves an unchaptered file behind.
|
|
141
|
+
for i, item in enumerate(proj["chapters"]):
|
|
142
|
+
check_keys(item, "chapters[]", f"chapters[{i}]")
|
|
143
|
+
if not isinstance(item, dict) or item.get("at") is None or not str(item.get("title") or "").strip():
|
|
144
|
+
die(f'chapters[{i}]: needs {{"at": TIME, "title": STR}}')
|
|
129
145
|
for name in ("clips", "graphics", "overlays"):
|
|
130
146
|
items = proj.get(name)
|
|
131
147
|
if isinstance(items, list):
|
|
@@ -287,6 +303,9 @@ def main() -> int:
|
|
|
287
303
|
p = str(p)
|
|
288
304
|
return p if os.path.isabs(p) else str(base / p)
|
|
289
305
|
|
|
306
|
+
if isinstance(proj.get("chapters"), str) and not os.path.exists(rel(proj["chapters"])):
|
|
307
|
+
die(f"chapters file not found: {rel(proj['chapters'])}")
|
|
308
|
+
|
|
290
309
|
clips = proj.get("clips") or []
|
|
291
310
|
if not clips:
|
|
292
311
|
die("project.clips is empty")
|
|
@@ -475,17 +494,33 @@ def main() -> int:
|
|
|
475
494
|
return 0
|
|
476
495
|
|
|
477
496
|
# ---- audio
|
|
478
|
-
au = proj.get("audio")
|
|
497
|
+
au = dict(proj.get("audio") or {})
|
|
479
498
|
if au:
|
|
499
|
+
# "stems": one level per element of the mix, the way a mixing desk names them. Each maps
|
|
500
|
+
# to the flag that already exists (dialogue = the main track's gain, music = the bed's
|
|
501
|
+
# level, effects = the third file's level), so a stems block is a vocabulary, not a
|
|
502
|
+
# second code path -- and an explicit flag next to it wins, since it is the more specific
|
|
503
|
+
# statement of the same thing.
|
|
504
|
+
stems = au.pop("stems", None) or {}
|
|
505
|
+
if stems.get("effects") is not None and not au.get("effects"):
|
|
506
|
+
die('audio.stems.effects sets the level of "audio": {"effects": "sfx.wav"}, which this project does not have')
|
|
507
|
+
if stems.get("music") is not None and not au.get("music"):
|
|
508
|
+
die('audio.stems.music sets the level of "audio": {"music": "bed.mp3"}, which this project does not have')
|
|
509
|
+
for stem, key in (("dialogue", "gain"), ("music", "music_volume"), ("effects", "effects_volume")):
|
|
510
|
+
if stems.get(stem) is not None:
|
|
511
|
+
au.setdefault(key, stems[stem])
|
|
480
512
|
nxt = str(work / "audio.mp4")
|
|
481
513
|
argv = [current, "-o", nxt]
|
|
482
|
-
for k, flag in (("music", "--music"), ("replace", "--replace")):
|
|
514
|
+
for k, flag in (("music", "--music"), ("replace", "--replace"), ("effects", "--effects")):
|
|
483
515
|
if au.get(k):
|
|
484
516
|
argv += [flag, rel(au[k])]
|
|
485
|
-
for k, flag in (("music_volume", "--music-volume"), ("fade_in", "--fade-in"), ("fade_out", "--fade-out"), ("music_fade_out", "--music-fade-out"), ("gain", "--gain"), ("duck_amount", "--duck-amount")):
|
|
517
|
+
for k, flag in (("music_volume", "--music-volume"), ("effects_volume", "--effects-volume"), ("fade_in", "--fade-in"), ("fade_out", "--fade-out"), ("music_fade_out", "--music-fade-out"), ("gain", "--gain"), ("duck_amount", "--duck-amount"), ("duck_threshold", "--duck-threshold"), ("duck_attack", "--duck-attack"), ("duck_release", "--duck-release"), ("stereo_widen", "--stereo-widen")):
|
|
486
518
|
if au.get(k) is not None:
|
|
487
519
|
argv += [flag, str(au[k])]
|
|
488
|
-
|
|
520
|
+
# "voice": true is the medium chain; "voice": "light"|"medium"|"strong" names one
|
|
521
|
+
if au.get("voice") is not None and au.get("voice") is not False:
|
|
522
|
+
argv += ["--voice"] + ([] if au["voice"] is True else [str(au["voice"])])
|
|
523
|
+
for k, flag in (("denoise", "--denoise"), ("duck", "--duck"), ("music_loop", "--music-loop"), ("stereo", "--stereo"), ("mono", "--mono"), ("downmix", "--downmix")):
|
|
489
524
|
if au.get(k):
|
|
490
525
|
argv.append(flag)
|
|
491
526
|
sh("audio.py", *argv)
|
|
@@ -536,6 +571,24 @@ def main() -> int:
|
|
|
536
571
|
info(("[dry-run] would copy" if STATE.dry_run else "copied") + f" final stage to {output}")
|
|
537
572
|
current = output
|
|
538
573
|
|
|
574
|
+
# ---- chapters (metadata.py on the delivered file: streams copied, markers written)
|
|
575
|
+
ch = proj.get("chapters")
|
|
576
|
+
if ch:
|
|
577
|
+
# Planned exactly like the audio stage: the metadata.py command names the export's output,
|
|
578
|
+
# which a dry run has not written either. The plan is the run, so --dry-run shows the
|
|
579
|
+
# command and lists the stage (the child's own --dry-run prints rather than writes).
|
|
580
|
+
if isinstance(ch, list):
|
|
581
|
+
chapter_file = str(work / "chapters.txt")
|
|
582
|
+
lines = [f"{entry['at']} {entry['title']}" for entry in ch]
|
|
583
|
+
Path(chapter_file).write_text("\n".join(lines) + "\n", encoding="utf-8")
|
|
584
|
+
else:
|
|
585
|
+
chapter_file = rel(ch)
|
|
586
|
+
tagged = str(work / ("chapters" + Path(output).suffix))
|
|
587
|
+
sh("metadata.py", output, "--chapters", chapter_file, "-o", tagged)
|
|
588
|
+
if not STATE.dry_run:
|
|
589
|
+
place_output(tagged, output)
|
|
590
|
+
stages_done.append("chapters")
|
|
591
|
+
|
|
539
592
|
# ---- check
|
|
540
593
|
ck = proj.get("check")
|
|
541
594
|
check_result = None
|