ffmpeg-skill 2.18.0 → 2.20.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/SKILL.md +1 -1
- package/docs/contract.md +12 -2
- package/package.json +1 -1
- package/references/scripts.md +12 -1
- package/scripts/_common/asr.py +15 -3
- package/scripts/_common/decision.py +104 -18
- package/scripts/_common/fonts.py +51 -11
- package/scripts/_common/probe.py +16 -0
- package/scripts/_contract.py +1 -0
- package/scripts/export.py +14 -0
- package/scripts/graphics.py +1 -1
- package/scripts/join.py +19 -3
- package/scripts/loudness.py +39 -8
- package/scripts/silence.py +40 -3
package/README.md
CHANGED
|
@@ -393,12 +393,12 @@ The short list for humans. The agent-facing version, with the reasoning, is the
|
|
|
393
393
|
- **Phrase-aware caption breaking (1.16).** `caption.py`/`graphics.py --wrap phrase` (the default) never breaks inside a word or on the wrong side of a hyphen, never leaves a lone digit, kana or punctuation pair on a line, prefers Japanese sentence ends and particles over a mid-word break, and never ends a line on an article or preposition. All four are penalties over break positions that already fit, so no line is widened and the line count never changes; `--wrap measured` restores 1.15's width-only wrap. The text itself is never rewritten or shortened. Since 1.16.1 a Thai run and a katakana word are never broken inside (Thai writes no space inside a phrase and there is no dictionary: the break goes where you put a space or `|`), and a line wider than the safe width is reported as `overlong` with the fix named.
|
|
394
394
|
- **Caption size fitted to the cue (1.17, reachable from the templates since 1.17.1).** At a platform caption size a line holds about six em, so an ordinary sentence needs four lines and `--max-lines 2` used to cut it into consecutive cues — half the sentence arriving late. `caption.py --fit-size` (default `auto`) walks the size down until every cue fits, *before* laying the cues out, with a legibility floor of 4.5 % of the frame height (`--min-size`, default 13 ASS units). An explicit `--size` or a `brand.json` size is a statement about the look and is never overridden — which in 1.17.0 also silenced the fitter on every `render.py --template` run, since a template fills the size from the platform table; 1.17.1 marks that size as the default it is (`"fit_size": "on"` in the filled project), so the type shrinks and no cue is split. `--fit-size off` restores that earlier behaviour byte for byte, and the caption text is still never rewritten to make it fit.
|
|
395
395
|
- **Beat-synced cuts (1.17).** `scenes.py --beats` reports the measured grid — tempo, beat times, and a confidence built from how far the winning autocorrelation lag stands above the others and how many onsets land on it. `cut.py --snap beats` (and a `"snap"` block in a `render.py` project) moves in/out points to the nearest beat within `--snap-tolerance` — and only onto the grid points a measured onset actually marks, never onto the regular grid's continuation through a passage with no music in it. Below `--min-confidence` it **refuses**: a cut point may move to a measured beat and may not appear from one, so speech and ambience get an honest "no steady pulse here" instead of an invented grid.
|
|
396
|
-
- **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Whisper stays optional: `--transcribe` with no engine installed refuses and names the five installs.
|
|
396
|
+
- **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Built-in lists since 2.19: English, Japanese, Korean, Chinese, Spanish, German, French, Portuguese, Italian, Dutch, Polish, Russian and Turkish; any other language takes `--filler-words FILE`. Whisper stays optional: `--transcribe` with no engine installed refuses and names the five installs.
|
|
397
397
|
- **Throughput (1.17).** `batch.py --jobs N` runs several files at once, capped at `min(N, cpu_count, 8)` and sharing one `--timeout` budget rather than one per item; the per-item table keeps its order. `render.py --cache DIR` reuses stages whose inputs and arguments did not change, so swapping an export preset re-runs export only. The cache is opt-in with no default directory, and the ffmpeg, skill and contract versions are part of every key, so a cache is never reused across them.
|
|
398
398
|
- **Audiogram (1.16).** `waveform.py --image cover.png` (or `render.py --template audiogram`) puts the waveform over a still plate for an episode that has no picture, with `--platform` for the frame, `--title` and burnt-in captions. The image is a local file you give: nothing is fetched and no cover art is ever invented.
|
|
399
399
|
- **Emoji in captions and titles (1.15).** `caption.py`/`graphics.py --emoji-assets DIR` composites a PNG per emoji (Twemoji/Noto naming, `1f389.png`) on top of the text, because drawtext cannot load a colour emoji font at all and an installed one does not prove libass will draw it in colour — `doctor --json`'s `fonts.emoji` answers that from a render probe. Without assets the run still succeeds and reports `mode: mono`. Nothing is ever downloaded.
|
|
400
400
|
- **Indic and Thai text shaped correctly in titles and lower-thirds (1.15).** `graphics.py` renders Devanagari, Bengali, Tamil, Thai and Lao through libass automatically (`text_renderer: "ass"`), because drawtext never reorders matras or re-clusters marks; Arabic and Hebrew were already correct on a fribidi build. `--text-render drawtext` with such a script is refused, never rendered wrongly.
|
|
401
|
-
- **Non-Latin text picks a font by script (1.12).** Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari, Thai, Cyrillic and Greek cues, titles and overlays resolve a font file that covers them automatically, and a machine with no such font fails the job (`kind: input`) instead of rendering boxes. `doctor --json`'s `fonts.scripts` says which languages this machine can render; `--lang ja|ko` disambiguates Han-only text; an explicit `--font`/`--font-file` is always kept.
|
|
401
|
+
- **Non-Latin text picks a font by script (1.12).** Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari, Thai, Cyrillic and Greek cues, titles and overlays resolve a font file that covers them automatically, and a machine with no such font fails the job (`kind: input`) instead of rendering boxes. `doctor --json`'s `fonts.scripts` says which languages this machine can render; `--lang ja|ko` disambiguates Han-only text, and `--lang zh-TW` (or `zh-HK`, `zh-Hant`) picks a Traditional Chinese face (2.19); an explicit `--font`/`--font-file` is always kept.
|
|
402
402
|
- **Silence detection finds nothing?** The default threshold is −35 dBFS. The tool prints a hint with the track's measured level; raise the threshold (`silence.py --threshold -25`) or shorten `--min-silence`.
|
|
403
403
|
- **Sync results carry a confidence.** Below 0.3, or an offset near the edge of the analysis window, is probably wrong: enlarge `--analyze-seconds` or find a clap. Recordings over ten minutes from separate devices need `sync.py --fix-drift`.
|
|
404
404
|
- **Outputs are never overwritten silently.** An existing output path is refused (`kind: input`, `error.reason: output_exists`, before anything runs); pass `--overwrite` where a replacement is intended.
|
package/SKILL.md
CHANGED
|
@@ -34,7 +34,7 @@ Ask a short question only when the answer changes the output materially and the
|
|
|
34
34
|
- **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, say which. Ask when the content is a talk (trimming loses words) and the change is large.
|
|
35
35
|
- **Captions** without a text source: `--transcribe` if a local engine exists, else ask for the text or a timed file; never invent dialogue.
|
|
36
36
|
- **Fonts and brand**: if the user mentions a brand, colours or "our font", ask for or create `brand.json` once, reuse it.
|
|
37
|
-
- **CJK / non-Latin text**: let the tool pick the font by script (`--font` turns it off); `--lang ja|ko` for Han-only text. `doctor --json` `.fonts.scripts` says what renders here. Tofu is a failed job.
|
|
37
|
+
- **CJK / non-Latin text**: let the tool pick the font by script (`--font` turns it off); `--lang ja|ko|zh-TW` for Han-only text. `doctor --json` `.fonts.scripts` says what renders here. Tofu is a failed job.
|
|
38
38
|
- **Crop position** for `--fit crop`: centre by default, but when the request or the source names an off-centre subject ("keep the product on the right", someone visibly off-centre in the sheet) use `--crop-x`/`--crop-y` (0=left/top, 1=right/bottom) instead of guessing centre, or ask which edge to keep.
|
|
39
39
|
- Anything else (transition type, caption style): pick the conventional default, say what you picked, offer the alternative in one line.
|
|
40
40
|
|
package/docs/contract.md
CHANGED
|
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
|
|
|
21
21
|
| Field | Meaning | Changes when |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
|
|
24
|
-
| `skill.version` | the npm / package.json version (`2.
|
|
24
|
+
| `skill.version` | the npm / package.json version (`2.20.0`) | any release |
|
|
25
25
|
|
|
26
26
|
A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
|
|
27
27
|
ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
|
|
@@ -131,7 +131,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
|
|
|
131
131
|
"deprecated": [{"what": "running on Python 3.9 (the requirements.python floor)", "since": "2.10.9", "replacement": "Python 3.10 or later; ...", "removed_in": "3.0.0", "where": "behaviour"},
|
|
132
132
|
{"what": "running on FFmpeg 5.x (the requirements.ffmpeg floor)", "since": "2.14.0", "replacement": "FFmpeg 6.1 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
|
|
133
133
|
"removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
|
|
134
|
-
"skill": {"id": "ffmpeg-skill", "version": "2.
|
|
134
|
+
"skill": {"id": "ffmpeg-skill", "version": "2.20.0", "execution_mode": "local", "kind": "execution",
|
|
135
135
|
"entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
|
|
136
136
|
"not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
|
|
137
137
|
"requirements": {"python": ">=3.9 (standard library only; 3.9 deprecated since 2.10.9, removed in 3.0.0; 3.10+ recommended)", "ffmpeg": ">=5.0 (5.x deprecated since 2.14.0, removed in 3.0.0; 6.1+ recommended)", "ffprobe": ">=5.0"},
|
|
@@ -699,6 +699,16 @@ from where the expected value came, and what was not measured.
|
|
|
699
699
|
| `audio_substituted` | `join.py` (video join) | the clips that had no audio stream, whose span of the join is generated silence; `notes` says the same |
|
|
700
700
|
| loudness `tolerance.tp` | `loudness.py`, `export.py`, and `check.py`'s true-peak row | the one dBTP margin every tool applies (`_platforms.TP_TOLERANCE_DB`, 0.1 dB); before 2.18 it was 0.1, 0.05 and 0.0 in the three tools |
|
|
701
701
|
|
|
702
|
+
Keys added in 2.20, all additive, from the same paper's baseline experiment (each was a length or a
|
|
703
|
+
silence a tool reported as fine while another measurement disagreed):
|
|
704
|
+
|
|
705
|
+
| key | tool | what it holds |
|
|
706
|
+
|---|---|---|
|
|
707
|
+
| `verification[]` step `duration` | `export.py` (a preset with a platform maximum, without `--allow-long`) | the written file's length against the maximum, measured as `check.py` measures it (the container's duration, `tolerance` 0). A length that cannot be read fails the step with `reason: not_measured` |
|
|
708
|
+
| `verification[]` step `duration` (`property: audio_duration`) | `loudness.py` | the output's sound against the input's, to one AAC frame (at least 0.03 s): a levels pass changes no timing. A length that cannot be read fails the step with `reason: not_measured` |
|
|
709
|
+
| `inputs_duration`, `transition_shortening`; step `duration`'s `inputs_total` | `join.py` | the clips' own total and how much the transitions took out of it. `expected_duration` is derived from the transition the join applied, so it cannot show that on its own |
|
|
710
|
+
| `silence_kinds`, `silence_breakdown` | `silence.py` | per detected silence, `digital` (at or below -90 dBFS) or `below_threshold` (quiet sound under `--threshold`), and the count and seconds of each with the level that defines it |
|
|
711
|
+
|
|
702
712
|
Per-tool keys added after 2.10.11 ("nothing happens silently"), all additive:
|
|
703
713
|
|
|
704
714
|
| key | tool | what it holds |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "ffmpeg-skill",
|
|
3
|
-
"version": "2.
|
|
3
|
+
"version": "2.20.0",
|
|
4
4
|
"description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"ffmpeg",
|
package/references/scripts.md
CHANGED
|
@@ -578,7 +578,7 @@ each shorter clip's last frame (with silence) out to the longest.
|
|
|
578
578
|
### silence.py — remove dead air / jump cuts
|
|
579
579
|
```
|
|
580
580
|
silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [--speech-aware] [-o OUT]
|
|
581
|
-
silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|es|de|fr|pt|it]
|
|
581
|
+
silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|ko|zh|es|de|fr|pt|it|nl|pl|ru|tr]
|
|
582
582
|
[--filler-words FILE] [--filler-extra W,W] [--filler-keep W,W] [--filler-pad 0.02]
|
|
583
583
|
[--transcribe] [--engine auto|parakeet-mlx|parakeet.cpp|whisper.cpp|faster-whisper|openai-whisper]
|
|
584
584
|
[--model MODEL] [--filler-list] [--max-cuts 400]
|
|
@@ -598,6 +598,11 @@ per-word `start`/`end`) or `--transcribe` (the same local engine `caption.py`
|
|
|
598
598
|
uses, never required). There is no heuristic fallback — finding an "um" by
|
|
599
599
|
looking for short quiet blips would cut real speech, so the tool refuses
|
|
600
600
|
instead. `--filler-list` reports what would go and writes nothing.
|
|
601
|
+
Built-in lists (2.19): en, ja, ko, zh, es, de, fr, pt, it, nl, pl, ru, tr. They hold
|
|
602
|
+
hesitation sounds only; words that are as often content (`like`, 저, 那个, ну) are
|
|
603
|
+
`--filler-extra` material. Chinese and Japanese are cut only where the transcript shows
|
|
604
|
+
a word boundary on each side (punctuation or a 0.1 s pause), because whisper splits them
|
|
605
|
+
into tokens rather than words.
|
|
601
606
|
`filler.matched_untimed` counts listed words the transcript gives no usable
|
|
602
607
|
timing for (not removed, and a `filler_words_untimed` warning), and
|
|
603
608
|
`matched_too_long` those longer than a filler (a held vowel, kept).
|
|
@@ -608,11 +613,17 @@ Built-in lists, one per language:
|
|
|
608
613
|
|---|---|
|
|
609
614
|
| en | um, uh, erm, hmm, mm, mhm, er, ah |
|
|
610
615
|
| ja | えー, えーと, えっと, あの, あのー, その, そのー, まあ, なんか |
|
|
616
|
+
| ko | 어, 어어, 에, 음, 음음, 으음, 흠 |
|
|
617
|
+
| zh | 呃, 呃呃, 嗯, 嗯嗯, 唔 |
|
|
611
618
|
| es | eh, este, esto, mmm |
|
|
612
619
|
| de | äh, ähm, hm |
|
|
613
620
|
| fr | euh, hein |
|
|
614
621
|
| pt | é, hum |
|
|
615
622
|
| it | ehm |
|
|
623
|
+
| nl | eh, ehm, uh, uhm, hm |
|
|
624
|
+
| pl | yy, yyy, ee, eee, em, emm, hmm |
|
|
625
|
+
| ru | э, ээ, эм, эмм, мм, хм |
|
|
626
|
+
| tr | ıı, ııı, ee, eee, hmm |
|
|
616
627
|
|
|
617
628
|
**Discourse markers are not disfluencies.** `like`, `tipo` and `cioè` are
|
|
618
629
|
deliberately *not* in the default lists: they are grammatical words in most
|
package/scripts/_common/asr.py
CHANGED
|
@@ -12,10 +12,11 @@ from __future__ import annotations
|
|
|
12
12
|
import json
|
|
13
13
|
import os
|
|
14
14
|
import re
|
|
15
|
+
import unicodedata
|
|
15
16
|
from pathlib import Path
|
|
16
17
|
from typing import Any, Dict, List, NamedTuple, Optional, Tuple
|
|
17
18
|
|
|
18
|
-
from _common.decision import fmt_srt_time, parse_time
|
|
19
|
+
from _common.decision import _unspaced_script, fmt_srt_time, parse_time
|
|
19
20
|
from _common.emit import die, info, warn
|
|
20
21
|
from _common.runner import read_text_or_die
|
|
21
22
|
|
|
@@ -502,7 +503,11 @@ def _words_from_whisper_cpp_json(path: str) -> "List[Dict[str, Any]]":
|
|
|
502
503
|
|
|
503
504
|
Token text carries leading spaces and the model's special tokens ([_BEG_], [_TT_123]); those
|
|
504
505
|
are dropped, and a token that is a word continuation (no leading space) is glued onto the
|
|
505
|
-
previous word so "un" + "believable" is one word with one span, not two.
|
|
506
|
+
previous word so "un" + "believable" is one word with one span, not two. Chinese, Japanese
|
|
507
|
+
and Thai have no spaces, so there every token would continue the one before and a whole
|
|
508
|
+
segment became one "word": between two whole characters of such a script the tokens stay
|
|
509
|
+
apart, as openai-whisper and faster-whisper split them (filler_spans() finds the word
|
|
510
|
+
boundaries). A token that is part of a character (U+FFFD) is still glued.
|
|
506
511
|
"""
|
|
507
512
|
try:
|
|
508
513
|
doc = json.loads(Path(path).read_text(encoding="utf-8"))
|
|
@@ -519,7 +524,14 @@ def _words_from_whisper_cpp_json(path: str) -> "List[Dict[str, Any]]":
|
|
|
519
524
|
start, end = float(offsets["from"]) / 1000.0, float(offsets["to"]) / 1000.0
|
|
520
525
|
except (KeyError, TypeError, ValueError):
|
|
521
526
|
continue
|
|
522
|
-
|
|
527
|
+
prev = out[-1]["word"] if out else ""
|
|
528
|
+
# "嗯" + "," + "呃": the comma stays with the word before it and ends that word
|
|
529
|
+
core = prev
|
|
530
|
+
while core and unicodedata.category(core[-1]).startswith("P"):
|
|
531
|
+
core = core[:-1]
|
|
532
|
+
unspaced = (core and _unspaced_script(core[-1]) and _unspaced_script(text.lstrip()[:1] or " ")
|
|
533
|
+
and "\ufffd" not in prev[-1:] + text)
|
|
534
|
+
if out and not text.startswith(" ") and not unspaced:
|
|
523
535
|
out[-1]["word"] += text
|
|
524
536
|
out[-1]["end"] = end
|
|
525
537
|
else:
|
|
@@ -1158,6 +1158,18 @@ FILLER_WORDS = {
|
|
|
1158
1158
|
"fr": frozenset({"euh", "hein"}),
|
|
1159
1159
|
"pt": frozenset({"é", "hum"}),
|
|
1160
1160
|
"it": frozenset({"ehm"}),
|
|
1161
|
+
# 2.19: Korean is written with spaces, so a whisper word is a word and the list works as the
|
|
1162
|
+
# English one does. 그/저/뭐/이제 are fillers as often as they are "that"/"I"/"what"/"now":
|
|
1163
|
+
# discourse markers below, not defaults.
|
|
1164
|
+
"ko": frozenset({"음", "으음", "음음", "어", "어어", "에", "흠"}),
|
|
1165
|
+
# Chinese has no spaces: these match only as a whole token with a boundary on each side
|
|
1166
|
+
# (filler_spans), so the 额 of 金额 or the 呃 inside a word is never cut. 额 itself is left
|
|
1167
|
+
# out: it is a common character in its own right.
|
|
1168
|
+
"zh": frozenset({"嗯", "嗯嗯", "呃", "呃呃", "唔"}),
|
|
1169
|
+
"ru": frozenset({"э", "ээ", "эм", "эмм", "мм", "хм"}),
|
|
1170
|
+
"nl": frozenset({"eh", "ehm", "uh", "uhm", "hm"}),
|
|
1171
|
+
"pl": frozenset({"yyy", "yy", "eee", "ee", "em", "emm", "hmm"}),
|
|
1172
|
+
"tr": frozenset({"ıı", "ııı", "ee", "eee", "hmm"}),
|
|
1161
1173
|
}
|
|
1162
1174
|
|
|
1163
1175
|
# Words in a default list that are also ordinary vocabulary: every run that removes one says so.
|
|
@@ -1173,11 +1185,33 @@ FILLER_DISCOURSE_MARKERS = {
|
|
|
1173
1185
|
"en": ("like", "you know"),
|
|
1174
1186
|
"pt": ("tipo",),
|
|
1175
1187
|
"it": ("cioè",),
|
|
1188
|
+
"ko": ("그", "저", "뭐", "이제", "약간", "막", "그니까"),
|
|
1189
|
+
"zh": ("那个", "这个", "就是", "然后"),
|
|
1190
|
+
"ru": ("ну", "типа", "короче", "как бы"),
|
|
1191
|
+
"nl": ("nou", "dus", "zeg maar"),
|
|
1192
|
+
"pl": ("no", "jakby", "wiesz", "znaczy"),
|
|
1193
|
+
"tr": ("şey", "yani", "işte"),
|
|
1176
1194
|
}
|
|
1177
1195
|
|
|
1178
1196
|
FILLER_MAX_WORD = 1.2 # a longer "uhhh" is a held vowel someone meant
|
|
1179
1197
|
FILLER_MIN_GAP = 0.05 # spans closer than this become one span
|
|
1180
1198
|
FILLER_PAD = 0.02 # trimmed either side of the word
|
|
1199
|
+
FILLER_MAX_TOKENS = 4 # a listed word may span this many engine tokens ("えー" + "と", "you" + "know")
|
|
1200
|
+
FILLER_BOUNDARY_GAP = 0.1 # a pause this long between two tokens is a word boundary
|
|
1201
|
+
|
|
1202
|
+
|
|
1203
|
+
def _unspaced_script(text: str) -> bool:
|
|
1204
|
+
"""True when `text` has a character of a script written without spaces between words (Han,
|
|
1205
|
+
kana, Thai, Lao, Myanmar, Khmer). Whisper splits those languages into tokens, not words, so a
|
|
1206
|
+
token boundary there says nothing about a word boundary. Hangul is spaced and not listed."""
|
|
1207
|
+
for ch in text:
|
|
1208
|
+
o = ord(ch)
|
|
1209
|
+
if (0x3040 <= o <= 0x30FF or 0x31F0 <= o <= 0x31FF or 0xFF66 <= o <= 0xFF9F # kana
|
|
1210
|
+
or 0x3400 <= o <= 0x4DBF or 0x4E00 <= o <= 0x9FFF or 0xF900 <= o <= 0xFAFF # Han
|
|
1211
|
+
or 0x20000 <= o <= 0x323AF # Han extensions B-H
|
|
1212
|
+
or 0x0E00 <= o <= 0x0EFF or 0x1000 <= o <= 0x109F or 0x1780 <= o <= 0x17FF): # Thai, Lao, Myanmar, Khmer
|
|
1213
|
+
return True
|
|
1214
|
+
return False
|
|
1181
1215
|
|
|
1182
1216
|
|
|
1183
1217
|
def normalise_filler_token(word: str) -> str:
|
|
@@ -1195,37 +1229,89 @@ def filler_spans(words, wordlist, *, pad: float = FILLER_PAD, min_gap: float = F
|
|
|
1195
1229
|
max_word: float = FILLER_MAX_WORD, stats: "Optional[Dict[str, int]]" = None) -> "List[Dict[str, Any]]":
|
|
1196
1230
|
"""Time spans to remove, from measured word timings. Pure: no subprocess, no I/O.
|
|
1197
1231
|
|
|
1198
|
-
`words` is [{"word", "start", "end"}] as whisper emits. A
|
|
1199
|
-
|
|
1200
|
-
|
|
1201
|
-
|
|
1232
|
+
`words` is [{"word", "start", "end"}] as whisper emits. A run of up to FILLER_MAX_TOKENS
|
|
1233
|
+
consecutive entries is removed only when the run, joined with or without spaces and
|
|
1234
|
+
normalised, is in `wordlist` AND every entry carries a real start < end pair AND the run is
|
|
1235
|
+
<= max_word long. The longest run wins. Adjacent spans closer than min_gap merge. Returns
|
|
1236
|
+
[{"start", "end", "word"}] sorted and non-overlapping.
|
|
1237
|
+
|
|
1238
|
+
Whisper splits Chinese, Japanese and Thai into tokens rather than words ("えーと" arrives as
|
|
1239
|
+
"えー" + "と", "金额" as "金" + "额"), so a run in such a script also needs a word boundary on
|
|
1240
|
+
each side: the start or end of the transcript, punctuation, a spaced-script neighbour, or a
|
|
1241
|
+
pause of FILLER_BOUNDARY_GAP. Without one the run is part of a longer word and is kept.
|
|
1202
1242
|
|
|
1203
1243
|
`stats`, when given, receives the listed words that were kept anyway, so the caller can say
|
|
1204
1244
|
so: `matched_untimed` (no usable start < end: nothing to cut on) and `matched_too_long`
|
|
1205
1245
|
(longer than max_word: a held vowel someone meant).
|
|
1206
1246
|
"""
|
|
1247
|
+
import unicodedata as _ud
|
|
1207
1248
|
listed = {normalise_filler_token(w) for w in (wordlist or set())}
|
|
1208
1249
|
listed.discard("")
|
|
1209
|
-
|
|
1210
|
-
untimed = too_long = 0
|
|
1250
|
+
toks = []
|
|
1211
1251
|
for entry in words or []:
|
|
1212
1252
|
if not isinstance(entry, dict):
|
|
1213
1253
|
continue
|
|
1214
|
-
|
|
1215
|
-
if not token or token not in listed:
|
|
1216
|
-
continue
|
|
1254
|
+
raw = str(entry.get("word") or entry.get("text") or "").strip()
|
|
1217
1255
|
try:
|
|
1218
1256
|
start, end = float(entry["start"]), float(entry["end"])
|
|
1257
|
+
timed = end > start
|
|
1219
1258
|
except (KeyError, TypeError, ValueError):
|
|
1220
|
-
|
|
1221
|
-
|
|
1222
|
-
|
|
1223
|
-
|
|
1224
|
-
|
|
1225
|
-
|
|
1226
|
-
|
|
1227
|
-
|
|
1228
|
-
|
|
1259
|
+
start = end = 0.0
|
|
1260
|
+
timed = False
|
|
1261
|
+
toks.append({"norm": normalise_filler_token(raw), "start": start, "end": end, "timed": timed,
|
|
1262
|
+
"punct_before": bool(raw) and _ud.category(raw[0]).startswith("P"),
|
|
1263
|
+
"punct_after": bool(raw) and _ud.category(raw[-1]).startswith("P")})
|
|
1264
|
+
|
|
1265
|
+
def _bounded(i: int, j: int) -> bool:
|
|
1266
|
+
"""A word boundary before toks[i] and after toks[j - 1]."""
|
|
1267
|
+
left = (i == 0 or toks[i]["punct_before"] or toks[i - 1]["punct_after"] or not toks[i - 1]["norm"]
|
|
1268
|
+
or not _unspaced_script(toks[i - 1]["norm"])
|
|
1269
|
+
or (toks[i - 1]["timed"] and toks[i]["start"] - toks[i - 1]["end"] >= FILLER_BOUNDARY_GAP - 1e-6))
|
|
1270
|
+
right = (j == len(toks) or toks[j - 1]["punct_after"] or toks[j]["punct_before"] or not toks[j]["norm"]
|
|
1271
|
+
or not _unspaced_script(toks[j]["norm"])
|
|
1272
|
+
or (toks[j]["timed"] and toks[j]["start"] - toks[j - 1]["end"] >= FILLER_BOUNDARY_GAP - 1e-6))
|
|
1273
|
+
return left and right
|
|
1274
|
+
|
|
1275
|
+
hits = []
|
|
1276
|
+
untimed = too_long = 0
|
|
1277
|
+
i = 0
|
|
1278
|
+
while i < len(toks):
|
|
1279
|
+
found = 0
|
|
1280
|
+
rejected = None
|
|
1281
|
+
for k in range(min(FILLER_MAX_TOKENS, len(toks) - i), 0, -1):
|
|
1282
|
+
run = toks[i:i + k]
|
|
1283
|
+
norms = [t["norm"] for t in run]
|
|
1284
|
+
if not all(norms):
|
|
1285
|
+
continue
|
|
1286
|
+
unspaced = any(_unspaced_script(n) for n in norms)
|
|
1287
|
+
joined = ("" if unspaced else " ").join(norms)
|
|
1288
|
+
if joined not in listed or (unspaced and not _bounded(i, i + k)):
|
|
1289
|
+
continue
|
|
1290
|
+
# punctuation inside the run ("えー、" + "と") ends the word: the shorter run decides
|
|
1291
|
+
if any(run[m]["punct_after"] or run[m + 1]["punct_before"] for m in range(k - 1)):
|
|
1292
|
+
continue
|
|
1293
|
+
# so does a pause inside an unspaced run: the tokens were two words
|
|
1294
|
+
if unspaced and any(run[m]["timed"] and run[m + 1]["timed"]
|
|
1295
|
+
and run[m + 1]["start"] - run[m]["end"] >= FILLER_BOUNDARY_GAP - 1e-6
|
|
1296
|
+
for m in range(k - 1)):
|
|
1297
|
+
continue
|
|
1298
|
+
# a run listed out of time order has no span to cut on
|
|
1299
|
+
if not all(t["timed"] for t in run) or run[-1]["end"] <= run[0]["start"]:
|
|
1300
|
+
rejected = rejected or ("untimed", k)
|
|
1301
|
+
continue
|
|
1302
|
+
if run[-1]["end"] - run[0]["start"] > max_word:
|
|
1303
|
+
rejected = rejected or ("too_long", k)
|
|
1304
|
+
continue
|
|
1305
|
+
hits.append({"start": max(0.0, run[0]["start"] - pad), "end": run[-1]["end"] + pad, "word": joined})
|
|
1306
|
+
found = k
|
|
1307
|
+
break
|
|
1308
|
+
if not found and rejected:
|
|
1309
|
+
# the longest listed run that could not be cut is the one reported. Its prefixes were
|
|
1310
|
+
# tried here; moving on by one token lets a listed word later in it be tried too
|
|
1311
|
+
untimed += rejected[0] == "untimed"
|
|
1312
|
+
too_long += rejected[0] == "too_long"
|
|
1313
|
+
found = 1
|
|
1314
|
+
i += found or 1
|
|
1229
1315
|
hits.sort(key=lambda h: (h["start"], h["end"]))
|
|
1230
1316
|
merged: "List[Dict[str, Any]]" = []
|
|
1231
1317
|
for h in hits:
|
package/scripts/_common/fonts.py
CHANGED
|
@@ -147,13 +147,14 @@ LANGUAGE_NAMES = {
|
|
|
147
147
|
"ja": "Japanese", "zh": "Chinese", "ko": "Korean", "ar": "Arabic", "he": "Hebrew",
|
|
148
148
|
"hi": "Devanagari (Hindi/Marathi/Nepali)", "th": "Thai", "ru": "Cyrillic (Russian and others)",
|
|
149
149
|
"el": "Greek", "latin": "Latin", "bn": "Bengali", "ta": "Tamil", "lo": "Lao",
|
|
150
|
+
"zh-hant": "Traditional Chinese",
|
|
150
151
|
}
|
|
151
152
|
|
|
152
153
|
|
|
153
154
|
# fontconfig's own :lang= codes for each script we detect (zh uses zh-cn, the Simplified subset
|
|
154
155
|
# every CJK font that claims zh carries; the rest are the plain two-letter codes).
|
|
155
156
|
FC_LANG = {"ja": "ja", "zh": "zh-cn", "ko": "ko", "ar": "ar", "he": "he", "hi": "hi", "th": "th", "ru": "ru", "el": "el",
|
|
156
|
-
"bn": "bn", "ta": "ta", "lo": "lo"}
|
|
157
|
+
"bn": "bn", "ta": "ta", "lo": "lo", "zh-hant": "zh-tw"}
|
|
157
158
|
|
|
158
159
|
|
|
159
160
|
# Families tried in order, best first. The names are matched case-insensitively against the start
|
|
@@ -162,6 +163,8 @@ FC_LANG = {"ja": "ja", "zh": "zh-cn", "ko": "ko", "ar": "ar", "he": "he", "hi":
|
|
|
162
163
|
PREFERRED_FAMILIES = {
|
|
163
164
|
"ja": ["Noto Sans CJK JP", "Noto Serif CJK JP", "Noto Sans JP", "Source Han Sans", "IPAPGothic", "IPAGothic", "IPA", "VL Gothic", "TakaoGothic", "WenQuanYi Zen Hei"],
|
|
164
165
|
"zh": ["Noto Sans CJK SC", "Noto Serif CJK SC", "Noto Sans SC", "Source Han Sans", "WenQuanYi Zen Hei", "WenQuanYi Micro Hei", "Droid Sans Fallback"],
|
|
166
|
+
"zh-hant": ["Noto Sans CJK TC", "Noto Serif CJK TC", "Noto Sans CJK HK", "Noto Sans TC", "Noto Sans HK", "Source Han Sans TW",
|
|
167
|
+
"Source Han Sans HC", "AR PL UMing TW", "AR PL UKai TW", "PingFang TC", "WenQuanYi Zen Hei", "Droid Sans Fallback"],
|
|
165
168
|
"ko": ["Noto Sans CJK KR", "Noto Serif CJK KR", "Noto Sans KR", "Source Han Sans K", "NanumGothic", "Nanum Gothic", "Malgun Gothic", "WenQuanYi Zen Hei"],
|
|
166
169
|
"ar": ["Noto Sans Arabic", "Noto Naskh Arabic", "Amiri", "Scheherazade", "DejaVu Sans", "FreeSans", "FreeSerif"],
|
|
167
170
|
"he": ["Noto Sans Hebrew", "Noto Serif Hebrew", "DejaVu Sans", "FreeSans", "FreeSerif"],
|
|
@@ -179,6 +182,7 @@ PREFERRED_FAMILIES = {
|
|
|
179
182
|
WINDOWS_FONTS = {
|
|
180
183
|
"ko": [("malgun.ttf", "Malgun Gothic"), ("gulim.ttc", "Gulim"), ("batang.ttc", "Batang")],
|
|
181
184
|
"zh": [("msyh.ttc", "Microsoft YaHei"), ("simhei.ttf", "SimHei"), ("simsun.ttc", "SimSun")],
|
|
185
|
+
"zh-hant": [("msjh.ttc", "Microsoft JhengHei"), ("mingliu.ttc", "MingLiU")],
|
|
182
186
|
"ja": [("meiryo.ttc", "Meiryo"), ("YuGothM.ttc", "Yu Gothic Medium"), ("YuGothR.ttc", "Yu Gothic"), ("msgothic.ttc", "MS Gothic")],
|
|
183
187
|
"ar": [("tahoma.ttf", "Tahoma"), ("arial.ttf", "Arial")],
|
|
184
188
|
"he": [("tahoma.ttf", "Tahoma"), ("arial.ttf", "Arial")],
|
|
@@ -277,6 +281,22 @@ def detect_script(text: str, lang: "Optional[str]" = None) -> str:
|
|
|
277
281
|
return best
|
|
278
282
|
|
|
279
283
|
|
|
284
|
+
# 2.19: Han-only text is Chinese, and Chinese is written in two standard forms whose glyphs differ
|
|
285
|
+
# (骨, 直, 角 are drawn differently in Taipei and Beijing). A Simplified face draws a Taiwanese
|
|
286
|
+
# caption in the wrong form, so a --lang naming Traditional Chinese resolves a Traditional face.
|
|
287
|
+
# The script stays "zh" for wrapping; only the font lookup changes.
|
|
288
|
+
HANT_TAGS = ("zh-tw", "zh-hk", "zh-mo", "zh-hant", "yue")
|
|
289
|
+
|
|
290
|
+
|
|
291
|
+
def font_key(script: str, lang: "Optional[str]" = None) -> str:
|
|
292
|
+
"""The font-table key for `script` under the caller's `lang`: "zh-hant" for Chinese text the
|
|
293
|
+
caller says is Traditional (zh-TW, zh-HK, zh-MO, zh-Hant-*, yue), else `script` itself."""
|
|
294
|
+
tag = (lang or "").strip().lower().replace("_", "-")
|
|
295
|
+
if script == "zh" and any(tag == t or tag.startswith(t + "-") for t in HANT_TAGS):
|
|
296
|
+
return "zh-hant"
|
|
297
|
+
return script
|
|
298
|
+
|
|
299
|
+
|
|
280
300
|
_SCRIPT_FONT_CACHE: "Dict[Tuple[str, Optional[str]], Optional[Tuple[str, str]]]" = {}
|
|
281
301
|
|
|
282
302
|
|
|
@@ -359,7 +379,11 @@ def _script_font_uncached(script: str, family_hint: "Optional[str]" = None):
|
|
|
359
379
|
mono = 1 if "mono" in joined else 0
|
|
360
380
|
# regular weights before Bold/Italic/Oblique cuts, so a default caption is not bold by accident
|
|
361
381
|
styled = 1 if any(k in stem for k in ("bold", "italic", "oblique", "light", "thin", "black")) else 0
|
|
362
|
-
|
|
382
|
+
rank = _family_rank(families, preferred)
|
|
383
|
+
# a .ttc lists every face it holds (Noto CJK: JP, KR, SC, TC): name the one that ranked
|
|
384
|
+
family = next((f for f in families if rank < len(preferred) and f.lower().startswith(preferred[rank].lower())),
|
|
385
|
+
families[0])
|
|
386
|
+
scored.append((last_resort, rank, mono, styled, path, family))
|
|
363
387
|
scored.sort(key=lambda row: (row[0], row[1], row[2], row[3], row[4]))
|
|
364
388
|
best = scored[0]
|
|
365
389
|
return best[4], best[5]
|
|
@@ -453,6 +477,11 @@ def fonts_dir_covers_script(fonts_dir: str, script: str) -> "Optional[bool]":
|
|
|
453
477
|
return False
|
|
454
478
|
|
|
455
479
|
|
|
480
|
+
def _is_last_resort(entry: "Tuple[str, str]") -> bool:
|
|
481
|
+
"""Unifont, fontconfig's code-point-box face: it covers every script and renders none."""
|
|
482
|
+
return "unifont" in " ".join(str(x) for x in entry).lower()
|
|
483
|
+
|
|
484
|
+
|
|
456
485
|
def script_font_for_text(text: str, *, lang: "Optional[str]" = None, font: "Optional[str]" = None,
|
|
457
486
|
font_explicit: bool = False, font_file: "Optional[str]" = None,
|
|
458
487
|
fonts_dir: "Optional[str]" = None
|
|
@@ -476,30 +505,41 @@ def script_font_for_text(text: str, *, lang: "Optional[str]" = None, font: "Opti
|
|
|
476
505
|
return script, None, None
|
|
477
506
|
if font_file:
|
|
478
507
|
return script, None, None
|
|
508
|
+
key = font_key(script, lang)
|
|
479
509
|
if font_explicit and font:
|
|
480
|
-
if not font_covers_script(font,
|
|
481
|
-
warn(f"font: '{font}' does not cover {LANGUAGE_NAMES[
|
|
510
|
+
if not font_covers_script(font, key):
|
|
511
|
+
warn(f"font: '{font}' does not cover {LANGUAGE_NAMES[key]} text on this machine; keeping it as asked "
|
|
482
512
|
f"(drop --font, or {FONT_FLAG_HINT}, to pick one by script automatically)", "font_lacks_script",
|
|
483
513
|
font=font, script=script)
|
|
484
514
|
return script, None, None
|
|
485
515
|
if fonts_dir:
|
|
486
|
-
covered = fonts_dir_covers_script(fonts_dir,
|
|
516
|
+
covered = fonts_dir_covers_script(fonts_dir, key)
|
|
487
517
|
if covered:
|
|
488
518
|
return script, None, None
|
|
489
519
|
if covered is None:
|
|
490
|
-
warn(f"font: could not verify that {fonts_dir} covers {LANGUAGE_NAMES[
|
|
520
|
+
warn(f"font: could not verify that {fonts_dir} covers {LANGUAGE_NAMES[key]} text "
|
|
491
521
|
"(no fc-scan on this machine); using it as given", "font_coverage_unverified", fonts_dir=fonts_dir, script=script)
|
|
492
522
|
return script, None, None
|
|
493
|
-
record_fallback(f"font: no face in {fonts_dir} covers {LANGUAGE_NAMES[
|
|
523
|
+
record_fallback(f"font: no face in {fonts_dir} covers {LANGUAGE_NAMES[key]} text; "
|
|
494
524
|
"picking one by script instead (the directory is still searched first)", "fonts_dir_lacks_script",
|
|
495
525
|
fonts_dir=fonts_dir, script=script)
|
|
496
|
-
entry = _script_font_entry(
|
|
526
|
+
entry = _script_font_entry(key)
|
|
527
|
+
if key != script and (not entry or (entry is not FC_UNKNOWN and _is_last_resort(entry))):
|
|
528
|
+
# no Traditional face (Unifont's code-point boxes are none): the Simplified one still
|
|
529
|
+
# has the characters, in the other form
|
|
530
|
+
simplified = _script_font_entry(script)
|
|
531
|
+
if simplified and simplified is not FC_UNKNOWN and (not entry or not _is_last_resort(simplified)):
|
|
532
|
+
entry = simplified
|
|
533
|
+
record_fallback(f"font: no {LANGUAGE_NAMES[key]} face is installed; drawing with {entry[0]}, a "
|
|
534
|
+
"Simplified Chinese face whose glyph forms differ in places "
|
|
535
|
+
"(install fonts-noto-cjk or a TC font)", "font_variant_missing", lang=lang, file=entry[0])
|
|
536
|
+
key = script
|
|
497
537
|
if entry is FC_UNKNOWN:
|
|
498
|
-
warn(f"font: could not verify that this machine can render {LANGUAGE_NAMES[
|
|
538
|
+
warn(f"font: could not verify that this machine can render {LANGUAGE_NAMES[key]} text "
|
|
499
539
|
"(no working fontconfig); rendering with the font as given -- "
|
|
500
540
|
"doctor --json .fonts.scripts reports what is known", "font_coverage_unverified", script=script)
|
|
501
541
|
return script, None, None
|
|
502
542
|
if not entry:
|
|
503
|
-
die(f"no installed font covers {LANGUAGE_NAMES[
|
|
504
|
-
info(f"font: {entry[0]} (covers {
|
|
543
|
+
die(f"no installed font covers {LANGUAGE_NAMES[key]} text on this machine — {FONT_INSTALL_HINT}", kind="input")
|
|
544
|
+
info(f"font: {entry[0]} (covers {key})")
|
|
505
545
|
return script, entry[0], entry[1]
|
package/scripts/_common/probe.py
CHANGED
|
@@ -300,6 +300,9 @@ def probe(path: str, role: str = "input") -> Dict[str, Any]:
|
|
|
300
300
|
"bitrate": _to_int(audio.get("bit_rate")),
|
|
301
301
|
"duration": _to_float(audio.get("duration")),
|
|
302
302
|
"start_time": _to_float(audio.get("start_time")),
|
|
303
|
+
# Matroska/WebM give no stream duration, only a DURATION tag; the container's own length
|
|
304
|
+
# is the longest stream's, which is not the sound's when the picture runs longer (2.20)
|
|
305
|
+
"tagged_duration": _tag_seconds((audio.get("tags") or {}).get("DURATION")),
|
|
303
306
|
}
|
|
304
307
|
# every audio stream in file order: index n here is `-map 0:a:n` (audio.py --audio-stream n)
|
|
305
308
|
out["audio_streams"] = [{
|
|
@@ -497,6 +500,19 @@ def analyze_levels(path: str, seconds: float = 20.0) -> Dict[str, Any]:
|
|
|
497
500
|
}
|
|
498
501
|
|
|
499
502
|
|
|
503
|
+
def _tag_seconds(value: Any) -> "Optional[float]":
|
|
504
|
+
"""A Matroska DURATION tag ("00:00:04.008000000") in seconds, or None."""
|
|
505
|
+
try:
|
|
506
|
+
if "-" in str(value): # "-00:00:01": int("-00") is 0, so the sign would be lost
|
|
507
|
+
return None
|
|
508
|
+
h, m, sec = str(value).split(":")
|
|
509
|
+
seconds = int(h) * 3600 + int(m) * 60 + float(sec)
|
|
510
|
+
except (TypeError, ValueError):
|
|
511
|
+
return None
|
|
512
|
+
# float() reads "nan" and "inf": a length that is not a finite, non-negative number is none
|
|
513
|
+
return seconds if math.isfinite(seconds) and seconds >= 0 else None
|
|
514
|
+
|
|
515
|
+
|
|
500
516
|
def _to_float(v: Any) -> Optional[float]:
|
|
501
517
|
try:
|
|
502
518
|
return float(v) if v is not None else None
|
package/scripts/_contract.py
CHANGED
|
@@ -456,6 +456,7 @@ DIAGNOSTIC_CODES = {
|
|
|
456
456
|
"font_path_copied": ("fallbacks", "a --font-file under a non-ASCII path was copied to an ASCII one, because drawtext cannot open it on Windows; {font, file}"),
|
|
457
457
|
"font_path_not_ascii": ("warnings", "a --font-file under a non-ASCII path could not be given an ASCII form on Windows; drawtext may not open it; {font}"),
|
|
458
458
|
"fonts_dir_lacks_script": ("fallbacks", "no face in --fonts-dir covers the text's script, so one was picked by script; {fonts_dir, script}"),
|
|
459
|
+
"font_variant_missing": ("fallbacks", "--lang named Traditional Chinese but no Traditional face is installed, so a Simplified one draws it (some glyph forms differ); {lang, file}"),
|
|
459
460
|
"emoji_monochrome": ("warnings", "emoji drawn by the text font (monochrome at best): no colour path here"),
|
|
460
461
|
"emoji_stripped": ("warnings", "overlay.py --emoji none: emoji removed from the drawn text; {mode}"),
|
|
461
462
|
"emoji_assets_missing": ("warnings", "--emoji-assets has no PNG for these clusters; the text font draws them; {missing}"),
|
package/scripts/export.py
CHANGED
|
@@ -333,6 +333,20 @@ def main() -> int:
|
|
|
333
333
|
elif platform and has_audio and args.normalize:
|
|
334
334
|
spec = PLATFORMS[platform]
|
|
335
335
|
notes.append(f"[dry-run] --normalize: loudness.py -I {spec['lufs']:g} --tp {spec['tp']:g} would run on the written file if it misses {platform}'s spec")
|
|
336
|
+
if p["max"] and not args.allow_long and not STATE.dry_run:
|
|
337
|
+
# the preset's length promise, measured the way check.py measures it (the container's
|
|
338
|
+
# duration, no tolerance): before 2.20 nothing re-read the length, and a --normalize pass
|
|
339
|
+
# that left the sound 0.1 s long shipped a 90.10 s Reels file as verified
|
|
340
|
+
written = probe(output, role="output").get("duration")
|
|
341
|
+
if written is not None:
|
|
342
|
+
step = {"step": "duration", "property": "duration", "ok": written <= p["max"],
|
|
343
|
+
"expected": {"max": p["max"]}, "measured": round(written, 3),
|
|
344
|
+
"delta": round(written - p["max"], 3), "tolerance": 0.0, "provenance": "specification"}
|
|
345
|
+
else:
|
|
346
|
+
# the preset promised a length; one that could not be read is not one that held
|
|
347
|
+
step = {"step": "duration", "property": "duration", "ok": False, "expected": {"max": p["max"]},
|
|
348
|
+
"reason": "not_measured: the written file reports no duration"}
|
|
349
|
+
extra.setdefault("verification", []).append(step)
|
|
336
350
|
if notes:
|
|
337
351
|
extra["notes"] = notes
|
|
338
352
|
# the delivery keeps the source's picture and, except as a GIF, its sound
|
package/scripts/graphics.py
CHANGED
|
@@ -294,7 +294,7 @@ def main(force_route: Optional[str] = None) -> int:
|
|
|
294
294
|
if explicit:
|
|
295
295
|
return font_family_of_file(explicit) or args.font or brand.get("font")
|
|
296
296
|
if script_file:
|
|
297
|
-
return font_family_of_file(script_file) or font_family_for_script(_script)
|
|
297
|
+
return _family or font_family_of_file(script_file) or font_family_for_script(_script)
|
|
298
298
|
return args.font or brand.get("font", "DejaVu Sans")
|
|
299
299
|
|
|
300
300
|
def ass_fonts_dir() -> "Optional[str]":
|
package/scripts/join.py
CHANGED
|
@@ -81,6 +81,20 @@ def unpending_length(durs: List[float], d: float) -> Optional[float]:
|
|
|
81
81
|
return round(sum(durs) - d * (len(durs) - 1), 3)
|
|
82
82
|
|
|
83
83
|
|
|
84
|
+
def overlap_report(durs: List[float], expected: Optional[float]) -> Dict[str, Any]:
|
|
85
|
+
"""The clips' own total next to the joined length (2.20). The joined length is derived from
|
|
86
|
+
the transition the join itself applied, so a check against it alone cannot show that a fade
|
|
87
|
+
took time out of the material; the sum of the inputs is the yardstick the user holds."""
|
|
88
|
+
if expected is None or STATE_PENDING:
|
|
89
|
+
return {}
|
|
90
|
+
total = round(sum(durs), 3)
|
|
91
|
+
out: Dict[str, Any] = {"inputs_duration": total, "transition_shortening": round(total - expected, 3)}
|
|
92
|
+
if out["transition_shortening"] > 0.0005:
|
|
93
|
+
STATE_NOTES.append(f"the transitions overlap the clips: {fmt_secs(total)} of clips play as {fmt_secs(expected)} "
|
|
94
|
+
f"({out['transition_shortening']:.3f} s shorter)")
|
|
95
|
+
return out
|
|
96
|
+
|
|
97
|
+
|
|
84
98
|
def clip_length(meta: Dict[str, Any], fps: float) -> float:
|
|
85
99
|
"""The one length a clip holds in a join, for its picture and its sound alike: the
|
|
86
100
|
video stream's, or the audio stream's when the sound runs more than a frame past the picture
|
|
@@ -175,7 +189,8 @@ def join_audio(args: argparse.Namespace, metas: List[dict]) -> int:
|
|
|
175
189
|
die(f"{output} is {a.get('sample_rate')} Hz {a.get('channels')} ch, expected {rate} Hz {channels} ch")
|
|
176
190
|
info(f"wrote {output} ({fmt_secs(r['duration'])}, {expected_text(expected)}, audio {a.get('codec')} {channels}ch {rate}Hz, {n} clips, "
|
|
177
191
|
+ ("crossfade" if d else "butt join") + ")")
|
|
178
|
-
|
|
192
|
+
overlap = overlap_report(durs, expected)
|
|
193
|
+
emit(output, mode="audio", **overlap, **reported(), clips=n, transition=args.transition if d else "none", expected_duration=expected,
|
|
179
194
|
sample_rate=rate, channels=channels, video=False)
|
|
180
195
|
return 0
|
|
181
196
|
|
|
@@ -568,7 +583,7 @@ def main() -> int:
|
|
|
568
583
|
verification.append({"step": "duration", "property": "video_duration", "ok": abs(measured - expected) <= tol,
|
|
569
584
|
"measured": round(measured, 3), "expected": expected,
|
|
570
585
|
"delta": round(measured - expected, 3), "tolerance": round(tol, 3),
|
|
571
|
-
"provenance": "input_premeasurement"})
|
|
586
|
+
"provenance": "input_premeasurement", "inputs_total": round(sum(lens), 3)})
|
|
572
587
|
# a clip with no audio stream is joined over generated silence: the join is right to do that, but
|
|
573
588
|
# the report must say which clips' sound is not theirs (the paper's join finding: absence was
|
|
574
589
|
# filled and nothing in the result showed it)
|
|
@@ -577,7 +592,8 @@ def main() -> int:
|
|
|
577
592
|
STATE_NOTES.append(f"{len(substituted)} clip(s) had no audio stream; their span of the join is generated "
|
|
578
593
|
"silence: " + ", ".join(os.path.basename(p) for p in substituted))
|
|
579
594
|
info(f"wrote {output} ({fmt_secs(measured)}, {expected_text(expected)}, {w}x{h} @ {fps:g}fps, {n} clips, {args.transition})")
|
|
580
|
-
|
|
595
|
+
overlap = overlap_report(lens, expected)
|
|
596
|
+
emit(output, mode="video", **overlap, **reported(), clips=n, transition=args.transition, expected_duration=expected,
|
|
581
597
|
verification=verification, expect_streams={"video": True, "audio": True},
|
|
582
598
|
audio_substituted=substituted,
|
|
583
599
|
dropped_non_av_streams=any(m.get("subtitle_streams") or m.get("data_streams") for m in metas))
|
package/scripts/loudness.py
CHANGED
|
@@ -20,6 +20,8 @@ import re
|
|
|
20
20
|
import sys
|
|
21
21
|
|
|
22
22
|
from _platforms import TP_TOLERANCE_DB
|
|
23
|
+
|
|
24
|
+
AUDIO_LENGTH_TOLERANCE = 0.03 # seconds: one AAC frame at 48 kHz is 0.021 s
|
|
23
25
|
from _common import STATE, add_common, apply_common, emit, AUDIO_CODECS, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run, run_analysis, run_keeping_subtitles, dry_run_input_pending
|
|
24
26
|
|
|
25
27
|
|
|
@@ -93,6 +95,12 @@ def main() -> int:
|
|
|
93
95
|
bitrate = args.audio_bitrate or "192k"
|
|
94
96
|
|
|
95
97
|
dropped_streams = False
|
|
98
|
+
# loudnorm emits its last block whole, and through the AAC encoder that came out as 0.1 s more
|
|
99
|
+
# sound than the input had (a 90.000 s Reels export left at 90.100 s, which check.py fails):
|
|
100
|
+
# the sound is trimmed back to the input's own length. `duration=` counts from the first sample,
|
|
101
|
+
# so sound that starts late keeps its end (`end=` is a timestamp, and cut it). atrim follows
|
|
102
|
+
# loudnorm directly: an aresample between them fails channel-layout negotiation on FFmpeg 5.1
|
|
103
|
+
in_audio = audio_length(meta)
|
|
96
104
|
|
|
97
105
|
def encode(tp: float, bitrate: str) -> None:
|
|
98
106
|
nonlocal dropped_streams
|
|
@@ -101,6 +109,8 @@ def main() -> int:
|
|
|
101
109
|
f":measured_I={stats['input_i']}:measured_TP={stats['input_tp']}:measured_LRA={stats['input_lra']}"
|
|
102
110
|
f":measured_thresh={stats['input_thresh']}:offset={stats['target_offset']}:linear=true:print_format=summary"
|
|
103
111
|
)
|
|
112
|
+
if in_audio:
|
|
113
|
+
af += f",atrim=duration={float(in_audio):.6f}"
|
|
104
114
|
cmd = ffmpeg_base() + ["-i", args.input, "-af", af, "-ar", str(sr)]
|
|
105
115
|
if ext in AUDIO_CODECS or not meta.get("video"):
|
|
106
116
|
cmd += ["-vn"] + audio_codec_for(output, bitrate)
|
|
@@ -158,19 +168,40 @@ def main() -> int:
|
|
|
158
168
|
f"the encoder overshoots more than the loudnorm ceiling can absorb at this bitrate",
|
|
159
169
|
kind="verification", output=output, result=result,
|
|
160
170
|
hint="raise --audio-bitrate (e.g. 256k) or deliver a lossless format (wav/flac) and let the platform encode")
|
|
171
|
+
verification = [{"step": "loudness",
|
|
172
|
+
"ok": bool(after.get("silent")) or (abs(float(after["input_i"]) - args.lufs) <= 1.0 and float(after["input_tp"]) <= args.tp + TP_TOLERANCE_DB),
|
|
173
|
+
"lufs": float(after["input_i"]), "tp": float(after["input_tp"]), "target_lufs": args.lufs, "target_tp": args.tp,
|
|
174
|
+
"property": "loudness", "expected": {"lufs": args.lufs, "tp_max": args.tp},
|
|
175
|
+
"measured": {"lufs": float(after["input_i"]), "tp": float(after["input_tp"])},
|
|
176
|
+
"delta": {"lufs": round(float(after["input_i"]) - args.lufs, 2), "tp": round(float(after["input_tp"]) - args.tp, 2)},
|
|
177
|
+
"tolerance": {"lufs": 1.0, "tp": TP_TOLERANCE_DB}, "provenance": "user_specified"}]
|
|
178
|
+
written = probe(output, role="output")
|
|
179
|
+
out_audio = audio_length(written)
|
|
180
|
+
if in_audio and out_audio:
|
|
181
|
+
# a levels pass changes no timing: the sound is as long as the input's, to one AAC frame
|
|
182
|
+
tol = max(1024.0 / float(sr), AUDIO_LENGTH_TOLERANCE)
|
|
183
|
+
verification.append({"step": "duration", "property": "audio_duration", "ok": abs(out_audio - in_audio) <= tol,
|
|
184
|
+
"expected": round(float(in_audio), 3), "measured": round(float(out_audio), 3),
|
|
185
|
+
"delta": round(float(out_audio) - float(in_audio), 3), "tolerance": round(tol, 3),
|
|
186
|
+
"provenance": "input_premeasurement"})
|
|
187
|
+
else:
|
|
188
|
+
# a length that could not be read is not a length that matched
|
|
189
|
+
verification.append({"step": "duration", "property": "audio_duration", "ok": False,
|
|
190
|
+
"reason": "not_measured: the " + ("input" if not in_audio else "output") + " reports no duration"})
|
|
161
191
|
emit(output, result=result, dropped_non_av_streams=dropped_streams,
|
|
162
192
|
measured={k: stats[k] for k in ("input_i", "input_tp", "input_lra", "input_thresh", "target_offset", "silent")},
|
|
163
|
-
targets={"lufs": args.lufs, "tp": args.tp, "lra": args.lra},
|
|
164
|
-
verification=[{"step": "loudness",
|
|
165
|
-
"ok": bool(after.get("silent")) or (abs(float(after["input_i"]) - args.lufs) <= 1.0 and float(after["input_tp"]) <= args.tp + TP_TOLERANCE_DB),
|
|
166
|
-
"lufs": float(after["input_i"]), "tp": float(after["input_tp"]), "target_lufs": args.lufs, "target_tp": args.tp,
|
|
167
|
-
"property": "loudness", "expected": {"lufs": args.lufs, "tp_max": args.tp},
|
|
168
|
-
"measured": {"lufs": float(after["input_i"]), "tp": float(after["input_tp"])},
|
|
169
|
-
"delta": {"lufs": round(float(after["input_i"]) - args.lufs, 2), "tp": round(float(after["input_tp"]) - args.tp, 2)},
|
|
170
|
-
"tolerance": {"lufs": 1.0, "tp": TP_TOLERANCE_DB}, "provenance": "user_specified"}])
|
|
193
|
+
targets={"lufs": args.lufs, "tp": args.tp, "lra": args.lra}, verification=verification)
|
|
171
194
|
return 0
|
|
172
195
|
|
|
173
196
|
|
|
197
|
+
def audio_length(meta: dict) -> "Optional[float]":
|
|
198
|
+
"""The sound's own length: the stream's duration, else Matroska's DURATION tag. Never the
|
|
199
|
+
container's, which is the picture's when the picture runs longer (and would fail a correct
|
|
200
|
+
output against an expected length the sound never had)."""
|
|
201
|
+
audio = meta.get("audio") or {}
|
|
202
|
+
return audio.get("duration") or audio.get("tagged_duration")
|
|
203
|
+
|
|
204
|
+
|
|
174
205
|
def _kbps(value: str) -> int:
|
|
175
206
|
v = value.lower().rstrip("k")
|
|
176
207
|
try:
|
package/scripts/silence.py
CHANGED
|
@@ -15,7 +15,7 @@ import argparse
|
|
|
15
15
|
import json
|
|
16
16
|
import os
|
|
17
17
|
import sys
|
|
18
|
-
from typing import List, Tuple
|
|
18
|
+
from typing import Any, Dict, List, Tuple
|
|
19
19
|
|
|
20
20
|
# `detect` moved into _common/probe.py in 1.16.0 so metadata.py --auto-chapters can measure the
|
|
21
21
|
# same silences without importing this tool; the body is unchanged and the name still lives here.
|
|
@@ -44,6 +44,7 @@ def merge_spans(spans: List[Tuple[float, float]]) -> List[Tuple[float, float]]:
|
|
|
44
44
|
return out
|
|
45
45
|
|
|
46
46
|
|
|
47
|
+
DIGITAL_SILENCE_DB = -90.0 # 2.20: at or below this a span is digital silence (16-bit dither sits near -96)
|
|
47
48
|
BREATH_FLOOR = 0.12 # shortest gap silencedetect is asked for under --speech-aware
|
|
48
49
|
|
|
49
50
|
|
|
@@ -152,7 +153,10 @@ def resolve_filler(args, meta):
|
|
|
152
153
|
# (or one that says "auto", which is no language either) keeps the English list it always had
|
|
153
154
|
# the ISO 639-2 spellings of the languages that have a built-in list (the mux tag uses them)
|
|
154
155
|
iso3 = {"eng": "en", "spa": "es", "deu": "de", "ger": "de", "fra": "fr", "fre": "fr",
|
|
155
|
-
"ita": "it", "jpn": "ja", "por": "pt"
|
|
156
|
+
"ita": "it", "jpn": "ja", "por": "pt", "kor": "ko", "zho": "zh", "chi": "zh",
|
|
157
|
+
"rus": "ru", "nld": "nl", "dut": "nl", "pol": "pl", "tur": "tr"}
|
|
158
|
+
names = {"spanish": "es", "german": "de", "portuguese": "pt", "chinese": "zh", "mandarin": "zh",
|
|
159
|
+
"dutch": "nl", "polish": "pl", "turkish": "tr"}
|
|
156
160
|
|
|
157
161
|
def _code(value) -> str:
|
|
158
162
|
value = str(value or "").strip().lower().replace("_", "-")
|
|
@@ -160,10 +164,13 @@ def resolve_filler(args, meta):
|
|
|
160
164
|
return ""
|
|
161
165
|
# a regional tag names its primary language (ja-JP is ja); a three-letter code that is not
|
|
162
166
|
# in the table stays as it is (est is Estonian, not the Spanish "es"); a word such as
|
|
163
|
-
# "english" is cut to its two-letter code
|
|
167
|
+
# "english" is cut to its two-letter code; a name whose first two letters are not its
|
|
168
|
+
# code ("spanish" is es, "chinese" zh) is looked up, as whisper's own language table spells it
|
|
164
169
|
primary = value.split("-")[0]
|
|
165
170
|
if primary in iso3:
|
|
166
171
|
return iso3[primary]
|
|
172
|
+
if primary in names:
|
|
173
|
+
return names[primary]
|
|
167
174
|
return primary[:2] if len(primary) > 3 else primary
|
|
168
175
|
spoken = _code((raw or {}).get("language")) if isinstance(raw, dict) else ""
|
|
169
176
|
spoken = spoken or _code(transcription.get("language") or transcription.get("detected_language"))
|
|
@@ -249,6 +256,33 @@ def _words_from_transcript(data):
|
|
|
249
256
|
return out, had_segments, untimed
|
|
250
257
|
|
|
251
258
|
|
|
259
|
+
def silence_kinds(path: str, silences: "List[Tuple[float, float]]", duration: float,
|
|
260
|
+
threshold: float) -> "Dict[str, Any]":
|
|
261
|
+
"""Which detected silences are digital silence and which are quiet sound under the threshold
|
|
262
|
+
(2.20). Both are removed alike, but they are not the same finding: a room tone, a breath or a
|
|
263
|
+
whisper under --threshold is sound, and a report that called all of it "silence" hid that the
|
|
264
|
+
cut took audible material. One more silencedetect pass at DIGITAL_SILENCE_DB finds the digital
|
|
265
|
+
spans; a detected silence that is at least 99% digital counts as digital."""
|
|
266
|
+
if not silences or STATE.dry_run:
|
|
267
|
+
return {}
|
|
268
|
+
digital = [(s, duration if e == float("inf") else e) for s, e in detect(path, DIGITAL_SILENCE_DB, 0.02)]
|
|
269
|
+
labels, tally = [], {"digital": [0, 0.0], "below_threshold": [0, 0.0]}
|
|
270
|
+
for s, e in silences:
|
|
271
|
+
e = duration if e == float("inf") else e
|
|
272
|
+
span = max(e - s, 1e-9)
|
|
273
|
+
covered = sum(max(0.0, min(e, de) - max(s, ds)) for ds, de in digital)
|
|
274
|
+
kind = "digital" if covered / span >= 0.99 else "below_threshold"
|
|
275
|
+
labels.append(kind)
|
|
276
|
+
tally[kind][0] += 1
|
|
277
|
+
tally[kind][1] += e - s
|
|
278
|
+
return {"silence_kinds": labels,
|
|
279
|
+
"silence_breakdown": {"digital": {"count": tally["digital"][0], "seconds": round(tally["digital"][1], 3),
|
|
280
|
+
"level_db": DIGITAL_SILENCE_DB},
|
|
281
|
+
"below_threshold": {"count": tally["below_threshold"][0],
|
|
282
|
+
"seconds": round(tally["below_threshold"][1], 3),
|
|
283
|
+
"level_db": threshold}}}
|
|
284
|
+
|
|
285
|
+
|
|
252
286
|
def main() -> int:
|
|
253
287
|
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
|
254
288
|
ap.add_argument("input")
|
|
@@ -338,6 +372,7 @@ def main() -> int:
|
|
|
338
372
|
if args.filler:
|
|
339
373
|
silence_keeps = keep_ranges(merge_spans(list(silences)), duration, args.margin, args.min_keep)
|
|
340
374
|
silence_only = max(0.0, duration - sum(e - s for s, e in silence_keeps))
|
|
375
|
+
kinds = silence_kinds(args.input, silences, duration, args.threshold)
|
|
341
376
|
summary = {
|
|
342
377
|
"silences": [[round(s, 3), None if e == float("inf") else round(e, 3)] for s, e in silences],
|
|
343
378
|
"keep": [[round(s, 3), round(e, 3)] for s, e in keeps],
|
|
@@ -345,6 +380,8 @@ def main() -> int:
|
|
|
345
380
|
"kept_duration": round(kept, 3),
|
|
346
381
|
"removed_seconds": round(silence_only, 3),
|
|
347
382
|
}
|
|
383
|
+
if kinds:
|
|
384
|
+
summary.update(kinds)
|
|
348
385
|
if args.speech_aware:
|
|
349
386
|
summary["speech_aware"] = {
|
|
350
387
|
"min_silence": args.min_silence, "floor": min(BREATH_FLOOR, args.min_silence),
|