ffmpeg-skill 2.17.4 → 2.19.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -393,12 +393,12 @@ The short list for humans. The agent-facing version, with the reasoning, is the
393
393
  - **Phrase-aware caption breaking (1.16).** `caption.py`/`graphics.py --wrap phrase` (the default) never breaks inside a word or on the wrong side of a hyphen, never leaves a lone digit, kana or punctuation pair on a line, prefers Japanese sentence ends and particles over a mid-word break, and never ends a line on an article or preposition. All four are penalties over break positions that already fit, so no line is widened and the line count never changes; `--wrap measured` restores 1.15's width-only wrap. The text itself is never rewritten or shortened. Since 1.16.1 a Thai run and a katakana word are never broken inside (Thai writes no space inside a phrase and there is no dictionary: the break goes where you put a space or `|`), and a line wider than the safe width is reported as `overlong` with the fix named.
394
394
  - **Caption size fitted to the cue (1.17, reachable from the templates since 1.17.1).** At a platform caption size a line holds about six em, so an ordinary sentence needs four lines and `--max-lines 2` used to cut it into consecutive cues — half the sentence arriving late. `caption.py --fit-size` (default `auto`) walks the size down until every cue fits, *before* laying the cues out, with a legibility floor of 4.5 % of the frame height (`--min-size`, default 13 ASS units). An explicit `--size` or a `brand.json` size is a statement about the look and is never overridden — which in 1.17.0 also silenced the fitter on every `render.py --template` run, since a template fills the size from the platform table; 1.17.1 marks that size as the default it is (`"fit_size": "on"` in the filled project), so the type shrinks and no cue is split. `--fit-size off` restores that earlier behaviour byte for byte, and the caption text is still never rewritten to make it fit.
395
395
  - **Beat-synced cuts (1.17).** `scenes.py --beats` reports the measured grid — tempo, beat times, and a confidence built from how far the winning autocorrelation lag stands above the others and how many onsets land on it. `cut.py --snap beats` (and a `"snap"` block in a `render.py` project) moves in/out points to the nearest beat within `--snap-tolerance` — and only onto the grid points a measured onset actually marks, never onto the regular grid's continuation through a passage with no music in it. Below `--min-confidence` it **refuses**: a cut point may move to a measured beat and may not appear from one, so speech and ambience get an honest "no steady pulse here" instead of an invented grid.
396
- - **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Whisper stays optional: `--transcribe` with no engine installed refuses and names the five installs.
396
+ - **Filler words (1.17).** `silence.py --filler --words transcript.json` removes "um" and "uh" through the same cut graph the silences use. Never without measured word timings — there is no heuristic that finds an "um" without them that would not also cut real speech — and `like`, `tipo` and `cioè` are deliberately not in the built-in lists, because a discourse marker is a content word. Built-in lists since 2.19: English, Japanese, Korean, Chinese, Spanish, German, French, Portuguese, Italian, Dutch, Polish, Russian and Turkish; any other language takes `--filler-words FILE`. Whisper stays optional: `--transcribe` with no engine installed refuses and names the five installs.
397
397
  - **Throughput (1.17).** `batch.py --jobs N` runs several files at once, capped at `min(N, cpu_count, 8)` and sharing one `--timeout` budget rather than one per item; the per-item table keeps its order. `render.py --cache DIR` reuses stages whose inputs and arguments did not change, so swapping an export preset re-runs export only. The cache is opt-in with no default directory, and the ffmpeg, skill and contract versions are part of every key, so a cache is never reused across them.
398
398
  - **Audiogram (1.16).** `waveform.py --image cover.png` (or `render.py --template audiogram`) puts the waveform over a still plate for an episode that has no picture, with `--platform` for the frame, `--title` and burnt-in captions. The image is a local file you give: nothing is fetched and no cover art is ever invented.
399
399
  - **Emoji in captions and titles (1.15).** `caption.py`/`graphics.py --emoji-assets DIR` composites a PNG per emoji (Twemoji/Noto naming, `1f389.png`) on top of the text, because drawtext cannot load a colour emoji font at all and an installed one does not prove libass will draw it in colour — `doctor --json`'s `fonts.emoji` answers that from a render probe. Without assets the run still succeeds and reports `mode: mono`. Nothing is ever downloaded.
400
400
  - **Indic and Thai text shaped correctly in titles and lower-thirds (1.15).** `graphics.py` renders Devanagari, Bengali, Tamil, Thai and Lao through libass automatically (`text_renderer: "ass"`), because drawtext never reorders matras or re-clusters marks; Arabic and Hebrew were already correct on a fribidi build. `--text-render drawtext` with such a script is refused, never rendered wrongly.
401
- - **Non-Latin text picks a font by script (1.12).** Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari, Thai, Cyrillic and Greek cues, titles and overlays resolve a font file that covers them automatically, and a machine with no such font fails the job (`kind: input`) instead of rendering boxes. `doctor --json`'s `fonts.scripts` says which languages this machine can render; `--lang ja|ko` disambiguates Han-only text; an explicit `--font`/`--font-file` is always kept.
401
+ - **Non-Latin text picks a font by script (1.12).** Japanese, Chinese, Korean, Arabic, Hebrew, Devanagari, Thai, Cyrillic and Greek cues, titles and overlays resolve a font file that covers them automatically, and a machine with no such font fails the job (`kind: input`) instead of rendering boxes. `doctor --json`'s `fonts.scripts` says which languages this machine can render; `--lang ja|ko` disambiguates Han-only text, and `--lang zh-TW` (or `zh-HK`, `zh-Hant`) picks a Traditional Chinese face (2.19); an explicit `--font`/`--font-file` is always kept.
402
402
  - **Silence detection finds nothing?** The default threshold is −35 dBFS. The tool prints a hint with the track's measured level; raise the threshold (`silence.py --threshold -25`) or shorten `--min-silence`.
403
403
  - **Sync results carry a confidence.** Below 0.3, or an offset near the edge of the analysis window, is probably wrong: enlarge `--analyze-seconds` or find a clap. Recordings over ten minutes from separate devices need `sync.py --fix-drift`.
404
404
  - **Outputs are never overwritten silently.** An existing output path is refused (`kind: input`, `error.reason: output_exists`, before anything runs); pass `--overwrite` where a replacement is intended.
package/SKILL.md CHANGED
@@ -34,7 +34,7 @@ Ask a short question only when the answer changes the output materially and the
34
34
  - **Duration** ("make it 60 s") without a method: speed up for ≤1.5× changes, trim otherwise, say which. Ask when the content is a talk (trimming loses words) and the change is large.
35
35
  - **Captions** without a text source: `--transcribe` if a local engine exists, else ask for the text or a timed file; never invent dialogue.
36
36
  - **Fonts and brand**: if the user mentions a brand, colours or "our font", ask for or create `brand.json` once, reuse it.
37
- - **CJK / non-Latin text**: let the tool pick the font by script (`--font` turns it off); `--lang ja|ko` for Han-only text. `doctor --json` `.fonts.scripts` says what renders here. Tofu is a failed job.
37
+ - **CJK / non-Latin text**: let the tool pick the font by script (`--font` turns it off); `--lang ja|ko|zh-TW` for Han-only text. `doctor --json` `.fonts.scripts` says what renders here. Tofu is a failed job.
38
38
  - **Crop position** for `--fit crop`: centre by default, but when the request or the source names an off-centre subject ("keep the product on the right", someone visibly off-centre in the sheet) use `--crop-x`/`--crop-y` (0=left/top, 1=right/bottom) instead of guessing centre, or ask which edge to keep.
39
39
  - Anything else (transition type, caption style): pick the conventional default, say what you picked, offer the alternative in one line.
40
40
 
@@ -150,7 +150,6 @@ Timestamp flags (`--start`, `--end`, `--at`, `--from`, `--duration`, `--offset`,
150
150
  | "where does the subject move, so I can crop it myself" | `cropdetect.py input.mp4 --motion-centre` — motion centroid per second, report-only; the calling agent picks the crop |
151
151
  | "does it look like Log / S-Log?" | `probe.py clip.mp4 --analyze` (`looks_like_log`) then `color.py --lut` |
152
152
  | "test the tool on my real files" | `verify.py ~/Footage --report verify.md` |
153
- | "show me progress", "quick preview first" | any encoding script with `--progress` and/or `--fast` |
154
153
  | "it's a phone video with variable frame rate" | nothing extra: re-encodes conform VFR to constant fps; `fit.py --fps 30` picks the rate |
155
154
 
156
155
  ## Audio-only files
@@ -181,7 +180,7 @@ Look: final_sheet.png(字幕はセーフエリア内、ロゴは右上)
181
180
  Notes: 元は VFR だったので 30 fps に揃えた。音声はモノラルだったのでステレオにした
182
181
  ```
183
182
 
184
- Keep it to those five lines plus anything the user must decide. Never describe a fix you did not run.
183
+ Keep it to those five lines plus anything the user must decide. Never describe a fix you did not run. `verified` covers only what was measured: in `Notes:` name any `unverified` item the user would assume checked. Report a step with `attribution: unresolved`; do not re-run it.
185
184
 
186
185
  When a step fails, replace `Done:` with `Failed:` and keep the rest honest:
187
186
 
@@ -196,9 +195,7 @@ Notes: send a valid .cube, or say if you want the clip left as is
196
195
  Those filler lines are sentences, not labels: the same report for a Japanese request ends
197
196
  `Check: 検証するものなし` / `Look: 不要`.
198
197
 
199
- A refusal uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. A partial result is `Done:` with the shortfall in `Notes:`; a refusal that still delivers something is `Failed:` — never a third label like `Done (partially):`. Quote a failure's `error.hint` in `Notes:`.
200
-
201
- A failure's `--json` is `status: failed` plus `error.kind` and `error.message`; quote the message, never paraphrase it.
198
+ A refusal uses the same shape: `Failed:` names what was refused and why, `Steps:` lists what did run, `Look: not needed`. A partial result is `Done:` with the shortfall in `Notes:`; a refusal that still delivers something is `Failed:` — never a third label like `Done (partially):`. A failure's `--json` is `status: failed` plus `error.kind`, `error.message` and `error.hint`: quote the message in `Failed:`, never paraphrase it, and the hint in `Notes:`.
202
199
 
203
200
  ## Things that look right but are wrong
204
201
 
package/docs/contract.md CHANGED
@@ -21,7 +21,7 @@ The contract is derived from the code that runs, not maintained beside it:
21
21
  | Field | Meaning | Changes when |
22
22
  |---|---|---|
23
23
  | `contract_version` | shape of this document (`1.0`) | a key is renamed, removed or changes meaning |
24
- | `skill.version` | the npm / package.json version (`2.17.4`) | any release |
24
+ | `skill.version` | the npm / package.json version (`2.19.0`) | any release |
25
25
 
26
26
  A release that adds a tool or a flag keeps `contract_version`; a breaking change to the
27
27
  ToolSpec shape bumps it. Consumers pin on `contract_version` and read `skill.version`
@@ -131,7 +131,7 @@ JSON would have been rewritten for that. The flat keys, typed per tool in `outpu
131
131
  "deprecated": [{"what": "running on Python 3.9 (the requirements.python floor)", "since": "2.10.9", "replacement": "Python 3.10 or later; ...", "removed_in": "3.0.0", "where": "behaviour"},
132
132
  {"what": "running on FFmpeg 5.x (the requirements.ffmpeg floor)", "since": "2.14.0", "replacement": "FFmpeg 6.1 or later; ...", "removed_in": "3.0.0", "where": "behaviour"}],
133
133
  "removed": [{"what": "...", "since": "1.10.0", "replacement": "...", "removed_in": "2.0.0", "where": "cli | json | mcp | behaviour"}],
134
- "skill": {"id": "ffmpeg-skill", "version": "2.17.4", "execution_mode": "local", "kind": "execution",
134
+ "skill": {"id": "ffmpeg-skill", "version": "2.19.0", "execution_mode": "local", "kind": "execution",
135
135
  "entrypoints": {"cli": "...", "mcp": "...", "contract": "...", "doctor": "..."},
136
136
  "not_provided": ["AI reasoning", "decisions", "production plans", "project IR", "approvals", "network access", "transcription engine"]},
137
137
  "requirements": {"python": ">=3.9 (standard library only; 3.9 deprecated since 2.10.9, removed in 3.0.0; 3.10+ recommended)", "ffmpeg": ">=5.0 (5.x deprecated since 2.14.0, removed in 3.0.0; 6.1+ recommended)", "ffprobe": ">=5.0"},
@@ -169,6 +169,7 @@ One entry per tool under `tools`, sorted by id. Tool ids are stable:
169
169
  | `produces_artifact` | writes a file (media, PNG, HTML, EDL) |
170
170
  | `verification` | `{required, tools}`: which tools to run on the output afterwards |
171
171
  | `requires_visual_verification` | the picture changed; run `ffmpeg-skill/look` and inspect the PNG |
172
+ | `unverified_properties` | 2.18: the output properties this tool does not measure, the same list its results carry as `unverified` (empty when the tool names none) |
172
173
  | `reencodes_video`, `reencodes_audio` | `"always"` / `"never"` / `"conditional"`, meaning *when that stream is present in the input* — not whether the tool touches the file at all. `"conditional"` tools (`cut`, `export`, `render`, `batch`, `verify`, `caption`, `color`) carry a `reencode_note` explaining what it depends on — for `caption`, `--mode burn` (default) always re-encodes both streams, `--mode mux` copies both untouched; for `color`, `--strip-dovi` and `--retag` are a stream copy of both (retag only re-encodes if the copy attempt fails or ffprobe still reads the old tags on the copy), while `--to-sdr` / `--lut` / `--correct` always re-encode both. Several visual tools (`fit`, `overlay`, `graphics`, `join`, `multicam`, `silence`) are `"always"` on audio too: this codebase never mixes `-c:v` re-encode with `-c:a copy` in one call, so a caller cannot assume the original audio codec survives just because only the picture changed |
173
174
  | `audio_only` | accepts an audio-only input (WAV, MP3, M4A, FLAC, OGG, Opus) |
174
175
  | `video_required` | refuses an input without a video stream ("input has no video stream") |
@@ -685,6 +686,19 @@ Per-tool keys added after 2.2.2, all additive:
685
686
  | `vfr_check` | `cut.py` (a video input that could still stream-copy) | the sampled frame timing, `{heuristic: true, measured, method: "sampled", guard, windows, deltas}`; `measured` is `sampled_cfr`, `vfr` or `inconclusive`. `--vfr-guard average` (the default) keeps the 2.x guard: a source whose nominal and average frame rates differ (`variable_frame_rate_suspected`) re-encodes as `--accurate` (reason `vfr`, stderr says "variable-frame-rate"), and only then is the timing sampled and reported, with a `notes` line naming `--vfr-guard sampled` when it measured `sampled_cfr`; the key is absent otherwise. `--vfr-guard sampled` samples before every copy and re-encodes on `vfr` (reason `vfr`) or `inconclusive` (fewer than 20 intervals; `vfr_inconclusive`); `--vfr-guard off` samples and keeps the copy, with a `notes` line. Absent for audio-only input or output, `--accurate`, `--codec` and a pending `--dry-run` input |
686
687
  | `join_check`, `segment_end_snap_seconds` | `cut.py --segments` | `join_check`: a video stream-copy join measured by demuxing, `{packets, expected_packets, max_step_seconds, audio_offset_ms, audio_parts_checked, ok}`. `expected_packets` counts the source's video packets from each part's start keyframe to its end (or the end of the video); `max_step_seconds` is the largest presentation step, `null` when `vfr_check` measured anything but `sampled_cfr` (a `--vfr-guard off` join of variable timing: count only), and each step at a join must be the step its parts predict within 1 ms. `audio_offset_ms`: per part, its sound's shift minus its picture's in ms (+ = sound late), from audio packets matched by CRC32 to the source stream shaped like the joined audio (FFmpeg's default selection may copy one other than `a:0`; an MPEG-TS source's AAC is hashed without its ADTS headers); `null` for a part with no unique match (digital silence, a periodic tone), and as a whole when the source has no audio or the video check failed. `audio_parts_checked` counts the measured parts. `ok` is the conjunction (count, steps, every measured part within 5 ms); `ok: false` means the join was re-cut (`concat_fallback`) and this is the failed measurement. `null` for a single segment, audio, `--dry-run`, any re-encode forced for every segment, a segment that re-encoded, or a join re-cut before a copy was tried (open GOPs, a later B-frame segment starting between keyframes, parts that differ, a concat error). `segment_end_snap_seconds`: per segment, the signed seconds its end moved to the keyframe the copied part stops at, the first at or after the requested end (`null` for a part that reaches the end of the video, a part of a source with no B-frames, cut to its requested end as in 2.5.1, or a Matroska/MPEG-TS part, which is not snapped); `null` as a whole unless the output is those copied parts |
687
688
 
689
+ Keys added in 2.18, all additive. They make a result carry what the VDC paper
690
+ (Verification-Driven Coding) calls the co-reported items: what was measured against what, how far,
691
+ from where the expected value came, and what was not measured.
692
+
693
+ | key | tool | what it holds |
694
+ |---|---|---|
695
+ | `verification[].property`, `expected`, `measured`, `delta`, `tolerance`, `provenance` | steps that measure a property (join's `duration`, loudness's `loudness`, export's `loudness`, every `streams` step) | the property measured, its expected and measured values, their difference, the tolerance applied, and where the expected value came from: `specification` (the platform table), `user_specified` (a flag), `independent_measurement`, or `input_premeasurement` (the input's own probe) |
696
+ | `verification[]` step `streams` | `cut.py`, `join.py`, `export.py` | the output's video/audio streams against the ones the input had and the operation was not told to drop (an audio extension drops the picture, a GIF the sound). A missing stream is `ok: false` with `missing`, so the result is `verified: false` (#367: a TS cut that lost its video) |
697
+ | `verification[].attribution` | any step with `ok: false` | `unresolved` until a tool knows whether the artifact, the reference, the measuring instrument or the expected value's provenance is at fault. A caller does not re-run on an unresolved failure: a re-run can hide which one it was |
698
+ | `unverified` | every writing tool `UNVERIFIED` names (`_common/emit.py`) | the output's properties this tool did not measure, e.g. `caption`'s `rendered_placement`, `sync`'s `lip_sync`. `verified` is about the measured properties only; a report that passes the verdict on must pass this list with it. The contract carries the same list per tool as `unverified_properties` |
699
+ | `audio_substituted` | `join.py` (video join) | the clips that had no audio stream, whose span of the join is generated silence; `notes` says the same |
700
+ | loudness `tolerance.tp` | `loudness.py`, `export.py`, and `check.py`'s true-peak row | the one dBTP margin every tool applies (`_platforms.TP_TOLERANCE_DB`, 0.1 dB); before 2.18 it was 0.1, 0.05 and 0.0 in the three tools |
701
+
688
702
  Per-tool keys added after 2.10.11 ("nothing happens silently"), all additive:
689
703
 
690
704
  | key | tool | what it holds |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ffmpeg-skill",
3
- "version": "2.17.4",
3
+ "version": "2.19.0",
4
4
  "description": "Agent Skill that gives coding agents (Claude Code, Cursor, Codex) a local video editor: 42 FFmpeg tools with a machine-readable contract, contract-derived MCP server, FFmpeg capability detection, probe-first / verify-last workflow. Cut, join, silence removal, fit, captions and karaoke, overlays, motion graphics, HDR to SDR, LUTs, audio clean-up and typed dynamics, sync with drift correction, multicam, loudness, delivery checks, project rendering, batch. No API keys, no cloud, no dependencies.",
5
5
  "keywords": [
6
6
  "ffmpeg",
@@ -578,7 +578,7 @@ each shorter clip's last frame (with silence) out to the longest.
578
578
  ### silence.py — remove dead air / jump cuts
579
579
  ```
580
580
  silence.py INPUT [--threshold -35] [--min-silence 0.6] [--margin 0.15] [--min-keep 0.2] [--list] [--edl keep.txt] [--speech-aware] [-o OUT]
581
- silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|es|de|fr|pt|it]
581
+ silence.py INPUT --filler --words transcript.json [--filler-lang auto|en|ja|ko|zh|es|de|fr|pt|it|nl|pl|ru|tr]
582
582
  [--filler-words FILE] [--filler-extra W,W] [--filler-keep W,W] [--filler-pad 0.02]
583
583
  [--transcribe] [--engine auto|parakeet-mlx|parakeet.cpp|whisper.cpp|faster-whisper|openai-whisper]
584
584
  [--model MODEL] [--filler-list] [--max-cuts 400]
@@ -598,6 +598,11 @@ per-word `start`/`end`) or `--transcribe` (the same local engine `caption.py`
598
598
  uses, never required). There is no heuristic fallback — finding an "um" by
599
599
  looking for short quiet blips would cut real speech, so the tool refuses
600
600
  instead. `--filler-list` reports what would go and writes nothing.
601
+ Built-in lists (2.19): en, ja, ko, zh, es, de, fr, pt, it, nl, pl, ru, tr. They hold
602
+ hesitation sounds only; words that are as often content (`like`, 저, 那个, ну) are
603
+ `--filler-extra` material. Chinese and Japanese are cut only where the transcript shows
604
+ a word boundary on each side (punctuation or a 0.1 s pause), because whisper splits them
605
+ into tokens rather than words.
601
606
  `filler.matched_untimed` counts listed words the transcript gives no usable
602
607
  timing for (not removed, and a `filler_words_untimed` warning), and
603
608
  `matched_too_long` those longer than a filler (a held vowel, kept).
@@ -608,11 +613,17 @@ Built-in lists, one per language:
608
613
  |---|---|
609
614
  | en | um, uh, erm, hmm, mm, mhm, er, ah |
610
615
  | ja | えー, えーと, えっと, あの, あのー, その, そのー, まあ, なんか |
616
+ | ko | 어, 어어, 에, 음, 음음, 으음, 흠 |
617
+ | zh | 呃, 呃呃, 嗯, 嗯嗯, 唔 |
611
618
  | es | eh, este, esto, mmm |
612
619
  | de | äh, ähm, hm |
613
620
  | fr | euh, hein |
614
621
  | pt | é, hum |
615
622
  | it | ehm |
623
+ | nl | eh, ehm, uh, uhm, hm |
624
+ | pl | yy, yyy, ee, eee, em, emm, hmm |
625
+ | ru | э, ээ, эм, эмм, мм, хм |
626
+ | tr | ıı, ııı, ee, eee, hmm |
616
627
 
617
628
  **Discourse markers are not disfluencies.** `like`, `tipo` and `cioè` are
618
629
  deliberately *not* in the default lists: they are grammatical words in most
@@ -12,10 +12,11 @@ from __future__ import annotations
12
12
  import json
13
13
  import os
14
14
  import re
15
+ import unicodedata
15
16
  from pathlib import Path
16
17
  from typing import Any, Dict, List, NamedTuple, Optional, Tuple
17
18
 
18
- from _common.decision import fmt_srt_time, parse_time
19
+ from _common.decision import _unspaced_script, fmt_srt_time, parse_time
19
20
  from _common.emit import die, info, warn
20
21
  from _common.runner import read_text_or_die
21
22
 
@@ -502,7 +503,11 @@ def _words_from_whisper_cpp_json(path: str) -> "List[Dict[str, Any]]":
502
503
 
503
504
  Token text carries leading spaces and the model's special tokens ([_BEG_], [_TT_123]); those
504
505
  are dropped, and a token that is a word continuation (no leading space) is glued onto the
505
- previous word so "un" + "believable" is one word with one span, not two.
506
+ previous word so "un" + "believable" is one word with one span, not two. Chinese, Japanese
507
+ and Thai have no spaces, so there every token would continue the one before and a whole
508
+ segment became one "word": between two whole characters of such a script the tokens stay
509
+ apart, as openai-whisper and faster-whisper split them (filler_spans() finds the word
510
+ boundaries). A token that is part of a character (U+FFFD) is still glued.
506
511
  """
507
512
  try:
508
513
  doc = json.loads(Path(path).read_text(encoding="utf-8"))
@@ -519,7 +524,14 @@ def _words_from_whisper_cpp_json(path: str) -> "List[Dict[str, Any]]":
519
524
  start, end = float(offsets["from"]) / 1000.0, float(offsets["to"]) / 1000.0
520
525
  except (KeyError, TypeError, ValueError):
521
526
  continue
522
- if out and not text.startswith(" "):
527
+ prev = out[-1]["word"] if out else ""
528
+ # "嗯" + "," + "呃": the comma stays with the word before it and ends that word
529
+ core = prev
530
+ while core and unicodedata.category(core[-1]).startswith("P"):
531
+ core = core[:-1]
532
+ unspaced = (core and _unspaced_script(core[-1]) and _unspaced_script(text.lstrip()[:1] or " ")
533
+ and "\ufffd" not in prev[-1:] + text)
534
+ if out and not text.startswith(" ") and not unspaced:
523
535
  out[-1]["word"] += text
524
536
  out[-1]["end"] = end
525
537
  else:
@@ -1158,6 +1158,18 @@ FILLER_WORDS = {
1158
1158
  "fr": frozenset({"euh", "hein"}),
1159
1159
  "pt": frozenset({"é", "hum"}),
1160
1160
  "it": frozenset({"ehm"}),
1161
+ # 2.19: Korean is written with spaces, so a whisper word is a word and the list works as the
1162
+ # English one does. 그/저/뭐/이제 are fillers as often as they are "that"/"I"/"what"/"now":
1163
+ # discourse markers below, not defaults.
1164
+ "ko": frozenset({"음", "으음", "음음", "어", "어어", "에", "흠"}),
1165
+ # Chinese has no spaces: these match only as a whole token with a boundary on each side
1166
+ # (filler_spans), so the 额 of 金额 or the 呃 inside a word is never cut. 额 itself is left
1167
+ # out: it is a common character in its own right.
1168
+ "zh": frozenset({"嗯", "嗯嗯", "呃", "呃呃", "唔"}),
1169
+ "ru": frozenset({"э", "ээ", "эм", "эмм", "мм", "хм"}),
1170
+ "nl": frozenset({"eh", "ehm", "uh", "uhm", "hm"}),
1171
+ "pl": frozenset({"yyy", "yy", "eee", "ee", "em", "emm", "hmm"}),
1172
+ "tr": frozenset({"ıı", "ııı", "ee", "eee", "hmm"}),
1161
1173
  }
1162
1174
 
1163
1175
  # Words in a default list that are also ordinary vocabulary: every run that removes one says so.
@@ -1173,11 +1185,33 @@ FILLER_DISCOURSE_MARKERS = {
1173
1185
  "en": ("like", "you know"),
1174
1186
  "pt": ("tipo",),
1175
1187
  "it": ("cioè",),
1188
+ "ko": ("그", "저", "뭐", "이제", "약간", "막", "그니까"),
1189
+ "zh": ("那个", "这个", "就是", "然后"),
1190
+ "ru": ("ну", "типа", "короче", "как бы"),
1191
+ "nl": ("nou", "dus", "zeg maar"),
1192
+ "pl": ("no", "jakby", "wiesz", "znaczy"),
1193
+ "tr": ("şey", "yani", "işte"),
1176
1194
  }
1177
1195
 
1178
1196
  FILLER_MAX_WORD = 1.2 # a longer "uhhh" is a held vowel someone meant
1179
1197
  FILLER_MIN_GAP = 0.05 # spans closer than this become one span
1180
1198
  FILLER_PAD = 0.02 # trimmed either side of the word
1199
+ FILLER_MAX_TOKENS = 4 # a listed word may span this many engine tokens ("えー" + "と", "you" + "know")
1200
+ FILLER_BOUNDARY_GAP = 0.1 # a pause this long between two tokens is a word boundary
1201
+
1202
+
1203
+ def _unspaced_script(text: str) -> bool:
1204
+ """True when `text` has a character of a script written without spaces between words (Han,
1205
+ kana, Thai, Lao, Myanmar, Khmer). Whisper splits those languages into tokens, not words, so a
1206
+ token boundary there says nothing about a word boundary. Hangul is spaced and not listed."""
1207
+ for ch in text:
1208
+ o = ord(ch)
1209
+ if (0x3040 <= o <= 0x30FF or 0x31F0 <= o <= 0x31FF or 0xFF66 <= o <= 0xFF9F # kana
1210
+ or 0x3400 <= o <= 0x4DBF or 0x4E00 <= o <= 0x9FFF or 0xF900 <= o <= 0xFAFF # Han
1211
+ or 0x20000 <= o <= 0x323AF # Han extensions B-H
1212
+ or 0x0E00 <= o <= 0x0EFF or 0x1000 <= o <= 0x109F or 0x1780 <= o <= 0x17FF): # Thai, Lao, Myanmar, Khmer
1213
+ return True
1214
+ return False
1181
1215
 
1182
1216
 
1183
1217
  def normalise_filler_token(word: str) -> str:
@@ -1195,37 +1229,89 @@ def filler_spans(words, wordlist, *, pad: float = FILLER_PAD, min_gap: float = F
1195
1229
  max_word: float = FILLER_MAX_WORD, stats: "Optional[Dict[str, int]]" = None) -> "List[Dict[str, Any]]":
1196
1230
  """Time spans to remove, from measured word timings. Pure: no subprocess, no I/O.
1197
1231
 
1198
- `words` is [{"word", "start", "end"}] as whisper emits. A word is removed only when its
1199
- normalised form is in `wordlist` AND it carries a real start < end pair AND its length is
1200
- <= max_word. Adjacent spans closer than min_gap merge. Returns [{"start", "end", "word"}]
1201
- sorted and non-overlapping.
1232
+ `words` is [{"word", "start", "end"}] as whisper emits. A run of up to FILLER_MAX_TOKENS
1233
+ consecutive entries is removed only when the run, joined with or without spaces and
1234
+ normalised, is in `wordlist` AND every entry carries a real start < end pair AND the run is
1235
+ <= max_word long. The longest run wins. Adjacent spans closer than min_gap merge. Returns
1236
+ [{"start", "end", "word"}] sorted and non-overlapping.
1237
+
1238
+ Whisper splits Chinese, Japanese and Thai into tokens rather than words ("えーと" arrives as
1239
+ "えー" + "と", "金额" as "金" + "额"), so a run in such a script also needs a word boundary on
1240
+ each side: the start or end of the transcript, punctuation, a spaced-script neighbour, or a
1241
+ pause of FILLER_BOUNDARY_GAP. Without one the run is part of a longer word and is kept.
1202
1242
 
1203
1243
  `stats`, when given, receives the listed words that were kept anyway, so the caller can say
1204
1244
  so: `matched_untimed` (no usable start < end: nothing to cut on) and `matched_too_long`
1205
1245
  (longer than max_word: a held vowel someone meant).
1206
1246
  """
1247
+ import unicodedata as _ud
1207
1248
  listed = {normalise_filler_token(w) for w in (wordlist or set())}
1208
1249
  listed.discard("")
1209
- hits = []
1210
- untimed = too_long = 0
1250
+ toks = []
1211
1251
  for entry in words or []:
1212
1252
  if not isinstance(entry, dict):
1213
1253
  continue
1214
- token = normalise_filler_token(entry.get("word") or entry.get("text") or "")
1215
- if not token or token not in listed:
1216
- continue
1254
+ raw = str(entry.get("word") or entry.get("text") or "").strip()
1217
1255
  try:
1218
1256
  start, end = float(entry["start"]), float(entry["end"])
1257
+ timed = end > start
1219
1258
  except (KeyError, TypeError, ValueError):
1220
- untimed += 1
1221
- continue # no measured timing: nothing to cut
1222
- if not (end > start):
1223
- untimed += 1
1224
- continue
1225
- if (end - start) > max_word:
1226
- too_long += 1
1227
- continue
1228
- hits.append({"start": max(0.0, start - pad), "end": end + pad, "word": token})
1259
+ start = end = 0.0
1260
+ timed = False
1261
+ toks.append({"norm": normalise_filler_token(raw), "start": start, "end": end, "timed": timed,
1262
+ "punct_before": bool(raw) and _ud.category(raw[0]).startswith("P"),
1263
+ "punct_after": bool(raw) and _ud.category(raw[-1]).startswith("P")})
1264
+
1265
+ def _bounded(i: int, j: int) -> bool:
1266
+ """A word boundary before toks[i] and after toks[j - 1]."""
1267
+ left = (i == 0 or toks[i]["punct_before"] or toks[i - 1]["punct_after"] or not toks[i - 1]["norm"]
1268
+ or not _unspaced_script(toks[i - 1]["norm"])
1269
+ or (toks[i - 1]["timed"] and toks[i]["start"] - toks[i - 1]["end"] >= FILLER_BOUNDARY_GAP - 1e-6))
1270
+ right = (j == len(toks) or toks[j - 1]["punct_after"] or toks[j]["punct_before"] or not toks[j]["norm"]
1271
+ or not _unspaced_script(toks[j]["norm"])
1272
+ or (toks[j]["timed"] and toks[j]["start"] - toks[j - 1]["end"] >= FILLER_BOUNDARY_GAP - 1e-6))
1273
+ return left and right
1274
+
1275
+ hits = []
1276
+ untimed = too_long = 0
1277
+ i = 0
1278
+ while i < len(toks):
1279
+ found = 0
1280
+ rejected = None
1281
+ for k in range(min(FILLER_MAX_TOKENS, len(toks) - i), 0, -1):
1282
+ run = toks[i:i + k]
1283
+ norms = [t["norm"] for t in run]
1284
+ if not all(norms):
1285
+ continue
1286
+ unspaced = any(_unspaced_script(n) for n in norms)
1287
+ joined = ("" if unspaced else " ").join(norms)
1288
+ if joined not in listed or (unspaced and not _bounded(i, i + k)):
1289
+ continue
1290
+ # punctuation inside the run ("えー、" + "と") ends the word: the shorter run decides
1291
+ if any(run[m]["punct_after"] or run[m + 1]["punct_before"] for m in range(k - 1)):
1292
+ continue
1293
+ # so does a pause inside an unspaced run: the tokens were two words
1294
+ if unspaced and any(run[m]["timed"] and run[m + 1]["timed"]
1295
+ and run[m + 1]["start"] - run[m]["end"] >= FILLER_BOUNDARY_GAP - 1e-6
1296
+ for m in range(k - 1)):
1297
+ continue
1298
+ # a run listed out of time order has no span to cut on
1299
+ if not all(t["timed"] for t in run) or run[-1]["end"] <= run[0]["start"]:
1300
+ rejected = rejected or ("untimed", k)
1301
+ continue
1302
+ if run[-1]["end"] - run[0]["start"] > max_word:
1303
+ rejected = rejected or ("too_long", k)
1304
+ continue
1305
+ hits.append({"start": max(0.0, run[0]["start"] - pad), "end": run[-1]["end"] + pad, "word": joined})
1306
+ found = k
1307
+ break
1308
+ if not found and rejected:
1309
+ # the longest listed run that could not be cut is the one reported. Its prefixes were
1310
+ # tried here; moving on by one token lets a listed word later in it be tried too
1311
+ untimed += rejected[0] == "untimed"
1312
+ too_long += rejected[0] == "too_long"
1313
+ found = 1
1314
+ i += found or 1
1229
1315
  hits.sort(key=lambda h: (h["start"], h["end"]))
1230
1316
  merged: "List[Dict[str, Any]]" = []
1231
1317
  for h in hits:
@@ -372,6 +372,45 @@ def steps_encoder_report(steps: Sequence["Tuple[str, Any]"]) -> Dict[str, Any]:
372
372
  return out
373
373
 
374
374
 
375
+ # The properties each writing tool does NOT measure about its output, by tool name: the unverified
376
+ # range U that every --json result reports (docs/contract.md "unverified"). A property listed here
377
+ # is one a caller might assume was checked; naming it is the difference between "verified" meaning
378
+ # "the listed properties were measured" and "everything is fine". Kept beside emit() so a tool
379
+ # cannot report verified without this list travelling with it.
380
+ UNVERIFIED: Dict[str, List[str]] = {
381
+ "cut": ["picture_content"],
382
+ "join": ["picture_content", "transition_look"],
383
+ "fit": ["subject_framing"],
384
+ "crop": ["subject_framing"],
385
+ "caption": ["rendered_placement", "legibility"],
386
+ "overlay": ["rendered_placement"],
387
+ "graphics": ["rendered_placement"],
388
+ "color": ["grade_look"],
389
+ "export": ["picture_content"],
390
+ "render": ["picture_content", "rendered_placement"],
391
+ "silence": ["removed_span_content"],
392
+ "audio": ["mix_balance"],
393
+ "loudness": ["perceived_loudness"],
394
+ "sync": ["lip_sync"],
395
+ "multicam": ["lip_sync", "shot_choice"],
396
+ }
397
+
398
+
399
+ def stream_inventory_step(meta: Dict[str, Any], expect: Dict[str, bool]) -> Dict[str, Any]:
400
+ """The output's stream composition checked against what the operation should have produced
401
+ (#367: a cut of an MPEG-TS source lost its whole video stream and still said verified: true,
402
+ because the only properties measured were duration and start skew). `expect` comes from the
403
+ input's own probe and the tool's arguments (provenance "input_premeasurement"): a stream the
404
+ input had and the operation was not told to drop must still be there."""
405
+ measured = {kind: bool(meta.get(kind)) for kind in expect}
406
+ missing = [k for k, want in expect.items() if want and not measured[k]]
407
+ step: Dict[str, Any] = {"step": "streams", "property": "stream_inventory", "expected": dict(expect),
408
+ "measured": measured, "provenance": "input_premeasurement", "ok": not missing}
409
+ if missing:
410
+ step["missing"] = missing
411
+ return step
412
+
413
+
375
414
  def emit(output: Optional[str], *, ctx: "Optional[Context]" = None, **extra: Any) -> None:
376
415
  """Final stdout line: the output path, or a JSON document with --json.
377
416
 
@@ -392,13 +431,22 @@ def emit(output: Optional[str], *, ctx: "Optional[Context]" = None, **extra: Any
392
431
  # only when the file was written, probed, and every self-check met its target; a dry run
393
432
  # verified nothing. Spec failures the tool cannot fix on its own (export's loudness gap)
394
433
  # keep status completed and say verified: false, so a caller keys on one field.
434
+ expect_streams = extra.pop("expect_streams", None)
395
435
  steps: List[Dict[str, Any]] = ([{"step": "probe", "ok": True}] if meta else []) + list(extra.pop("verification", None) or [])
436
+ if meta and expect_streams:
437
+ steps.append(stream_inventory_step(meta, expect_streams))
396
438
  # a tool that ran others (render.py) verified what its stages verified too (absorb_stage())
397
439
  steps += [s for s in ctx.stage_verification if s not in steps]
398
440
  if output and not ctx.dry_run and os.path.splitext(output)[1].lower() not in MEDIA_EXT:
399
441
  steps.insert(0, {"step": "exists", "ok": True})
400
442
  doc["verified"] = not ctx.dry_run and bool(steps) and all(s.get("ok") for s in steps)
443
+ # a failed check asserts that something in artifact, reference, instrument or the expected
444
+ # value's provenance is wrong; until a tool knows which, it says it does not know
445
+ steps = [dict(s, attribution=s.get("attribution") or "unresolved") if s.get("ok") is False else s for s in steps]
401
446
  doc["verification"] = steps
447
+ tool = os.path.splitext(os.path.basename(sys.argv[0] or ""))[0]
448
+ if output and not ctx.dry_run and tool in UNVERIFIED and "unverified" not in extra:
449
+ doc["unverified"] = list(UNVERIFIED[tool])
402
450
  doc.update(_encoder_report(ctx))
403
451
  doc.update(extra)
404
452
  hw_notes = [n for n in _hw_notes_for(ctx) if n not in (doc.get("notes") or [])]
@@ -147,13 +147,14 @@ LANGUAGE_NAMES = {
147
147
  "ja": "Japanese", "zh": "Chinese", "ko": "Korean", "ar": "Arabic", "he": "Hebrew",
148
148
  "hi": "Devanagari (Hindi/Marathi/Nepali)", "th": "Thai", "ru": "Cyrillic (Russian and others)",
149
149
  "el": "Greek", "latin": "Latin", "bn": "Bengali", "ta": "Tamil", "lo": "Lao",
150
+ "zh-hant": "Traditional Chinese",
150
151
  }
151
152
 
152
153
 
153
154
  # fontconfig's own :lang= codes for each script we detect (zh uses zh-cn, the Simplified subset
154
155
  # every CJK font that claims zh carries; the rest are the plain two-letter codes).
155
156
  FC_LANG = {"ja": "ja", "zh": "zh-cn", "ko": "ko", "ar": "ar", "he": "he", "hi": "hi", "th": "th", "ru": "ru", "el": "el",
156
- "bn": "bn", "ta": "ta", "lo": "lo"}
157
+ "bn": "bn", "ta": "ta", "lo": "lo", "zh-hant": "zh-tw"}
157
158
 
158
159
 
159
160
  # Families tried in order, best first. The names are matched case-insensitively against the start
@@ -162,6 +163,8 @@ FC_LANG = {"ja": "ja", "zh": "zh-cn", "ko": "ko", "ar": "ar", "he": "he", "hi":
162
163
  PREFERRED_FAMILIES = {
163
164
  "ja": ["Noto Sans CJK JP", "Noto Serif CJK JP", "Noto Sans JP", "Source Han Sans", "IPAPGothic", "IPAGothic", "IPA", "VL Gothic", "TakaoGothic", "WenQuanYi Zen Hei"],
164
165
  "zh": ["Noto Sans CJK SC", "Noto Serif CJK SC", "Noto Sans SC", "Source Han Sans", "WenQuanYi Zen Hei", "WenQuanYi Micro Hei", "Droid Sans Fallback"],
166
+ "zh-hant": ["Noto Sans CJK TC", "Noto Serif CJK TC", "Noto Sans CJK HK", "Noto Sans TC", "Noto Sans HK", "Source Han Sans TW",
167
+ "Source Han Sans HC", "AR PL UMing TW", "AR PL UKai TW", "PingFang TC", "WenQuanYi Zen Hei", "Droid Sans Fallback"],
165
168
  "ko": ["Noto Sans CJK KR", "Noto Serif CJK KR", "Noto Sans KR", "Source Han Sans K", "NanumGothic", "Nanum Gothic", "Malgun Gothic", "WenQuanYi Zen Hei"],
166
169
  "ar": ["Noto Sans Arabic", "Noto Naskh Arabic", "Amiri", "Scheherazade", "DejaVu Sans", "FreeSans", "FreeSerif"],
167
170
  "he": ["Noto Sans Hebrew", "Noto Serif Hebrew", "DejaVu Sans", "FreeSans", "FreeSerif"],
@@ -179,6 +182,7 @@ PREFERRED_FAMILIES = {
179
182
  WINDOWS_FONTS = {
180
183
  "ko": [("malgun.ttf", "Malgun Gothic"), ("gulim.ttc", "Gulim"), ("batang.ttc", "Batang")],
181
184
  "zh": [("msyh.ttc", "Microsoft YaHei"), ("simhei.ttf", "SimHei"), ("simsun.ttc", "SimSun")],
185
+ "zh-hant": [("msjh.ttc", "Microsoft JhengHei"), ("mingliu.ttc", "MingLiU")],
182
186
  "ja": [("meiryo.ttc", "Meiryo"), ("YuGothM.ttc", "Yu Gothic Medium"), ("YuGothR.ttc", "Yu Gothic"), ("msgothic.ttc", "MS Gothic")],
183
187
  "ar": [("tahoma.ttf", "Tahoma"), ("arial.ttf", "Arial")],
184
188
  "he": [("tahoma.ttf", "Tahoma"), ("arial.ttf", "Arial")],
@@ -277,6 +281,22 @@ def detect_script(text: str, lang: "Optional[str]" = None) -> str:
277
281
  return best
278
282
 
279
283
 
284
+ # 2.19: Han-only text is Chinese, and Chinese is written in two standard forms whose glyphs differ
285
+ # (骨, 直, 角 are drawn differently in Taipei and Beijing). A Simplified face draws a Taiwanese
286
+ # caption in the wrong form, so a --lang naming Traditional Chinese resolves a Traditional face.
287
+ # The script stays "zh" for wrapping; only the font lookup changes.
288
+ HANT_TAGS = ("zh-tw", "zh-hk", "zh-mo", "zh-hant", "yue")
289
+
290
+
291
+ def font_key(script: str, lang: "Optional[str]" = None) -> str:
292
+ """The font-table key for `script` under the caller's `lang`: "zh-hant" for Chinese text the
293
+ caller says is Traditional (zh-TW, zh-HK, zh-MO, zh-Hant-*, yue), else `script` itself."""
294
+ tag = (lang or "").strip().lower().replace("_", "-")
295
+ if script == "zh" and any(tag == t or tag.startswith(t + "-") for t in HANT_TAGS):
296
+ return "zh-hant"
297
+ return script
298
+
299
+
280
300
  _SCRIPT_FONT_CACHE: "Dict[Tuple[str, Optional[str]], Optional[Tuple[str, str]]]" = {}
281
301
 
282
302
 
@@ -359,7 +379,11 @@ def _script_font_uncached(script: str, family_hint: "Optional[str]" = None):
359
379
  mono = 1 if "mono" in joined else 0
360
380
  # regular weights before Bold/Italic/Oblique cuts, so a default caption is not bold by accident
361
381
  styled = 1 if any(k in stem for k in ("bold", "italic", "oblique", "light", "thin", "black")) else 0
362
- scored.append((last_resort, _family_rank(families, preferred), mono, styled, path, families[0]))
382
+ rank = _family_rank(families, preferred)
383
+ # a .ttc lists every face it holds (Noto CJK: JP, KR, SC, TC): name the one that ranked
384
+ family = next((f for f in families if rank < len(preferred) and f.lower().startswith(preferred[rank].lower())),
385
+ families[0])
386
+ scored.append((last_resort, rank, mono, styled, path, family))
363
387
  scored.sort(key=lambda row: (row[0], row[1], row[2], row[3], row[4]))
364
388
  best = scored[0]
365
389
  return best[4], best[5]
@@ -453,6 +477,11 @@ def fonts_dir_covers_script(fonts_dir: str, script: str) -> "Optional[bool]":
453
477
  return False
454
478
 
455
479
 
480
+ def _is_last_resort(entry: "Tuple[str, str]") -> bool:
481
+ """Unifont, fontconfig's code-point-box face: it covers every script and renders none."""
482
+ return "unifont" in " ".join(str(x) for x in entry).lower()
483
+
484
+
456
485
  def script_font_for_text(text: str, *, lang: "Optional[str]" = None, font: "Optional[str]" = None,
457
486
  font_explicit: bool = False, font_file: "Optional[str]" = None,
458
487
  fonts_dir: "Optional[str]" = None
@@ -476,30 +505,41 @@ def script_font_for_text(text: str, *, lang: "Optional[str]" = None, font: "Opti
476
505
  return script, None, None
477
506
  if font_file:
478
507
  return script, None, None
508
+ key = font_key(script, lang)
479
509
  if font_explicit and font:
480
- if not font_covers_script(font, script):
481
- warn(f"font: '{font}' does not cover {LANGUAGE_NAMES[script]} text on this machine; keeping it as asked "
510
+ if not font_covers_script(font, key):
511
+ warn(f"font: '{font}' does not cover {LANGUAGE_NAMES[key]} text on this machine; keeping it as asked "
482
512
  f"(drop --font, or {FONT_FLAG_HINT}, to pick one by script automatically)", "font_lacks_script",
483
513
  font=font, script=script)
484
514
  return script, None, None
485
515
  if fonts_dir:
486
- covered = fonts_dir_covers_script(fonts_dir, script)
516
+ covered = fonts_dir_covers_script(fonts_dir, key)
487
517
  if covered:
488
518
  return script, None, None
489
519
  if covered is None:
490
- warn(f"font: could not verify that {fonts_dir} covers {LANGUAGE_NAMES[script]} text "
520
+ warn(f"font: could not verify that {fonts_dir} covers {LANGUAGE_NAMES[key]} text "
491
521
  "(no fc-scan on this machine); using it as given", "font_coverage_unverified", fonts_dir=fonts_dir, script=script)
492
522
  return script, None, None
493
- record_fallback(f"font: no face in {fonts_dir} covers {LANGUAGE_NAMES[script]} text; "
523
+ record_fallback(f"font: no face in {fonts_dir} covers {LANGUAGE_NAMES[key]} text; "
494
524
  "picking one by script instead (the directory is still searched first)", "fonts_dir_lacks_script",
495
525
  fonts_dir=fonts_dir, script=script)
496
- entry = _script_font_entry(script)
526
+ entry = _script_font_entry(key)
527
+ if key != script and (not entry or (entry is not FC_UNKNOWN and _is_last_resort(entry))):
528
+ # no Traditional face (Unifont's code-point boxes are none): the Simplified one still
529
+ # has the characters, in the other form
530
+ simplified = _script_font_entry(script)
531
+ if simplified and simplified is not FC_UNKNOWN and (not entry or not _is_last_resort(simplified)):
532
+ entry = simplified
533
+ record_fallback(f"font: no {LANGUAGE_NAMES[key]} face is installed; drawing with {entry[0]}, a "
534
+ "Simplified Chinese face whose glyph forms differ in places "
535
+ "(install fonts-noto-cjk or a TC font)", "font_variant_missing", lang=lang, file=entry[0])
536
+ key = script
497
537
  if entry is FC_UNKNOWN:
498
- warn(f"font: could not verify that this machine can render {LANGUAGE_NAMES[script]} text "
538
+ warn(f"font: could not verify that this machine can render {LANGUAGE_NAMES[key]} text "
499
539
  "(no working fontconfig); rendering with the font as given -- "
500
540
  "doctor --json .fonts.scripts reports what is known", "font_coverage_unverified", script=script)
501
541
  return script, None, None
502
542
  if not entry:
503
- die(f"no installed font covers {LANGUAGE_NAMES[script]} text on this machine — {FONT_INSTALL_HINT}", kind="input")
504
- info(f"font: {entry[0]} (covers {script})")
543
+ die(f"no installed font covers {LANGUAGE_NAMES[key]} text on this machine — {FONT_INSTALL_HINT}", kind="input")
544
+ info(f"font: {entry[0]} (covers {key})")
505
545
  return script, entry[0], entry[1]
@@ -456,6 +456,7 @@ DIAGNOSTIC_CODES = {
456
456
  "font_path_copied": ("fallbacks", "a --font-file under a non-ASCII path was copied to an ASCII one, because drawtext cannot open it on Windows; {font, file}"),
457
457
  "font_path_not_ascii": ("warnings", "a --font-file under a non-ASCII path could not be given an ASCII form on Windows; drawtext may not open it; {font}"),
458
458
  "fonts_dir_lacks_script": ("fallbacks", "no face in --fonts-dir covers the text's script, so one was picked by script; {fonts_dir, script}"),
459
+ "font_variant_missing": ("fallbacks", "--lang named Traditional Chinese but no Traditional face is installed, so a Simplified one draws it (some glyph forms differ); {lang, file}"),
459
460
  "emoji_monochrome": ("warnings", "emoji drawn by the text font (monochrome at best): no colour path here"),
460
461
  "emoji_stripped": ("warnings", "overlay.py --emoji none: emoji removed from the drawn text; {mode}"),
461
462
  "emoji_assets_missing": ("warnings", "--emoji-assets has no PNG for these clusters; the text font draws them; {missing}"),
@@ -536,7 +537,8 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str
536
537
  "hw": {"type": "object", "description": "present when VideoToolbox was asked for (--hw, $FFMPEG_SKILL_HW, a render.py stage, a batch.py item), on export.py when $FFMPEG_SKILL_HW=1 is set but its delivery preset does not take it, and when this ffmpeg build has no libx264/libx265 and an encode ran on VideoToolbox instead: {requested: whether this run's encodes were asked onto VideoToolbox (false for that export, and for the build without libx264/libx265), source: flag|env|auto (auto: nobody asked, the build has no libx264/libx265 and this Apple Silicon machine ran VideoToolbox in its place), used: true|false whether the encoder that ran last is VideoToolbox, null under --dry-run (nothing ran) and when this process ran no encode itself (batch.py: its stages are child processes; its top-level hw gathers the items' requested/source/fallback/notes and each results row carries its item's encoder and hw), fallback: a job VideoToolbox refused (encoder open or compression session failure; nothing else is retried) was re-encoded on the CPU, notes: why an encode stayed on or fell back to the CPU, and, when $FFMPEG_SKILL_HW chose the GPU and it ran, what that costs and that --no-hw opts out}. A VideoToolbox encode also adds a top-level notes line: its quality is not CRF-equivalent. A kind: ffmpeg failure under --hw / $FFMPEG_SKILL_HW carries encoder and hw too"},
537
538
  "plan": {"type": "string", "description": "with --plan FILE: the plan document written (the run itself is a dry run)"},
538
539
  "verified": {"type": "boolean", "description": "true only when the artifact was written, probed, and every self-check the tool ran (verification) met its target; false under --dry-run"},
539
- "verification": {"type": "array", "items": {"type": "object", "properties": {"step": {"type": "string"}, "ok": {"type": "boolean"}, "stage": {"type": "string"}}}, "description": "what the tool itself verified: probe, plus loudness (loudness.py, export platform presets; ok false with reason not_measured when it could not be read), check (render), report.py's sections (loudness, compliance, contact_sheet) and batch.py's items (item). render.py also lists every stage's own steps beyond its artifact probe, tagged with `stage` (a cached stage's as recorded), so its `verified` is the conjunction of every stage's"},
540
+ "verification": {"type": "array", "items": {"type": "object", "properties": {"step": {"type": "string"}, "ok": {"type": "boolean"}, "stage": {"type": "string"}, "property": {"type": "string"}, "expected": {}, "measured": {}, "delta": {}, "tolerance": {}, "provenance": {"enum": ["specification", "user_specified", "independent_measurement", "input_premeasurement"]}, "attribution": {"enum": ["artifact", "reference", "instrument", "provenance", "unresolved"]}}}, "description": "what the tool itself verified: probe, plus loudness (loudness.py, export platform presets; ok false with reason not_measured when it could not be read), check (render), report.py's sections (loudness, compliance, contact_sheet) and batch.py's items (item). render.py also lists every stage's own steps beyond its artifact probe, tagged with `stage` (a cached stage's as recorded), so its `verified` is the conjunction of every stage's. Since 2.18 a step may carry the property it measured with expected, measured, delta, tolerance and the provenance of the expected value; cut, join and export add a `streams` step comparing the output's video/audio streams with the input's; a step with ok false carries `attribution` (unresolved until a tool knows the cause)"},
541
+ "unverified": {"type": "array", "items": {"type": "string"}, "description": "2.18: the properties of the output this tool did not measure (the unverified range), e.g. caption's rendered_placement; `verified` covers the measured properties only"},
540
542
  "diagnostics": DIAGNOSTICS_SCHEMA}
541
543
  extra: Dict[str, Any] = {}
542
544
  if name == "check":
@@ -683,6 +685,13 @@ def output_schema(name: str, meta: Dict[str, Any], hw: bool = False) -> Dict[str
683
685
  return {"type": "object", "properties": props, "required": required, "additionalProperties": True}
684
686
 
685
687
  @functools.lru_cache(maxsize=1)
688
+ def _unverified_table() -> Dict[str, List[str]]:
689
+ """emit()'s per-tool unverified range, read where the results are made so the two cannot differ."""
690
+ sys.path.insert(0, str(ROOT / "scripts"))
691
+ from _common.emit import UNVERIFIED
692
+ return UNVERIFIED
693
+
694
+
686
695
  def skill_examples() -> Dict[str, List[Dict[str, Any]]]:
687
696
  """1.20.0: per-tool `examples`, parsed from SKILL.md's "User says" / "Do" table -- the
688
697
  same facts that table already states, machine-readable instead of prose-only. A row
@@ -1547,6 +1556,8 @@ def tool_spec(name: str, version: str) -> Dict[str, Any]:
1547
1556
  "reencodes_audio": REENCODE_META.get(name, {}).get("audio", "conditional"),
1548
1557
  **({"reencode_note": REENCODE_META[name]["note"]} if REENCODE_META.get(name, {}).get("note") else {}),
1549
1558
  "requires_visual_verification": meta["visual"],
1559
+ # 2.18: what `verified` does not cover for this tool (the result's `unverified`)
1560
+ "unverified_properties": list(_unverified_table().get(name, [])),
1550
1561
  "audio_only": meta["audio_only"],
1551
1562
  "video_required": meta["video_required"],
1552
1563
  "deterministic_inputs": meta["deterministic"],
@@ -55,6 +55,13 @@ _NO_SAFE = {"top": 0.0, "bottom": 0.0, "left": 0.0, "right": 0.0}
55
55
  _CAP_VERTICAL = {"size": 0.0833, "position": "bottom", "box": False, "outline": 2, "animate": "pop"}
56
56
  _CAP_WIDE = {"size": 0.0694, "position": "bottom", "box": False, "outline": 2, "animate": "none"}
57
57
 
58
+ # The one true-peak tolerance every tool applies to a dBTP ceiling, reported beside the verdict.
59
+ # Before 2.18 loudness.py allowed +0.1 dB, check.py +0.05 and export.py 0.0, so "verified" meant a
60
+ # different scale in each tool and export could fail a file check passed (the VDC paper's
61
+ # sensitivity run). 0.1 dB is loudness.py's margin: two loudnorm passes on the same encode differ
62
+ # by up to that much, so a tighter rule fails a file on measurement noise.
63
+ TP_TOLERANCE_DB = 0.1
64
+
58
65
  PLATFORMS: Dict[str, Dict[str, Any]] = {
59
66
  "tiktok": {
60
67
  "title": "TikTok",
package/scripts/check.py CHANGED
@@ -34,7 +34,7 @@ import sys
34
34
  from fractions import Fraction
35
35
  from typing import Any, Dict, List
36
36
 
37
- from _platforms import PLATFORMS, PLATFORM_CHOICES, spec_of, resolve as resolve_platform
37
+ from _platforms import PLATFORMS, PLATFORM_CHOICES, TP_TOLERANCE_DB, spec_of, resolve as resolve_platform
38
38
  from _common.probe import SILENT_PEAK_DBFS, detect_content, measured_level_dbfs
39
39
  from _common import STATE, add_common, apply_common, die, emit, info, probe, require_tool, run, run_analysis, dry_run_input_pending
40
40
 
@@ -249,7 +249,7 @@ def main() -> int:
249
249
  diff = abs(lm["lufs"] - spec["lufs"])
250
250
  row("loudness", "PASS" if diff <= spec["lufs_tol"] else "FAIL", f"{lm['lufs']:.1f} LUFS", f"{spec['lufs']:g} ± {spec['lufs_tol']:g} LUFS", f"loudness.py -I {spec['lufs']:g} for speech or music; leave ambience/near-silence (<= -40 LUFS) alone and say so",
251
251
  reason="the platform will auto-normalise it to its own target anyway, which can pump or duck the mix in ways you did not choose")
252
- row("true peak", "PASS" if lm["tp"] <= spec["tp"] + 0.05 else "FAIL", f"{lm['tp']:.1f} dBTP", f"<= {spec['tp']:g} dBTP", f"loudness.py --tp {spec['tp']:g}")
252
+ row("true peak", "PASS" if lm["tp"] <= spec["tp"] + TP_TOLERANCE_DB else "FAIL", f"{lm['tp']:.1f} dBTP", f"<= {spec['tp']:g} dBTP (+{TP_TOLERANCE_DB:g} dB tolerance)", f"loudness.py --tp {spec['tp']:g}")
253
253
  else:
254
254
  # ffmpeg's loudnorm JSON didn't parse out of stderr (unexpected/garbled output).
255
255
  # Dropping the loudness/true-peak rows silently here would report an overall PASS
package/scripts/cut.py CHANGED
@@ -1677,7 +1677,11 @@ def main() -> int:
1677
1677
  notes.append(skew_note)
1678
1678
  info(f"wrote {output} ({fmt_secs(got)}, expected ~{expected:.3f}s, "
1679
1679
  + ("re-encoded" if reencoded else "lossless stream copy") + f", {precision} precision)")
1680
+ # the streams the source had and this cut was not told to drop (an audio extension drops the
1681
+ # picture): #367, a TS cut that lost its video must not say verified
1682
+ expect_streams = {"video": bool(meta.get("video")) and not is_audio_output(output), "audio": bool(meta.get("audio"))}
1680
1683
  emit(output, expected_duration=round(expected, 6), duration_error_ms=error_ms, precision=precision, reencoded=reencoded,
1684
+ expect_streams=expect_streams,
1681
1685
  dropped_non_av_streams=bool(DROPPED_STREAMS),
1682
1686
  requested_start=round(requested[0][0], 6) if len(requested) == 1 else None,
1683
1687
  requested_end=round(requested[0][1], 6) if len(requested) == 1 else None,
package/scripts/export.py CHANGED
@@ -40,10 +40,22 @@ from _common import STATE, add_common, apply_common, bt709_matrix_args, bt709_ta
40
40
  from _common.decision import hw_preset_video, restate_last_swap
41
41
  from _common.emit import note, record_skip, warn
42
42
  from check import SPECS as PLATFORMS, measure_loudness
43
- from _platforms import ALIASES as _ALIASES, PLATFORMS as PLATFORM_TABLE, resolve as resolve_platform
43
+ from _platforms import ALIASES as _ALIASES, PLATFORMS as PLATFORM_TABLE, TP_TOLERANCE_DB, resolve as resolve_platform
44
44
  # Frame and duration limit come from the one platform table (scripts/_platforms.py) rather than
45
45
  # from a literal restated here: before 1.14 they were typed twice and the facebook preset had
46
46
  # already drifted (no duration cap against the table's 14400 s).
47
+
48
+ def _loudness_step(ok: bool, platform: str, spec: dict, m: dict) -> dict:
49
+ """export's loudness verification step: the platform's targets, what was measured, the
50
+ difference and the tolerance each was held to (docs/contract.md, 2.18 keys)."""
51
+ expected = {"lufs": spec["lufs"], "tp_max": spec["tp"]}
52
+ measured = {"lufs": m["lufs"], "tp": m["tp"]}
53
+ delta = {"lufs": round(m["lufs"] - spec["lufs"], 2) if math.isfinite(m["lufs"]) else None,
54
+ "tp": round(m["tp"] - spec["tp"], 2) if math.isfinite(m["tp"]) else None}
55
+ return {"step": "loudness", "ok": ok, "platform": platform, "property": "loudness",
56
+ "expected": expected, "measured": measured, "delta": delta,
57
+ "tolerance": {"lufs": spec["lufs_tol"], "tp": TP_TOLERANCE_DB}, "provenance": "specification"}
58
+
47
59
  def _from_table(dest: str, **over) -> Dict:
48
60
  """A preset's w/h/max read from PLATFORMS[dest], with the encoder settings given here.
49
61
 
@@ -268,9 +280,9 @@ def main() -> int:
268
280
  if not m:
269
281
  _loudness_unmeasured(extra, notes, platform, spec)
270
282
  if m:
271
- ok = abs(m["lufs"] - spec["lufs"]) <= spec["lufs_tol"] and m["tp"] <= spec["tp"]
283
+ ok = abs(m["lufs"] - spec["lufs"]) <= spec["lufs_tol"] and m["tp"] <= spec["tp"] + TP_TOLERANCE_DB
272
284
  extra["loudness"] = {"lufs": m["lufs"], "tp": m["tp"], "target_lufs": spec["lufs"], "target_tp": spec["tp"], "ok": ok}
273
- extra["verification"] = [{"step": "loudness", "ok": ok, "platform": platform}]
285
+ extra["verification"] = [_loudness_step(ok, platform, spec, m)]
274
286
  if not math.isfinite(m["lufs"]):
275
287
  # a silent output measures -inf LUFS: no gain reaches the target, so recommending
276
288
  # (or running, under --normalize) loudness.py only returns "nothing to normalise"
@@ -305,9 +317,9 @@ def main() -> int:
305
317
  _loudness_unmeasured(extra, notes, platform, spec, normalized=True)
306
318
  ok = True # nothing more to say about a level that was not measured
307
319
  else:
308
- ok = abs(m["lufs"] - spec["lufs"]) <= spec["lufs_tol"] and m["tp"] <= spec["tp"]
320
+ ok = abs(m["lufs"] - spec["lufs"]) <= spec["lufs_tol"] and m["tp"] <= spec["tp"] + TP_TOLERANCE_DB
309
321
  extra["loudness"] = {"lufs": m["lufs"], "tp": m["tp"], "target_lufs": spec["lufs"], "target_tp": spec["tp"], "ok": ok, "normalized": True}
310
- extra["verification"] = [{"step": "loudness", "ok": ok, "platform": platform}]
322
+ extra["verification"] = [_loudness_step(ok, platform, spec, m)]
311
323
  if not ok:
312
324
  notes.append(f"loudness is still {m['lufs']:.1f} LUFS / {m['tp']:+.1f} dBTP after loudness.py (target {spec['lufs']:g} LUFS / {spec['tp']:g} dBTP): {child.get('result', {}).get('note') or 'the encoder overshoots the ceiling'}")
313
325
  warn(notes[-1], "loudness_outside_spec", platform=platform, lufs=m["lufs"], tp=m["tp"], normalized=True)
@@ -323,6 +335,8 @@ def main() -> int:
323
335
  notes.append(f"[dry-run] --normalize: loudness.py -I {spec['lufs']:g} --tp {spec['tp']:g} would run on the written file if it misses {platform}'s spec")
324
336
  if notes:
325
337
  extra["notes"] = notes
338
+ # the delivery keeps the source's picture and, except as a GIF, its sound
339
+ extra["expect_streams"] = {"video": bool(meta.get("video")), "audio": has_audio and args.preset != "gif"}
326
340
  emit(output, **extra)
327
341
  return 0
328
342
 
@@ -294,7 +294,7 @@ def main(force_route: Optional[str] = None) -> int:
294
294
  if explicit:
295
295
  return font_family_of_file(explicit) or args.font or brand.get("font")
296
296
  if script_file:
297
- return font_family_of_file(script_file) or font_family_for_script(_script)
297
+ return _family or font_family_of_file(script_file) or font_family_for_script(_script)
298
298
  return args.font or brand.get("font", "DejaVu Sans")
299
299
 
300
300
  def ass_fonts_dir() -> "Optional[str]":
package/scripts/join.py CHANGED
@@ -565,11 +565,21 @@ def main() -> int:
565
565
  verification = []
566
566
  if expected is not None and measured is not None and not STATE.dry_run:
567
567
  tol = max(1.5 / fps, 0.05)
568
- verification.append({"step": "duration", "ok": abs(measured - expected) <= tol,
569
- "measured": round(measured, 3), "expected": expected, "tolerance": round(tol, 3)})
568
+ verification.append({"step": "duration", "property": "video_duration", "ok": abs(measured - expected) <= tol,
569
+ "measured": round(measured, 3), "expected": expected,
570
+ "delta": round(measured - expected, 3), "tolerance": round(tol, 3),
571
+ "provenance": "input_premeasurement"})
572
+ # a clip with no audio stream is joined over generated silence: the join is right to do that, but
573
+ # the report must say which clips' sound is not theirs (the paper's join finding: absence was
574
+ # filled and nothing in the result showed it)
575
+ substituted = [p for p, m in zip(args.inputs, metas) if not m.get("audio")]
576
+ if substituted:
577
+ STATE_NOTES.append(f"{len(substituted)} clip(s) had no audio stream; their span of the join is generated "
578
+ "silence: " + ", ".join(os.path.basename(p) for p in substituted))
570
579
  info(f"wrote {output} ({fmt_secs(measured)}, {expected_text(expected)}, {w}x{h} @ {fps:g}fps, {n} clips, {args.transition})")
571
580
  emit(output, mode="video", **reported(), clips=n, transition=args.transition, expected_duration=expected,
572
- verification=verification,
581
+ verification=verification, expect_streams={"video": True, "audio": True},
582
+ audio_substituted=substituted,
573
583
  dropped_non_av_streams=any(m.get("subtitle_streams") or m.get("data_streams") for m in metas))
574
584
  return 0
575
585
 
@@ -19,6 +19,7 @@ import os
19
19
  import re
20
20
  import sys
21
21
 
22
+ from _platforms import TP_TOLERANCE_DB
22
23
  from _common import STATE, add_common, apply_common, emit, AUDIO_CODECS, audio_codec_for, default_output, die, ffmpeg_base, info, probe, require_tool, run, run_analysis, run_keeping_subtitles, dry_run_input_pending
23
24
 
24
25
 
@@ -128,7 +129,7 @@ def main() -> int:
128
129
  # integrated loudness -- reported, never hidden.
129
130
  ceiling, rounds = args.tp, 0
130
131
  steps = [] if bitrate_pinned or ext in AUDIO_CODECS and "aac" not in AUDIO_CODECS[ext] else [b for b in ("256k", "320k") if _kbps(b) > _kbps(bitrate)]
131
- while not after.get("silent") and float(after["input_tp"]) > args.tp + 0.1 and rounds < 5:
132
+ while not after.get("silent") and float(after["input_tp"]) > args.tp + TP_TOLERANCE_DB and rounds < 5:
132
133
  rounds += 1
133
134
  overshoot = float(after["input_tp"]) - args.tp
134
135
  if steps:
@@ -152,7 +153,7 @@ def main() -> int:
152
153
  result["note"] = (f"integrated loudness is {float(after['input_i']) - args.lufs:+.1f} LU from the target because the "
153
154
  f"true-peak ceiling had to absorb the encoder's overshoot; a lossless delivery (wav/flac) or a pinned "
154
155
  f"higher --audio-bitrate keeps both")
155
- if not after.get("silent") and float(after["input_tp"]) > args.tp + 0.1:
156
+ if not after.get("silent") and float(after["input_tp"]) > args.tp + TP_TOLERANCE_DB:
156
157
  die(f"true peak is still {float(after['input_tp']):.2f} dBTP after {rounds + 1} encodes (requested <= {args.tp:g}); "
157
158
  f"the encoder overshoots more than the loudnorm ceiling can absorb at this bitrate",
158
159
  kind="verification", output=output, result=result,
@@ -161,8 +162,12 @@ def main() -> int:
161
162
  measured={k: stats[k] for k in ("input_i", "input_tp", "input_lra", "input_thresh", "target_offset", "silent")},
162
163
  targets={"lufs": args.lufs, "tp": args.tp, "lra": args.lra},
163
164
  verification=[{"step": "loudness",
164
- "ok": bool(after.get("silent")) or (abs(float(after["input_i"]) - args.lufs) <= 1.0 and float(after["input_tp"]) <= args.tp + 0.1),
165
- "lufs": float(after["input_i"]), "tp": float(after["input_tp"]), "target_lufs": args.lufs, "target_tp": args.tp}])
165
+ "ok": bool(after.get("silent")) or (abs(float(after["input_i"]) - args.lufs) <= 1.0 and float(after["input_tp"]) <= args.tp + TP_TOLERANCE_DB),
166
+ "lufs": float(after["input_i"]), "tp": float(after["input_tp"]), "target_lufs": args.lufs, "target_tp": args.tp,
167
+ "property": "loudness", "expected": {"lufs": args.lufs, "tp_max": args.tp},
168
+ "measured": {"lufs": float(after["input_i"]), "tp": float(after["input_tp"])},
169
+ "delta": {"lufs": round(float(after["input_i"]) - args.lufs, 2), "tp": round(float(after["input_tp"]) - args.tp, 2)},
170
+ "tolerance": {"lufs": 1.0, "tp": TP_TOLERANCE_DB}, "provenance": "user_specified"}])
166
171
  return 0
167
172
 
168
173
 
@@ -152,7 +152,10 @@ def resolve_filler(args, meta):
152
152
  # (or one that says "auto", which is no language either) keeps the English list it always had
153
153
  # the ISO 639-2 spellings of the languages that have a built-in list (the mux tag uses them)
154
154
  iso3 = {"eng": "en", "spa": "es", "deu": "de", "ger": "de", "fra": "fr", "fre": "fr",
155
- "ita": "it", "jpn": "ja", "por": "pt"}
155
+ "ita": "it", "jpn": "ja", "por": "pt", "kor": "ko", "zho": "zh", "chi": "zh",
156
+ "rus": "ru", "nld": "nl", "dut": "nl", "pol": "pl", "tur": "tr"}
157
+ names = {"spanish": "es", "german": "de", "portuguese": "pt", "chinese": "zh", "mandarin": "zh",
158
+ "dutch": "nl", "polish": "pl", "turkish": "tr"}
156
159
 
157
160
  def _code(value) -> str:
158
161
  value = str(value or "").strip().lower().replace("_", "-")
@@ -160,10 +163,13 @@ def resolve_filler(args, meta):
160
163
  return ""
161
164
  # a regional tag names its primary language (ja-JP is ja); a three-letter code that is not
162
165
  # in the table stays as it is (est is Estonian, not the Spanish "es"); a word such as
163
- # "english" is cut to its two-letter code
166
+ # "english" is cut to its two-letter code; a name whose first two letters are not its
167
+ # code ("spanish" is es, "chinese" zh) is looked up, as whisper's own language table spells it
164
168
  primary = value.split("-")[0]
165
169
  if primary in iso3:
166
170
  return iso3[primary]
171
+ if primary in names:
172
+ return names[primary]
167
173
  return primary[:2] if len(primary) > 3 else primary
168
174
  spoken = _code((raw or {}).get("language")) if isinstance(raw, dict) else ""
169
175
  spoken = spoken or _code(transcription.get("language") or transcription.get("detected_language"))