@dsh-gis/dsh-video-studio-skill 0.7.2 → 0.9.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/CHANGELOG.md +1109 -0
  2. package/README.md +76 -10
  3. package/THIRD_PARTY_NOTICES.md +1 -0
  4. package/docs/styles/sheet-8-styles.png +0 -0
  5. package/docs/styles/sheet-before-after.png +0 -0
  6. package/index.js +4 -1
  7. package/package.json +7 -3
  8. package/schemas/film.schema.json +329 -2
  9. package/skills/video-studio/SKILL.md +270 -13
  10. package/skills/video-studio/assets/golden_frames.json +65 -0
  11. package/skills/video-studio/assets/scenes.py +2162 -1190
  12. package/skills/video-studio/assets/web/engine.js +63 -2
  13. package/skills/video-studio/assets/web/film.js +224 -101
  14. package/skills/video-studio/assets/web/index.html +4 -2
  15. package/skills/video-studio/assets/web/qa_text.js +36 -8
  16. package/skills/video-studio/references/audio-variation.md +4 -3
  17. package/skills/video-studio/references/audio.md +2 -2
  18. package/skills/video-studio/references/change-impact.md +1 -1
  19. package/skills/video-studio/references/quality-gates.md +25 -3
  20. package/skills/video-studio/references/scene-grammar.md +306 -0
  21. package/skills/video-studio/references/style-packs.md +108 -3
  22. package/skills/video-studio/references/v08-quality.md +256 -0
  23. package/skills/video-studio/references/web-engine.md +19 -2
  24. package/skills/video-studio/references/word-timeline.md +147 -0
  25. package/skills/video-studio/requirements.txt +14 -0
  26. package/skills/video-studio/scripts/filmcheck.py +218 -0
  27. package/skills/video-studio/scripts/filmkit.py +521 -38
  28. package/skills/video-studio/scripts/gates.py +88 -4
  29. package/skills/video-studio/scripts/grammar.py +559 -0
  30. package/skills/video-studio/scripts/preflight.py +774 -0
  31. package/skills/video-studio/scripts/qa_frames.py +49 -8
  32. package/skills/video-studio/scripts/qaasr.py +1563 -0
  33. package/skills/video-studio/scripts/qavmaf.py +1442 -0
  34. package/skills/video-studio/scripts/render.py +764 -32
  35. package/skills/video-studio/scripts/scorekit.py +16 -30
  36. package/skills/video-studio/scripts/selfcheck.py +483 -0
  37. package/skills/video-studio/scripts/sfxkit.py +130 -14
  38. package/skills/video-studio/scripts/stagewatch.py +19 -6
  39. package/skills/video-studio/scripts/stylekit.py +242 -12
  40. package/skills/video-studio/scripts/voicekit.py +15 -7
  41. package/skills/video-studio/scripts/webengine.py +1119 -72
  42. package/tools/ci_check.py +173 -31
  43. package/tools/estimate.py +130 -19
  44. package/tools/golden_frames.py +664 -0
  45. package/tools/silhouette_qa.py +910 -0
@@ -0,0 +1,1563 @@
1
+ # -*- coding: utf-8 -*-
2
+ """qaasr.py — ASR 回读校验(v0.8.0)。
3
+
4
+ v0.7 及以前的质检只看"容器 / 响度 / 帧 / 音画同步",都建立在一个假设上:
5
+ **成片音轨里念的就是剧本那几句话**。可是 TTS 缓存串了、换声线时漏了某句、
6
+ 混音把某一段压没了、重新渲了但音轨是旧的——这些从容器和波形上都看不出来。
7
+ 本模块把成片音轨转回文字,和剧本逐句对账:
8
+
9
+ 1. 引擎优先级:faster-whisper(本机已装,import 失败才降级)> ffmpeg whisper 滤镜
10
+ (ffmpeg 9.0.2-full_build 带 --enable-whisper,实测有 whisper 滤镜、没有 asr 滤镜)
11
+ > 两者都不可用则 skipped。**不许把"没查"写成 pass**。
12
+ 2. 匹配:按 timeline.json 每段 start/end 取 ASR 结果里落在该窗口内的文本,去标点 / 空白 /
13
+ 数字写法差异 / 常用繁体(whisper 中文输出常夹繁体)后做**字符级多重集召回**
14
+ (中文没有词边界,词级召回不成立;多重集而不是集合,是因为"是"这种字重复出现要被计数)。
15
+ 3. 偏移:报告每段 ASR 首字相对 timeline 段语音起点(voice_start,缺省用 start)的偏移;
16
+ |偏移| > 0.35 s 给 warning。偏移只提示"对不齐",不改召回率本身。
17
+ ⚠ 没写 voice_start 的老 timeline 会退化成"相对段 start",于是每句都多算一个 lead
18
+ (本项目默认 0.55 s)→ 全片报偏移 warning。这是**判据的已知限制**,不是片子坏了:
19
+ filmkit.build_timeline 写的 timeline 一定带 voice_start;只有手搓 timeline 才会缺。
20
+ 4. 三档语义(与 gates.py 的 pass / fail / skipped 完全一致):
21
+ * 引擎跑不了(ffmpeg 缺失 / 无音轨 / 两个引擎都不可用 / 模型下载失败)-> skipped,
22
+ errors 里写清原因,warnings 里写"没查不等于通过";
23
+ * recall < recall_warn(默认 0.72,与 render.py 的 G12 同口径)-> fail;
24
+ * recall_warn <= recall < recall_pass(默认 0.85)-> pass + warning(别装没事);
25
+ * recall >= recall_pass -> pass。
26
+
27
+ 为什么召回率用字符级多重集:本项目旁白是中文,whisper 的错字通常是同音字("逐句"→"逐取"),
28
+ 一字之差不该让整句判死;而"整句丢了"会直接让召回率掉一大截。字符级对中文天然合适。
29
+
30
+ 为什么偏移要看 voice_start 而不是段 start:timeline 的 start 是"字幕/画面段起点",
31
+ 旁白在 lead 之后才开口,拿段 start 当期望首字时刻会系统性地多算一个 lead(默认 0.55 s),
32
+ 把正常的对齐全部报成超差。
33
+
34
+ CLI:
35
+ python qaasr.py <film.mp4> --timeline work/timeline.json [--script film.json]
36
+ [--model small] [--lang zh] [--limit-seconds 30]
37
+ [--out work/qa/asr.json] [--engine auto|faster-whisper|ffmpeg]
38
+ [--whisper-model <ggml-small.bin>] [--vad] [--strict]
39
+ python qaasr.py <film.mp4> --timeline ... --asr-json <已有 ASR 结果.json>
40
+ # 离线回放,不加载模型(CI / 自测用)
41
+ python qaasr.py --selftest
42
+ # 内置自检,不联网
43
+
44
+ 依赖:stdlib + numpy(项目已有);faster-whisper 可选(未装时自动降级)。
45
+ 所有 ffmpeg / ffprobe 调用都走 FILMKIT_FFMPEG 环境变量覆盖 + fail-closed 探测
46
+ (候选文件存在 != 能用,必须真的跑通 "-version" 且 returncode=0);
47
+ 一个都不可用时返回空串,调用方必须按"没查"处理,不许回退成裸命令硬跑。
48
+
49
+ 实测(本机:Windows 11 / Python 3.12.16 / CPU-only Radeon 890M / 59.6 GB 内存):
50
+ * 素材:3 句 edge-tts 中文旁白拼成的 19.87 s 成片(.team/asr_fixture/fixture.mp4,
51
+ timeline 的 voice_start = 0.55 / 7.55 / 14.05)
52
+ * faster-whisper tiny(int8):加载 1.36 s,转写 0.50 s(RTF 0.025),召回 0.923,
53
+ 三句偏移 0.00 s(tiny 的段起点整秒截断,词级时间戳也只给整秒)
54
+ * faster-whisper small(int8):加载 2.0–3.3 s、转写 3.2–3.5 s(RTF 0.16–0.18);
55
+ 召回 0.985(64/65,唯一的字是"鞋稿"错成"写稿");词级首字偏移
56
+ -0.05 / -0.06 / -0.04 s(即 whisper 觉得旁白比 lead 早约 50 ms,属正常估计误差)
57
+ * --limit-seconds 12(small):召回 0.956(2 段 43/45),转写 3.56 s
58
+ * ffmpeg whisper 滤镜(ggml-tiny.bin,use_gpu=0):19.87 s 音频 3.75 s(RTF 0.19),
59
+ 召回 0.723(3 s 定长切块 + 尾部幻觉),偏移 0.00 / +0.02 / 0.00 s——只作后备引擎
60
+ * 丢句反证:把 l03 从音轨里剪掉、timeline 仍写 3 句 -> small 召回 0.677、
61
+ missing_lines=["l03"]、gate=fail、CLI exit 1
62
+ * 确定性:tiny 同一输入跑两次(--limit-seconds 12)segments / offsets 完全一致
63
+ 结论:CPU-only 跑 small 完全可行(20 s 片连加载约 6–8 s;30 s 片约 9–11 s);
64
+ 模型 484 MB(实测 HF 缓存 972 MB 是 fp16 + int8 两份),内存峰值 < 1 GB。
65
+ tiny 只适合快速冒烟(错字多、时间戳只到整秒)。默认 model_size="small" 就是这个原因。
66
+ """
67
+ from __future__ import annotations
68
+
69
+ __version__ = "0.8.0"
70
+
71
+ import argparse
72
+ import json
73
+ import math
74
+ import os
75
+ import re
76
+ import shutil
77
+ import subprocess
78
+ import sys
79
+ import tempfile
80
+ import time
81
+ import unicodedata
82
+ import wave
83
+
84
+ HERE = os.path.dirname(os.path.abspath(__file__))
85
+
86
+ # 同目录的 filmkit 提供 FFMPEG / ffprobe 解析;独立拷贝本文件时仍要能跑
87
+ try:
88
+ import filmkit as _K
89
+ except Exception: # noqa: BLE001
90
+ _K = None
91
+
92
+ try:
93
+ import numpy as np
94
+ except Exception: # noqa: BLE001 —— 只有读 wav 用到
95
+ np = None
96
+
97
+ # ---------------------------------------------------------------- 可调参数
98
+ # 抽取给 ASR 的统一音频格式:16 kHz 单声道(whisper 系列模型的输入口径)
99
+ AUDIO_SR = 16000
100
+ # 偏移告警阈值(秒):旁白首字和 timeline 期望起点的差
101
+ OFFSET_WARN_S = 0.35
102
+ # 召回率的 pass / fail 线。**权威值在 render.py**(QA_ASR_RECALL_PASS / QA_ASR_RECALL_WARN,
103
+ # G12 门禁从这里读),本模块只是把默认值对齐,避免"同一部片、CLI 判 fail、门禁判 pass":
104
+ # 同一事实只能有一个权威值(CHANGELOG v0.4 D 节)。
105
+ # 为什么是 0.85 / 0.72 而不是 0.95 / 0.90:中文旁白实测(模板片,faster-whisper small)
106
+ # 干净成片召回只有 0.81–0.83 —— 中英混排专名("DeepSeek 4.1 Flash"→DIXX 变体)是 ASR 的
107
+ # 正常误差,不是片子坏了;低于 0.72 才说明"整句丢了 / 音轨跟剧本不是一套"。
108
+ QA_RECALL_PASS = 0.85
109
+ QA_RECALL_WARN = 0.72
110
+ # 行级"疑似丢句"阈值:单行字符召回低于它,就进 missing_lines 让人复核。
111
+ # 注意:它**不参与门禁判定**(门禁只看全片字符召回),只是一个定位线索——
112
+ # 短句错一个字就可能掉到 0.8,拿全局 fail 线(0.72)去卡行级会满屏假阳性。
113
+ MISS_LINE_RECALL = 0.60
114
+ # 引擎名(进报告,别在别处硬编码字符串)
115
+ ENGINE_FW = "faster-whisper"
116
+ ENGINE_FF = "ffmpeg-whisper"
117
+ # 常用繁体 -> 简体(whisper 中文输出常夹繁体,实测 tiny 输出"視頻創作/從寫導到配音")。
118
+ # 只收高频字,不做全量 OpenCC:多一个字表就多一份维护成本,而我们的判据是召回率,
119
+ # 剩下的生僻差异本来就会被召回率吸收。
120
+ _T2S_PAIRS = """
121
+ 創创 寫写 導导 動动 於于 單单 檢检 還还 對对 們们 這这 個个 時时 間间 軸轴 會会 樣样 麼么
122
+ 為为 開开 關关 說说 讀读 聽听 產产 術术 數数 據据 應应 該该 讓让 給给 與与 從从 見见 覺觉
123
+ 東东 車车 長长 門门 問问 題题 話话 語语 詞词 義义 體体 點点 線线 網网 頁页 風风 飛飞 馬马
124
+ 魚鱼 鳥鸟 麗丽 龐庞 謝谢 傳传 統统 實实 現现 發发 員员 專专 業业 務务 內内 兩两 準准 確确
125
+ 認认 識识 環环 節节 圖图 檔档 標标 質质 進进 過过 運运 邊边 際际 復复 雜杂 處处 將将 軍军
126
+ 醫医 藥药 價价 錢钱 買买 賣卖 貨货 資资 費费 華华 國国 陸陆 續续 斷断 歸归 隨随 離离 難难
127
+ 響响 較较 願愿 齊齐 魯鲁 爾尔 牆墙 殘残 殺杀 氣气 漢汉 溫温 濕湿 燈灯 燒烧 熱热 嗎吗
128
+ 視视 頻频 頭头 無无 萬万 種种 學学 聲声 靜静 連连 結结 構构 證证 驗验 試试 測测 範范 圍围
129
+ 錢钱 銀银 鋼钢 鐵铁 鐘钟 鍵键 盤盘 碼码 絡络 紀纪 錄录 訊讯 號号 頭头 髮发 隻只 醜丑 龍龙
130
+ 鳳凤 寶宝 貴贵 團团 黨党 齡龄 麵面 麼么 麼么
131
+ """
132
+ _T2S = {}
133
+ for _pair in _T2S_PAIRS.split():
134
+ if len(_pair) == 2:
135
+ _T2S[_pair[0]] = _pair[1]
136
+
137
+ # 数字写法归一:阿拉伯数字 -> 中文逐位写法("2024"->"二零二四"、"第3季度"->"第三季度")。
138
+ # 选这个方向而不是反过来,是因为单字用例("3"->"三")反过来做不了——中文数字转阿拉伯
139
+ # 需要区分"三"是数字还是"一起"里的量词,而逐位映射是确定性的、不会误伤。
140
+ # 已知盲区:数量词写法("3000" 读作"三千")会归一成"三零零零"和"三千"对不上;
141
+ # 这种差异只会丢极少数几个字的召回,不影响整句判定,也不值得为它引入词表。
142
+ _CN_DIGITS = "零一二三四五六七八九"
143
+ _ARABIC_RUN = re.compile(r"\d+")
144
+
145
+
146
+ # ---------------------------------------------------------------- ffmpeg / ffprobe
147
+ _FFMPEG_CACHE = None # None = 还没探测;"" = 探测过,一个都不可用(fail-closed)
148
+ _FFPROBE_CACHE = None
149
+ _FFMPEG_FILTERS_CACHE = None
150
+
151
+
152
+ def _probe_exe(exe) -> str:
153
+ """候选可执行文件是否真的能用:必须跑通 "<exe> -version" 且 returncode == 0。
154
+
155
+ fail-closed:文件不存在 / 不是可执行 / 权限拒绝 / 超时 / 非 0 退出,一律返回 ""。
156
+ 探测阶段吞掉异常是必须的——探测失败只代表"这个候选不可用",不该把 import 炸掉。
157
+ """
158
+ try:
159
+ if not exe:
160
+ return ""
161
+ exe = str(exe).strip().strip('"').strip("'")
162
+ if not exe:
163
+ return ""
164
+ cand = exe
165
+ if not os.path.isfile(cand):
166
+ cand = shutil.which(exe) or ""
167
+ if not cand:
168
+ return ""
169
+ p = subprocess.run([cand, "-version"], capture_output=True, timeout=15)
170
+ if p.returncode != 0:
171
+ return ""
172
+ return cand
173
+ except Exception: # noqa: BLE001
174
+ return ""
175
+
176
+
177
+ def _ffmpeg() -> str:
178
+ """解析可用的 ffmpeg 可执行文件;一个都不可用返回 ""(调用方必须当"没查")。
179
+
180
+ 顺序:FILMKIT_FFMPEG 环境变量 > filmkit.FFMPEG > PATH > WinGet/系统固定路径。
181
+ 每个候选都要真的跑通 -version 才算数(见 _probe_exe)。
182
+ """
183
+ global _FFMPEG_CACHE
184
+ if _FFMPEG_CACHE is not None:
185
+ return _FFMPEG_CACHE
186
+ cands = [os.environ.get("FILMKIT_FFMPEG")]
187
+ if _K is not None:
188
+ try:
189
+ cands.append(getattr(_K, "FFMPEG", None))
190
+ except Exception: # noqa: BLE001
191
+ pass
192
+ cands.append(shutil.which("ffmpeg"))
193
+ local = os.environ.get("LOCALAPPDATA", "")
194
+ user = os.environ.get("USERPROFILE", "")
195
+ if local:
196
+ cands.append(os.path.join(local, "Microsoft", "WinGet", "Links", "ffmpeg.exe"))
197
+ if user:
198
+ cands.append(os.path.join(user, "AppData", "Local", "Microsoft", "WinGet",
199
+ "Links", "ffmpeg.exe"))
200
+ cands += ["/usr/bin/ffmpeg", "/usr/local/bin/ffmpeg", "/opt/homebrew/bin/ffmpeg"]
201
+ seen, out = set(), ""
202
+ for c in cands:
203
+ key = os.path.normcase(str(c or ""))
204
+ if not c or key in seen:
205
+ continue
206
+ seen.add(key)
207
+ hit = _probe_exe(c)
208
+ if hit:
209
+ out = hit
210
+ break
211
+ _FFMPEG_CACHE = out
212
+ return out
213
+
214
+
215
+ def _ffprobe() -> str:
216
+ """解析 ffprobe:先找 ffmpeg 同目录的 ffprobe,再 PATH。找不到返回 ""。"""
217
+ global _FFPROBE_CACHE
218
+ if _FFPROBE_CACHE is not None:
219
+ return _FFPROBE_CACHE
220
+ cands = []
221
+ ff = _ffmpeg()
222
+ if ff:
223
+ base = os.path.basename(ff)
224
+ for name in (("ffprobe.exe", "ffprobe") if base.lower().endswith(".exe")
225
+ else ("ffprobe", "ffprobe.exe")):
226
+ cands.append(os.path.join(os.path.dirname(os.path.abspath(ff)), name))
227
+ cands += [shutil.which("ffprobe"), "ffprobe"]
228
+ seen, out = set(), ""
229
+ for c in cands:
230
+ key = os.path.normcase(str(c or ""))
231
+ if not c or key in seen:
232
+ continue
233
+ seen.add(key)
234
+ hit = _probe_exe(c)
235
+ if hit:
236
+ out = hit
237
+ break
238
+ _FFPROBE_CACHE = out
239
+ return out
240
+
241
+
242
+ def _ffmpeg_has_whisper() -> bool:
243
+ """本机 ffmpeg 是否带 whisper 滤镜(9.0.2-full_build 带 --enable-whisper,实测有)。
244
+
245
+ 只认"转写"滤镜:9.0.2 里没有 asr 滤镜(实测 -filters 只匹配到 whisper),
246
+ 所以别按 "asr" 去找——那样永远找不到,白白把后路断掉。
247
+ """
248
+ global _FFMPEG_FILTERS_CACHE
249
+ if _FFMPEG_FILTERS_CACHE is not None:
250
+ return _FFMPEG_FILTERS_CACHE
251
+ ff = _ffmpeg()
252
+ ok = False
253
+ if ff:
254
+ try:
255
+ p = subprocess.run([ff, "-hide_banner", "-filters"], capture_output=True, timeout=30)
256
+ txt = (p.stdout or b"").decode("utf-8", "replace")
257
+ txt += (p.stderr or b"").decode("utf-8", "replace")
258
+ for line in txt.splitlines():
259
+ s = line.strip()
260
+ # 形如 ".. whisper A->A Transcribe audio using whisper.cpp."
261
+ if " whisper " in (" " + s + " ") and "->" in s:
262
+ ok = True
263
+ break
264
+ except Exception: # noqa: BLE001
265
+ ok = False
266
+ _FFMPEG_FILTERS_CACHE = ok
267
+ return ok
268
+
269
+
270
+ # ---------------------------------------------------------------- 文本归一化 / 召回
271
+ def _digits_to_cn(digits: str) -> str:
272
+ """阿拉伯数字串 -> 中文逐位写法("2024"->"二零二四"、"007"->"零零七")。
273
+
274
+ 逐位而不是读成数值:中文旁白里年份(2024 读"二零二四")、编号("第007号")
275
+ 都是逐位念的,而 CTranslate2/whisper 输出的也正是"二零二四"这种写法。
276
+ 读数值反而会引入"两千零二十四"这种没人会念的形式。
277
+ """
278
+ return "".join(_CN_DIGITS[int(ch)] for ch in str(digits) if ch.isdigit())
279
+
280
+
281
+ def _norm_text(text) -> str:
282
+ """把一段文本归一化到"可比的字符序列"。
283
+
284
+ 做四件事,全部是**写法差异**,不改变语义:
285
+ 1. NFKC(全角空格/全角数字/连字等 -> 半角);
286
+ 2. 去标点 / 空白 / 控制字符 / 符号(unicodedata 分类 P* / Z* / C* / S*;S* 是符号,
287
+ "①"、"+" 这类也去,避免"加号"vs"+"的假差异);
288
+ 3. 常用繁体 -> 简体(whisper 中文输出常夹繁体);
289
+ 4. 数字写法归一:阿拉伯数字串 -> 中文逐位写法("2024"->"二零二四",
290
+ "第3季度"->"第三季度")。方向见 _CN_DIGITS 的注释。
291
+ 归一后长度可能为 0(纯标点行),调用方要能处理。
292
+ """
293
+ s = unicodedata.normalize("NFKC", str(text or ""))
294
+ out = []
295
+ for ch in s:
296
+ cat = unicodedata.category(ch)
297
+ if cat[0] in ("P", "Z", "C", "S"):
298
+ continue # 标点 / 分隔符 / 控制符 / 符号:不算内容
299
+ out.append(_T2S.get(ch, ch))
300
+ s = "".join(out)
301
+
302
+ s = _ARABIC_RUN.sub(lambda m: _digits_to_cn(m.group(0)), s)
303
+ return s
304
+
305
+
306
+ def _char_recall(ref_norm: str, hyp_norm: str) -> float:
307
+ """字符级多重集召回:|ref ∩ hyp| / |ref|(Counter 计数交集,重复字按出现次数算)。
308
+
309
+ ref_norm 已归一化;空 ref 返回 1.0(没有可查的内容,不制造假 FAIL——由调用方
310
+ 在**全片**层面判"没有可对照文本"并记 skipped)。
311
+ """
312
+ if not ref_norm:
313
+ return 1.0
314
+ if not hyp_norm:
315
+ return 0.0
316
+ from collections import Counter
317
+ a, b = Counter(ref_norm), Counter(hyp_norm)
318
+ hit = sum((a & b).values())
319
+ return hit / float(len(ref_norm))
320
+
321
+
322
+ # ---------------------------------------------------------------- 时间轴
323
+ def _load_json_arg(obj, what="参数"):
324
+ """接受 dict 或路径:dict 原样返回,路径读 JSON。读不了抛 ValueError(不吞)。"""
325
+ if obj is None:
326
+ return None
327
+ if isinstance(obj, dict):
328
+ return obj
329
+ path = str(obj)
330
+ if not os.path.exists(path):
331
+ raise ValueError("%s 不存在:%s" % (what, path))
332
+ try:
333
+ with open(path, encoding="utf-8") as fp:
334
+ return json.load(fp)
335
+ except Exception as e: # noqa: BLE001
336
+ raise ValueError("%s 解析失败(%s):%s" % (what, path, e)) from e
337
+
338
+
339
+ def _scenes_of(tl, script=None) -> list:
340
+ """从 timeline(+ 可选剧本)取出要对照的段落列表。
341
+
342
+ 剧本优先:lines[].text 是"原稿",timeline 只是排轴产物;改过台词没重跑 layout 时
343
+ 两者会不一致——那种情况下应该拿**原稿**当基准,由召回率去暴露时间轴没跟上。
344
+ 缺 text 的段直接跳过(老版本 timeline 不一定带 text),但全部缺就报错。
345
+
346
+ 返回 [{"id","start","end","voice_start","voice_end","text"}],按 start 排序。
347
+ """
348
+ if isinstance(tl, list):
349
+ # 便利入口:直接传 scenes 列表(自测 / 只关心段落时用;契约上仍是 timeline 路径或 dict)
350
+ scenes = [s for s in tl if isinstance(s, dict)]
351
+ else:
352
+ tl = _load_json_arg(tl, "timeline") or {}
353
+ if not isinstance(tl, dict):
354
+ raise ValueError("timeline 既不是 dict 也不是路径:%r" % (type(tl),))
355
+ scenes = [s for s in (tl.get("scenes") or []) if isinstance(s, dict)]
356
+ if not scenes:
357
+ raise ValueError("timeline 里没有 scenes(形如 {\"scenes\":[{id,start,end}, ...]})")
358
+ by_id = {}
359
+ if script:
360
+ sc = _load_json_arg(script, "剧本") or {}
361
+ for ln in (sc.get("lines") or []):
362
+ if isinstance(ln, dict) and ln.get("id") is not None:
363
+ by_id[str(ln["id"])] = str(ln.get("text") or "")
364
+ out = []
365
+ for s in scenes:
366
+ try:
367
+ start = float(s.get("start"))
368
+ end = float(s.get("end"))
369
+ except (TypeError, ValueError):
370
+ continue
371
+ if not (math.isfinite(start) and math.isfinite(end)) or end <= start:
372
+ continue
373
+ sid = str(s.get("id") if s.get("id") is not None else "s%02d" % (len(out) + 1))
374
+ text = by_id.get(sid)
375
+ if text is None:
376
+ text = str(s.get("text") or "")
377
+ row = {"id": sid, "start": start, "end": end, "text": text,
378
+ "voice_start": None, "voice_end": None}
379
+ # voice_start 是"旁白开口"的时刻;偏移判据以它为期望起点(见模块 docstring)
380
+ for key in ("voice_start", "voice_end"):
381
+ try:
382
+ v = float(s.get(key))
383
+ except (TypeError, ValueError):
384
+ v = None
385
+ if v is not None and math.isfinite(v):
386
+ row[key] = v
387
+ if row["voice_start"] is None:
388
+ row["voice_start"] = row["start"]
389
+ if row["voice_end"] is None:
390
+ row["voice_end"] = row["end"]
391
+ out.append(row)
392
+ if not out:
393
+ raise ValueError("timeline 的 scenes 里没有任何可用段(需要数值 start/end 且 end > start)")
394
+ out.sort(key=lambda r: r["start"])
395
+ return out
396
+
397
+
398
+ # ---------------------------------------------------------------- 音频抽取 / 探测
399
+ def _audio_info(film) -> dict:
400
+ """探测成片音轨:{"has_audio","duration","sample_rate","probe_ok"}。ffprobe 缺失时用 ffmpeg -i 兜底。
401
+
402
+ probe_ok 区分两种"has_audio=False":**真的没有音轨** 与 **没有工具可以探测**。
403
+ 以前两者都按"成片没有音轨"报,工具缺失时错误归因到片子上(skipped 的档位不变,
404
+ 但原因写错了,排障会往错误方向走)。
405
+ """
406
+ info = {"has_audio": False, "duration": None, "sample_rate": None, "probe_ok": False}
407
+ film = str(film)
408
+ fp = _ffprobe()
409
+ if fp:
410
+ info["probe_ok"] = True
411
+ try:
412
+ p = subprocess.run([fp, "-v", "error", "-show_entries",
413
+ "stream=codec_type,sample_rate:format=duration",
414
+ "-of", "json", film], capture_output=True, timeout=120)
415
+ if p.returncode == 0 and p.stdout:
416
+ data = json.loads(p.stdout.decode("utf-8", "replace"))
417
+ for st in (data.get("streams") or []):
418
+ if str(st.get("codec_type")) == "audio":
419
+ info["has_audio"] = True
420
+ try:
421
+ info["sample_rate"] = int(st.get("sample_rate") or 0) or None
422
+ except (TypeError, ValueError):
423
+ pass
424
+ break
425
+ try:
426
+ d = float((data.get("format") or {}).get("duration") or 0.0)
427
+ if d > 0:
428
+ info["duration"] = d
429
+ except (TypeError, ValueError):
430
+ pass
431
+ except Exception: # noqa: BLE001
432
+ pass
433
+ if not info["has_audio"]:
434
+ # ffprobe 缺失 / 没解析出音轨时用 ffmpeg -i 的 stderr 兜底:
435
+ # 宁可多跑一次,也不能因为"没探测到音轨"就把有音轨的片子判 skipped
436
+ ff = _ffmpeg()
437
+ if ff:
438
+ info["probe_ok"] = True
439
+ try:
440
+ q = subprocess.run([ff, "-hide_banner", "-i", film],
441
+ capture_output=True, timeout=180)
442
+ txt = (q.stderr or b"").decode("utf-8", "replace")
443
+ if "Audio:" in txt:
444
+ info["has_audio"] = True
445
+ if info["duration"] is None:
446
+ m = re.search(r"Duration:\s*(\d+):(\d+):([\d.]+)", txt)
447
+ if m:
448
+ info["duration"] = (int(m.group(1)) * 3600 + int(m.group(2)) * 60
449
+ + float(m.group(3)))
450
+ except Exception: # noqa: BLE001
451
+ pass
452
+ return info
453
+
454
+
455
+ def _extract_wav(film, out_wav, *, limit_seconds=None) -> None:
456
+ """把成片音轨抽成 16 kHz 单声道 s16 wav(两个引擎共用一份输入,保证同输入同结果)。"""
457
+ ff = _ffmpeg()
458
+ if not ff:
459
+ raise RuntimeError("找不到可用的 ffmpeg(FILMKIT_FFMPEG / PATH 都没有跑得通的 ffmpeg)")
460
+ cmd = [ff, "-v", "error", "-nostdin", "-y"]
461
+ if limit_seconds:
462
+ cmd += ["-t", "%.3f" % float(limit_seconds)]
463
+ cmd += ["-i", str(film), "-vn", "-sn", "-dn", "-ac", "1", "-ar", str(AUDIO_SR),
464
+ "-c:a", "pcm_s16le", str(out_wav)]
465
+ p = subprocess.run(cmd, capture_output=True, timeout=1800)
466
+ if p.returncode != 0 or not os.path.exists(out_wav) or os.path.getsize(out_wav) < 1000:
467
+ raise RuntimeError("抽取音轨失败(%s):%s"
468
+ % (film, (p.stderr or b"").decode("utf-8", "replace")[-500:]))
469
+
470
+
471
+ def _read_wav_f32(path):
472
+ """读 16 kHz 单声道 s16 wav -> (numpy float32 数组, 采样率)。只认 PCM16。"""
473
+ if np is None:
474
+ raise RuntimeError("numpy 不可用,读不了 wav")
475
+ with wave.open(str(path), "rb") as w:
476
+ ch, sw, sr, n = w.getnchannels(), w.getsampwidth(), w.getframerate(), w.getnframes()
477
+ raw = w.readframes(n)
478
+ if sw != 2:
479
+ raise RuntimeError("wav 采样宽度不是 16-bit(%d),请检查 ffmpeg 输出" % sw)
480
+ x = np.frombuffer(raw, dtype="<i2").astype(np.float32) / 32768.0
481
+ if ch > 1:
482
+ x = x.reshape(-1, ch).mean(axis=1)
483
+ return x, sr
484
+
485
+
486
+ # ---------------------------------------------------------------- 引擎:faster-whisper
487
+ _FW_IMPORT = None # None = 没试过;True/False = 结果
488
+ _FW_IMPORT_ERR = ""
489
+
490
+
491
+ def _faster_whisper_available() -> bool:
492
+ """faster_whisper 能否 import(不加载模型、不联网)。"""
493
+ global _FW_IMPORT, _FW_IMPORT_ERR
494
+ if _FW_IMPORT is not None:
495
+ return _FW_IMPORT
496
+ try:
497
+ import faster_whisper # noqa: F401
498
+ _FW_IMPORT = True
499
+ except Exception as e: # noqa: BLE001
500
+ _FW_IMPORT = False
501
+ _FW_IMPORT_ERR = "%s: %s" % (type(e).__name__, e)
502
+ return _FW_IMPORT
503
+
504
+
505
+ def _model_cache_bytes(model_size):
506
+ """faster-whisper 模型的本地缓存大小(HF hub 目录,排除 .incomplete 半截文件)。
507
+
508
+ 只用于报告"用了多大的模型",查不到返回 None(不猜数字)。
509
+ """
510
+ root = os.path.join(os.path.expanduser("~"), ".cache", "huggingface", "hub",
511
+ "models--Systran--faster-whisper-%s" % model_size)
512
+ if not os.path.isdir(root):
513
+ return None
514
+ total = 0
515
+ for dirpath, _dirnames, filenames in os.walk(root):
516
+ for fn in filenames:
517
+ if fn.endswith(".incomplete"):
518
+ continue
519
+ try:
520
+ total += os.path.getsize(os.path.join(dirpath, fn))
521
+ except OSError:
522
+ pass
523
+ return total or None
524
+
525
+
526
+ def _asr_faster_whisper(wav, *, model_size="small", lang="zh", vad=False, log=print) -> dict:
527
+ """faster-whisper 转写。
528
+
529
+ 注意:**不要用 transcribe(文件路径)**。本机 faster-whisper 1.2.1 + 新版 PyAV 不兼容,
530
+ 它的 decode_audio 会 TypeError(av.open() got an unexpected keyword 'metadata_errors')。
531
+ 所以这里统一走"ffmpeg 抽 wav -> numpy float32 -> transcribe(数组)",既绕开 av 的坑,
532
+ 又不引入新依赖(numpy 项目已有)。
533
+
534
+ 确定性:temperature=0、condition_on_previous_text=False。CPU int8 下多线程浮点归约
535
+ 理论上可能有极小抖动(同一句偶发一字之差),实测两次结果一致,但没有穷举验证——
536
+ 报告里不承诺位级确定性,只承诺"不影响门禁结论"。
537
+ """
538
+ from faster_whisper import WhisperModel
539
+
540
+ t0 = time.time()
541
+ model = WhisperModel(str(model_size), device="cpu", compute_type="int8")
542
+ load_s = time.time() - t0
543
+ audio, sr = _read_wav_f32(wav)
544
+ if sr != AUDIO_SR:
545
+ raise RuntimeError("wav 采样率 %d != %d,抽取参数不对" % (sr, AUDIO_SR))
546
+ audio_s = audio.size / float(AUDIO_SR)
547
+ if log:
548
+ log(" [ASR] faster-whisper %s:加载 %.2fs,音频 %.2fs,开始转写…"
549
+ % (model_size, load_s, audio_s))
550
+ t1 = time.time()
551
+ seg_iter, info = model.transcribe(
552
+ audio, language=str(lang or "zh"), beam_size=5, temperature=0.0,
553
+ vad_filter=bool(vad), condition_on_previous_text=False,
554
+ # 词级时间戳只为拿"这段第一个字到底落在哪一秒"。实测 segment.start 会被
555
+ # whisper 的 1 s/2 s 栅格吃掉(模板片 19.87 s 出来的是 0 / 7 / 14,全整秒),
556
+ # 拿它算偏移只会得到 0.00 的假精确;词的 start 是真实对齐的。
557
+ word_timestamps=True)
558
+ segs = []
559
+ for s in seg_iter:
560
+ txt = str(getattr(s, "text", "") or "").strip()
561
+ try:
562
+ a, b = float(s.start), float(s.end)
563
+ except (TypeError, ValueError):
564
+ continue
565
+ if not (math.isfinite(a) and math.isfinite(b)):
566
+ continue
567
+ seg = {"start": round(a, 3), "end": round(b, 3), "text": txt}
568
+ try:
569
+ words = list(getattr(s, "words", None) or [])
570
+ except Exception: # noqa: BLE001
571
+ words = []
572
+ if words:
573
+ try:
574
+ w0 = float(getattr(words[0], "start", float("nan")))
575
+ if math.isfinite(w0):
576
+ seg["first_word_t"] = round(w0, 3)
577
+ except (TypeError, ValueError):
578
+ pass
579
+ segs.append(seg)
580
+ asr_s = time.time() - t1
581
+ return {"engine": ENGINE_FW, "model": str(model_size), "segments": segs,
582
+ "load_s": round(load_s, 2), "asr_s": round(asr_s, 2),
583
+ "audio_s": round(audio_s, 3), "model_bytes": _model_cache_bytes(model_size),
584
+ "lang": str(getattr(info, "language", lang) or lang),
585
+ "lang_prob": round(float(getattr(info, "language_probability", 0.0) or 0.0), 3),
586
+ "detail": "faster-whisper %s / CPU int8%s"
587
+ % (model_size, " / vad" if vad else "")}
588
+
589
+
590
+ # ---------------------------------------------------------------- 引擎:ffmpeg whisper
591
+ def _default_whisper_models(model_size) -> list:
592
+ """ffmpeg whisper 滤镜用的 ggml 模型候选(本地文件,**本工具不自动下载**)。
593
+
594
+ 为什么不自动下载:whisper.cpp 的 ggml 模型和 faster-whisper 的 CTranslate2 模型不是
595
+ 一个格式,自动下 75–500 MB 的二进制属于"眼睛之外的动作";这里只探测,找不到就
596
+ 让 ffmpeg 引擎不可用(fail-closed),由用户 --whisper-model 或环境变量显式指定。
597
+
598
+ 候选顺序:QAASR_WHISPER_MODEL / FILMKIT_WHISPER_MODEL > <cwd 及向上 3 层>/{models,
599
+ work/models,tools/whisper}/ggml-<size>.bin。
600
+ """
601
+ out = []
602
+ for env in ("QAASR_WHISPER_MODEL", "FILMKIT_WHISPER_MODEL"):
603
+ v = (os.environ.get(env) or "").strip().strip('"').strip("'")
604
+ if v:
605
+ out.append(v)
606
+ roots = []
607
+ try:
608
+ d = os.path.abspath(os.getcwd())
609
+ except OSError:
610
+ d = ""
611
+ for _ in range(4):
612
+ if not d:
613
+ break
614
+ roots.append(d)
615
+ parent = os.path.dirname(d)
616
+ if parent == d:
617
+ break
618
+ d = parent
619
+ name = "ggml-%s.bin" % model_size
620
+ for root in roots:
621
+ for sub in ("models", os.path.join("work", "models"), os.path.join("tools", "whisper")):
622
+ out.append(os.path.join(root, sub, name))
623
+ # 本项目约定:<repo>/skills/video-studio/assets/models/ggml-<size>.bin
624
+ out.append(os.path.join(HERE, "..", "..", "assets", "models", name))
625
+ out.append(os.path.join(HERE, "models", name))
626
+ return out
627
+
628
+
629
+ def _find_whisper_model(model_size, explicit=None) -> str:
630
+ """找 ffmpeg whisper 用的 ggml 模型文件;找不到返回 ""。"""
631
+ cands = []
632
+ if explicit:
633
+ cands.append(str(explicit))
634
+ cands += _default_whisper_models(model_size)
635
+ for c in cands:
636
+ try:
637
+ if c and os.path.isfile(c) and os.path.getsize(c) > 1_000_000:
638
+ return os.path.abspath(c)
639
+ except OSError:
640
+ continue
641
+ return ""
642
+
643
+
644
+ def _asr_ffmpeg_whisper(wav, *, model_path, lang="zh", log=print) -> dict:
645
+ """用 ffmpeg 的 whisper 滤镜转写(后备引擎,实测可用但比 faster-whisper 慢约 20 倍)。
646
+
647
+ 两个实测踩过的坑,代码里必须处理:
648
+ 1. **Windows 盘符冒号**:滤镜参数是 key=value:key=value,路径里的 "C:" 会被当成
649
+ 参数分隔符。各种反斜杠转义写法在 ffmpeg 9.0.2 上实测直接报
650
+ "No option name near ..."(滤镜解析器不认 C: 这种转义)。最稳的办法是
651
+ 在一个临时目录里跑,模型和音频都用**相对文件名**,destination 也写相对名。
652
+ 2. **输出格式**:format=json 时 destination 文件是**每行一个 JSON 对象**(不是一个大
653
+ 数组),start/end 单位是**毫秒整数**,且按 3 s 定长窗口切块、窗口之间会重叠。
654
+ 这里逐行解析,遇到解析不了的行直接跳过并计数(不静默:返回 parse_skipped)。
655
+ """
656
+ ff = _ffmpeg()
657
+ if not ff:
658
+ raise RuntimeError("找不到可用的 ffmpeg")
659
+ if not _ffmpeg_has_whisper():
660
+ raise RuntimeError("本机 ffmpeg 没有 whisper 滤镜(需要 --enable-whisper 的构建)")
661
+ model_path = os.path.abspath(str(model_path))
662
+ tmp = tempfile.mkdtemp(prefix="qaasr_ff_")
663
+ try:
664
+ # 模型与音频用相对名放临时目录:只为了让滤镜参数里没有盘符冒号
665
+ target_model = os.path.join(tmp, "model_ggml.bin")
666
+ try:
667
+ os.link(model_path, target_model)
668
+ except OSError:
669
+ shutil.copyfile(model_path, target_model)
670
+ local_wav = os.path.join(tmp, "in.wav")
671
+ shutil.copyfile(str(wav), local_wav)
672
+ dest = "asr_out.json"
673
+ af = ("whisper=model=model_ggml.bin:language=%s:translate=0:use_gpu=0:format=json:"
674
+ "destination=%s" % (str(lang or "zh"), dest))
675
+ t0 = time.time()
676
+ p = subprocess.run([ff, "-v", "info", "-nostats", "-nostdin", "-y", "-i", "in.wav",
677
+ "-af", af, "-f", "null", "-"],
678
+ cwd=tmp, capture_output=True, timeout=3600)
679
+ wall = time.time() - t0
680
+ out_path = os.path.join(tmp, dest)
681
+ if p.returncode != 0 or not os.path.exists(out_path):
682
+ raise RuntimeError("ffmpeg whisper 滤镜失败(rc=%s):%s"
683
+ % (p.returncode,
684
+ (p.stderr or b"").decode("utf-8", "replace")[-600:]))
685
+ segs, bad = [], 0
686
+ with open(out_path, encoding="utf-8", errors="replace") as fp:
687
+ for line in fp:
688
+ line = line.strip()
689
+ if not line:
690
+ continue
691
+ try:
692
+ row = json.loads(line)
693
+ a = float(row.get("start")) / 1000.0 # 毫秒 -> 秒
694
+ b = float(row.get("end")) / 1000.0
695
+ txt = str(row.get("text") or "").strip()
696
+ except Exception: # noqa: BLE001
697
+ bad += 1
698
+ continue
699
+ if math.isfinite(a) and math.isfinite(b) and txt:
700
+ segs.append({"start": round(a, 3), "end": round(b, 3), "text": txt})
701
+ audio_s = None
702
+ try:
703
+ with wave.open(local_wav, "rb") as w:
704
+ audio_s = round(w.getnframes() / float(w.getframerate()), 3)
705
+ except Exception: # noqa: BLE001
706
+ pass
707
+ return {"engine": ENGINE_FF, "model": os.path.basename(model_path), "segments": segs,
708
+ "load_s": None, "asr_s": round(wall, 2), "audio_s": audio_s,
709
+ "model_bytes": os.path.getsize(model_path), "lang": str(lang or "zh"),
710
+ "lang_prob": None, "parse_skipped": bad,
711
+ "detail": "ffmpeg whisper 滤镜(ggml %s,use_gpu=0)"
712
+ % os.path.basename(model_path)}
713
+ finally:
714
+ shutil.rmtree(tmp, ignore_errors=True)
715
+
716
+
717
+ # ---------------------------------------------------------------- 片段 -> 段落归属
718
+ def _sentence_windows(scenes) -> list:
719
+ """每段的"语音窗口"(句子级别):voice_start..voice_end,缺省 start..end。"""
720
+ out = []
721
+ for sc in scenes:
722
+ a = float(sc.get("voice_start") if sc.get("voice_start") is not None else sc["start"])
723
+ b = float(sc.get("voice_end") if sc.get("voice_end") is not None else sc["end"])
724
+ if b <= a:
725
+ a, b = float(sc["start"]), float(sc["end"])
726
+ out.append((a, b))
727
+ return out
728
+
729
+
730
+ def _assign_text(scenes, segments) -> list:
731
+ """把 ASR 片段按时间窗归属到 timeline 段落,返回每段 {"text","first_t","anchor"}。
732
+
733
+ 归属规则(**按音节在语音窗口里的占比裁文本**,不是"整段归给重叠最多的段"):
734
+ whisper 的 segment 边界很粗(faster-whisper 按语音活动切、ffmpeg whisper 干脆 3 s
735
+ 定长切),一个片段常常横跨两句旁白。整段归给最大重叠段会把上一句的尾字算进下一句,
736
+ 既污染召回率也污染偏移。这里假设"一个片段内的字在它的语音区间里匀速说出",
737
+ 把片段语音区间与该片段所有重叠窗口按时间铺开,字符按比例分给各段:
738
+
739
+ T = Σ 重叠时长;第 j 个字(共 n 个)的虚拟位置 = j / n × T;
740
+ 落在第 i 段窗口内那一段位置的字符就归第 i 段。
741
+
742
+ 这是"没有强制对齐信息"时最合理的近似,也是本项目能接受的口径——
743
+ 召回率对少量边界错配不敏感(丢一个字 = 掉 1/n),而偏移另由 first_t 单独估。
744
+ first_t 是该段 ASR 首字的估算时刻(窗口起点 + 该字在窗口内的偏移)。
745
+ anchor 记录来源片段下标(-1 = 没有重叠片段)。
746
+ """
747
+ n = len(scenes)
748
+ acc = [{"text": "", "first_t": None, "anchor": -1} for _ in range(n)]
749
+ if not segments:
750
+ return acc
751
+ wins = _sentence_windows(scenes)
752
+ for si, seg in enumerate(segments):
753
+ try:
754
+ a, b = float(seg["start"]), float(seg["end"])
755
+ except (KeyError, TypeError, ValueError):
756
+ continue
757
+ if not (math.isfinite(a) and math.isfinite(b)) or b <= a:
758
+ continue
759
+ chars = [ch for ch in str(seg.get("text") or "") if not ch.isspace()]
760
+ if not chars:
761
+ continue
762
+ # 词级首字时刻(faster-whisper 给),没有就退回按比例估(ffmpeg whisper 没有词级)
763
+ try:
764
+ first_word_t = float(seg.get("first_word_t"))
765
+ if not math.isfinite(first_word_t):
766
+ first_word_t = None
767
+ except (TypeError, ValueError):
768
+ first_word_t = None
769
+ overlaps = []
770
+ for i, (wa, wb) in enumerate(wins):
771
+ lo, hi = max(a, wa), min(b, wb)
772
+ if hi - lo > 1e-6:
773
+ overlaps.append((i, lo, hi, hi - lo))
774
+ if not overlaps:
775
+ continue
776
+ overlaps.sort(key=lambda r: r[1])
777
+ total = sum(r[3] for r in overlaps)
778
+ cnt = len(chars)
779
+ cum = 0.0
780
+ for i, lo, hi, dur in overlaps:
781
+ before = cum # 本窗口之前的累计重叠时长
782
+ c0 = int(round(before / total * cnt)) if total > 0 else 0
783
+ cum += dur
784
+ c1 = int(round(cum / total * cnt)) if total > 0 else cnt
785
+ c0 = max(0, min(c0, cnt))
786
+ c1 = max(c0, min(c1, cnt))
787
+ piece = "".join(chars[c0:c1])
788
+ if not piece:
789
+ continue
790
+ acc[i]["text"] += piece
791
+ if acc[i]["anchor"] >= 0:
792
+ continue
793
+ # 首字时刻:优先用词级时间戳(真实对齐);没有就按"字在窗口内匀速"估
794
+ off = (c0 / float(cnt)) * total - before if total > 0 else 0.0
795
+ off = max(0.0, min(float(dur), off))
796
+ if first_word_t is not None and c0 == 0:
797
+ acc[i]["first_t"] = first_word_t
798
+ else:
799
+ acc[i]["first_t"] = lo + off
800
+ acc[i]["anchor"] = si
801
+ return acc
802
+
803
+
804
+ def _judge(scenes, segments, *, recall_pass=QA_RECALL_PASS, recall_warn=QA_RECALL_WARN,
805
+ offset_warn=OFFSET_WARN_S) -> dict:
806
+ """纯函数:由段落 + ASR 片段算出召回率 / 缺句 / 偏移 / 三档结论。
807
+
808
+ 单独拎出来是为了 --selftest 和 --asr-json 离线回放能**不加载模型**验证判分逻辑:
809
+ 引擎只是"产生片段"的手段,判分对错必须和引擎解耦,否则没网/没模型时门禁就没法测。
810
+ """
811
+ recall_pass = float(recall_pass)
812
+ recall_warn = float(recall_warn)
813
+ if not (0.0 < recall_warn <= 1.0) or not (0.0 < recall_pass <= 1.0):
814
+ raise ValueError("recall_pass / recall_warn 必须落在 (0, 1],收到 %r / %r"
815
+ % (recall_pass, recall_warn))
816
+ if recall_warn > recall_pass:
817
+ raise ValueError("recall_warn(%.3f) 不能大于 recall_pass(%.3f)"
818
+ % (recall_warn, recall_pass))
819
+ from collections import Counter
820
+
821
+ assigned = _assign_text(scenes, segments)
822
+ per_line, missing, offsets, offset_ref = [], [], {}, {}
823
+ tot_ref, tot_hit = 0, 0
824
+ for sc, got in zip(scenes, assigned):
825
+ ref = _norm_text(sc.get("text"))
826
+ hyp = _norm_text(got["text"])
827
+ hit = sum((Counter(ref) & Counter(hyp)).values()) if ref else 0
828
+ r = (hit / float(len(ref))) if ref else 1.0
829
+ tot_ref += len(ref)
830
+ tot_hit += min(hit, len(ref))
831
+ per_line.append({
832
+ "id": sc["id"], "start": round(float(sc["start"]), 3),
833
+ "end": round(float(sc["end"]), 3), "ref_chars": len(ref),
834
+ "asr_chars": len(hyp), "hit_chars": hit, "recall": round(r, 4),
835
+ "asr_text": got["text"][:200],
836
+ })
837
+ if ref and r < MISS_LINE_RECALL:
838
+ missing.append(sc["id"])
839
+ if got["first_t"] is not None and ref:
840
+ vs = sc.get("voice_start")
841
+ expected = float(vs if vs is not None else sc["start"])
842
+ offsets[sc["id"]] = round(float(got["first_t"]) - expected, 3)
843
+ # 期望起点用的是哪个字段要如实说:voice_start 缺省(或等于 start)时就是 start
844
+ offset_ref[sc["id"]] = ("voice_start"
845
+ if vs is not None and abs(float(vs) - float(sc["start"])) > 1e-9
846
+ else "start")
847
+ recall = (tot_hit / float(tot_ref)) if tot_ref else 0.0
848
+ warnings, errors = [], []
849
+
850
+ for lid in missing:
851
+ row = next((p for p in per_line if p["id"] == lid), None)
852
+ if row is not None:
853
+ warnings.append("段落 %s 的字符召回只有 %.2f(%d/%d),疑似丢句/错词:ASR=%r"
854
+ % (lid, row["recall"], row["hit_chars"], row["ref_chars"],
855
+ row["asr_text"][:60]))
856
+ for lid, dt in sorted(offsets.items()):
857
+ if abs(float(dt)) > float(offset_warn) + 1e-9:
858
+ warnings.append("段落 %s 的旁白首字比 timeline 的 %s 偏 %+.2fs(阈值 ±%.2fs):"
859
+ "音画/字幕可能对不齐(通常是 TTS 缓存串了或混音起点变了)"
860
+ % (lid, offset_ref.get(lid, "voice_start"), float(dt),
861
+ float(offset_warn)))
862
+
863
+ if recall < recall_warn - 1e-12:
864
+ errors.append("ASR 回读字符召回 %.3f 低于 fail 线 %.2f(原稿 %d 字,对上 %d 字):"
865
+ "成片旁白与剧本不一致——先查 TTS 缓存 / 混音是否丢了句子"
866
+ % (recall, recall_warn, tot_ref, tot_hit))
867
+ status = "fail"
868
+ elif recall < recall_pass - 1e-12:
869
+ warnings.append("ASR 回读字符召回 %.3f 低于 pass 线 %.2f(但高于 fail 线 %.2f):"
870
+ "pass 但记 warning,别当完全没问题" % (recall, recall_pass, recall_warn))
871
+ status = "pass"
872
+ else:
873
+ status = "pass"
874
+
875
+ detail = "字符召回 %.3f(%d/%d 字,%d 段,缺句 %d 段)" % (
876
+ recall, tot_hit, tot_ref, len(per_line), len(missing))
877
+ if offsets:
878
+ worst = max(offsets.items(), key=lambda kv: abs(float(kv[1])))
879
+ detail += ";最大偏移 %s %+.2fs" % (worst[0], float(worst[1]))
880
+ return {
881
+ "recall": round(recall, 4), "ref_chars": tot_ref, "hit_chars": tot_hit,
882
+ "per_line": per_line, "missing_lines": missing,
883
+ "offsets": offsets, "offset_ref": offset_ref, "offset_warn_s": float(offset_warn),
884
+ "gate": {"id": "asr", "title": "ASR 回读(原稿 vs 成片旁白)",
885
+ "status": status, "detail": detail},
886
+ "warnings": warnings, "errors": errors,
887
+ }
888
+
889
+
890
+ # ---------------------------------------------------------------- 报告
891
+ def _write_json(rep, out_json) -> None:
892
+ """写报告 JSON(allow_nan=False:报告里出现 NaN 说明有算法 bug,宁可当场炸)。"""
893
+ out_json = str(out_json)
894
+ d = os.path.dirname(os.path.abspath(out_json))
895
+ if d:
896
+ os.makedirs(d, exist_ok=True)
897
+ txt = json.dumps(rep, ensure_ascii=False, indent=1, allow_nan=False)
898
+ with open(out_json, "w", encoding="utf-8") as fp:
899
+ fp.write(txt + "\n")
900
+
901
+
902
+ def report(film_mp4, tl, script=None, *, model_size="small", lang="zh", out_json=None,
903
+ recall_pass=QA_RECALL_PASS, recall_warn=QA_RECALL_WARN, log=print,
904
+ offset_warn=OFFSET_WARN_S,
905
+ limit_seconds=None, engine="auto", asr_json=None, whisper_model=None,
906
+ vad=False) -> dict:
907
+ """成片旁白的 ASR 回读校验,返回可直接进门禁的 dict。
908
+
909
+ 参数
910
+ film_mp4 成片路径(有音轨即可,视频轨不参与)
911
+ tl timeline:路径或 dict,需有 scenes[{id,start,end,(voice_start,voice_end,text)}]
912
+ script 可选剧本(film.json 或其 dict):给了就优先用 lines[].text 作为原稿
913
+ model_size faster-whisper 模型(tiny/base/small/medium/large-v3;默认 small)
914
+ lang ASR 语言(默认 zh)
915
+ out_json 报告 JSON 输出路径(可选)
916
+ recall_pass >= 它 -> pass(默认 0.85,与 render.py 的 G12 同口径)
917
+ recall_warn < 它 -> fail;介于两者 -> pass + warning(默认 0.72)
918
+ offset_warn 段落 ASR 首字相对 timeline 语音起点的偏移告警阈值(默认 0.35 s)
919
+ limit_seconds 只对前 N 秒音轨做 ASR(自测提速用;timeline 会被同步裁剪)
920
+ engine "auto"(默认,faster-whisper 优先)/ "faster-whisper" / "ffmpeg"
921
+ asr_json 已有 ASR 结果(路径或 dict,形如 {"segments":[{start,end,text}]}):
922
+ 给了就直接回放,不加载模型、不抽音频——CI / 自测 / 排查用
923
+ whisper_model ffmpeg whisper 滤镜用的 ggml 模型路径(缺省按 _default_whisper_models 探测)
924
+ vad 是否开 faster-whisper 的 VAD(默认关:静音段幻觉文本不影响字符召回,
925
+ 而开 VAD 多一个模型要下载,破坏"零网络也能跑")
926
+
927
+ 返回字段:judged / engine / recall / per_line / missing_lines / offsets / offset_ref /
928
+ gate{status} / warnings / errors,以及耗时与模型信息。
929
+ """
930
+ t_start = time.time()
931
+ film = str(film_mp4)
932
+ rep = {
933
+ "schema": "qaasr/1", "version": __version__,
934
+ "film": film, "judged": False, "engine": None, "engine_detail": None,
935
+ "model_size": str(model_size), "model_bytes": None, "lang": str(lang or "zh"),
936
+ "audio_s": None, "limit_seconds": (float(limit_seconds) if limit_seconds else None),
937
+ "load_s": None, "asr_s": None, "elapsed_s": 0.0,
938
+ "recall": None, "ref_chars": 0, "hit_chars": 0,
939
+ "recall_pass": float(recall_pass), "recall_warn": float(recall_warn),
940
+ "offset_warn_s": float(offset_warn),
941
+ "per_line": [], "missing_lines": [], "offsets": {}, "offset_ref": {},
942
+ "asr_segments": [],
943
+ "gate": {"id": "asr", "title": "ASR 回读(原稿 vs 成片旁白)", "status": "skipped",
944
+ "detail": "未执行"},
945
+ "warnings": [], "errors": [],
946
+ }
947
+
948
+ def finish() -> dict:
949
+ rep["elapsed_s"] = round(time.time() - t_start, 2)
950
+ if out_json:
951
+ try:
952
+ _write_json(rep, out_json)
953
+ except Exception as e: # noqa: BLE001
954
+ rep["warnings"].append("报告 JSON 写出失败(%s):%s" % (out_json, e))
955
+ if log:
956
+ try:
957
+ _log_summary(rep, log)
958
+ except Exception: # noqa: BLE001
959
+ pass
960
+ return rep
961
+
962
+ # ---- 0. 前置条件:成片存在、timeline 可用 ----
963
+ if not os.path.exists(film):
964
+ rep["errors"].append("成片不存在:%s" % film)
965
+ rep["gate"]["detail"] = "成片不存在,ASR 回读没有执行"
966
+ rep["warnings"].append("ASR 回读**没有执行**,这不等于通过")
967
+ return finish()
968
+ try:
969
+ scenes = _scenes_of(tl, script)
970
+ except Exception as e: # noqa: BLE001
971
+ rep["errors"].append("时间轴不可用:%s" % e)
972
+ rep["gate"]["detail"] = "时间轴不可用,ASR 回读没有执行"
973
+ rep["warnings"].append("ASR 回读**没有执行**,这不等于通过")
974
+ return finish()
975
+ if limit_seconds:
976
+ lim = float(limit_seconds)
977
+ if lim <= 0:
978
+ raise ValueError("limit_seconds 必须 > 0(收到 %r)" % (limit_seconds,))
979
+ clipped = []
980
+ for sc in scenes:
981
+ if float(sc["start"]) >= lim - 1e-9:
982
+ continue
983
+ row = dict(sc)
984
+ row["end"] = min(float(sc["end"]), lim)
985
+ row["voice_end"] = min(float(sc["voice_end"]), lim)
986
+ if row["end"] - row["start"] <= 1e-6:
987
+ continue
988
+ clipped.append(row)
989
+ scenes = clipped
990
+ if not scenes:
991
+ rep["errors"].append("limit_seconds=%.3f 把时间轴裁空了" % lim)
992
+ rep["gate"]["detail"] = "时间轴被裁空,ASR 回读没有执行"
993
+ rep["warnings"].append("ASR 回读**没有执行**,这不等于通过")
994
+ return finish()
995
+ ref_chars_total = sum(len(_norm_text(sc.get("text"))) for sc in scenes)
996
+ if ref_chars_total <= 0:
997
+ rep["errors"].append("timeline/剧本里没有任何可对照的文本(去标点后为空)")
998
+ rep["gate"]["detail"] = "没有可对照文本,ASR 回读没有执行"
999
+ rep["warnings"].append("ASR 回读**没有执行**,这不等于通过")
1000
+ return finish()
1001
+
1002
+ # ---- 1. 取 ASR 结果:离线回放 > 指定引擎 > auto ----
1003
+ asr = None
1004
+ audio_info = _audio_info(film)
1005
+ if not audio_info.get("has_audio"):
1006
+ if asr_json is not None:
1007
+ # 回放只验判分链路,不碰音轨:必须说清楚,别让人以为"音轨查过了"
1008
+ rep["warnings"].append("离线回放模式不检查音轨:本片探测不到音轨,"
1009
+ "但判分用的是 --asr-json 里的片段(回放 != 成片有音轨)")
1010
+ elif not audio_info.get("probe_ok"):
1011
+ # 探测工具不可用时 _audio_info 也只能返回 has_audio=False,但不能把它写成
1012
+ # "成片没有音轨"——那是把环境缺件错误归因到片子上。档位同样是 skipped,
1013
+ # 区别只在原因必须写对(排障方向完全不同)。
1014
+ rep["errors"].append("既没有可用的 ffprobe / ffmpeg 探测音轨,"
1015
+ "也没有任何音轨信息:%s" % film)
1016
+ rep["gate"]["detail"] = "音轨探测工具不可用,ASR 回读没有执行(skipped != 通过)"
1017
+ rep["warnings"].append("ASR 回读**没有执行**(缺 ffprobe/ffmpeg),这不等于通过")
1018
+ return finish()
1019
+ else:
1020
+ rep["errors"].append("成片没有音轨:%s" % film)
1021
+ rep["gate"]["detail"] = "成片无音轨,ASR 回读没有执行(没有东西可转写)"
1022
+ rep["warnings"].append("ASR 回读**没有执行**(无音轨),这不等于通过")
1023
+ return finish()
1024
+ if asr_json is not None:
1025
+ data = _load_json_arg(asr_json, "asr_json")
1026
+ segs = data.get("segments") if isinstance(data, dict) else data
1027
+ if not isinstance(segs, list):
1028
+ rep["errors"].append("asr_json 里没有 segments 列表")
1029
+ rep["gate"]["detail"] = "asr_json 结构不对,ASR 回读没有执行"
1030
+ rep["warnings"].append("ASR 回读**没有执行**,这不等于通过")
1031
+ return finish()
1032
+ asr = {"engine": "replay", "model": None, "segments": [], "load_s": None,
1033
+ "asr_s": None, "audio_s": None, "model_bytes": None,
1034
+ "lang": str(lang or "zh"), "lang_prob": None,
1035
+ "detail": "回放已有 ASR 结果"}
1036
+ for s in segs:
1037
+ if not isinstance(s, dict):
1038
+ continue
1039
+ try:
1040
+ a, b = float(s.get("start")), float(s.get("end"))
1041
+ except (TypeError, ValueError):
1042
+ continue
1043
+ if math.isfinite(a) and math.isfinite(b) and b > a:
1044
+ seg = {"start": round(a, 3), "end": round(b, 3),
1045
+ "text": str(s.get("text") or "")}
1046
+ # 报告里的 asr_segments 带 first_word_t(词级首字时刻),回放时要原样带上,
1047
+ # 否则"把 qaasr 自己的报告回放一遍"会丢掉词级精度、偏移退回按比例估。
1048
+ try:
1049
+ fw = float(s.get("first_word_t"))
1050
+ if math.isfinite(fw):
1051
+ seg["first_word_t"] = round(fw, 3)
1052
+ except (TypeError, ValueError):
1053
+ pass
1054
+ asr["segments"].append(seg)
1055
+
1056
+ ff = _ffmpeg()
1057
+ if asr is None and not ff:
1058
+ rep["errors"].append(
1059
+ "找不到可用的 ffmpeg(FILMKIT_FFMPEG 与 PATH 里都没有能跑通 \"ffmpeg -version\" 的"
1060
+ "可执行文件),音轨抽不出来,ASR 回读没有执行")
1061
+ rep["gate"]["detail"] = "找不到可用的 ffmpeg,ASR 回读没有执行(skipped != 通过)"
1062
+ rep["warnings"].append("ASR 回读**没有执行**(缺 ffmpeg),这不等于通过")
1063
+ return finish()
1064
+
1065
+ if asr is None:
1066
+ fw_ok = _faster_whisper_available()
1067
+ use_fw = fw_ok and engine in ("auto", "faster-whisper", ENGINE_FW)
1068
+ whisper_path = _find_whisper_model(model_size, whisper_model)
1069
+ ff_whisper = _ffmpeg_has_whisper()
1070
+ use_ff = bool(whisper_path) and ff_whisper and engine in ("auto", "ffmpeg",
1071
+ ENGINE_FF)
1072
+ if not use_fw and not use_ff:
1073
+ why = []
1074
+ if not fw_ok:
1075
+ why.append("faster_whisper 不可用(%s)" % (_FW_IMPORT_ERR or "import 失败"))
1076
+ elif not use_fw:
1077
+ why.append("faster_whisper 可用但被 --engine 排除")
1078
+ if not ff_whisper:
1079
+ why.append("本机 ffmpeg 没有 whisper 滤镜")
1080
+ elif not whisper_path:
1081
+ why.append("找不到 ffmpeg whisper 用的 ggml 模型(用 --whisper-model 指定)")
1082
+ else:
1083
+ why.append("ffmpeg whisper 可用但被 --engine 排除")
1084
+ rep["errors"].append("两种 ASR 引擎都不可用:%s" % ";".join(why))
1085
+ rep["warnings"].append("ASR 回读**没有执行**(没有可用引擎),这不等于通过")
1086
+ rep["gate"]["detail"] = "没有可用的 ASR 引擎,回读没有执行(skipped != 通过)"
1087
+ return finish()
1088
+
1089
+ tmpdir = tempfile.mkdtemp(prefix="qaasr_")
1090
+ wav = os.path.join(tmpdir, "audio.wav")
1091
+ try:
1092
+ if log:
1093
+ log("== ASR 回读:%s" % os.path.basename(film))
1094
+ try:
1095
+ _extract_wav(film, wav, limit_seconds=limit_seconds)
1096
+ except Exception as e: # noqa: BLE001
1097
+ # 抽不出音轨(音轨为空/损坏、写盘失败、编码不支持)也是"没查成":
1098
+ # 三档语义里记 skipped + 原因。以前这里直接抛出去,render.py 的
1099
+ # catch-all 会把它记成 G12 **fail**——环境/媒资问题被说成"门禁自身失败",
1100
+ # 而且 report() 承诺"总是返回 dict"的契约也断了。
1101
+ rep["errors"].append("音轨抽取失败(%s: %s)" % (type(e).__name__, e))
1102
+ rep["gate"]["detail"] = "音轨抽取失败,ASR 回读没有执行(skipped != 通过)"
1103
+ rep["warnings"].append("ASR 回读**没有执行**(抽轨失败),这不等于通过")
1104
+ return finish()
1105
+ if use_fw:
1106
+ try:
1107
+ asr = _asr_faster_whisper(wav, model_size=model_size, lang=lang,
1108
+ vad=vad,
1109
+ log=log or (lambda *_a, **_k: None))
1110
+ except Exception as e: # noqa: BLE001
1111
+ msg = "faster-whisper 转写失败(%s: %s)" % (type(e).__name__, e)
1112
+ if use_ff:
1113
+ rep["warnings"].append(msg + ";已降级用 ffmpeg whisper 滤镜重试")
1114
+ if log:
1115
+ log(" [WARN] " + msg + ",降级 ffmpeg whisper…")
1116
+ else:
1117
+ rep["errors"].append(msg + ";且没有 ffmpeg whisper 的后备模型")
1118
+ rep["gate"]["detail"] = ("faster-whisper 模型加载/下载失败,"
1119
+ "ASR 回读没有完成(skipped)")
1120
+ rep["warnings"].append("ASR 回读**没有完成**,这不等于通过")
1121
+ return finish()
1122
+ if asr is None:
1123
+ try:
1124
+ asr = _asr_ffmpeg_whisper(wav, model_path=whisper_path, lang=lang, log=log)
1125
+ except Exception as e: # noqa: BLE001
1126
+ # 同抽轨失败:后备引擎跑挂了 = 本次没查成,记 skipped 而不是抛异常。
1127
+ rep["errors"].append("ffmpeg whisper 转写失败(%s: %s)"
1128
+ % (type(e).__name__, e))
1129
+ rep["gate"]["detail"] = "ffmpeg whisper 引擎失败,ASR 回读没有完成(skipped)"
1130
+ rep["warnings"].append("ASR 回读**没有完成**,这不等于通过")
1131
+ return finish()
1132
+ finally:
1133
+ shutil.rmtree(tmpdir, ignore_errors=True)
1134
+
1135
+ # ---- 2. 判分 ----
1136
+ rep["engine"] = asr.get("engine")
1137
+ rep["engine_detail"] = asr.get("detail")
1138
+ rep["asr_segments"] = asr.get("segments") or []
1139
+ rep["load_s"] = asr.get("load_s")
1140
+ rep["asr_s"] = asr.get("asr_s")
1141
+ rep["model_bytes"] = asr.get("model_bytes")
1142
+ if asr.get("audio_s"):
1143
+ rep["audio_s"] = asr.get("audio_s")
1144
+ else:
1145
+ rep["audio_s"] = audio_info.get("duration") # 前面探测过,不重复跑 ffprobe
1146
+ judged = _judge(scenes, rep["asr_segments"], recall_pass=recall_pass,
1147
+ recall_warn=recall_warn, offset_warn=offset_warn)
1148
+ for key in ("recall", "ref_chars", "hit_chars", "per_line", "missing_lines",
1149
+ "offsets", "offset_ref", "gate", "warnings", "errors"):
1150
+ rep[key] = judged[key]
1151
+ if asr.get("parse_skipped"):
1152
+ rep["warnings"].append("ffmpeg whisper 输出里有 %d 行解析不了(格式变了?已跳过)"
1153
+ % asr["parse_skipped"])
1154
+ if not rep["asr_segments"]:
1155
+ rep["errors"].append("ASR 没有转出任何文字(音轨可能整段静音/只有音乐):"
1156
+ "剧本有 %d 个字要对上,召回按 0 处理" % rep["ref_chars"])
1157
+ rep["gate"]["status"] = "fail"
1158
+ rep["gate"]["detail"] = "ASR 输出为空(召回 0),旁白可能整段丢了"
1159
+ rep["judged"] = True
1160
+ return finish()
1161
+
1162
+
1163
+ def _log_summary(rep, log=print) -> None:
1164
+ """把报告打印成人能读的中文摘要(风格与 qa_frames 的 _print_summary 一致)。"""
1165
+ g = rep.get("gate") or {}
1166
+ st = g.get("status")
1167
+ tag = {"pass": "[PASS]", "fail": "[FAIL]", "skipped": "[SKIP]"}.get(st, "[??]")
1168
+ log("%s %s — %s" % (tag, g.get("title", "ASR 回读"), g.get("detail", "")))
1169
+ if rep.get("engine"):
1170
+ parts = ["引擎 %s(%s)" % (rep.get("engine"), rep.get("engine_detail") or "")]
1171
+ if rep.get("audio_s"):
1172
+ parts.append("音频 %.2fs" % float(rep["audio_s"]))
1173
+ if rep.get("asr_s") is not None:
1174
+ parts.append("转写 %.2fs" % float(rep["asr_s"]))
1175
+ if rep.get("load_s") is not None:
1176
+ parts.append("加载模型 %.2fs" % float(rep["load_s"]))
1177
+ log(" " + ",".join(parts))
1178
+ if rep.get("recall") is not None:
1179
+ log(" 字符召回 %.3f(pass 线 %.2f / fail 线 %.2f),缺句 %d 段"
1180
+ % (float(rep["recall"]), float(rep["recall_pass"]),
1181
+ float(rep["recall_warn"]), len(rep.get("missing_lines") or [])))
1182
+ for e in (rep.get("errors") or [])[:6]:
1183
+ log(" [ERROR] %s" % e)
1184
+ for w in (rep.get("warnings") or [])[:6]:
1185
+ log(" [WARN] %s" % w)
1186
+
1187
+
1188
+ # ---------------------------------------------------------------- 自检(不联网)
1189
+ def _make_tiny_mp4(path, *, seconds=2.0, with_audio=True) -> bool:
1190
+ """造一个极小的 mp4(lavfi color + anullsrc),自检用;造不出来返回 False。
1191
+
1192
+ 不联网、不需要素材:只验证"有音轨/无音轨"两条路径的探测与抽轨。
1193
+ """
1194
+ ff = _ffmpeg()
1195
+ if not ff:
1196
+ return False
1197
+ cmd = [ff, "-y", "-v", "error", "-f", "lavfi", "-i",
1198
+ "color=c=black:s=64x36:r=10:d=%.2f" % seconds]
1199
+ if with_audio:
1200
+ cmd += ["-f", "lavfi", "-i", "anullsrc=r=16000:cl=mono:d=%.2f" % seconds,
1201
+ "-c:a", "aac", "-b:a", "32k", "-shortest"]
1202
+ cmd += ["-c:v", "libx264", "-preset", "ultrafast", "-pix_fmt", "yuv420p", str(path)]
1203
+ try:
1204
+ p = subprocess.run(cmd, capture_output=True, timeout=120)
1205
+ return p.returncode == 0 and os.path.exists(path)
1206
+ except Exception: # noqa: BLE001
1207
+ return False
1208
+
1209
+
1210
+ def selftest() -> int:
1211
+ """内置自检:**不联网、不加载 ASR 模型**(用 --asr-json 离线回放路径测判分)。
1212
+
1213
+ 覆盖:文本归一化、字符召回、三档判定边界、偏移告警、跨句片段归属、
1214
+ 引擎全不可用时 skipped、无音轨 skipped、CLI 端到端、import 无副作用。
1215
+ """
1216
+ results = []
1217
+ t_start = time.perf_counter()
1218
+ tmp = tempfile.mkdtemp(prefix="qaasr_st_")
1219
+
1220
+ def check(name, fn):
1221
+ try:
1222
+ ok, detail = fn()
1223
+ except Exception as e: # noqa: BLE001
1224
+ ok, detail = False, "%s: %s" % (type(e).__name__, e)
1225
+ st = "SKIP" if ok is None else ("PASS" if ok else "FAIL")
1226
+ results.append((name, st, detail))
1227
+ print(" [%s] %-36s %s" % (st, name, detail))
1228
+
1229
+ scenes = [
1230
+ {"id": "l01", "start": 0.0, "end": 7.0, "voice_start": 0.55, "voice_end": 6.2,
1231
+ "text": "人工智能正在改变视频创作,从写稿到配音都能自动完成。"},
1232
+ {"id": "l02", "start": 7.0, "end": 13.5, "voice_start": 7.55, "voice_end": 12.7,
1233
+ "text": "但自动化不等于放弃检查,成片仍然需要逐句回读。"},
1234
+ {"id": "l03", "start": 13.5, "end": 19.9, "voice_start": 14.05, "voice_end": 19.0,
1235
+ "text": "只有时间轴对得上,字幕和旁白才不会是两张皮。"},
1236
+ ]
1237
+ good_segs = [
1238
+ {"start": 0.37, "end": 5.19,
1239
+ "text": "人工智能正在改变视频创作,从写稿到配音都能自动完成。"},
1240
+ {"start": 7.58, "end": 11.82, "text": "但自动化不等于放弃检查,成片仍然需要逐句回读。"},
1241
+ {"start": 14.17, "end": 18.13, "text": "只有时间轴对得上,字幕和旁白才不会是两张皮。"},
1242
+ ]
1243
+
1244
+ def c_norm():
1245
+ # 用 whisper tiny 的真实输出(实测 .team/asr_fixture 那次):"視頻創作/從寫到配音/自動"
1246
+ a = _norm_text("人工智能正在改变視頻創作,从寫到配音都能自動完成。")
1247
+ b = _norm_text("人工智能正在改变视频创作,从写到配音都能自动完成")
1248
+ if a != b:
1249
+ return False, "归一化不一致:%r vs %r" % (a, b)
1250
+ if _norm_text("2024年 第3季度") != _norm_text("二零二四年第三季度"):
1251
+ return False, "数字写法归一不一致:%r vs %r" % (
1252
+ _norm_text("2024年 第3季度"), _norm_text("二零二四年第三季度"))
1253
+ if _norm_text(",。! ——") != "":
1254
+ return False, "纯标点没归零:%r" % _norm_text(",。! ——")
1255
+ if _norm_text("一起") != "一起":
1256
+ return False, "单个'一'被误当数字:%r" % _norm_text("一起")
1257
+ return True, "繁简/标点/数字/单字数字都正确"
1258
+
1259
+ def c_recall():
1260
+ if abs(_char_recall("你好世界", "你好世界") - 1.0) > 1e-9:
1261
+ return False, "完全一致应为 1.0"
1262
+ if abs(_char_recall("你好世界", "你好") - 0.5) > 1e-9:
1263
+ return False, "丢掉一半应为 0.5,实际 %r" % _char_recall("你好世界", "你好")
1264
+ if abs(_char_recall("是的是的", "是的") - 0.5) > 1e-9:
1265
+ return False, "多重集重复字应算 0.5,实际 %r" % _char_recall("是的是的", "是的")
1266
+ if abs(_char_recall("", "x") - 1.0) > 1e-9:
1267
+ return False, "空参考应为 1.0(由调用方判 skipped)"
1268
+ return True, "1.0 / 0.5 / 重复字 / 空参考都对"
1269
+
1270
+ def c_assign():
1271
+ # 一个横跨 l01 尾与 l02 头的片段:按语音窗口比例裁切,两段都该拿到字
1272
+ segs = [{"start": 6.0, "end": 8.0, "text": "自动完成但自动化"}]
1273
+ got = _assign_text(scenes, segs)
1274
+ if not got[0]["text"] or not got[1]["text"]:
1275
+ return False, "跨句片段没有分给两段:%r / %r" % (got[0]["text"], got[1]["text"])
1276
+ # 语音窗口:l01 到 6.2(重叠 0.2 s)、l02 从 7.55 开始(重叠 0.45 s),
1277
+ # 8 个字按 0.2:0.45 分 -> 2 个字给 l01、6 个字给 l02
1278
+ if got[0]["text"] != "自动" or got[1]["text"] != "完成但自动化":
1279
+ return False, "跨句裁切比例不对:%r / %r(预期 '自动' / '完成但自动化')" % (
1280
+ got[0]["text"], got[1]["text"])
1281
+ if got[0]["first_t"] is None or got[1]["first_t"] is None:
1282
+ return False, "跨句片段的 first_t 没算出来"
1283
+ return True, "跨句片段切成 %r / %r" % (got[0]["text"], got[1]["text"])
1284
+
1285
+ def c_pass():
1286
+ j = _judge(scenes, good_segs)
1287
+ if j["gate"]["status"] != "pass":
1288
+ return False, "完美召回却判 %s:%s" % (j["gate"]["status"], j["errors"])
1289
+ if j["recall"] < 0.99:
1290
+ return False, "召回 %.3f 偏低" % j["recall"]
1291
+ if j["missing_lines"]:
1292
+ return False, "误报缺句:%s" % (j["missing_lines"],)
1293
+ if any("首字" in w for w in j["warnings"]):
1294
+ return False, "正常偏移被误报:%s" % j["warnings"]
1295
+ return True, "召回 %.3f,pass,无缺句/偏移告警" % j["recall"]
1296
+
1297
+ def c_warn():
1298
+ # 少数几个字的写法差异 -> 召回落在 [warn, pass) 区间(pass 线刻意设成 1.0)
1299
+ segs = [
1300
+ {"start": 0.37, "end": 5.19, "text": "人工智能正在改变视频创作,从写稿到配音都能自动"},
1301
+ {"start": 7.58, "end": 11.82, "text": "但自动化不等于放弃检查,成片仍然需要逐句回读。"},
1302
+ {"start": 14.17, "end": 18.13, "text": "只有时间轴对得上,字幕和旁白才不会是两张皮。"},
1303
+ ]
1304
+ j = _judge(scenes, segs, recall_pass=1.0, recall_warn=0.80)
1305
+ if j["gate"]["status"] != "pass" or not any("低于 pass 线" in w for w in j["warnings"]):
1306
+ return False, "介于两线之间应 pass + warning,实际 %s / %s" % (
1307
+ j["gate"]["status"], j["warnings"])
1308
+ return True, "召回 %.3f -> pass + %d 条 warning" % (j["recall"], len(j["warnings"]))
1309
+
1310
+ def c_fail():
1311
+ segs = [good_segs[0],
1312
+ {"start": 7.58, "end": 11.82, "text": "但自动化不等于放弃检查"}]
1313
+ j = _judge(scenes, segs)
1314
+ if j["gate"]["status"] != "fail":
1315
+ return False, "丢一句应 fail,实际 %s(recall=%.3f)" % (
1316
+ j["gate"]["status"], j["recall"])
1317
+ if "l03" not in j["missing_lines"]:
1318
+ return False, "丢句没进 missing_lines:%s" % (j["missing_lines"],)
1319
+ if not any("低于 fail 线" in e for e in j["errors"]):
1320
+ return False, "fail 没写 errors:%s" % (j["errors"],)
1321
+ return True, "丢一句 -> fail(recall %.3f),缺句 %s" % (j["recall"], j["missing_lines"])
1322
+
1323
+ def c_offset():
1324
+ segs = [dict(s) for s in good_segs]
1325
+ segs[1]["start"] += 0.6 # l02 整体晚 0.6 s
1326
+ segs[1]["end"] += 0.6
1327
+ j = _judge(scenes, segs)
1328
+ dt = j["offsets"].get("l02")
1329
+ if dt is None or abs(dt - 0.63) > 0.12:
1330
+ return False, "偏移算错:%r(预期约 +0.63)" % (dt,)
1331
+ if not any("l02" in w and "偏" in w for w in j["warnings"]):
1332
+ return False, "偏移超阈没告警:%s" % (j["warnings"],)
1333
+ if j["gate"]["status"] != "pass":
1334
+ return False, "偏移只该 warning,不该改判定(%s)" % j["gate"]["status"]
1335
+ return True, "l02 偏移 %+.2fs 记 warning,判定仍 pass" % dt
1336
+
1337
+ def c_no_film():
1338
+ rep = report(os.path.join(tmp, "nope.mp4"), scenes, log=None)
1339
+ if rep["gate"]["status"] != "skipped":
1340
+ return False, "成片不存在应 skipped,实际 %s" % rep["gate"]["status"]
1341
+ if not rep["errors"]:
1342
+ return False, "skipped 却没写 errors"
1343
+ if not any("没有执行" in w for w in rep["warnings"]):
1344
+ return False, "skipped 没给'没查不等于通过'的 warning"
1345
+ return True, "成片不存在 -> skipped + errors + warning"
1346
+
1347
+ def c_no_engine():
1348
+ """两个引擎都不可用时 skipped,且不能冒充 pass。"""
1349
+ old = (_FW_IMPORT, _FW_IMPORT_ERR)
1350
+ try:
1351
+ globals()["_FW_IMPORT"] = False
1352
+ globals()["_FW_IMPORT_ERR"] = "selftest 注入:import 失败"
1353
+ mp4 = os.path.join(tmp, "with_audio.mp4")
1354
+ if not _make_tiny_mp4(mp4):
1355
+ return None, "没有 ffmpeg,造不出测试 mp4"
1356
+ rep = report(mp4, scenes, engine="faster-whisper", log=None,
1357
+ whisper_model=os.path.join(tmp, "no_model.bin"))
1358
+ if rep["gate"]["status"] != "skipped":
1359
+ return False, "引擎不可用应 skipped,实际 %s" % rep["gate"]["status"]
1360
+ if not rep["errors"]:
1361
+ return False, "引擎不可用没写 errors"
1362
+ if rep["judged"] is not False:
1363
+ return False, "skipped 时 judged 应为 False(没查不算查过)"
1364
+ return True, "引擎不可用 -> skipped(不冒充 pass),原因已写明"
1365
+ finally:
1366
+ globals()["_FW_IMPORT"], globals()["_FW_IMPORT_ERR"] = old
1367
+
1368
+ def c_no_audio():
1369
+ mp4 = os.path.join(tmp, "no_audio.mp4")
1370
+ if not _make_tiny_mp4(mp4, with_audio=False):
1371
+ return None, "没有 ffmpeg,造不出无音轨 mp4"
1372
+ # 不传 asr_json:必须走真实引擎分支,验证"无音轨 -> skipped"
1373
+ rep = report(mp4, scenes, log=None)
1374
+ if rep["gate"]["status"] != "skipped":
1375
+ return False, "无音轨应 skipped,实际 %s" % rep["gate"]["status"]
1376
+ if not any("音轨" in e for e in rep["errors"]):
1377
+ return False, "无音轨原因没写进 errors:%s" % (rep["errors"],)
1378
+ return True, "无音轨 -> skipped + 说明"
1379
+
1380
+ def c_replay():
1381
+ mp4 = os.path.join(tmp, "audio_only.mp4")
1382
+ if not _make_tiny_mp4(mp4):
1383
+ return None, "没有 ffmpeg,造不出测试 mp4"
1384
+ outj = os.path.join(tmp, "asr.json")
1385
+ rep = report(mp4, scenes, asr_json={"segments": good_segs}, out_json=outj, log=None)
1386
+ if rep["gate"]["status"] != "pass" or rep["engine"] != "replay":
1387
+ return False, "离线回放判分错:%s / %s" % (rep["gate"]["status"], rep["engine"])
1388
+ if not (os.path.exists(outj) and os.path.getsize(outj) > 200):
1389
+ return False, "报告 JSON 没写出来"
1390
+ data = json.load(open(outj, encoding="utf-8"))
1391
+ need = {"judged", "engine", "recall", "per_line", "missing_lines", "offsets",
1392
+ "gate", "warnings", "errors"}
1393
+ lack = need - set(data)
1394
+ if lack:
1395
+ return False, "报告缺字段:%s" % sorted(lack)
1396
+ if data["gate"]["status"] not in ("pass", "fail", "skipped"):
1397
+ return False, "gate.status 非法:%r" % data["gate"]["status"]
1398
+ return True, "离线回放 pass,JSON 字段齐全(%d 字节)" % os.path.getsize(outj)
1399
+
1400
+ def c_cli():
1401
+ mp4 = os.path.join(tmp, "cli.mp4")
1402
+ if not _make_tiny_mp4(mp4):
1403
+ return None, "没有 ffmpeg,造不出测试 mp4"
1404
+ tlj = os.path.join(tmp, "timeline.json")
1405
+ json.dump({"total": 19.9, "scenes": scenes}, open(tlj, "w", encoding="utf-8"),
1406
+ ensure_ascii=False)
1407
+ asrj = os.path.join(tmp, "replay.json")
1408
+ json.dump({"segments": good_segs}, open(asrj, "w", encoding="utf-8"),
1409
+ ensure_ascii=False)
1410
+ outj = os.path.join(tmp, "cli_asr.json")
1411
+ p = subprocess.run([sys.executable, os.path.join(HERE, "qaasr.py"), mp4,
1412
+ "--timeline", tlj, "--asr-json", asrj, "--out", outj],
1413
+ cwd=HERE, capture_output=True, timeout=180)
1414
+ out = (p.stdout or b"").decode("utf-8", "replace")
1415
+ if p.returncode != 0:
1416
+ return False, "pass 时应 exit 0,实际 %d;%s" % (
1417
+ p.returncode, out[-300:] + (p.stderr or b"").decode("utf-8", "replace")[-200:])
1418
+ if "字符召回" not in out:
1419
+ return False, "CLI 输出缺少中文摘要:%r" % out[:200]
1420
+ if not os.path.exists(outj):
1421
+ return False, "CLI 没写出 --out JSON"
1422
+ badj = os.path.join(tmp, "bad.json")
1423
+ json.dump({"segments": good_segs[:1]}, open(badj, "w", encoding="utf-8"),
1424
+ ensure_ascii=False)
1425
+ p2 = subprocess.run([sys.executable, os.path.join(HERE, "qaasr.py"), mp4,
1426
+ "--timeline", tlj, "--asr-json", badj],
1427
+ cwd=HERE, capture_output=True, timeout=180)
1428
+ if p2.returncode != 1:
1429
+ return False, "fail 时应 exit 1,实际 %d" % p2.returncode
1430
+ # 用法级错误(成片不存在)-> exit 2
1431
+ p3 = subprocess.run([sys.executable, os.path.join(HERE, "qaasr.py"),
1432
+ os.path.join(tmp, "missing.mp4"), "--timeline", tlj,
1433
+ "--asr-json", asrj], cwd=HERE, capture_output=True, timeout=180)
1434
+ if p3.returncode != 2:
1435
+ return False, "成片不存在应 exit 2,实际 %d" % p3.returncode
1436
+ return True, "pass -> 0;丢句 -> 1;用法错误(成片缺失)-> 2"
1437
+
1438
+ def c_import():
1439
+ p = subprocess.run([sys.executable, "-c", "import qaasr"], cwd=HERE,
1440
+ capture_output=True, timeout=120)
1441
+ if p.returncode != 0:
1442
+ return False, "import 失败:%s" % (
1443
+ (p.stderr or b"").decode("utf-8", "replace")[-300:])
1444
+ if (p.stdout or b"").strip():
1445
+ return False, "import 有打印:%r" % (p.stdout[:200],)
1446
+ return True, "干净退出,无打印"
1447
+
1448
+ try:
1449
+ print("== qaasr.py selftest(不联网、不加载 ASR 模型)")
1450
+ check("文本归一化(繁简/标点/数字)", c_norm)
1451
+ check("字符级多重集召回", c_recall)
1452
+ check("跨句片段按语音窗口裁切", c_assign)
1453
+ check("完美召回 -> pass", c_pass)
1454
+ check("介于两线之间 -> pass + warning", c_warn)
1455
+ check("丢一句 -> fail", c_fail)
1456
+ check("偏移超阈 -> warning 不改判定", c_offset)
1457
+ check("成片不存在 -> skipped", c_no_film)
1458
+ check("引擎不可用 -> skipped(不冒充 pass)", c_no_engine)
1459
+ check("无音轨 -> skipped", c_no_audio)
1460
+ check("离线回放 + 报告 JSON 字段", c_replay)
1461
+ check("CLI 端到端(退出码)", c_cli)
1462
+ check("import 无副作用", c_import)
1463
+ finally:
1464
+ shutil.rmtree(tmp, ignore_errors=True)
1465
+
1466
+ npass = sum(1 for _n, st, _d in results if st == "PASS")
1467
+ nfail = sum(1 for _n, st, _d in results if st == "FAIL")
1468
+ nskip = sum(1 for _n, st, _d in results if st == "SKIP")
1469
+ elapsed = time.perf_counter() - t_start
1470
+ print("== 结果:%d PASS / %d FAIL / %d SKIP,耗时 %.1f s"
1471
+ % (npass, nfail, nskip, elapsed))
1472
+ if nfail:
1473
+ return 1
1474
+ if nskip:
1475
+ print(" [WARN] 有 %d 项因环境缺件跳过(%s),跳过的项不算通过"
1476
+ % (nskip, "、".join(n for n, st, _d in results if st == "SKIP")))
1477
+ return 0
1478
+
1479
+
1480
+ # ---------------------------------------------------------------- CLI
1481
+ def _utf8_stdout() -> None:
1482
+ """被管道/文件重定向时切 UTF-8(与 render.py / qa_frames.py 同一口径)。"""
1483
+ for stream in (sys.stdout, sys.stderr):
1484
+ try:
1485
+ if stream is not None and not stream.isatty():
1486
+ stream.reconfigure(encoding="utf-8", errors="replace")
1487
+ except Exception: # noqa: BLE001
1488
+ pass
1489
+
1490
+
1491
+ def main(argv=None) -> int:
1492
+ ap = argparse.ArgumentParser(
1493
+ prog="qaasr.py",
1494
+ description="ASR 回读校验:把成片音轨转回文字,与 timeline/剧本做字符级召回与偏移检查"
1495
+ "(v0.8.0)")
1496
+ ap.add_argument("film", nargs="?", help="成片路径(有音轨即可,视频轨不参与)")
1497
+ ap.add_argument("--timeline", default=None,
1498
+ help="work/timeline.json(scenes[].id/start/end,可选 voice_start/voice_end/text)")
1499
+ ap.add_argument("--script", default=None, help="film.json(给了就用 lines[].text 当原稿)")
1500
+ ap.add_argument("--model", default="small", help="faster-whisper 模型(默认 small)")
1501
+ ap.add_argument("--lang", default="zh", help="ASR 语言(默认 zh)")
1502
+ ap.add_argument("--out", default=None, help="报告 JSON 输出路径")
1503
+ ap.add_argument("--recall-pass", type=float, default=QA_RECALL_PASS,
1504
+ help="pass 线(默认 %.2f,与 render.py 的 G12 门禁同一口径)"
1505
+ % QA_RECALL_PASS)
1506
+ ap.add_argument("--recall-warn", type=float, default=QA_RECALL_WARN,
1507
+ help="fail 线(默认 %.2f,与 render.py 的 G12 门禁同一口径)"
1508
+ % QA_RECALL_WARN)
1509
+ ap.add_argument("--offset-warn", type=float, default=OFFSET_WARN_S,
1510
+ help="首字偏移告警阈值秒(默认 %.2f)" % OFFSET_WARN_S)
1511
+ ap.add_argument("--limit-seconds", type=float, default=None,
1512
+ help="只对前 N 秒做 ASR(自测提速用,timeline 会同步裁剪)")
1513
+ ap.add_argument("--engine", default="auto",
1514
+ choices=["auto", "faster-whisper", "ffmpeg"], help="ASR 引擎(默认 auto)")
1515
+ ap.add_argument("--asr-json", default=None,
1516
+ help="已有 ASR 结果 JSON(离线回放,不加载模型/不抽音频)")
1517
+ ap.add_argument("--whisper-model", default=None,
1518
+ help="ffmpeg whisper 滤镜用的 ggml 模型路径")
1519
+ ap.add_argument("--vad", action="store_true",
1520
+ help="faster-whisper 开 VAD(会多要一个模型)")
1521
+ ap.add_argument("--strict", action="store_true",
1522
+ help="skipped 也算失败(与 render.py 的 --strict-gates 同义)")
1523
+ ap.add_argument("--selftest", action="store_true", help="跑内置自检(不联网,≤60 s)")
1524
+ a = ap.parse_args(argv)
1525
+ if a.selftest:
1526
+ return selftest()
1527
+ if not a.film or not a.timeline:
1528
+ ap.error("需要成片路径与 --timeline(或用 --selftest)")
1529
+ _utf8_stdout()
1530
+ # 用法级错误单独判(exit 2):路径打错和"环境缺件导致没查"不是一回事。
1531
+ # 后者由 report() 记 skipped + errors,默认 exit 0,--strict 才升级成失败,
1532
+ # 与 render.py 的 --strict-gates 语义保持同一口径。
1533
+ if not os.path.exists(str(a.film)):
1534
+ print("[FAIL] 成片不存在:%s" % a.film, file=sys.stderr)
1535
+ return 2
1536
+ if not os.path.exists(str(a.timeline)):
1537
+ print("[FAIL] timeline 不存在:%s" % a.timeline, file=sys.stderr)
1538
+ return 2
1539
+ try:
1540
+ rep = report(a.film, a.timeline, a.script, model_size=a.model, lang=a.lang,
1541
+ out_json=a.out, recall_pass=a.recall_pass, recall_warn=a.recall_warn,
1542
+ offset_warn=a.offset_warn, limit_seconds=a.limit_seconds,
1543
+ engine=a.engine, asr_json=a.asr_json, whisper_model=a.whisper_model,
1544
+ vad=a.vad)
1545
+ except Exception as e: # noqa: BLE001
1546
+ print("[FAIL] ASR 回读失败:%s" % e, file=sys.stderr)
1547
+ return 1
1548
+ status = (rep.get("gate") or {}).get("status")
1549
+ if status == "fail":
1550
+ return 1
1551
+ if status == "skipped":
1552
+ if a.strict:
1553
+ print(" [FAIL] --strict:ASR 回读本次没有执行(skipped != 通过)",
1554
+ file=sys.stderr)
1555
+ return 1
1556
+ print(" [SKIP] ASR 回读没有执行(skipped != 通过;加 --strict 可让它失败)",
1557
+ file=sys.stderr)
1558
+ return 0
1559
+ return 0
1560
+
1561
+
1562
+ if __name__ == "__main__":
1563
+ sys.exit(main())