@speclip/pi-subtitles 0.2.2 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +61 -3
- package/extensions/subtitles/index.ts +24 -6
- package/package.json +1 -1
- package/prompts/add-subtitles.md +2 -0
- package/skills/add-subtitles/SKILL.md +12 -13
- package/skills/add-subtitles/references/timing-and-layout.md +3 -1
- package/src/contracts.ts +49 -0
- package/src/cues.ts +54 -15
- package/src/export.ts +25 -6
- package/src/project.ts +71 -66
- package/src/timeline.ts +84 -0
- package/src/validation.ts +66 -0
package/README.md
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
## 能力边界
|
|
8
8
|
|
|
9
9
|
- 输入:`pi-speech` 兼容的词级 JSON 转录,也兼容常见 `{ word, start, end }` 秒级结构。
|
|
10
|
-
- A-roll
|
|
10
|
+
- A-roll 映射:直接读取渲染回执,支持多源、回接、重复片段和实际帧/采样边界;生成、修订及导出均检查剪辑点。
|
|
11
11
|
- 自动分段:结合标点、停顿、字数、时长和阅读速度生成初稿。
|
|
12
12
|
- Agent 审阅:通过不可变 revision 修正 ASR 文本、标点和断句;横屏项目强制单行,审阅阶段也不能绕过。
|
|
13
13
|
- 导出:标准 SRT;可选带横屏/竖屏安全区预设的 ASS;两者都不覆盖已有文件。
|
|
@@ -39,6 +39,24 @@ pi install npm:@speclip/pi-subtitles
|
|
|
39
39
|
|
|
40
40
|
### 1. 创建字幕项目
|
|
41
41
|
|
|
42
|
+
A-roll 已渲染时,优先直接传官方回执和原始转录,不需要自行生成派生转录:
|
|
43
|
+
|
|
44
|
+
```js
|
|
45
|
+
subtitles_create {
|
|
46
|
+
projectId: "launch-captions",
|
|
47
|
+
renderReceiptPath: "analysis/render_receipt.json",
|
|
48
|
+
transcriptPath: "transcripts/launch.json",
|
|
49
|
+
layoutPreset: "landscape",
|
|
50
|
+
protectedPhrases: ["跷跷板"]
|
|
51
|
+
}
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
回执必须含 `pi-media` 的 `timelineTiming`,工具核对成片和源文件哈希,按实际输出边界映射。回接和重复片段分别保留独立字词 ID;帧取整造成的字尾截短返回 `render-tail-clipped` 听校提示,真正跨原片剪点的词仍会报错。
|
|
55
|
+
|
|
56
|
+
多个主素材用 `sourceTranscripts: [{ sourcePath, transcriptPath }]` 替代单个 `transcriptPath`,每个主素材恰好绑定一份转录。回执模式不能同时传 `sourceDurationMs` 或 `timelineSegments`。
|
|
57
|
+
|
|
58
|
+
以下是兼容保留的旧输入方式;它没有经过渲染校验的跨插件时间轴绑定。
|
|
59
|
+
|
|
42
60
|
无 A-roll 剪辑时,只传源时长:
|
|
43
61
|
|
|
44
62
|
```js
|
|
@@ -50,7 +68,7 @@ subtitles_create {
|
|
|
50
68
|
}
|
|
51
69
|
```
|
|
52
70
|
|
|
53
|
-
|
|
71
|
+
旧模式有剪辑时,传按播放顺序排列的保留源区间,可回接、可重复:
|
|
54
72
|
|
|
55
73
|
```js
|
|
56
74
|
subtitles_create {
|
|
@@ -89,6 +107,24 @@ subtitles_get {
|
|
|
89
107
|
|
|
90
108
|
### 3. 写入 Agent 审阅版本
|
|
91
109
|
|
|
110
|
+
优先读取 `subtitles_get { projectId, view: "words", offset: 0, limit: 200 }` 的所有字词页,按稳定 ID 提交完整分组,让工具计算时间:
|
|
111
|
+
|
|
112
|
+
```js
|
|
113
|
+
subtitles_apply {
|
|
114
|
+
projectId: "launch-captions",
|
|
115
|
+
expectedRevision: 1,
|
|
116
|
+
changeReason: "按语义重新断句,保持全部字词",
|
|
117
|
+
groups: [
|
|
118
|
+
{ startWordId: "a-001:word-0", endWordId: "a-001:word-8" }
|
|
119
|
+
// 继续列出全部保留字词,不能只提交一页
|
|
120
|
+
]
|
|
121
|
+
}
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
分组不得遗漏、重复、乱序或跨剪辑片段。可选 `text` 用于标点、空格或听校后的纠错;实质文字变化必须说明 `correctionReason`。工具按组内真实字词时间生成字幕,短字幕仅借用同片段内的静音。`protectedPhrases` 用于初稿的短语保护,原 ASR 多字词也优先保持整体。
|
|
125
|
+
|
|
126
|
+
`groups` 与下方兼容的完整 `cues` 二选一。直接提交 `cues` 可精确改显示时间,但文字覆盖变化会产生 `text-changed` 提示。所有修订均校验非重叠、片段边界、视频末端和行长。
|
|
127
|
+
|
|
92
128
|
```js
|
|
93
129
|
subtitles_apply {
|
|
94
130
|
projectId: "launch-captions",
|
|
@@ -102,6 +138,8 @@ subtitles_apply {
|
|
|
102
138
|
|
|
103
139
|
`cues` 必须是完整列表,不是某一页。时间必须递增、不重叠,且不能超过成片时长。`expectedRevision` 防止并发审阅互相覆盖;每次成功都会产生新的不可变快照。
|
|
104
140
|
|
|
141
|
+
`subtitles_get` 同时返回 `findings` 和 `review`。阅读速度、短时显示等是审阅提示;不能用它们掩盖结构错误。`review.text`、`review.sync`、`review.visual` 分别记录听校、同步和视觉结果,可在修订时传 `{ status: "passed", evidence: "实际核对的音频/视频及范围" }`;未提供的项目会重置为 pending。结构通过不代表听校通过。
|
|
142
|
+
|
|
105
143
|
### 4. 导出标准轨道
|
|
106
144
|
|
|
107
145
|
```js
|
|
@@ -119,6 +157,25 @@ subtitles_export {
|
|
|
119
157
|
|
|
120
158
|
本包的完成状态是“字幕轨道已生成并校验”,不是“字幕已经烧进视频”。最后应把轨道交给支持字幕的渲染器,并检查实际成片中的同步、遮挡、安全区、字体回退和漏字。
|
|
121
159
|
|
|
160
|
+
### 5. 预览与烧录交接
|
|
161
|
+
|
|
162
|
+
导出返回 `subtitleTrack: { sourcePath, format, mode: "burn-in", exportReceiptPath }`(回执模式才有 `exportReceiptPath`)。直接放入 `pi-media` 的 timeline `edit_apply`,保留其他编辑字段。渲染器核对轨道哈希和主音频时间轴指纹;B-roll/BGM 调整不改变该指纹,主素材、源区间、顺序或输出帧率改变则拒绝旧字幕。
|
|
163
|
+
|
|
164
|
+
先用支持新接口的 `pi-media` 生成抽样证据:
|
|
165
|
+
|
|
166
|
+
```js
|
|
167
|
+
media_subtitle_preview {
|
|
168
|
+
videoPath: "deliverables/clean.mp4",
|
|
169
|
+
exportReceiptPath: ".subtitles/projects/launch-captions/exports/<exportId>.json",
|
|
170
|
+
sampleTimesSeconds: [5, 55, 77],
|
|
171
|
+
clip: { startSeconds: 54, durationSeconds: 4 }
|
|
172
|
+
}
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
选择视频内 1–6 个时间点,短视频最长 15 秒。返回 PNG、可听短视频、字体选择日志和报告路径;视觉与同步状态保持 pending,由实际查看/听校后更新。最终视频仍需 `render` + `review`。字幕渲染环境需有 libass;可用 `PI_MEDIA_SUBTITLE_FFMPEG_BINARY` 指定二进制。
|
|
176
|
+
|
|
177
|
+
内部保留精确毫秒,导出时 SRT 量化为毫秒、ASS 为厘秒,且时间不能越过镜头边界。格式精度导致零时长时明确失败,不静默输出坏字幕。
|
|
178
|
+
|
|
122
179
|
## 工作区数据
|
|
123
180
|
|
|
124
181
|
项目状态保存在:
|
|
@@ -126,11 +183,12 @@ subtitles_export {
|
|
|
126
183
|
```text
|
|
127
184
|
.subtitles/projects/<projectId>/
|
|
128
185
|
├── project.json
|
|
186
|
+
├── words.json # 不可变字词映射、源字词位置、片段边界与诊断
|
|
129
187
|
├── snapshots/<revision>.json
|
|
130
188
|
└── exports/<exportId>.json
|
|
131
189
|
```
|
|
132
190
|
|
|
133
|
-
|
|
191
|
+
新字段均为可选字段,旧项目与原有四个工具仍兼容;升级不会原地改写旧项目。源转录与导出文件都不会被覆盖。源转录字节变化后,旧项目会拒绝继续导出,避免把新内容误配到旧时间轴。
|
|
134
192
|
|
|
135
193
|
## 开发
|
|
136
194
|
|
|
@@ -5,6 +5,7 @@ import {
|
|
|
5
5
|
createSubtitleProject,
|
|
6
6
|
exportSubtitleProject,
|
|
7
7
|
getSubtitleProject,
|
|
8
|
+
getProjectWords,
|
|
8
9
|
} from "../../src/project.ts";
|
|
9
10
|
|
|
10
11
|
function result(details: unknown) {
|
|
@@ -60,6 +61,8 @@ const assStyle = Type.Object({
|
|
|
60
61
|
marginV: Type.Optional(Type.Integer({ minimum: 0, maximum: 4_000 })),
|
|
61
62
|
}, { additionalProperties: false });
|
|
62
63
|
|
|
64
|
+
const reviewEvidence = Type.Object({ status: Type.Union([Type.Literal("pending"), Type.Literal("passed")]), evidence: Type.Optional(Type.String({ minLength: 1, maxLength: 2000 })) }, { additionalProperties: false });
|
|
65
|
+
|
|
63
66
|
export default function subtitles(pi: ExtensionAPI): void {
|
|
64
67
|
pi.registerTool({
|
|
65
68
|
name: "subtitles_create",
|
|
@@ -67,8 +70,11 @@ export default function subtitles(pi: ExtensionAPI): void {
|
|
|
67
70
|
description: "Create an immutable workspace subtitle project from a word-timestamp JSON transcript. Optionally remaps retained A-roll source segments onto the final edited timeline, then generates readable cue groups. Landscape is the default and strictly remains single-line.",
|
|
68
71
|
parameters: Type.Object({
|
|
69
72
|
projectId: Type.String({ minLength: 1, maxLength: 128, pattern: "^[a-z0-9](?:[a-z0-9._-]{0,126}[a-z0-9])?$" }),
|
|
70
|
-
transcriptPath: Type.String({ minLength: 1, description: "Existing workspace-relative word-timestamp JSON
|
|
71
|
-
|
|
73
|
+
transcriptPath: Type.Optional(Type.String({ minLength: 1, description: "Existing workspace-relative word-timestamp JSON; single primary source only." })),
|
|
74
|
+
renderReceiptPath: Type.Optional(Type.String({ minLength: 1, description: "Exact pi-media render receipt with timelineTiming. Preferred after A-roll editing; mutually exclusive with sourceDurationMs/timelineSegments." })),
|
|
75
|
+
sourceTranscripts: Type.Optional(Type.Array(Type.Object({ sourcePath: Type.String({ minLength: 1 }), transcriptPath: Type.String({ minLength: 1 }) }, { additionalProperties: false }), { minItems: 1, maxItems: 1000 })),
|
|
76
|
+
protectedPhrases: Type.Optional(Type.Array(Type.String({ minLength: 1, maxLength: 80 }), { maxItems: 200 })),
|
|
77
|
+
sourceDurationMs: Type.Optional(Type.Number({ exclusiveMinimum: 0 })),
|
|
72
78
|
layoutPreset: Type.Optional(Type.Union([
|
|
73
79
|
Type.Literal("landscape"),
|
|
74
80
|
Type.Literal("portrait"),
|
|
@@ -76,7 +82,7 @@ export default function subtitles(pi: ExtensionAPI): void {
|
|
|
76
82
|
timelineSegments: Type.Optional(Type.Array(timelineSegment, {
|
|
77
83
|
minItems: 1,
|
|
78
84
|
maxItems: 1_000,
|
|
79
|
-
description: "
|
|
85
|
+
description: "Legacy ranges in playback order, including reordered/repeated source ranges. Prefer renderReceiptPath for frame-accurate editing.",
|
|
80
86
|
})),
|
|
81
87
|
policy: Type.Optional(policy),
|
|
82
88
|
}, { additionalProperties: false }),
|
|
@@ -97,19 +103,25 @@ export default function subtitles(pi: ExtensionAPI): void {
|
|
|
97
103
|
parameters: Type.Object({
|
|
98
104
|
projectId: Type.String({ minLength: 1, maxLength: 128 }),
|
|
99
105
|
revision: Type.Optional(Type.Integer({ minimum: 1 })),
|
|
106
|
+
view: Type.Optional(Type.Union([Type.Literal("cues"), Type.Literal("words")], { description: "words returns stable occurrence IDs for complete word-group review." })),
|
|
100
107
|
offset: Type.Optional(Type.Integer({ minimum: 0, default: 0 })),
|
|
101
108
|
limit: Type.Optional(Type.Integer({ minimum: 1, maximum: 200, default: 100 })),
|
|
102
109
|
}, { additionalProperties: false }),
|
|
103
110
|
async execute(_toolCallId, params, signal, _onUpdate, ctx) {
|
|
104
111
|
signal?.throwIfAborted();
|
|
105
112
|
const loaded = await getSubtitleProject(ctx.cwd, params.projectId, params.revision);
|
|
113
|
+
const mapped = params.view === "words" ? await getProjectWords(ctx.cwd, loaded.project) : undefined;
|
|
114
|
+
const items = mapped?.words ?? loaded.snapshot.cues;
|
|
106
115
|
const offset = params.offset ?? 0;
|
|
107
116
|
const limit = params.limit ?? 100;
|
|
108
117
|
return result({
|
|
109
118
|
project: loaded.project,
|
|
110
119
|
revision: loaded.snapshot.revision,
|
|
111
120
|
snapshotSha256: loaded.snapshot.snapshotSha256,
|
|
112
|
-
total:
|
|
121
|
+
total: items.length,
|
|
122
|
+
view: params.view ?? "cues",
|
|
123
|
+
review: loaded.snapshot.review,
|
|
124
|
+
findings: loaded.snapshot.findings,
|
|
113
125
|
offset,
|
|
114
126
|
limit,
|
|
115
127
|
hasMore: offset + limit < loaded.snapshot.cues.length,
|
|
@@ -121,11 +133,17 @@ export default function subtitles(pi: ExtensionAPI): void {
|
|
|
121
133
|
pi.registerTool({
|
|
122
134
|
name: "subtitles_apply",
|
|
123
135
|
label: "Apply subtitle review",
|
|
124
|
-
description: "Create a new immutable subtitle revision from a
|
|
136
|
+
description: "Create a new immutable subtitle revision from complete word groups (preferred; automatic timing and coverage checks) or a full cue list. Uses optimistic locking and requires a human-readable change reason.",
|
|
125
137
|
parameters: Type.Object({
|
|
126
138
|
projectId: Type.String({ minLength: 1, maxLength: 128 }),
|
|
127
139
|
expectedRevision: Type.Integer({ minimum: 1 }),
|
|
128
|
-
cues: Type.Array(subtitleCue, { minItems: 1, maxItems: 10_000 }),
|
|
140
|
+
cues: Type.Optional(Type.Array(subtitleCue, { minItems: 1, maxItems: 10_000 })),
|
|
141
|
+
groups: Type.Optional(Type.Array(Type.Object({
|
|
142
|
+
startWordId: Type.String({ minLength: 1 }), endWordId: Type.String({ minLength: 1 }),
|
|
143
|
+
text: Type.Optional(Type.String({ minLength: 1, maxLength: 2000 })),
|
|
144
|
+
correctionReason: Type.Optional(Type.String({ minLength: 1, maxLength: 2000 })),
|
|
145
|
+
}, { additionalProperties: false }), { minItems: 1, maxItems: 10000, description: "Complete ordered word ranges; timing is computed by the tool. Mutually exclusive with cues." })),
|
|
146
|
+
review: Type.Optional(Type.Object({ text: Type.Optional(reviewEvidence), sync: Type.Optional(reviewEvidence), visual: Type.Optional(reviewEvidence) }, { additionalProperties: false })),
|
|
129
147
|
changeReason: Type.String({ minLength: 1, maxLength: 2_000 }),
|
|
130
148
|
}, { additionalProperties: false }),
|
|
131
149
|
async execute(_toolCallId, params, signal, _onUpdate, ctx) {
|
package/package.json
CHANGED
package/prompts/add-subtitles.md
CHANGED
|
@@ -5,3 +5,5 @@ description: Add reviewed subtitles to an edited talking-head video
|
|
|
5
5
|
Use the `add-subtitles` skill to map a word-level transcript onto the final A-roll timeline, inspect every generated cue page, preserve spoken meaning, apply only evidence-backed text or grouping corrections, and export an exact SRT plus optional ASS track for this request: $@
|
|
6
6
|
|
|
7
7
|
Finish A-roll first. Do not time subtitles from B-roll assets, submit a partial cue page as a complete revision, overwrite existing tracks, or claim that the video contains subtitles until a renderer has applied and visually reviewed the exported track.
|
|
8
|
+
|
|
9
|
+
Prefer renderReceiptPath with original transcripts, then word-ID groups for review. Let tools compute timing and validate coverage. Use media_subtitle_preview for sample images/font logs and a short audio/video clip, then hand the returned subtitleTrack to pi-media. Keep text, sync and visual review status separate.
|
|
@@ -5,18 +5,17 @@ description: Build and review subtitles from word-level ASR timestamps after A-r
|
|
|
5
5
|
|
|
6
6
|
# Add subtitles
|
|
7
7
|
|
|
8
|
-
Use `pi-speech` for word timestamps
|
|
8
|
+
Use `pi-speech` for word timestamps, `pi-subtitles` for timing and review, and `pi-media` for subtitle previews and final rendering.
|
|
9
9
|
|
|
10
|
-
1. Finish
|
|
11
|
-
2.
|
|
12
|
-
3.
|
|
13
|
-
4.
|
|
14
|
-
5.
|
|
15
|
-
6.
|
|
16
|
-
7.
|
|
17
|
-
8.
|
|
18
|
-
9.
|
|
10
|
+
1. Finish A-roll and obtain the official render receipt containing `timelineTiming`. Use `subtitles_create` with `renderReceiptPath` and the ORIGINAL `transcriptPath`. For multiple primary sources, pass `sourceTranscripts: [{ sourcePath, transcriptPath }]` instead. Do not copy time ranges, recalculate frame rounding, manufacture a derived transcript, or change playback order to satisfy the subtitle tool. The receipt importer handles reordered and repeated clips, actual frame/sample boundaries and source provenance.
|
|
11
|
+
2. Without an edited render receipt, the legacy `transcriptPath` + `sourceDurationMs` + optional `timelineSegments` input remains available. Ranges are in playback order. Do not combine these timing inputs with `renderReceiptPath`. Legacy projects have no verified renderer handoff binding.
|
|
12
|
+
3. Choose `landscape` (default, one line) or `portrait` (up to two lines). Add `protectedPhrases` for known phrases that must not be split. These phrases must fit the layout; they do not correct transcription.
|
|
13
|
+
4. Read ALL cue pages using `subtitles_get`, inspect `findings`, and check spoken words, terminology, punctuation and semantic grouping against audio. Automatic grouping and structural checks do not establish text accuracy or sync. A `render-tail-clipped` finding means actual rendering shortened a word; listen to that join. A real source cut through a word is an error requiring an edit correction.
|
|
14
|
+
5. Prefer `subtitles_get` with `view: "words"` when regrouping. Read all pages, then submit the complete ordered `groups` list to `subtitles_apply`: each group contains `startWordId` and `endWordId`. The tool calculates timing, checks full word coverage, and extends short cues only within the same clip's available silence. Do not write Python to match text to timestamps. Word IDs identify clip occurrences, including repeated use of a source word.
|
|
15
|
+
6. A group's optional `text` adjusts punctuation/spacing or corrects spoken text. Changes beyond punctuation/spacing require `correctionReason` supported by audio. Preserve meaning, fillers, emphasis, terms and numbers. Never delete words merely to make a shorter cue. The complete legacy `cues` input remains available for precise timing edits; its text-change findings require review. Never submit one page as a full revision. Use `expectedRevision` and a concrete `changeReason`; reconcile concurrent revisions.
|
|
16
|
+
7. Review dimensions are separate: `review.text`, `review.sync`, `review.visual`. Each is `pending` or `passed`; `passed` requires concrete `evidence`. A new revision resets unspecified dimensions to pending. Do not report a pass if you cannot actually hear or view the relevant output. Structural validation runs automatically at creation, apply and export.
|
|
17
|
+
8. Export the exact revision with `subtitles_export`, requesting ASS for styling and burn-in. Both layouts use Source Han Sans SC Heavy with white glyphs and a softened shadow. Keep canonical ASS PlayRes rather than changing it to the input resolution. The exporter quantizes SRT/ASS within edit and video boundaries, and rejects timing that would collapse at format precision.
|
|
18
|
+
9. Call `media_subtitle_preview` with the clean `videoPath`, the returned `receipt.receiptPath` as `exportReceiptPath`, 1–6 `sampleTimesSeconds`, and optionally `clip: { startSeconds, durationSeconds }` (up to 15 seconds). Inspect images, actual font-resolution logs and the audio/video clip. Preview generation leaves sync/visual approval pending. If the runtime lacks libass, report the actionable configuration error instead of writing ad hoc FFmpeg scripts.
|
|
19
|
+
10. For final burn-in, pass the exported `subtitleTrack` object unchanged into the existing pi-media timeline's `edit_apply`, preserving the other timeline fields. Its `exportReceiptPath` binds the track hash and primary timeline. B-roll overlays can change; A-roll source/range/order/frame-rate changes require remapping. Then `render` and `review` the precise edit revision and inspect/listen to the result. Independent tracks and sample previews are not a completed burned video.
|
|
19
20
|
|
|
20
|
-
Never overwrite
|
|
21
|
-
|
|
22
|
-
Read [timing and layout](references/timing-and-layout.md) when reviewing dense, fast, bilingual, or unusually sparse speech.
|
|
21
|
+
Never overwrite source transcripts, tracks or renders. Transcript or primary timeline changes require a new subtitle project. Read [timing and layout](references/timing-and-layout.md) for dense or bilingual speech.
|
|
@@ -5,9 +5,11 @@
|
|
|
5
5
|
For a retained A-roll source range, final cue time is:
|
|
6
6
|
|
|
7
7
|
```text
|
|
8
|
-
final_time =
|
|
8
|
+
final_time = receipt.outputStartSeconds * 1000 + source_word_time - range_source_start
|
|
9
9
|
```
|
|
10
10
|
|
|
11
|
+
Use actual receipt output frame/sample boundaries. Do not accumulate requested source durations after rendering, and do not stretch all words to fit frame rounding: pi-media pads or trims the tail without changing speech speed. Small rendered-tail truncations produce findings; real source-word cuts remain errors.
|
|
12
|
+
|
|
11
13
|
A cut is also a subtitle boundary. Even when the two retained words become adjacent in the final video, a cue must not bridge across the edit because it can expose removed language or create misleading timing.
|
|
12
14
|
|
|
13
15
|
## Talking-head layout policy
|
package/src/contracts.ts
CHANGED
|
@@ -9,10 +9,43 @@ export interface TimelineSegment {
|
|
|
9
9
|
id: string;
|
|
10
10
|
sourceStartMs: number;
|
|
11
11
|
sourceEndMs: number;
|
|
12
|
+
sourcePath?: string;
|
|
13
|
+
outputStartMs?: number;
|
|
14
|
+
outputEndMs?: number;
|
|
12
15
|
}
|
|
13
16
|
|
|
14
17
|
export interface TimelineWord extends TranscriptWord {
|
|
15
18
|
timelineSegmentId?: string;
|
|
19
|
+
id?: string;
|
|
20
|
+
sourceWordIndex?: number;
|
|
21
|
+
sourceBeginMs?: number;
|
|
22
|
+
sourceEndMs?: number;
|
|
23
|
+
timelineSegmentEndMs?: number;
|
|
24
|
+
}
|
|
25
|
+
|
|
26
|
+
export interface SubtitleFinding {
|
|
27
|
+
code: string;
|
|
28
|
+
severity: "warning" | "info";
|
|
29
|
+
message: string;
|
|
30
|
+
cueId?: string;
|
|
31
|
+
}
|
|
32
|
+
|
|
33
|
+
export interface WordGroup {
|
|
34
|
+
startWordId: string;
|
|
35
|
+
endWordId: string;
|
|
36
|
+
text?: string;
|
|
37
|
+
correctionReason?: string;
|
|
38
|
+
}
|
|
39
|
+
|
|
40
|
+
export interface ReviewEvidence {
|
|
41
|
+
status: "pending" | "passed";
|
|
42
|
+
evidence?: string;
|
|
43
|
+
}
|
|
44
|
+
|
|
45
|
+
export interface SubtitleReview {
|
|
46
|
+
text?: ReviewEvidence;
|
|
47
|
+
sync?: ReviewEvidence;
|
|
48
|
+
visual?: ReviewEvidence;
|
|
16
49
|
}
|
|
17
50
|
|
|
18
51
|
export interface SubtitlePolicy {
|
|
@@ -76,6 +109,15 @@ export interface SubtitleProject {
|
|
|
76
109
|
transcript: FileRef;
|
|
77
110
|
timelineSegments: TimelineSegment[];
|
|
78
111
|
policy: SubtitlePolicy;
|
|
112
|
+
wordMap?: FileRef;
|
|
113
|
+
sources?: Array<{ source: FileRef; transcript: FileRef }>;
|
|
114
|
+
renderBinding?: {
|
|
115
|
+
receipt: FileRef;
|
|
116
|
+
video: FileRef;
|
|
117
|
+
projectId: string;
|
|
118
|
+
snapshotRevision: number;
|
|
119
|
+
primaryTimelineSha256: string;
|
|
120
|
+
};
|
|
79
121
|
}
|
|
80
122
|
|
|
81
123
|
export interface SubtitleSnapshot {
|
|
@@ -86,6 +128,9 @@ export interface SubtitleSnapshot {
|
|
|
86
128
|
createdAt: string;
|
|
87
129
|
changeReason: string;
|
|
88
130
|
cues: SubtitleCue[];
|
|
131
|
+
groups?: WordGroup[];
|
|
132
|
+
review?: SubtitleReview;
|
|
133
|
+
findings?: SubtitleFinding[];
|
|
89
134
|
snapshotSha256: string;
|
|
90
135
|
}
|
|
91
136
|
|
|
@@ -97,6 +142,10 @@ export interface SubtitleExportReceipt {
|
|
|
97
142
|
createdAt: string;
|
|
98
143
|
snapshotSha256: string;
|
|
99
144
|
tracks: { srt: FileRef; ass?: FileRef };
|
|
145
|
+
primaryTimelineSha256?: string;
|
|
146
|
+
video?: FileRef;
|
|
147
|
+
review?: SubtitleReview;
|
|
148
|
+
structure?: "passed";
|
|
100
149
|
receiptPath: string;
|
|
101
150
|
receiptSha256: string;
|
|
102
151
|
}
|
package/src/cues.ts
CHANGED
|
@@ -4,6 +4,7 @@ import type {
|
|
|
4
4
|
TimelineSegment,
|
|
5
5
|
TimelineWord,
|
|
6
6
|
TranscriptWord,
|
|
7
|
+
SubtitleFinding,
|
|
7
8
|
} from "./contracts.ts";
|
|
8
9
|
|
|
9
10
|
export const DEFAULT_SUBTITLE_POLICY: SubtitlePolicy = {
|
|
@@ -49,39 +50,54 @@ function validateWords(words: TranscriptWord[]): void {
|
|
|
49
50
|
|
|
50
51
|
function validateTimeline(segments: TimelineSegment[]): void {
|
|
51
52
|
if (segments.length === 0) throw new Error("Timeline must retain at least one A-roll segment");
|
|
52
|
-
let previousEnd =
|
|
53
|
+
let previousEnd = segments[0]?.outputStartMs ?? 0;
|
|
53
54
|
const ids = new Set<string>();
|
|
54
55
|
for (const segment of segments) {
|
|
55
56
|
if (!segment.id || ids.has(segment.id) || !finiteNonNegative(segment.sourceStartMs)
|
|
56
57
|
|| !finiteNonNegative(segment.sourceEndMs) || segment.sourceEndMs <= segment.sourceStartMs) {
|
|
57
58
|
throw new Error("Timeline contains an invalid or duplicate segment");
|
|
58
59
|
}
|
|
59
|
-
if (segment.
|
|
60
|
+
if ((segment.outputStartMs === undefined) !== (segment.outputEndMs === undefined)) throw new Error("Timeline output range must have both endpoints");
|
|
61
|
+
if (segment.outputStartMs !== undefined && (!finiteNonNegative(segment.outputStartMs)
|
|
62
|
+
|| !finiteNonNegative(segment.outputEndMs) || segment.outputEndMs <= segment.outputStartMs
|
|
63
|
+
|| Math.abs(segment.outputStartMs - previousEnd) > 0.001)) throw new Error("Timeline output ranges must be contiguous in playback order");
|
|
60
64
|
ids.add(segment.id);
|
|
61
|
-
previousEnd = segment.sourceEndMs;
|
|
65
|
+
previousEnd = segment.outputEndMs ?? previousEnd + segment.sourceEndMs - segment.sourceStartMs;
|
|
62
66
|
}
|
|
63
67
|
}
|
|
64
68
|
|
|
65
|
-
export function remapWordsToTimeline(words: TranscriptWord[], segments: TimelineSegment[]): TimelineWord[] {
|
|
69
|
+
export function remapWordsToTimeline(words: TranscriptWord[], segments: TimelineSegment[], findings: SubtitleFinding[] = []): TimelineWord[] {
|
|
66
70
|
validateWords(words);
|
|
67
71
|
validateTimeline(segments);
|
|
68
72
|
const remapped: TimelineWord[] = [];
|
|
69
73
|
let outputOffsetMs = 0;
|
|
70
74
|
for (const segment of segments) {
|
|
71
|
-
|
|
75
|
+
const outputStart = segment.outputStartMs ?? outputOffsetMs;
|
|
76
|
+
const outputEnd = segment.outputEndMs ?? outputStart + segment.sourceEndMs - segment.sourceStartMs;
|
|
77
|
+
for (const [wordIndex, word] of words.entries()) {
|
|
72
78
|
const overlaps = word.beginMs < segment.sourceEndMs && word.endMs > segment.sourceStartMs;
|
|
73
79
|
if (!overlaps) continue;
|
|
74
|
-
if (word.beginMs < segment.sourceStartMs || word.endMs > segment.sourceEndMs) {
|
|
75
|
-
throw new Error(`Word timestamp crosses A-roll edit boundary: ${word.text}`);
|
|
80
|
+
if (word.beginMs < segment.sourceStartMs - 0.001 || word.endMs > segment.sourceEndMs + 0.001) {
|
|
81
|
+
throw new Error(`Word timestamp crosses A-roll edit boundary: ${segment.id}, word ${wordIndex} (${word.text}), ${word.beginMs}–${word.endMs}ms`);
|
|
76
82
|
}
|
|
83
|
+
const beginMs = Math.max(outputStart, outputStart + word.beginMs - segment.sourceStartMs);
|
|
84
|
+
const rawEnd = outputStart + word.endMs - segment.sourceStartMs;
|
|
85
|
+
const endMs = Math.min(outputEnd, rawEnd);
|
|
86
|
+
if (endMs <= beginMs) throw new Error(`Rendered segment ${segment.id} removes word ${wordIndex} (${word.text}); adjust the edit boundary`);
|
|
87
|
+
if (rawEnd > outputEnd + 0.001) findings.push({ code: "render-tail-clipped", severity: "warning", message: `${segment.id}: ${word.text} shortened by ${rawEnd - outputEnd}ms by rendered timing; listen to the boundary` });
|
|
77
88
|
remapped.push({
|
|
78
89
|
...word,
|
|
79
|
-
|
|
80
|
-
|
|
90
|
+
id: `${segment.id}:word-${wordIndex}`,
|
|
91
|
+
sourceWordIndex: wordIndex,
|
|
92
|
+
sourceBeginMs: word.beginMs,
|
|
93
|
+
sourceEndMs: word.endMs,
|
|
94
|
+
beginMs,
|
|
95
|
+
endMs,
|
|
81
96
|
timelineSegmentId: segment.id,
|
|
97
|
+
timelineSegmentEndMs: outputEnd,
|
|
82
98
|
});
|
|
83
99
|
}
|
|
84
|
-
outputOffsetMs
|
|
100
|
+
outputOffsetMs = outputEnd;
|
|
85
101
|
}
|
|
86
102
|
return remapped;
|
|
87
103
|
}
|
|
@@ -105,12 +121,13 @@ function splitTiming(word: TimelineWord, policy: SubtitlePolicy): TimedToken[] {
|
|
|
105
121
|
punctuation: "",
|
|
106
122
|
}, policy);
|
|
107
123
|
if (nested[0] && start > 0) nested[0].spaceBefore = true;
|
|
124
|
+
if (nested.at(-1) && match === matches.at(-1)) nested.at(-1)!.text += word.punctuation;
|
|
108
125
|
return nested;
|
|
109
126
|
});
|
|
110
127
|
}
|
|
111
128
|
const asciiWord = isAsciiWord(text);
|
|
112
|
-
const mustSplit =
|
|
113
|
-
const units =
|
|
129
|
+
const mustSplit = visibleChars(text + word.punctuation) > policy.maxLineChars * policy.maxLines || durationMs > policy.maxDurationMs;
|
|
130
|
+
const units = !mustSplit ? [text] : [...text];
|
|
114
131
|
const stepMs = durationMs / units.length;
|
|
115
132
|
return units.map((unit, index) => ({
|
|
116
133
|
text: `${unit}${index === units.length - 1 ? word.punctuation : ""}`,
|
|
@@ -233,23 +250,45 @@ export function buildSubtitleCues(
|
|
|
233
250
|
words: TimelineWord[],
|
|
234
251
|
timelineEndMs: number,
|
|
235
252
|
policy: Partial<SubtitlePolicy> = {},
|
|
253
|
+
protectedPhrases: string[] = [],
|
|
236
254
|
): SubtitleCue[] {
|
|
237
255
|
if (!finiteNonNegative(timelineEndMs)) throw new Error("timelineEndMs must be non-negative");
|
|
238
256
|
validateWords(words);
|
|
239
257
|
const resolved = { ...DEFAULT_SUBTITLE_POLICY, ...policy };
|
|
258
|
+
if (protectedPhrases.some(p => !p.trim() || visibleChars(p) > resolved.maxLineChars * resolved.maxLines)) throw new Error("Protected phrase must fit the subtitle layout");
|
|
240
259
|
const groups: TimelineWord[][] = [];
|
|
241
260
|
for (const word of words) {
|
|
242
261
|
const previous = groups.at(-1);
|
|
243
262
|
if (!previous || previous[0]!.timelineSegmentId !== word.timelineSegmentId) groups.push([word]);
|
|
244
263
|
else previous.push(word);
|
|
245
264
|
}
|
|
246
|
-
const raw = groups.flatMap((group) => buildGroup(group.flatMap((word) => splitTiming(word, resolved)), resolved)
|
|
265
|
+
const raw = groups.flatMap((group) => buildGroup(protectPhrases(group, protectedPhrases).flatMap((word) => splitTiming(word, resolved)), resolved)
|
|
266
|
+
.map((cue) => ({ ...cue, segmentEnd: group[0]!.timelineSegmentEndMs ?? timelineEndMs })));
|
|
247
267
|
return raw.map((cue, index) => {
|
|
248
|
-
const nextStart = raw[index + 1]?.beginMs ??
|
|
268
|
+
const nextStart = Math.min(raw[index + 1]?.beginMs ?? timelineEndMs, cue.segmentEnd, timelineEndMs);
|
|
249
269
|
return {
|
|
250
270
|
id: `cue-${String(index + 1).padStart(3, "0")}`,
|
|
251
|
-
|
|
271
|
+
beginMs: cue.beginMs,
|
|
272
|
+
text: cue.text,
|
|
252
273
|
endMs: Math.max(cue.endMs, Math.min(cue.beginMs + resolved.minDurationMs, cue.beginMs + resolved.maxDurationMs, nextStart)),
|
|
253
274
|
};
|
|
254
275
|
});
|
|
255
276
|
}
|
|
277
|
+
|
|
278
|
+
function protectPhrases(words: TimelineWord[], phrases: string[]): TimelineWord[] {
|
|
279
|
+
const result: TimelineWord[] = [];
|
|
280
|
+
for (let i = 0; i < words.length; i++) {
|
|
281
|
+
let matchEnd = i;
|
|
282
|
+
for (const phrase of phrases) {
|
|
283
|
+
let text = "";
|
|
284
|
+
for (let end = i; end < words.length && text.length < phrase.length; end++) {
|
|
285
|
+
text += words[end]!.text;
|
|
286
|
+
if (text === phrase) matchEnd = Math.max(matchEnd, end);
|
|
287
|
+
if (words[end]!.punctuation) break;
|
|
288
|
+
}
|
|
289
|
+
}
|
|
290
|
+
result.push({ ...words[i]!, text: words.slice(i, matchEnd + 1).map(w => w.text).join(""), endMs: words[matchEnd]!.endMs, punctuation: words[matchEnd]!.punctuation });
|
|
291
|
+
i = matchEnd;
|
|
292
|
+
}
|
|
293
|
+
return result;
|
|
294
|
+
}
|
package/src/export.ts
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
|
-
import type { AssStyle, FileRef, SubtitleCue, SubtitleLayoutPreset } from "./contracts.ts";
|
|
1
|
+
import type { AssStyle, FileRef, SubtitleCue, SubtitleLayoutPreset, SubtitleProject } from "./contracts.ts";
|
|
2
|
+
import { segmentRanges, validateCues } from "./validation.ts";
|
|
2
3
|
import { rm } from "node:fs/promises";
|
|
3
4
|
import { resolveExistingWorkspaceFile, writeNewWorkspaceFile } from "./workspace.ts";
|
|
4
5
|
|
|
@@ -87,7 +88,7 @@ function assertCues(cues: SubtitleCue[]): void {
|
|
|
87
88
|
}
|
|
88
89
|
|
|
89
90
|
export function renderSrt(cues: SubtitleCue[]): string {
|
|
90
|
-
|
|
91
|
+
cues = quantizeCues(cues, 1);
|
|
91
92
|
return `${cues.map((cue, index) => `${index + 1}\n${srtTime(cue.beginMs)} --> ${srtTime(cue.endMs)}\n${cue.text}`).join("\n\n")}\n`;
|
|
92
93
|
}
|
|
93
94
|
|
|
@@ -96,7 +97,7 @@ function assText(value: string): string {
|
|
|
96
97
|
}
|
|
97
98
|
|
|
98
99
|
export function renderAss(cues: SubtitleCue[], style: Partial<AssStyle> = {}): string {
|
|
99
|
-
|
|
100
|
+
cues = quantizeCues(cues, 10);
|
|
100
101
|
const resolved = { ...DEFAULT_ASS_STYLE, ...style };
|
|
101
102
|
const header = `[Script Info]\nScriptType: v4.00+\nPlayResX: ${resolved.playResX}\nPlayResY: ${resolved.playResY}\nWrapStyle: 2\nScaledBorderAndShadow: yes\n\n[V4+ Styles]\nFormat: Name, Fontname, Fontsize, PrimaryColour, SecondaryColour, OutlineColour, BackColour, Bold, Italic, Underline, StrikeOut, ScaleX, ScaleY, Spacing, Angle, BorderStyle, Outline, Shadow, Alignment, MarginL, MarginR, MarginV, Encoding\nStyle: Default,${resolved.fontName},${resolved.fontSize},${resolved.primaryColour},${resolved.primaryColour},${resolved.outlineColour},${resolved.shadowColour},${resolved.bold ? -1 : 0},0,0,0,100,100,0,0,1,${resolved.outline},${resolved.shadow},${resolved.alignment},${resolved.marginL},${resolved.marginR},${resolved.marginV},1\n\n[Events]\nFormat: Layer, Start, End, Style, Name, MarginL, MarginR, MarginV, Effect, Text`;
|
|
102
103
|
const blur = resolved.blur > 0 ? `{\\blur${resolved.blur}}` : "";
|
|
@@ -107,18 +108,36 @@ export function renderAss(cues: SubtitleCue[], style: Partial<AssStyle> = {}): s
|
|
|
107
108
|
export async function exportSubtitleTracks(
|
|
108
109
|
cwd: string,
|
|
109
110
|
cues: SubtitleCue[],
|
|
110
|
-
input: { srtPath: string; assPath?: string; assStyle?: Partial<AssStyle
|
|
111
|
+
input: { srtPath: string; assPath?: string; assStyle?: Partial<AssStyle>; project?: SubtitleProject },
|
|
111
112
|
): Promise<{ srt: FileRef; ass?: FileRef }> {
|
|
112
113
|
assertCues(cues);
|
|
113
114
|
if (!input.srtPath.toLowerCase().endsWith(".srt")) throw new Error("srtPath must end in .srt");
|
|
114
115
|
if (input.assPath !== undefined && !input.assPath.toLowerCase().endsWith(".ass")) throw new Error("assPath must end in .ass");
|
|
115
|
-
const
|
|
116
|
+
const srtText = renderSrt(quantizeCues(cues, 1, input.project));
|
|
117
|
+
const assText = input.assPath ? renderAss(quantizeCues(cues, 10, input.project), input.assStyle) : undefined;
|
|
118
|
+
const srt = await writeNewWorkspaceFile(cwd, input.srtPath, srtText, "application/x-subrip; charset=utf-8");
|
|
116
119
|
if (!input.assPath) return { srt };
|
|
117
120
|
try {
|
|
118
|
-
const ass = await writeNewWorkspaceFile(cwd, input.assPath,
|
|
121
|
+
const ass = await writeNewWorkspaceFile(cwd, input.assPath, assText!, "text/x-ssa; charset=utf-8");
|
|
119
122
|
return { srt, ass };
|
|
120
123
|
} catch (error) {
|
|
121
124
|
await rm(await resolveExistingWorkspaceFile(cwd, srt.path), { force: true });
|
|
122
125
|
throw error;
|
|
123
126
|
}
|
|
124
127
|
}
|
|
128
|
+
|
|
129
|
+
export function quantizeCues(cues: SubtitleCue[], unitMs: number, project?: SubtitleProject): SubtitleCue[] {
|
|
130
|
+
assertCues(cues);
|
|
131
|
+
if (project) validateCues(cues, project);
|
|
132
|
+
const ranges = project ? segmentRanges(project.timelineSegments) : [];
|
|
133
|
+
const rounded = cues.map(c => {
|
|
134
|
+
const range = ranges.find(s => c.beginMs >= s.beginMs && c.endMs <= s.endMs);
|
|
135
|
+
return { ...c,
|
|
136
|
+
beginMs: Math.max(Math.round(c.beginMs / unitMs) * unitMs, range ? Math.ceil(range.beginMs / unitMs) * unitMs : 0),
|
|
137
|
+
endMs: Math.min(Math.round(c.endMs / unitMs) * unitMs, range ? Math.floor(range.endMs / unitMs) * unitMs : Infinity),
|
|
138
|
+
};
|
|
139
|
+
});
|
|
140
|
+
try { assertCues(rounded); } catch { throw new Error(`Subtitle timing cannot be represented at ${unitMs}ms precision without overlap or zero duration`); }
|
|
141
|
+
if (project) validateCues(rounded, project);
|
|
142
|
+
return rounded;
|
|
143
|
+
}
|
package/src/project.ts
CHANGED
|
@@ -1,6 +1,9 @@
|
|
|
1
1
|
import { createHash, randomUUID } from "node:crypto";
|
|
2
|
-
import { mkdir,
|
|
2
|
+
import { mkdir, rename, rm, writeFile } from "node:fs/promises";
|
|
3
3
|
import { dirname, join } from "node:path";
|
|
4
|
+
import { importRenderTimeline, parseTranscript } from "./timeline.ts";
|
|
5
|
+
import type { SourceTranscriptInput } from "./timeline.ts";
|
|
6
|
+
import { validateCues, validateReview, cuesFromGroups, textCoverageFinding } from "./validation.ts";
|
|
4
7
|
import { buildSubtitleCues, DEFAULT_SUBTITLE_POLICY, remapWordsToTimeline } from "./cues.ts";
|
|
5
8
|
import type {
|
|
6
9
|
AssStyle,
|
|
@@ -11,9 +14,12 @@ import type {
|
|
|
11
14
|
SubtitleLayoutPreset,
|
|
12
15
|
SubtitleSnapshot,
|
|
13
16
|
TimelineSegment,
|
|
14
|
-
|
|
17
|
+
TimelineWord,
|
|
18
|
+
WordGroup,
|
|
19
|
+
SubtitleReview,
|
|
20
|
+
SubtitleFinding,
|
|
15
21
|
} from "./contracts.ts";
|
|
16
|
-
import { exportSubtitleTracks,
|
|
22
|
+
import { exportSubtitleTracks, resolveAssStyle } from "./export.ts";
|
|
17
23
|
import {
|
|
18
24
|
readWorkspaceJson,
|
|
19
25
|
resolveExistingWorkspaceFile,
|
|
@@ -24,15 +30,13 @@ import {
|
|
|
24
30
|
|
|
25
31
|
const PROJECT_ID = /^[a-z0-9](?:[a-z0-9._-]{0,126}[a-z0-9])?$/u;
|
|
26
32
|
|
|
27
|
-
interface RawTranscript {
|
|
28
|
-
words?: unknown[];
|
|
29
|
-
sentences?: Array<{ words?: unknown[] }>;
|
|
30
|
-
}
|
|
31
|
-
|
|
32
33
|
export interface CreateSubtitleProjectInput {
|
|
33
34
|
projectId: string;
|
|
34
|
-
transcriptPath
|
|
35
|
-
sourceDurationMs
|
|
35
|
+
transcriptPath?: string;
|
|
36
|
+
sourceDurationMs?: number;
|
|
37
|
+
renderReceiptPath?: string;
|
|
38
|
+
sourceTranscripts?: SourceTranscriptInput[];
|
|
39
|
+
protectedPhrases?: string[];
|
|
36
40
|
layoutPreset?: SubtitleLayoutPreset;
|
|
37
41
|
timelineSegments?: TimelineSegment[];
|
|
38
42
|
policy?: Partial<SubtitlePolicy>;
|
|
@@ -41,7 +45,9 @@ export interface CreateSubtitleProjectInput {
|
|
|
41
45
|
export interface ApplySubtitleRevisionInput {
|
|
42
46
|
projectId: string;
|
|
43
47
|
expectedRevision: number;
|
|
44
|
-
cues
|
|
48
|
+
cues?: SubtitleCue[];
|
|
49
|
+
groups?: WordGroup[];
|
|
50
|
+
review?: SubtitleReview;
|
|
45
51
|
changeReason: string;
|
|
46
52
|
}
|
|
47
53
|
|
|
@@ -67,32 +73,6 @@ function projectRelative(projectId: string, path = ""): string {
|
|
|
67
73
|
return `.subtitles/projects/${projectId}${path ? `/${path}` : ""}`;
|
|
68
74
|
}
|
|
69
75
|
|
|
70
|
-
function normalizedWord(value: unknown): TranscriptWord | undefined {
|
|
71
|
-
if (!value || typeof value !== "object") return undefined;
|
|
72
|
-
const item = value as Record<string, unknown>;
|
|
73
|
-
const text = typeof item.text === "string" ? item.text : typeof item.word === "string" ? item.word : undefined;
|
|
74
|
-
const beginMs = typeof item.beginMs === "number" ? item.beginMs : typeof item.start === "number" ? item.start * 1_000 : undefined;
|
|
75
|
-
const endMs = typeof item.endMs === "number" ? item.endMs : typeof item.end === "number" ? item.end * 1_000 : undefined;
|
|
76
|
-
if (!text || beginMs === undefined || endMs === undefined) return undefined;
|
|
77
|
-
const trimmed = text.trim();
|
|
78
|
-
if (!trimmed || /^(?:<sil>|\[sil\])$/iu.test(trimmed)) return undefined;
|
|
79
|
-
return {
|
|
80
|
-
text: trimmed,
|
|
81
|
-
beginMs,
|
|
82
|
-
endMs,
|
|
83
|
-
punctuation: typeof item.punctuation === "string" ? item.punctuation : "",
|
|
84
|
-
};
|
|
85
|
-
}
|
|
86
|
-
|
|
87
|
-
function transcriptWords(transcript: RawTranscript): TranscriptWord[] {
|
|
88
|
-
const raw = Array.isArray(transcript.sentences)
|
|
89
|
-
? transcript.sentences.flatMap((sentence) => Array.isArray(sentence.words) ? sentence.words : [])
|
|
90
|
-
: Array.isArray(transcript.words) ? transcript.words : [];
|
|
91
|
-
const words = raw.map(normalizedWord).filter((word): word is TranscriptWord => word !== undefined);
|
|
92
|
-
if (words.length === 0) throw new Error("Transcript contains no usable word timestamps");
|
|
93
|
-
return words;
|
|
94
|
-
}
|
|
95
|
-
|
|
96
76
|
function validateSourceAndTimeline(sourceDurationMs: number, timelineSegments: TimelineSegment[]): void {
|
|
97
77
|
if (!Number.isFinite(sourceDurationMs) || sourceDurationMs <= 0) throw new Error("sourceDurationMs must be positive");
|
|
98
78
|
if (timelineSegments.some((segment) => segment.sourceEndMs > sourceDurationMs)) {
|
|
@@ -101,6 +81,8 @@ function validateSourceAndTimeline(sourceDurationMs: number, timelineSegments: T
|
|
|
101
81
|
}
|
|
102
82
|
|
|
103
83
|
function resolvedPolicy(layoutPreset: SubtitleLayoutPreset, input: Partial<SubtitlePolicy> | undefined): SubtitlePolicy {
|
|
84
|
+
if (layoutPreset !== "landscape" && layoutPreset !== "portrait") throw new Error("Invalid subtitle layout preset");
|
|
85
|
+
if (layoutPreset === "portrait" && input?.maxLines !== undefined && input.maxLines > 2) throw new Error("Portrait subtitles allow at most two lines");
|
|
104
86
|
if (layoutPreset === "landscape" && input?.maxLines !== undefined && input.maxLines !== 1) {
|
|
105
87
|
throw new Error("Landscape subtitles must use one line");
|
|
106
88
|
}
|
|
@@ -124,7 +106,7 @@ function timelineDuration(segments: TimelineSegment[]): number {
|
|
|
124
106
|
return segments.reduce((total, segment) => total + segment.sourceEndMs - segment.sourceStartMs, 0);
|
|
125
107
|
}
|
|
126
108
|
|
|
127
|
-
function snapshot(projectId: string, revision: number, parentRevision: number | null, cues: SubtitleCue[], changeReason: string): SubtitleSnapshot {
|
|
109
|
+
function snapshot(projectId: string, revision: number, parentRevision: number | null, cues: SubtitleCue[], changeReason: string, extras: Pick<SubtitleSnapshot, "groups" | "review" | "findings"> = {}): SubtitleSnapshot {
|
|
128
110
|
const base = {
|
|
129
111
|
schemaVersion: 1 as const,
|
|
130
112
|
projectId,
|
|
@@ -133,6 +115,7 @@ function snapshot(projectId: string, revision: number, parentRevision: number |
|
|
|
133
115
|
createdAt: new Date().toISOString(),
|
|
134
116
|
changeReason,
|
|
135
117
|
cues,
|
|
118
|
+
...extras,
|
|
136
119
|
};
|
|
137
120
|
return { ...base, snapshotSha256: sha256Json(base) };
|
|
138
121
|
}
|
|
@@ -151,15 +134,22 @@ async function writeInternalJson(cwd: string, relativePath: string, value: unkno
|
|
|
151
134
|
|
|
152
135
|
export async function createSubtitleProject(cwd: string, input: CreateSubtitleProjectInput) {
|
|
153
136
|
assertProjectId(input.projectId);
|
|
154
|
-
|
|
155
|
-
const
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
137
|
+
if (input.renderReceiptPath && (input.timelineSegments || input.sourceDurationMs !== undefined)) throw new Error("renderReceiptPath is mutually exclusive with legacy timelineSegments/sourceDurationMs");
|
|
138
|
+
const imported = input.renderReceiptPath ? await importRenderTimeline(cwd, input.renderReceiptPath, input.transcriptPath, input.sourceTranscripts) : undefined;
|
|
139
|
+
if (!imported && (!input.transcriptPath || input.sourceDurationMs === undefined || input.sourceTranscripts)) throw new Error("Legacy input requires transcriptPath and sourceDurationMs; sourceTranscripts requires renderReceiptPath");
|
|
140
|
+
const transcript = imported?.sources[0]!.transcript ?? await snapshotExistingWorkspaceFile(cwd, input.transcriptPath!, "application/json");
|
|
141
|
+
if (!imported && input.timelineSegments?.some(s => s.outputStartMs !== undefined || s.outputEndMs !== undefined || s.sourcePath !== undefined)) throw new Error("Explicit output timing requires a render receipt");
|
|
142
|
+
const segments = imported?.segments ?? input.timelineSegments ?? [{ id: "a-1", sourceStartMs: 0, sourceEndMs: input.sourceDurationMs! }];
|
|
143
|
+
if (!imported) validateSourceAndTimeline(input.sourceDurationMs!, segments);
|
|
144
|
+
const durationMs = imported?.durationMs ?? timelineDuration(segments);
|
|
145
|
+
const findings: SubtitleFinding[] = imported?.findings ?? [];
|
|
146
|
+
const words = imported?.words ?? remapWordsToTimeline(parseTranscript(await readWorkspaceJson(cwd, input.transcriptPath!, "word transcript")), segments, findings);
|
|
159
147
|
const layoutPreset = input.layoutPreset ?? "landscape";
|
|
160
148
|
const policy = resolvedPolicy(layoutPreset, input.policy);
|
|
161
|
-
const cues = buildSubtitleCues(
|
|
149
|
+
const cues = buildSubtitleCues(words, durationMs, policy, input.protectedPhrases);
|
|
162
150
|
if (cues.length === 0) throw new Error("Retained A-roll timeline contains no transcribed words");
|
|
151
|
+
const wordMapText = `${JSON.stringify({ words, findings }, null, 2)}\n`;
|
|
152
|
+
const wordMap = { path: projectRelative(input.projectId, "words.json"), bytes: Buffer.byteLength(wordMapText), sha256: createHash("sha256").update(wordMapText).digest("hex"), mimeType: "application/json" };
|
|
163
153
|
const now = new Date().toISOString();
|
|
164
154
|
const project: SubtitleProject = {
|
|
165
155
|
schemaVersion: 1,
|
|
@@ -168,13 +158,15 @@ export async function createSubtitleProject(cwd: string, input: CreateSubtitlePr
|
|
|
168
158
|
updatedAt: now,
|
|
169
159
|
currentRevision: 1,
|
|
170
160
|
layoutPreset,
|
|
171
|
-
sourceDurationMs: input.sourceDurationMs,
|
|
161
|
+
sourceDurationMs: input.sourceDurationMs ?? Math.max(...segments.map(s => s.sourceEndMs)),
|
|
172
162
|
timelineDurationMs: durationMs,
|
|
173
163
|
transcript,
|
|
174
164
|
timelineSegments: segments,
|
|
175
165
|
policy,
|
|
166
|
+
wordMap,
|
|
167
|
+
...(imported ? { sources: imported.sources, renderBinding: imported.binding } : {}),
|
|
176
168
|
};
|
|
177
|
-
const first = snapshot(input.projectId, 1, null, cues, "Initial subtitle generation from word timestamps");
|
|
169
|
+
const first = snapshot(input.projectId, 1, null, cues, "Initial subtitle generation from word timestamps", { review: validateReview(), findings: [...findings, ...validateCues(cues, project)] });
|
|
178
170
|
const projectsMarker = await resolveNewWorkspaceFile(cwd, ".subtitles/projects/.marker");
|
|
179
171
|
const projectsRoot = dirname(projectsMarker);
|
|
180
172
|
await mkdir(projectsRoot, { recursive: true });
|
|
@@ -182,6 +174,7 @@ export async function createSubtitleProject(cwd: string, input: CreateSubtitlePr
|
|
|
182
174
|
const temporary = join(projectsRoot, `.${input.projectId}.${randomUUID()}.tmp`);
|
|
183
175
|
await mkdir(join(temporary, "snapshots"), { recursive: true });
|
|
184
176
|
try {
|
|
177
|
+
await writeFile(join(temporary, "words.json"), wordMapText, { flag: "wx" });
|
|
185
178
|
await writeFile(join(temporary, "project.json"), `${JSON.stringify(project, null, 2)}\n`, { flag: "wx" });
|
|
186
179
|
await writeFile(join(temporary, "snapshots/1.json"), `${JSON.stringify(first, null, 2)}\n`, { flag: "wx" });
|
|
187
180
|
await rename(temporary, target);
|
|
@@ -210,20 +203,11 @@ export async function getSubtitleProject(cwd: string, projectId: string, revisio
|
|
|
210
203
|
return { project, snapshot: selectedSnapshot };
|
|
211
204
|
}
|
|
212
205
|
|
|
213
|
-
function
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
if (cues.length === 0 || cues.length > 10_000) throw new Error("cues must contain 1-10000 items");
|
|
219
|
-
renderSrt(cues);
|
|
220
|
-
if (cues.at(-1)!.endMs > project.timelineDurationMs) throw new Error("Subtitle cue exceeds the final timeline duration");
|
|
221
|
-
if (project.layoutPreset === "landscape") {
|
|
222
|
-
if (cues.some((cue) => /[\r\n]/u.test(cue.text))) throw new Error("Landscape subtitles must use one line");
|
|
223
|
-
if (cues.some((cue) => visibleCharacters(cue.text) > project.policy.maxLineChars)) {
|
|
224
|
-
throw new Error(`Landscape subtitle exceeds ${project.policy.maxLineChars} visible characters`);
|
|
225
|
-
}
|
|
226
|
-
}
|
|
206
|
+
export async function getProjectWords(cwd: string, project: SubtitleProject): Promise<{ words: TimelineWord[]; findings: SubtitleFinding[] }> {
|
|
207
|
+
if (!project.wordMap) return { words: remapWordsToTimeline(parseTranscript(await readWorkspaceJson(cwd, project.transcript.path, "transcript")), project.timelineSegments), findings: [] };
|
|
208
|
+
const current = await snapshotExistingWorkspaceFile(cwd, project.wordMap.path, "application/json");
|
|
209
|
+
if (current.sha256 !== project.wordMap.sha256) throw new Error("Subtitle word map changed");
|
|
210
|
+
return await readWorkspaceJson(cwd, project.wordMap.path, "subtitle word map");
|
|
227
211
|
}
|
|
228
212
|
|
|
229
213
|
async function withProjectLock<T>(cwd: string, projectId: string, action: (directory: string) => Promise<T>): Promise<T> {
|
|
@@ -249,9 +233,13 @@ export async function applySubtitleRevision(cwd: string, input: ApplySubtitleRev
|
|
|
249
233
|
if (project.currentRevision !== input.expectedRevision) {
|
|
250
234
|
throw new Error(`Project ${input.projectId} expected revision ${input.expectedRevision} but current revision is ${project.currentRevision}`);
|
|
251
235
|
}
|
|
252
|
-
|
|
236
|
+
if ((input.cues === undefined) === (input.groups === undefined)) throw new Error("Submit exactly one of cues or groups");
|
|
237
|
+
await assertTranscriptUnchanged(cwd, project);
|
|
238
|
+
const mapped = await getProjectWords(cwd, project);
|
|
239
|
+
const cues = input.groups ? cuesFromGroups(input.groups, mapped.words, project) : input.cues!;
|
|
240
|
+
const findings = [...mapped.findings, ...validateCues(cues, project), ...textCoverageFinding(cues, mapped.words)];
|
|
253
241
|
const revision = project.currentRevision + 1;
|
|
254
|
-
const nextSnapshot = snapshot(input.projectId, revision, project.currentRevision,
|
|
242
|
+
const nextSnapshot = snapshot(input.projectId, revision, project.currentRevision, cues, input.changeReason.trim(), { ...(input.groups ? { groups: input.groups } : {}), review: validateReview(input.review), findings });
|
|
255
243
|
const snapshotPath = join(directory, `snapshots/${revision}.json`);
|
|
256
244
|
await writeFile(snapshotPath, `${JSON.stringify(nextSnapshot, null, 2)}\n`, { flag: "wx" });
|
|
257
245
|
const nextProject: SubtitleProject = { ...project, currentRevision: revision, updatedAt: new Date().toISOString() };
|
|
@@ -269,9 +257,10 @@ export async function applySubtitleRevision(cwd: string, input: ApplySubtitleRev
|
|
|
269
257
|
}
|
|
270
258
|
|
|
271
259
|
async function assertTranscriptUnchanged(cwd: string, project: SubtitleProject): Promise<void> {
|
|
272
|
-
const
|
|
273
|
-
|
|
274
|
-
|
|
260
|
+
const refs = project.sources?.map(s => s.transcript) ?? [project.transcript];
|
|
261
|
+
for (const ref of refs) {
|
|
262
|
+
const current = await snapshotExistingWorkspaceFile(cwd, ref.path, ref.mimeType);
|
|
263
|
+
if (current.sha256 !== ref.sha256 || current.bytes !== ref.bytes) throw new Error("Source transcript has changed since subtitle project creation");
|
|
275
264
|
}
|
|
276
265
|
}
|
|
277
266
|
|
|
@@ -284,6 +273,15 @@ export async function exportSubtitleProject(cwd: string, input: {
|
|
|
284
273
|
}) {
|
|
285
274
|
const { project, snapshot: selectedSnapshot } = await getSubtitleProject(cwd, input.projectId, input.revision);
|
|
286
275
|
await assertTranscriptUnchanged(cwd, project);
|
|
276
|
+
validateCues(selectedSnapshot.cues, project);
|
|
277
|
+
await getProjectWords(cwd, project);
|
|
278
|
+
if (project.renderBinding) {
|
|
279
|
+
const refs = [project.renderBinding.receipt, project.renderBinding.video, ...(project.sources ?? []).map(s => s.source)];
|
|
280
|
+
for (const ref of refs) {
|
|
281
|
+
const current = await snapshotExistingWorkspaceFile(cwd, ref.path, ref.mimeType);
|
|
282
|
+
if (current.sha256 !== ref.sha256 || current.bytes !== ref.bytes) throw new Error(`Subtitle render provenance changed: ${ref.path}`);
|
|
283
|
+
}
|
|
284
|
+
}
|
|
287
285
|
const resolvedAssStyle = input.assPath
|
|
288
286
|
? resolveAssStyle(project.layoutPreset ?? "landscape", input.assStyle)
|
|
289
287
|
: undefined;
|
|
@@ -291,6 +289,7 @@ export async function exportSubtitleProject(cwd: string, input: {
|
|
|
291
289
|
srtPath: input.srtPath,
|
|
292
290
|
...(input.assPath ? { assPath: input.assPath } : {}),
|
|
293
291
|
...(resolvedAssStyle ? { assStyle: resolvedAssStyle } : {}),
|
|
292
|
+
project,
|
|
294
293
|
});
|
|
295
294
|
const exportId = randomUUID();
|
|
296
295
|
const receiptPath = projectRelative(input.projectId, `exports/${exportId}.json`);
|
|
@@ -302,9 +301,15 @@ export async function exportSubtitleProject(cwd: string, input: {
|
|
|
302
301
|
createdAt: new Date().toISOString(),
|
|
303
302
|
snapshotSha256: selectedSnapshot.snapshotSha256,
|
|
304
303
|
tracks,
|
|
304
|
+
structure: "passed" as const,
|
|
305
|
+
review: selectedSnapshot.review ?? validateReview(),
|
|
306
|
+
...(project.renderBinding ? { primaryTimelineSha256: project.renderBinding.primaryTimelineSha256, video: project.renderBinding.video } : {}),
|
|
305
307
|
receiptPath,
|
|
306
308
|
};
|
|
307
309
|
const receipt: SubtitleExportReceipt = { ...base, receiptSha256: sha256Json(base) };
|
|
308
310
|
await writeInternalJson(cwd, receiptPath, receipt);
|
|
309
|
-
return { project, snapshot: selectedSnapshot, receipt
|
|
311
|
+
return { project, snapshot: selectedSnapshot, receipt, subtitleTrack: {
|
|
312
|
+
sourcePath: (tracks.ass ?? tracks.srt).path, format: tracks.ass ? "ass" as const : "srt" as const, mode: "burn-in" as const,
|
|
313
|
+
...(project.renderBinding ? { exportReceiptPath: receiptPath } : {}),
|
|
314
|
+
} };
|
|
310
315
|
}
|
package/src/timeline.ts
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
import { createHash } from 'node:crypto';
|
|
2
|
+
import type { FileRef, SubtitleFinding, TimelineSegment, TimelineWord, TranscriptWord } from './contracts.ts';
|
|
3
|
+
import { remapWordsToTimeline } from './cues.ts';
|
|
4
|
+
import { readWorkspaceJson, snapshotExistingWorkspaceFile } from './workspace.ts';
|
|
5
|
+
|
|
6
|
+
export function parseTranscript(value: unknown): TranscriptWord[] {
|
|
7
|
+
const input = value as { words?: unknown[]; sentences?: Array<{ words?: unknown[] }> };
|
|
8
|
+
const raw = Array.isArray(input?.sentences) ? input.sentences.flatMap(s => s.words ?? []) : input?.words;
|
|
9
|
+
if (!Array.isArray(raw)) throw new Error('Transcript contains no usable word timestamps');
|
|
10
|
+
const words: TranscriptWord[] = [];
|
|
11
|
+
for (const [index, value] of raw.entries()) {
|
|
12
|
+
const w = value as Record<string, unknown>;
|
|
13
|
+
const text = typeof w?.text === 'string' ? w.text : w?.word;
|
|
14
|
+
if (typeof text === 'string' && /^(?:<sil>|\[sil\])$/iu.test(text.trim())) continue;
|
|
15
|
+
const beginMs = typeof w?.beginMs === 'number' ? w.beginMs : typeof w?.start === 'number' ? w.start * 1000 : NaN;
|
|
16
|
+
const endMs = typeof w?.endMs === 'number' ? w.endMs : typeof w?.end === 'number' ? w.end * 1000 : NaN;
|
|
17
|
+
if (typeof text !== 'string' || !text.trim() || !Number.isFinite(beginMs) || !Number.isFinite(endMs)) throw new Error(`Invalid transcript word ${index}; refusing to silently discard text`);
|
|
18
|
+
words.push({ text: text.trim(), beginMs, endMs, punctuation: typeof w.punctuation === 'string' ? w.punctuation : '' });
|
|
19
|
+
}
|
|
20
|
+
if (!words.length) throw new Error('Transcript contains no usable word timestamps');
|
|
21
|
+
return words;
|
|
22
|
+
}
|
|
23
|
+
|
|
24
|
+
export interface SourceTranscriptInput { sourcePath: string; transcriptPath: string }
|
|
25
|
+
interface Timing {
|
|
26
|
+
policy: string; frameRate: number; sampleRate: number; totalFrames: number; totalSamples: number; outputDurationSeconds: number;
|
|
27
|
+
segments: Array<{ id: string; sourcePath: string; sourceStartSeconds: number; sourceEndSeconds: number; outputStartSeconds: number; outputEndSeconds: number; outputStartFrame: number; outputEndFrame: number; outputStartSample: number; outputEndSample: number; durationAdjustmentSeconds: number }>;
|
|
28
|
+
}
|
|
29
|
+
interface Receipt { primaryTimelineSha256?: string; path: string; bytes: number; sha256: string; projectId: string; snapshotRevision: number; sourceSha256s: string[]; timelineTiming: Timing }
|
|
30
|
+
|
|
31
|
+
// Versioned, ordered tuple shared with pi-media. Excludes overlays, BGM, IDs and output file bytes.
|
|
32
|
+
export function primaryTimelineHash(timing: Timing, sourceHashes: Map<string, string>): string {
|
|
33
|
+
return createHash('sha256').update(JSON.stringify(['primary-timeline-v1', timing.frameRate, timing.sampleRate,
|
|
34
|
+
timing.segments.map(s => [sourceHashes.get(s.sourcePath), s.sourceStartSeconds, s.sourceEndSeconds, s.outputStartSample, s.outputEndSample])])).digest('hex');
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
export async function importRenderTimeline(cwd: string, receiptPath: string, transcriptPath?: string, inputs?: SourceTranscriptInput[]) {
|
|
38
|
+
const receiptRef = await snapshotExistingWorkspaceFile(cwd, receiptPath, 'application/json');
|
|
39
|
+
const receipt = await readWorkspaceJson<Receipt>(cwd, receiptPath, 'render receipt');
|
|
40
|
+
const t = receipt.timelineTiming;
|
|
41
|
+
if (!t || t.policy !== 'cumulative-frame-rounding' || !Number.isFinite(t.frameRate) || t.frameRate < 1 || t.frameRate > 240 || t.sampleRate !== 48000
|
|
42
|
+
|| !Array.isArray(t.segments) || !t.segments.length || t.segments.length > 1000 || !Array.isArray(receipt.sourceSha256s)
|
|
43
|
+
|| !receipt.projectId || !Number.isInteger(receipt.snapshotRevision) || receipt.snapshotRevision < 1) throw new Error('Invalid or unsupported render timeline receipt');
|
|
44
|
+
let frames = 0, samples = 0, ideal = 0;
|
|
45
|
+
const ids = new Set<string>();
|
|
46
|
+
for (const s of t.segments) {
|
|
47
|
+
if (!s.id || ids.has(s.id) || !s.sourcePath || !Number.isFinite(s.sourceStartSeconds) || s.sourceStartSeconds < 0
|
|
48
|
+
|| !Number.isFinite(s.sourceEndSeconds) || s.sourceEndSeconds <= s.sourceStartSeconds) throw new Error('Invalid render segment');
|
|
49
|
+
ids.add(s.id);
|
|
50
|
+
ideal += s.sourceEndSeconds - s.sourceStartSeconds;
|
|
51
|
+
const endFrame = Math.round(ideal * t.frameRate + 1e-8), endSample = Math.round(endFrame / t.frameRate * t.sampleRate);
|
|
52
|
+
if (endFrame <= frames || s.outputStartFrame !== frames || s.outputEndFrame !== endFrame || s.outputStartSample !== samples || s.outputEndSample !== endSample
|
|
53
|
+
|| !Number.isFinite(s.outputStartSeconds) || Math.abs(s.outputStartSeconds - frames / t.frameRate) > 1e-9
|
|
54
|
+
|| !Number.isFinite(s.outputEndSeconds) || Math.abs(s.outputEndSeconds - endFrame / t.frameRate) > 1e-9
|
|
55
|
+
|| !Number.isFinite(s.durationAdjustmentSeconds) || Math.abs(s.durationAdjustmentSeconds - (endFrame - frames) / t.frameRate + s.sourceEndSeconds - s.sourceStartSeconds) > 1e-9) throw new Error(`Inconsistent render timing for ${s.id}`);
|
|
56
|
+
frames = endFrame; samples = endSample;
|
|
57
|
+
}
|
|
58
|
+
if (t.totalFrames !== frames || t.totalSamples !== samples || !Number.isFinite(t.outputDurationSeconds) || Math.abs(t.outputDurationSeconds - frames / t.frameRate) > 1e-9) throw new Error('Inconsistent render duration');
|
|
59
|
+
const video = await snapshotExistingWorkspaceFile(cwd, receipt.path, 'video/mp4');
|
|
60
|
+
if (video.sha256 !== receipt.sha256 || video.bytes !== receipt.bytes) throw new Error('Rendered video differs from its receipt');
|
|
61
|
+
const paths = [...new Set(t.segments.map(s => s.sourcePath))];
|
|
62
|
+
if (transcriptPath && inputs) throw new Error('Use transcriptPath or sourceTranscripts, not both');
|
|
63
|
+
if (transcriptPath && paths.length !== 1) throw new Error('Multi-source timeline requires sourceTranscripts for every primary source');
|
|
64
|
+
const bindings = inputs ?? (transcriptPath ? [{ sourcePath: paths[0]!, transcriptPath }] : []);
|
|
65
|
+
if (bindings.length !== paths.length || new Set(bindings.map(b => b.sourcePath)).size !== bindings.length || paths.some(p => !bindings.some(b => b.sourcePath === p))) throw new Error('sourceTranscripts must match every primary source exactly once');
|
|
66
|
+
const sources: Array<{ source: FileRef; transcript: FileRef }> = [];
|
|
67
|
+
const wordsByPath = new Map<string, TranscriptWord[]>(), sourceHashes = new Map<string, string>();
|
|
68
|
+
for (const b of bindings) {
|
|
69
|
+
const source = await snapshotExistingWorkspaceFile(cwd, b.sourcePath, 'application/octet-stream');
|
|
70
|
+
if (!receipt.sourceSha256s.includes(source.sha256)) throw new Error(`Source changed since render: ${b.sourcePath}`);
|
|
71
|
+
const transcript = await snapshotExistingWorkspaceFile(cwd, b.transcriptPath, 'application/json');
|
|
72
|
+
wordsByPath.set(b.sourcePath, parseTranscript(await readWorkspaceJson(cwd, b.transcriptPath, 'word transcript')));
|
|
73
|
+
sourceHashes.set(b.sourcePath, source.sha256); sources.push({ source, transcript });
|
|
74
|
+
}
|
|
75
|
+
const fingerprint = primaryTimelineHash(t, sourceHashes);
|
|
76
|
+
if (receipt.primaryTimelineSha256 && receipt.primaryTimelineSha256 !== fingerprint) throw new Error("Render primary timeline fingerprint differs from its sources");
|
|
77
|
+
const segments: TimelineSegment[] = t.segments.map(s => ({ id: s.id, sourcePath: s.sourcePath,
|
|
78
|
+
sourceStartMs: s.sourceStartSeconds * 1000, sourceEndMs: s.sourceEndSeconds * 1000,
|
|
79
|
+
outputStartMs: s.outputStartSeconds * 1000, outputEndMs: s.outputEndSeconds * 1000 }));
|
|
80
|
+
const findings: SubtitleFinding[] = [], words: TimelineWord[] = [];
|
|
81
|
+
for (const s of segments) words.push(...remapWordsToTimeline(wordsByPath.get(s.sourcePath!)!, [s], findings));
|
|
82
|
+
return { segments, words, sources, findings, durationMs: t.outputDurationSeconds * 1000,
|
|
83
|
+
binding: { receipt: receiptRef, video, projectId: receipt.projectId, snapshotRevision: receipt.snapshotRevision, primaryTimelineSha256: fingerprint } };
|
|
84
|
+
}
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
import type { SubtitleCue, SubtitleFinding, SubtitleProject, SubtitleReview, TimelineSegment, TimelineWord, WordGroup } from './contracts.ts';
|
|
2
|
+
|
|
3
|
+
export function segmentRanges(segments: TimelineSegment[]) {
|
|
4
|
+
let offset = 0;
|
|
5
|
+
return segments.map(s => {
|
|
6
|
+
const beginMs = s.outputStartMs ?? offset;
|
|
7
|
+
const endMs = s.outputEndMs ?? beginMs + s.sourceEndMs - s.sourceStartMs;
|
|
8
|
+
offset = endMs;
|
|
9
|
+
return { id: s.id, beginMs, endMs };
|
|
10
|
+
});
|
|
11
|
+
}
|
|
12
|
+
export function visibleCharacters(text: string) { return [...text].filter(c => !/[\s\p{Cf}]/u.test(c)).length; }
|
|
13
|
+
export function validateCues(cues: SubtitleCue[], project: SubtitleProject): SubtitleFinding[] {
|
|
14
|
+
if (!Array.isArray(cues) || !cues.length || cues.length > 10000) throw new Error('cues must contain 1-10000 items');
|
|
15
|
+
const ranges = segmentRanges(project.timelineSegments), findings: SubtitleFinding[] = [], ids = new Set<string>();
|
|
16
|
+
let previousEnd = 0;
|
|
17
|
+
for (const c of cues) {
|
|
18
|
+
if (!c.id || ids.has(c.id) || typeof c.text !== 'string' || !c.text.trim() || !Number.isFinite(c.beginMs) || !Number.isFinite(c.endMs)
|
|
19
|
+
|| c.beginMs < 0 || c.endMs <= c.beginMs || c.beginMs < previousEnd) throw new Error(`Invalid or overlapping subtitle cue: ${c.id}`);
|
|
20
|
+
ids.add(c.id); previousEnd = c.endMs;
|
|
21
|
+
if (c.endMs > project.timelineDurationMs) throw new Error(`Subtitle cue exceeds the final timeline duration: ${c.id}`);
|
|
22
|
+
if (!ranges.some(s => c.beginMs >= s.beginMs && c.endMs <= s.endMs)) throw new Error(`Subtitle cue crosses A-roll edit boundary: ${c.id}`);
|
|
23
|
+
if (project.layoutPreset === 'landscape' && /[\r\n]/u.test(c.text)) throw new Error('Landscape subtitles must use one line');
|
|
24
|
+
const lines = c.text.split(/\r?\n/u);
|
|
25
|
+
if (lines.length > project.policy.maxLines || lines.some(l => visibleCharacters(l) > project.policy.maxLineChars)) throw new Error(`Subtitle ${c.id} exceeds ${project.policy.maxLineChars} visible characters per line or line count`);
|
|
26
|
+
const duration = c.endMs - c.beginMs, count = visibleCharacters(c.text);
|
|
27
|
+
if (duration < project.policy.minDurationMs) findings.push({ code: 'short-cue', severity: 'warning', cueId: c.id, message: `${duration}ms; no safe room to extend` });
|
|
28
|
+
if (duration > project.policy.maxDurationMs) findings.push({ code: 'long-cue', severity: 'warning', cueId: c.id, message: `${duration}ms exceeds preferred duration` });
|
|
29
|
+
if (count / (duration / 1000) > project.policy.targetCps) findings.push({ code: 'fast-cue', severity: 'warning', cueId: c.id, message: `${(count * 1000 / duration).toFixed(2)} characters/second` });
|
|
30
|
+
if (count === 1) findings.push({ code: 'orphan-cue', severity: 'warning', cueId: c.id, message: 'Single-character cue requires semantic review' });
|
|
31
|
+
}
|
|
32
|
+
return findings;
|
|
33
|
+
}
|
|
34
|
+
export function validateReview(review: SubtitleReview = {}): SubtitleReview {
|
|
35
|
+
for (const entry of Object.values(review)) {
|
|
36
|
+
if (!entry || !['pending', 'passed'].includes(entry.status) || (entry.status === 'passed' && !entry.evidence?.trim())) throw new Error('Passed review requires concrete listening/visual evidence');
|
|
37
|
+
}
|
|
38
|
+
return { text: { status: 'pending' }, sync: { status: 'pending' }, visual: { status: 'pending' }, ...review };
|
|
39
|
+
}
|
|
40
|
+
function normalize(text: string) { return text.replace(/[\s\p{P}\p{Cf}]/gu, ''); }
|
|
41
|
+
export function textCoverageFinding(cues: SubtitleCue[], words: TimelineWord[]): SubtitleFinding[] {
|
|
42
|
+
return normalize(cues.map(c => c.text).join('')) === normalize(words.map(w => w.text).join('')) ? []
|
|
43
|
+
: [{ code: 'text-changed', severity: 'warning', message: 'Reviewed text differs from ASR; verify corrections against audio. Use word groups for exact coverage checks.' }];
|
|
44
|
+
}
|
|
45
|
+
export function cuesFromGroups(groups: WordGroup[], words: TimelineWord[], project: SubtitleProject): SubtitleCue[] {
|
|
46
|
+
if (!groups.length || groups.length > 10000) throw new Error('groups must contain 1-10000 complete ranges');
|
|
47
|
+
const positions = new Map(words.map((w, i) => [w.id, i]));
|
|
48
|
+
let cursor = 0;
|
|
49
|
+
const cues = groups.map((g, i) => {
|
|
50
|
+
const start = positions.get(g.startWordId), end = positions.get(g.endWordId);
|
|
51
|
+
if (start === undefined || end === undefined || start !== cursor || end < start) throw new Error(`Group ${i + 1} has missing, duplicated, out-of-order or unknown words`);
|
|
52
|
+
const selected = words.slice(start, end + 1), first = selected[0]!, last = selected.at(-1)!;
|
|
53
|
+
if (selected.some(w => w.timelineSegmentId !== first.timelineSegmentId)) throw new Error(`Group ${i + 1} crosses A-roll edit boundary`);
|
|
54
|
+
let original = '';
|
|
55
|
+
for (const w of selected) original += `${original && /[A-Za-z0-9]$/u.test(original) && /^[A-Za-z0-9]/u.test(w.text) ? ' ' : ''}${w.text}${w.punctuation}`;
|
|
56
|
+
if (g.text !== undefined && normalize(g.text) !== normalize(original) && !g.correctionReason?.trim()) throw new Error(`Group ${i + 1}: text correction requires correctionReason based on audio`);
|
|
57
|
+
cursor = end + 1;
|
|
58
|
+
return { id: `cue-${String(i + 1).padStart(3, '0')}`, beginMs: first.beginMs, endMs: last.endMs, text: g.text ?? original };
|
|
59
|
+
});
|
|
60
|
+
if (cursor !== words.length) throw new Error('Groups omit trailing transcript words');
|
|
61
|
+
const ranges = segmentRanges(project.timelineSegments);
|
|
62
|
+
return cues.map((c, i) => {
|
|
63
|
+
const range = ranges.find(r => c.beginMs >= r.beginMs && c.endMs <= r.endMs)!;
|
|
64
|
+
return { ...c, endMs: Math.max(c.endMs, Math.min(c.beginMs + project.policy.minDurationMs, cues[i + 1]?.beginMs ?? project.timelineDurationMs, range.endMs)) };
|
|
65
|
+
});
|
|
66
|
+
}
|