@wji.ai/cli 0.7.1 → 0.8.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -404,13 +404,36 @@ world, same faces, right size, nobody extra), fix, then continue. An episode is
404
404
  a wrong direction costs the whole batch — which is why this is opt-in and the default references
405
405
  the sheets directly.
406
406
 
407
- ## Soundtrack node — Seed Audio is not a TTS
407
+ ## Soundtrack — scored over the edit, not per clip
408
+
409
+ Each segment is a separate generation, so music and sound design made inside a clip stop at its
410
+ last frame: a cue cannot swell across a cut into the next segment, and every clip invents its own
411
+ score. So by default (`params.audioTrack: post`) **the clips carry no music**: the video prompt
412
+ says `No music.` (H3 `non_diegetic_music: N/A`, Wan `无配乐`), keeps the dialogue, the ambience and
413
+ the in-picture sfx, and drops every sfx marked `post: true` (stingers, risers, whooshes, impact
414
+ hits — sound design outside the story world). The storyboard still writes `sound.music` (intent and
415
+ cues) and the `post: true` hits: they are the **spotting notes**. When every clip is in, the episode
416
+ is edited, and the Sound Designer scores the edit as one piece on its timecode:
417
+
418
+ ```bash
419
+ $SD prompt <project> score E01 # planned spotting sheet: SMPTE rows + ≤120 s Seed Audio sections
420
+ ```
421
+
422
+ The sheet lays the segments end to end — scene beds (BGE), every line (where the music ducks), each
423
+ segment's music cues at their episode time, the `post` sound design, the sfx already in the clips
424
+ (never doubled) and the hook / turns / 爽点 / cliff as the 起承转合 anchors. It is the *planned*
425
+ timeline: the Sound Designer conforms it to the edit's real cuts
426
+ (`mix-over-video.md`) before any prompt is written. `manifest`
427
+ then asks for the episode's `wji.assets.edit` and `wji.assets.mix`.
428
+
429
+ ### Per-segment soundtrack node (`audioTrack: bed` / `full`)
408
430
 
409
431
  `seed-audio-1.0` (the voice-sample model of stage 2) generates a whole **audio scene** from one
410
432
  prompt: speech, sound effects, ambience and music in one track, up to 120 s, and a
411
433
  **`[start s:end s]` timecode in front of an element pins it to that interval** (100 ms
412
- precision). So every segment gets a soundtrack node next to its video node, and the prompt is
413
- derived — not written — from `segment.sound` and the cut timing:
434
+ precision). In `bed` / `full` every segment gets a soundtrack node next to its video node and the
435
+ music goes into the clip prompt too; the prompt is derived — not written — from `segment.sound` and
436
+ the cut timing:
414
437
 
415
438
  ```bash
416
439
  node {baseDir}/scripts/short-drama.mjs prompt <project> audio E01-05 # bed (default)
@@ -427,13 +450,13 @@ Music, instrumental: the low cello note.
427
450
  No dialogue, no vocals, no narration.
428
451
  ```
429
452
 
430
- `params.audioTrack` decides what the node makes: **`bed`** (default) — ambience + sfx + timed
453
+ `params.audioTrack` decides what the node makes: **`bed`** — ambience + sfx + timed
431
454
  music, no voices: the editor's music-and-effects track, and what the video model's same-pass sound
432
455
  is checked against; **`full`** — the bed plus every line at its cut (start = cut start +
433
456
  `speechLeadSeconds`, length from the reading rate), each read in the connected voice sample
434
457
  (`--from` the voice display nodes in speaking order → `@Audio1..3`; Seed Audio takes at most 3, so a
435
- segment with four speakers stays `bed`); **`none`** — no node. Music dynamics are written as cues on
436
- `sound.music`:
458
+ segment with four speakers stays `bed`); **`none`** / **`post`** — no node. Music dynamics are
459
+ written as cues on `sound.music` in every mode:
437
460
 
438
461
  ```json
439
462
  "music": { "intent": "the low cello note",
@@ -109,11 +109,11 @@ By default the video node references the sheets directly — **no storyboard ima
109
109
  video node, and the video node then references those instead of the sheets.
110
110
 
111
111
  - Display nodes: `canvas add image --asset <assetId> -n "C01 sheet"` and `canvas add audio --asset <voiceAsset> -n "C01 voice"` — record both in the episode's `wji.displayNodes` (`"C01:sheet"`, `"C01:voice"`) so every generate node can `--from` them. Pictures become `@imageN`, voices `@audioN`, numbered per type in connection order.
112
- - Per segment, beside the video node, a **soundtrack audio node** (`canvas add audio -n "E01-01 track" -m seed-audio-1.0 --prompt-file …`, prompt from `prompt audio E01-01`) → `storyboard.segments.E01-01.wji.nodes.track`; its asset joins the shot dataset's `images[]`.
112
+ - Soundtrack: in `audioTrack: post` (default) no per-segment node — after the edit, the episode canvas gets one **score node per ≤ 120 s section** (`canvas add audio -n "E01 score 1" -m seed-audio-1.0 --prompt-file …`, from the conformed `prompt score E01` sheet), and the episode binds `wji.assets.edit` / `wji.assets.mix`. In `bed` / `full`, per segment, beside the video node, a **soundtrack audio node** (`canvas add audio -n "E01-01 track" -m seed-audio-1.0 --prompt-file …`, prompt from `prompt audio E01-01`) → `storyboard.segments.E01-01.wji.nodes.track`; its asset joins the shot dataset's `images[]`.
113
113
  - Video: `canvas add video -n "E01-01" --from <sheet display nodes in legend order> <voice display nodes of everyone who speaks> -m <the mode's model> -r <params.ratio> [-d <Σ cut seconds>] --prompt-file prompt.txt`. The model and its resolution spelling come from `prompt segment` (`video.model`, `video.resolution`). `-d` **only in `videoDuration: fixed`** — it switches auto-duration off, which pins the clip to the storyboard's timestamps but also fixes the price; in `auto` (the default) leave it out and the model matches the clip to the prompt.
114
114
  - Prompt text node: `canvas add text -n "E01-01 prompt" --text-file prompt.txt` beside the video node, for people reading the canvas.
115
115
  - Keyframes (opt-in only): `canvas add image -n "E01-01 f<k>" --from <sheet display nodes> -m <image model> -r <params.ratio> -p "<plan prefix + frame>"`, and the video node then `--from f1…fN` instead of the sheets.
116
- - Shot dataset: `dataset new <shot/E01 folder> "E01-01" -t shot -d "<one line>"`, `--set video_prompt=@prompt.txt`, images = the soundtrack (and keyframes, if any), `primaryVideo` = the clip.
116
+ - Shot dataset: `dataset new <shot/E01 folder> "E01-01" -t shot -d "<one line>"`, `--set video_prompt=@prompt.txt`, images = the soundtrack in `bed` / `full` (and keyframes, if any), `primaryVideo` = the clip.
117
117
 
118
118
  Placement: `canvas add` puts a node right of its `--from` nodes; pass `--at x,y`
119
119
  for the row starts (title nodes) so blocks do not overlap. Take ids from the
@@ -17,7 +17,7 @@ import {
17
17
  speechDirection, cameraPhrase, lightPhrase, foregroundPhrase, focalOf, LIGHT_DIRECTIONS, identityOf, isEnglish,
18
18
  renderGuideline, renderDialogue, scriptSelfCheck, beatOffsets, turnPoints, parseAge, talkAndAction, normalizeRegister, REGISTERS, REGISTER_ACCENT,
19
19
  cutStart, segmentCharacters, keyScenes, shotTexts, motionSentence, cameraClause, PURPOSE_KINDS, ANGLES, HEIGHTS, NO_SUBTITLES, LIPS_CLOSED,
20
- audioPrompt, musicOf, cueSeconds, AUDIO_MODEL, AUDIO_MAX_CHARS,
20
+ audioPrompt, musicOf, cueSeconds, AUDIO_MODEL, AUDIO_MAX_CHARS, AUDIO_MAX_SECONDS, scoreSheet, smpte,
21
21
  VIDEO_MODELS, VIDEO_MODES, DEFAULT_PARAMS, videoModel, videoResolutionFor, nearestResolution, segmentPlan, videoReferences,
22
22
  } from './short-drama.mjs';
23
23
 
@@ -1394,21 +1394,24 @@ const breaks = (gateId, project, episode, needle) => {
1394
1394
  eq(cueSeconds(3, seg5.cuts), 6, 'a cue at a cut number is that cut\'s start');
1395
1395
  eq(cueSeconds('10.5s', seg5.cuts), 10.5, 'a cue in seconds is read as seconds');
1396
1396
  eq(cueSeconds('later', seg5.cuts), null, 'a cue that is neither is null');
1397
- const bed = audioPrompt(PROJECT, E01, seg5);
1398
- eq(bed.mode, 'bed', 'the default track is a bed');
1397
+ // A per-segment soundtrack node is the `bed` / `full` mode; the default is `post` (scored over the edit).
1398
+ const PB = clone(PROJECT); PB.params.audioTrack = 'bed';
1399
+ eq(paramsOf(PROJECT).audioTrack, 'post', 'the default soundtrack is made in post, over the edit');
1400
+ const bed = audioPrompt(PB, E01, seg5);
1401
+ eq(bed.mode, 'bed', 'bed mode makes a bed');
1399
1402
  eq(bed.model, AUDIO_MODEL, 'the soundtrack node is Seed Audio');
1400
1403
  ok(bed.prompt.startsWith('[0.0s:11.0s] Ambience, continuous: the low hum'), 'the bed opens with the timecoded ambience');
1401
1404
  ok(bed.prompt.includes('[6.0s:9.0s] SFX: a single ring of the brass wind chimes'), 'sfx are timecoded to their cut');
1402
1405
  ok(bed.prompt.includes('[6.0s:10.5s] the music swells — on the chime.') && bed.prompt.includes('[10.5s:11.0s] the music falls silent'), 'music cues become timecoded states');
1403
1406
  ok(bed.prompt.includes('No dialogue, no vocals, no narration.'), 'a bed carries no voices');
1404
1407
  eq(bed.references.length, 0, 'a bed references no voices');
1405
- const full = audioPrompt(PROJECT, E01, E01.storyboard.segments[1], { mode: 'full' });
1408
+ const full = audioPrompt(PB, E01, E01.storyboard.segments[1], { mode: 'full' });
1406
1409
  eq(full.references.map((r) => r.label).join(','), '@Audio1,@Audio2', 'full mode references the speakers as @AudioN');
1407
1410
  ok(full.prompt.includes('[5.5s:7.1s] @Audio1 (the stocky man in his sixties with a grey crew cut), slow and gravelly'), 'a line is timecoded into its cut after the lead-in, in its voice and direction');
1408
1411
  ok(full.prompt.includes('says: 又是你值夜班。'), 'the line is verbatim');
1409
- const noMusic = audioPrompt(PROJECT, E01, E01.storyboard.segments[0]);
1412
+ const noMusic = audioPrompt(PB, E01, E01.storyboard.segments[0]);
1410
1413
  ok(noMusic.prompt.includes('No music.'), 'N/A music is stated');
1411
- const md = renderSegment(PROJECT, E01, 'E01-05');
1414
+ const md = renderSegment(PB, E01, 'E01-05');
1412
1415
  ok(md.includes('## 音轨提示词(seed-audio-1.0 · bed · 11 s)') && md.includes('配乐:the low cello note(镜1 enter,镜3 swell,10.5s silence)'), 'segment markdown carries the soundtrack prompt and the cues');
1413
1416
  // gates
1414
1417
  const e = clone(E01); e.storyboard.segments[4].sound.music = { intent: '', cues: [] };
@@ -1418,7 +1421,7 @@ const breaks = (gateId, project, episode, needle) => {
1418
1421
  const e3 = clone(E01); e3.storyboard.segments[4].sound.music.cues[0].at = '40s';
1419
1422
  breaks('sound', PROJECT, e3, 'inside the segment');
1420
1423
  const e4 = clone(E01); e4.storyboard.segments[4].sound.ambience = 'x'.repeat(AUDIO_MAX_CHARS);
1421
- breaks('audio-track', PROJECT, e4, 'max 2048');
1424
+ breaks('audio-track', PB, e4, 'max 2048');
1422
1425
  const p = clone(PROJECT); p.params.audioTrack = 'full';
1423
1426
  p.characters.push({ id: 'C04', name: '路人', tier: 'functional', role: 'r', from: ['原创'], identityEn: 'a passer-by in a wet coat', voice: { videoDirection: 'flat' } });
1424
1427
  const e5 = clone(E01); e5.scenes[1].characters.push('C03', 'C04');
@@ -1428,9 +1431,51 @@ const breaks = (gateId, project, episode, needle) => {
1428
1431
  eq(failures('audio-track', p2, E01).length, 0, 'audioTrack none skips the gate');
1429
1432
  const p3 = clone(PROJECT); p3.params.audioTrack = 'stems';
1430
1433
  breaks('params', p3, null, 'audioTrack');
1431
- const items = manifest(PROJECT, { 1: E01 });
1432
- ok(items.some((i) => i.item === 'storyboard.segments.E01-01.wji.nodes.track'), 'manifest asks for the soundtrack node');
1434
+ const items = manifest(PB, { 1: E01 });
1435
+ ok(items.some((i) => i.item === 'storyboard.segments.E01-01.wji.nodes.track'), 'bed mode: manifest asks for the soundtrack node');
1433
1436
  ok(!manifest(p2, { 1: E01 }).some((i) => i.item.includes('.nodes.track')), 'audioTrack none asks for no soundtrack node');
1437
+ // post (default): no music in any clip, no per-segment track, the episode is scored over the edit
1438
+ ok(!manifest(PROJECT, { 1: E01 }).some((i) => i.item.includes('.nodes.track')), 'post asks for no per-segment soundtrack node');
1439
+ eq(failures('audio-track', PROJECT, E01).length, 0, 'post skips the per-segment audio gate');
1440
+ for (const video of ['h3', 'seedance2', 'wan']) {
1441
+ for (const references of ['keyframes', 'sheets']) {
1442
+ const pv = clone(PROJECT); pv.params.video = video; pv.params.references = references;
1443
+ const vp = promptSkeleton(pv, E01, 'E01-05').video.prompt;
1444
+ ok(!vp.includes('cello'), `post: the music intent stays out of the clip (${video}/${references})`);
1445
+ ok(video === 'h3' ? vp.includes('non_diegetic_music: N/A') : video === 'wan' ? vp.includes('无配乐') : vp.includes('No music.'), `post: the clip is told there is no music (${video}/${references})`);
1446
+ const pb = clone(pv); pb.params.audioTrack = 'bed';
1447
+ ok(promptSkeleton(pb, E01, 'E01-05').video.prompt.includes('cello'), `bed: the music intent is still in the clip (${video}/${references})`);
1448
+ }
1449
+ }
1450
+ const ePost = clone(E01); ePost.storyboard.segments[4].sound.sfx.push({ cut: 3, event: 'SPECIAL: a reversed-cymbal riser into a hard stop', post: true });
1451
+ ok(!promptSkeleton(PROJECT, ePost, 'E01-05').video.prompt.includes('reversed-cymbal'), 'a post: true sfx stays out of the clip');
1452
+ ok(audioPrompt(PB, ePost, ePost.storyboard.segments[4]).prompt.includes('reversed-cymbal'), 'a post: true sfx is still in a per-segment bed');
1453
+ ok(renderSegment(PROJECT, E01, 'E01-05').includes('prompt score E01'), 'post: segment markdown points at the episode score sheet');
1454
+ const sheet = scoreSheet(PROJECT, ePost);
1455
+ eq(sheet.seconds, 68, 'the score sheet lays the segments end to end');
1456
+ eq(sheet.fps, 24, 'the score sheet defaults to 24 fps');
1457
+ eq(smpte(63.5, 24), '00:01:03:12', 'SMPTE is HH:MM:SS:FF');
1458
+ eq(smpte(3725, 25), '01:02:05:00', 'SMPTE carries hours');
1459
+ ok(sheet.rows.some((r) => r.layer === 'BGM' && r.at === 63 && r.what.startsWith('swell')), 'a music cue lands at its segment offset + cut start');
1460
+ ok(sheet.rows.some((r) => r.layer === 'design' && r.at === 63 && r.what.includes('reversed-cymbal')), 'post sfx are sound design on the sheet');
1461
+ ok(sheet.rows.some((r) => r.layer === 'sfx in clip' && r.what.includes('wind chimes')), 'in-clip sfx are listed so they are not doubled');
1462
+ ok(sheet.rows.some((r) => r.layer === 'DX' && r.at === 3.5), 'lines are spotted after the lead-in, for ducking');
1463
+ eq(sheet.rows.filter((r) => r.layer === 'BGE').length, 2, 'one ambience bed per scene, chained across its segments');
1464
+ eq(sheet.rows.find((r) => r.layer === 'BGE' && r.segment === 'E01-02').end, 68, 'the scene bed runs to the scene end');
1465
+ ok(sheet.rows.some((r) => r.layer === 'story' && r.what.startsWith('hook')) && sheet.rows.some((r) => r.layer === 'story' && r.what.startsWith('cliff')), 'the hook and the cliff anchor 起 / 合');
1466
+ eq(sheet.sections.length, 1, 'a 68 s episode is one Seed Audio section');
1467
+ const eLong = clone(E01);
1468
+ for (const sg of eLong.storyboard.segments) for (const c of sg.cuts) c.seconds *= 2; // 136 s
1469
+ const long = scoreSheet(PROJECT, eLong, { fps: 25 });
1470
+ ok(long.sections.length === 2 && long.sections.every((x) => x.seconds <= AUDIO_MAX_SECONDS), 'a long episode splits into ≤ 120 s sections');
1471
+ eq(long.sections[0].segments.join(','), 'E01-01', 'the split falls on the scene change');
1472
+ eq(long.fps, 25, '--fps overrides the frame rate');
1473
+ const pp = clone(PROJECT); const eDone = clone(E01);
1474
+ for (const sg of eDone.storyboard.segments) sg.wji = { ...(sg.wji || {}), assets: { ...(sg.wji?.assets || {}), video: 'a1' } };
1475
+ ok(manifest(pp, { 1: eDone }).some((i) => i.item === 'wji.assets.edit'), 'post: once every clip exists the manifest asks for the edit');
1476
+ eDone.wji = { ...(eDone.wji || {}), assets: { edit: 'e1' } };
1477
+ ok(manifest(pp, { 1: eDone }).some((i) => i.item === 'wji.assets.mix'), 'post: after the edit it asks for the scored mix');
1478
+ ok(!manifest(pp, { 1: E01 }).some((i) => i.item.startsWith('wji.assets.')), 'post: no edit item while clips are missing');
1434
1479
  ok(voiceSamplePrompt(PROJECT.characters[0], paramsOf(PROJECT)).includes('干声,无配乐无音效'), 'the voice sample is asked for dry');
1435
1480
  }
1436
1481
 
@@ -27,6 +27,8 @@
27
27
  // prompt <project> segment E01-01 derived video-prompt skeleton + keyframe reference plan
28
28
  // prompt <project> voice C01 the acted sample text for the voice node
29
29
  // prompt <project> audio E01-01 [--full] the segment's Seed Audio soundtrack prompt (timecoded ambience / sfx / music, + lines with --full)
30
+ // prompt <project> score E01 [--fps 24] the episode's spotting sheet for the post soundtrack (audioTrack post): planned
31
+ // timecodes of scenes, lines, music cues and post sfx, and the ≤120 s Seed Audio sections
30
32
  // manifest <project> [--stage …] [--ep N] unbound WJI objects, in execution order
31
33
  // bind <project> [--ep N] <path> <value> write a WJI id back (path segments may be entity ids)
32
34
  // slug <name>
@@ -98,8 +100,12 @@ export const DEFAULT_PARAMS = Object.freeze({
98
100
  speechLeadSeconds: 0.5,
99
101
  subtitles: false,
100
102
  maxFrameChars: 900,
101
- // A Seed Audio node per segment: the soundtrack bed by default (ambience, sfx, timed music).
102
- audioTrack: 'bed',
103
+ // Where the score lives. `post` (default): the clips carry dialogue, ambience and in-picture sfx but
104
+ // no music — music and sound design cannot carry across separately generated segments, so they are
105
+ // made once over the edited episode, on its timecode. `bed` / `full`: a Seed Audio node per segment.
106
+ audioTrack: 'post',
107
+ // Frame rate the post spotting sheet writes SMPTE timecode at (conform it to the edit's real fps).
108
+ scoreFps: 24,
103
109
  thresholds: { maxLeads: 5, maxSupport: 10, maxFunctional: 10, maxBeatGap: 3, maxProps: 8 },
104
110
  });
105
111
 
@@ -177,8 +183,14 @@ export const AUDIO_MODEL = VOICE_MODEL;
177
183
  export const AUDIO_MAX_SECONDS = 120;
178
184
  export const AUDIO_MAX_CHARS = 2048;
179
185
  export const AUDIO_MAX_REFS = 3;
180
- /** What the per-segment audio node generates: nothing, a bed (ambience + sfx + music) or the full track with the lines. */
181
- export const AUDIO_TRACK_MODES = ['none', 'bed', 'full'];
186
+ /**
187
+ * Where the soundtrack is made. `post`: no music in the clips and no per-segment audio node — the
188
+ * score and sound design are made once over the edited episode (`prompt score`). `none`: nothing.
189
+ * `bed` (ambience + sfx + music) / `full` (the bed plus the lines): a Seed Audio node per segment.
190
+ */
191
+ export const AUDIO_TRACK_MODES = ['post', 'none', 'bed', 'full'];
192
+ /** No per-segment audio node in these modes. */
193
+ export const NO_SEGMENT_TRACK = ['post', 'none'];
182
194
  /** What the music does at a cue point. */
183
195
  export const MUSIC_CUES = ['enter', 'swell', 'duck', 'drop', 'silence', 'out'];
184
196
  /** `fixed` passes `-d` = Σ cut seconds; `auto` leaves it off and lets the model match the prompt. */
@@ -1385,6 +1397,100 @@ export function audioPrompt(project, episode, seg, opts = {}) {
1385
1397
  return { segment: seg.id, mode, seconds: total, prompt, references, model: AUDIO_MODEL };
1386
1398
  }
1387
1399
 
1400
+ /** Seconds → SMPTE `HH:MM:SS:FF` at a whole-number frame rate. */
1401
+ export function smpte(seconds, fps = 24) {
1402
+ const f = Math.max(0, Math.round(Number(seconds) * fps));
1403
+ const ff = f % fps, total = Math.floor(f / fps);
1404
+ return [Math.floor(total / 3600), Math.floor(total / 60) % 60, total % 60, ff].map(pad2).join(':');
1405
+ }
1406
+
1407
+ /**
1408
+ * The episode's spotting sheet for the post soundtrack (`audioTrack: post`): the storyboard laid end to
1409
+ * end on one timeline — scene beds (BGE), every line (where the music ducks), each segment's music
1410
+ * intent and cues, the `post: true` sound design, the sfx already in the clips (never doubled), and the
1411
+ * hook / turns / 爽点 / cliff as the 起承转合 anchors. It is the PLANNED timeline: the Sound Designer
1412
+ * conforms it to the edited video's real cuts before writing the Seed Audio prompts, one per section
1413
+ * of ≤ AUDIO_MAX_SECONDS, split at a scene change where one fits.
1414
+ */
1415
+ export function scoreSheet(project, episode, opts = {}) {
1416
+ const params = paramsOf(project);
1417
+ const fps = Math.round(Number(opts.fps || params.scoreFps)) || 24;
1418
+ const segs = (episode.storyboard?.segments || []).filter((s) => (s.cuts || []).length);
1419
+ const rows = [];
1420
+ const spans = [];
1421
+ const beatAtTime = new Map();
1422
+ let t = 0, bed = null;
1423
+ for (const seg of segs) {
1424
+ const cuts = seg.cuts;
1425
+ const starts = cutStarts(cuts);
1426
+ const total = cuts.reduce((sum, c) => sum + (Number(c.seconds) || 0), 0);
1427
+ const si = Number(seg.sceneIndex);
1428
+ const scene = (episode.scenes || [])[si - 1] || {};
1429
+ const sound = isObject(seg.sound) ? seg.sound : {};
1430
+ const newScene = !spans.length || spans[spans.length - 1].scene !== si;
1431
+ spans.push({ segment: seg.id, scene: si, start: t, end: t + total });
1432
+ rows.push({ at: t, end: t + total, layer: 'segment', segment: seg.id, what: `S${si}${newScene ? ' — new scene' : ''}` });
1433
+ if (newScene || !bed) {
1434
+ bed = { at: t, end: t + total, layer: 'BGE', segment: seg.id, what: sound.ambience || '…' };
1435
+ rows.push(bed);
1436
+ } else bed.end = t + total;
1437
+ cuts.forEach((c, k) => {
1438
+ const [from, to] = c.beats || [0, -1];
1439
+ for (let b = from; b <= to; b++) if (!beatAtTime.has(`${si}:${b}`)) beatAtTime.set(`${si}:${b}`, t + starts[k]);
1440
+ let at = t + starts[k] + (Number(params.speechLeadSeconds) || 0);
1441
+ for (const l of claimedLines(scene, [c])) {
1442
+ const secs = lineSeconds(l.line, params);
1443
+ const who = l.speaker === 'VO' ? 'VO' : characterById(project, l.speaker)?.name || l.speaker;
1444
+ rows.push({ at, end: Math.min(t + total, at + secs), layer: 'DX', segment: seg.id, what: `${who}: ${l.line}` });
1445
+ at += secs;
1446
+ }
1447
+ });
1448
+ for (const x of asList(sound.sfx)) {
1449
+ const k = Number(x?.cut);
1450
+ if (!(k >= 1 && k <= cuts.length)) continue;
1451
+ const at = t + starts[k - 1];
1452
+ rows.push({ at, end: at + (Number(cuts[k - 1].seconds) || 0), layer: x.post === true ? 'design' : 'sfx in clip', segment: seg.id, what: String(x.event) });
1453
+ }
1454
+ const music = musicOf(sound);
1455
+ if (music.intent) {
1456
+ const cues = music.cues.map((c) => ({ at: cueSeconds(c?.at, cuts), do: c?.do, note: c?.note })).filter((c) => c.at != null).sort((a, b) => a.at - b.at);
1457
+ if (!cues.length) rows.push({ at: t, end: t + total, layer: 'BGM', segment: seg.id, what: `bed — ${music.intent}` });
1458
+ cues.forEach((c, i) => rows.push({ at: t + c.at, end: t + (i + 1 < cues.length ? cues[i + 1].at : total), layer: 'BGM', segment: seg.id, what: `${c.do} — ${music.intent}${c.note ? ` (${c.note})` : ''}` }));
1459
+ }
1460
+ t += total;
1461
+ }
1462
+ const point = (ref, what) => {
1463
+ const at = Array.isArray(ref) ? beatAtTime.get(`${Number(ref[0])}:${Number(ref[1])}`) : undefined;
1464
+ if (at != null) rows.push({ at, end: at, layer: 'story', segment: '', what });
1465
+ };
1466
+ point(episode.hookBeat, 'hook (起)');
1467
+ (episode.scenes || []).forEach((sc, si) => (sc.flow || []).forEach((b, bi) => { if (b.turn === true) point([si + 1, bi + 1], 'turn (转)'); }));
1468
+ point(episode.satisfactionBeat, '爽点');
1469
+ if (segs.length) rows.push({ at: t, end: t, layer: 'story', segment: '', what: 'cliff (合) — last frame' });
1470
+ const order = ['story', 'segment', 'BGE', 'BGM', 'design', 'sfx in clip', 'DX'];
1471
+ rows.sort((a, b) => a.at - b.at || order.indexOf(a.layer) - order.indexOf(b.layer));
1472
+ // Seed Audio sections: ≤ AUDIO_MAX_SECONDS, cut at a segment boundary, the latest scene change when one fits.
1473
+ const sections = [];
1474
+ let open = [];
1475
+ for (const sp of spans) {
1476
+ if (open.length && sp.end - open[0].start > AUDIO_MAX_SECONDS) {
1477
+ let cut = open.length;
1478
+ for (let i = open.length - 1; i > 0; i--) if (open[i].scene !== open[i - 1].scene) { cut = i; break; }
1479
+ sections.push(open.slice(0, cut));
1480
+ open = open.slice(cut);
1481
+ }
1482
+ open.push(sp);
1483
+ }
1484
+ if (open.length) sections.push(open);
1485
+ return {
1486
+ episode: `E${pad2(episode.ep)}`,
1487
+ fps,
1488
+ seconds: t,
1489
+ rows: rows.map((r) => ({ ...r, tcIn: smpte(r.at, fps), tcOut: smpte(r.end, fps) })),
1490
+ sections: sections.map((g, i) => ({ n: i + 1, start: g[0].start, end: g[g.length - 1].end, seconds: g[g.length - 1].end - g[0].start, segments: g.map((x) => x.segment), tcIn: smpte(g[0].start, fps), tcOut: smpte(g[g.length - 1].end, fps) })),
1491
+ };
1492
+ }
1493
+
1388
1494
  // ── gates ───────────────────────────────────────────────────────────────────
1389
1495
  //
1390
1496
  // A gate is {id, stage, label, run(ctx) -> string[]} where the strings are the
@@ -3233,7 +3339,7 @@ export const GATES = [
3233
3339
  gate('audio-track', 'storyboard', 'Seed Audio node per segment: the derived prompt fits the model (≤ 120 s, ≤ 2048 chars, ≤ 3 voices in full mode)', (ctx) => {
3234
3340
  const { project, params } = ctx;
3235
3341
  const out = [];
3236
- if (params.audioTrack === 'none') return out;
3342
+ if (NO_SEGMENT_TRACK.includes(params.audioTrack)) return out;
3237
3343
  for (const [, e] of ctx.episodes) {
3238
3344
  for (const seg of e.storyboard?.segments || []) {
3239
3345
  if (!(seg.cuts || []).length || !isObject(seg.sound)) continue;
@@ -4086,7 +4192,10 @@ export function promptSkeleton(project, episode, segmentId) {
4086
4192
  const actionLine = cutLines.length ? [...cutLines, ''] : [];
4087
4193
  const accent = String(params.speechAccent || '').trim();
4088
4194
  const sound = isObject(seg.sound) ? seg.sound : {};
4089
- const sfxOf = (k) => asList(sound.sfx).filter((x) => Number(x?.cut) === k + 1).map((x) => String(x.event)).join('; ');
4195
+ // `post: true` sfx (stingers, risers, whooshes — sound design outside the story world) are made over
4196
+ // the edit, never in the clip; in `post` mode the music is too, so the clip is told there is none.
4197
+ const sfxOf = (k) => asList(sound.sfx).filter((x) => x?.post !== true && Number(x?.cut) === k + 1).map((x) => String(x.event)).join('; ');
4198
+ const clipMusic = params.audioTrack === 'post' ? '' : musicOf(sound).intent;
4090
4199
  // Who else is on screen while `speaker` has a line: the H3 idiom keeps the lip-sync off them.
4091
4200
  // In keyframes mode the @imageN labels are frames, so a person is named by identity instead.
4092
4201
  const othersOnScreen = (c, speaker) => (c.characters || []).filter((id) => id !== speaker).map((id) => (params.references === 'sheets' && sheetLabels.get(id) ? `the person in ${sheetLabels.get(id)}` : identityOf(characterById(project, id))));
@@ -4116,7 +4225,7 @@ export function promptSkeleton(project, episode, segmentId) {
4116
4225
  : [h3AlignmentLine(cuts), ...voiceLegend];
4117
4226
  const look = sheets ? [`Look: ${lookOf()}.`, ''] : [];
4118
4227
  const soundscape = [sound.ambience || '…', ...cuts.map((c, k) => (sfxOf(k) ? `at [Shot ${k + 1}] ${sfxOf(k)}` : '')).filter(Boolean)].join('; ');
4119
- video = [...lead, '', ...look, ...actionLine, 'integrated_multimodal_description:', ...shots, '', `overall_soundscape: ${soundscape}.${accent && dialogue.flat().length ? ` All dialogue in ${accent}.` : ''}${params.subtitles ? '' : ` ${NO_SUBTITLES.en[0].toUpperCase()}${NO_SUBTITLES.en.slice(1)}.`}`, '', `non_diegetic_music: ${musicOf(sound).intent || 'N/A'}`].join('\n');
4228
+ video = [...lead, '', ...look, ...actionLine, 'integrated_multimodal_description:', ...shots, '', `overall_soundscape: ${soundscape}.${accent && dialogue.flat().length ? ` All dialogue in ${accent}.` : ''}${params.subtitles ? '' : ` ${NO_SUBTITLES.en[0].toUpperCase()}${NO_SUBTITLES.en.slice(1)}.`}`, '', `non_diegetic_music: ${clipMusic || 'N/A'}`].join('\n');
4120
4229
  } else if (mode.family === 'wan') {
4121
4230
  const blocks = cuts.map((c, k) => {
4122
4231
  const talk = dialogue[k].map((l) => `台词(${l.speakerRef},${l.direction || '…'}):「${l.line}」`);
@@ -4131,7 +4240,7 @@ export function promptSkeleton(project, episode, segmentId) {
4131
4240
  ...blocks.flatMap((b) => [b, '']),
4132
4241
  ...(cine && params.references === 'sheets' ? [`画面:${cine.videoPhraseZh}${params.lookPhraseZh ? `,${params.lookPhraseZh}` : ''}。`, ''] : []),
4133
4242
  ...(grammarLines(seg, params, 'zh').length ? [...grammarLines(seg, params, 'zh'), ''] : []),
4134
- `音频:对白为${accent || '…'},环境音${sound.ambience || '…'}${sfxAll ? `;${sfxAll}` : ''}${musicOf(sound).intent ? `;配乐:${musicOf(sound).intent}` : ''}。${params.subtitles ? '' : `${NO_SUBTITLES.zh}。`}`,
4243
+ `音频:对白为${accent || '…'},环境音${sound.ambience || '…'}${sfxAll ? `;${sfxAll}` : ''}${clipMusic ? `;配乐:${clipMusic}` : params.audioTrack === 'post' ? ';无配乐' : ''}。${params.subtitles ? '' : `${NO_SUBTITLES.zh}。`}`,
4135
4244
  ].join('\n');
4136
4245
  } else if (params.references === 'sheets') {
4137
4246
  const legend = allRefs.map((r) => `${r.label} is ${r.use}.`);
@@ -4141,7 +4250,7 @@ export function promptSkeleton(project, episode, segmentId) {
4141
4250
  return `Shot ${k + 1} (${starts[k].toFixed(1)}–${(starts[k] + Number(c.seconds)).toFixed(1)} s, ${SIZES[c.size] || c.size}, ${c.camera}): ${c.frame || '…'} ${motionSentence(c)} ${cameraClause(c)} ${talk}`.replace(/\s+/g, ' ').trim();
4142
4251
  });
4143
4252
  const sfxAll = cuts.map((c, k) => (sfxOf(k) ? `shot ${k + 1}: ${sfxOf(k)}` : '')).filter(Boolean).join('; ');
4144
- video = [...legend, '', ...shots, '', ...actionLine, `Look: ${lookOf()}. Maintain continuity in character identities, clothing, prop ownership, scene layout and the camera axis across every shot.`, `Sound: ${sound.ambience || '…'}${sfxAll ? `; ${sfxAll}` : ''}.${accent && dialogue.flat().length ? ` All dialogue in ${accent}.` : ''}${musicOf(sound).intent ? ` Music: ${musicOf(sound).intent}.` : ' No music.'}${params.subtitles ? '' : ` ${NO_SUBTITLES.en[0].toUpperCase()}${NO_SUBTITLES.en.slice(1)}.`}`].join('\n');
4253
+ video = [...legend, '', ...shots, '', ...actionLine, `Look: ${lookOf()}. Maintain continuity in character identities, clothing, prop ownership, scene layout and the camera axis across every shot.`, `Sound: ${sound.ambience || '…'}${sfxAll ? `; ${sfxAll}` : ''}.${accent && dialogue.flat().length ? ` All dialogue in ${accent}.` : ''}${clipMusic ? ` Music: ${clipMusic}.` : ' No music.'}${params.subtitles ? '' : ` ${NO_SUBTITLES.en[0].toUpperCase()}${NO_SUBTITLES.en.slice(1)}.`}`].join('\n');
4145
4254
  } else {
4146
4255
  const anchors = [...seedanceAnchors(cuts.length), ...allRefs.filter((r) => r.kind !== 'image').map((r) => `${r.label} is ${r.use}.`)];
4147
4256
  const shotOf = (c) => `${SIZES[c.size] || c.size}, ${c.camera}, ${cameraPhrase(c, params)}; ${lightPhrase(c)}${foregroundPhrase(c)}`;
@@ -4151,7 +4260,7 @@ export function promptSkeleton(project, episode, segmentId) {
4151
4260
  return `From @image${k + 1} (${starts[k].toFixed(1)}–${(starts[k] + Number(c.seconds)).toFixed(1)} s): ${motionSentence(c)} ${cameraClause(c)} ${talk}`.replace(/\s+/g, ' ').trim();
4152
4261
  });
4153
4262
  const sfxAll = cuts.map((c, k) => (sfxOf(k) ? `from @image${k + 1}: ${sfxOf(k)}` : '')).filter(Boolean).join('; ');
4154
- video = [...anchors, ...states, '', ...script, '', ...actionLine, 'Maintain continuity in character identities, clothing, prop ownership, scene layout and the camera axis.', `Sound: ${sound.ambience || '…'}${sfxAll ? `; ${sfxAll}` : ''}.${accent && dialogue.flat().length ? ` All dialogue in ${accent}.` : ''}${musicOf(sound).intent ? ` Music: ${musicOf(sound).intent}.` : ' No music.'}${params.subtitles ? '' : ` ${NO_SUBTITLES.en[0].toUpperCase()}${NO_SUBTITLES.en.slice(1)}.`}`].join('\n');
4263
+ video = [...anchors, ...states, '', ...script, '', ...actionLine, 'Maintain continuity in character identities, clothing, prop ownership, scene layout and the camera axis.', `Sound: ${sound.ambience || '…'}${sfxAll ? `; ${sfxAll}` : ''}.${accent && dialogue.flat().length ? ` All dialogue in ${accent}.` : ''}${clipMusic ? ` Music: ${clipMusic}.` : ' No music.'}${params.subtitles ? '' : ` ${NO_SUBTITLES.en[0].toUpperCase()}${NO_SUBTITLES.en.slice(1)}.`}`].join('\n');
4155
4264
  }
4156
4265
 
4157
4266
  const sceneEntity = sceneById(project, scene.sceneId) || {};
@@ -4506,7 +4615,9 @@ export function renderSegment(project, episode, segmentId) {
4506
4615
  lines.push('## 画面提示词(起始瞬间)', '', ...cuts.flatMap((c, k) => [`**${k + 1}.** ${c.frame || ''}`, '']));
4507
4616
  if (seg.note) lines.push(`> ${seg.note}`, '');
4508
4617
  lines.push('## 视频提示词', '', '```', seg.prompt || '', '```', '');
4509
- if (params.audioTrack !== 'none' && isObject(seg.sound)) {
4618
+ if (params.audioTrack === 'post' && isObject(seg.sound)) {
4619
+ lines.push('## 音轨', '', `配乐与声音设计在剪辑后整集制作(audioTrack post)— 本段的配乐意图与 post 音效写入整集的 spotting sheet:\`prompt score E${pad2(episode.ep)}\`。`, '');
4620
+ } else if (params.audioTrack !== 'none' && isObject(seg.sound)) {
4510
4621
  const a = audioPrompt(project, episode, seg);
4511
4622
  lines.push(`## 音轨提示词(${AUDIO_MODEL} · ${a.mode} · ${round2(a.seconds)} s)`, '', '```', a.prompt, '```', '');
4512
4623
  }
@@ -4590,7 +4701,7 @@ export function manifest(project, episodes, opts = {}) {
4590
4701
  if (!sw.nodes?.keyframes?.[k]) add('storyboard', seg.id, `storyboard.segments.${seg.id}.wji.nodes.keyframes.${k}`, `wji canvas add image -c <${E}> -n "${seg.id} f${k + 1}" --from <display nodes>`);
4591
4702
  else if (!sw.assets?.keyframes?.[k]) add('storyboard', seg.id, `storyboard.segments.${seg.id}.wji.assets.keyframes.${k}`, `wji canvas gen ${sw.nodes.keyframes[k]} → asset`);
4592
4703
  }
4593
- if (paramsOf(project).audioTrack !== 'none' && isObject(seg.sound)) {
4704
+ if (!NO_SEGMENT_TRACK.includes(paramsOf(project).audioTrack) && isObject(seg.sound)) {
4594
4705
  const full = paramsOf(project).audioTrack === 'full';
4595
4706
  if (!sw.nodes?.track) add('storyboard', seg.id, `storyboard.segments.${seg.id}.wji.nodes.track`, `wji canvas add audio -c <${E}> -n "${seg.id} track" -m ${AUDIO_MODEL}${full ? ' --from <voice display nodes, speaking order>' : ''} --prompt-file <prompt audio ${seg.id}>`);
4596
4707
  else if (!sw.assets?.track) add('storyboard', seg.id, `storyboard.segments.${seg.id}.wji.assets.track`, `wji canvas gen ${sw.nodes.track} → asset → fs set <shot dataset> --add-images <asset>`);
@@ -4598,6 +4709,11 @@ export function manifest(project, episodes, opts = {}) {
4598
4709
  if (!sw.nodes?.video) add('storyboard', seg.id, `storyboard.segments.${seg.id}.wji.nodes.video`, `wji canvas add video -c <${E}> -n "${seg.id}" --from <${paramsOf(project).references === 'sheets' ? 'sheet display nodes' : 'keyframes'}>`);
4599
4710
  else if (!sw.assets?.video) add('storyboard', seg.id, `storyboard.segments.${seg.id}.wji.assets.video`, `wji canvas gen ${sw.nodes.video} (video — ask first) → asset`);
4600
4711
  }
4712
+ // Post soundtrack: once every clip exists, the episode is edited, then scored and mixed as one.
4713
+ if (paramsOf(project).audioTrack === 'post' && segs.length && segs.every((s) => s.wji?.assets?.video)) {
4714
+ if (!ew.assets?.edit) add('storyboard', E, 'wji.assets.edit', `edit the clips into the locked ${E} cut (no music) → wji asset upload <${E} edit.mp4>`);
4715
+ else if (!ew.assets?.mix) add('storyboard', E, 'wji.assets.mix', `prompt score ${E} → conform to the edit → Seed Audio per section (wji canvas add audio -c <${E}> -n "${E} score N" -m ${AUDIO_MODEL} --prompt-file …; ask first) → mix under the edit → wji asset upload <${E} mix.mp4>`);
4716
+ }
4601
4717
  }
4602
4718
  return items;
4603
4719
  }
@@ -4898,6 +5014,28 @@ function main(argv) {
4898
5014
  return 0;
4899
5015
  }
4900
5016
 
5017
+ if (cmd === 'prompt' && pos[1] === 'score') {
5018
+ const ep = Number((pos[2] || '').match(/^E(\d+)$/)?.[1]);
5019
+ const { project, episodes } = loadProject(input);
5020
+ if (!ep || !episodes[ep]) throw new Error('usage: prompt <project> score E01 [--fps 24] [--json]');
5021
+ const out = scoreSheet(project, episodes[ep], { fps: flag(rest, '--fps') });
5022
+ if (flag(rest, '--json')) console.log(JSON.stringify(out, null, 2));
5023
+ else {
5024
+ console.log(`# ${out.episode} · post soundtrack spotting sheet · ${round2(out.seconds)} s · ${smpte(out.seconds, out.fps)} @ ${out.fps} fps`);
5025
+ console.log('');
5026
+ console.log('PLANNED timeline (storyboard end to end). Conform every row to the edited video\'s real cuts before writing prompts.');
5027
+ console.log('Layers: BGE / BGM / design are made in post; "sfx in clip" and DX are already in the video — never double them; BGM ducks under DX.');
5028
+ console.log('');
5029
+ console.log('| TC in | TC out | sec | layer | segment | what |');
5030
+ console.log('| --- | --- | --- | --- | --- | --- |');
5031
+ for (const r of out.rows) console.log(`| ${r.tcIn} | ${r.tcOut} | ${r.at.toFixed(1)}–${r.end.toFixed(1)} | ${r.layer} | ${r.segment} | ${String(r.what).replace(/\|/g, '/')} |`);
5032
+ console.log('');
5033
+ console.log(`Seed Audio sections (≤ ${AUDIO_MAX_SECONDS} s each, local time 0 = section start):`);
5034
+ for (const x of out.sections) console.log(` ${x.n}. ${x.tcIn} → ${x.tcOut} (${round2(x.seconds)} s) · ${x.segments.join(', ')}`);
5035
+ }
5036
+ return 0;
5037
+ }
5038
+
4901
5039
  if (cmd === 'prompt' && pos[1] === 'voice') {
4902
5040
  const { project } = loadProject(input);
4903
5041
  const c = characterById(project, pos[2]);