@wji.ai/cli 0.7.1 → 0.8.1

package/skill/short-drama/reference/stage-storyboard.md

@@ -404,13 +404,36 @@ world, same faces, right size, nobody extra), fix, then continue. An episode is
 a wrong direction costs the whole batch — which is why this is opt-in and the default references
 the sheets directly.
 
-## Soundtrack node — Seed Audio is not a TTS
+## Soundtrack — scored over the edit, not per clip
+
+Each segment is a separate generation, so music and sound design made inside a clip stop at its
+last frame: a cue cannot swell across a cut into the next segment, and every clip invents its own
+score. So by default (`params.audioTrack: post`) **the clips carry no music**: the video prompt
+says `No music.` (H3 `non_diegetic_music: N/A`, Wan `无配乐`), keeps the dialogue, the ambience and
+the in-picture sfx, and drops every sfx marked `post: true` (stingers, risers, whooshes, impact
+hits — sound design outside the story world). The storyboard still writes `sound.music` (intent and
+cues) and the `post: true` hits: they are the **spotting notes**. When every clip is in, the episode
+is edited, and the Sound Designer scores the edit as one piece on its timecode:
+
+```bash
+$SD prompt <project> score E01            # planned spotting sheet: SMPTE rows + ≤120 s Seed Audio sections
+```
+
+The sheet lays the segments end to end — scene beds (BGE), every line (where the music ducks), each
+segment's music cues at their episode time, the `post` sound design, the sfx already in the clips
+(never doubled) and the hook / turns / 爽点 / cliff as the 起承转合 anchors. It is the *planned*
+timeline: the Sound Designer conforms it to the edit's real cuts
+(`mix-over-video.md`) before any prompt is written. `manifest`
+then asks for the episode's `wji.assets.edit` and `wji.assets.mix`.
+
+### Per-segment soundtrack node (`audioTrack: bed` / `full`)
 
 `seed-audio-1.0` (the voice-sample model of stage 2) generates a whole **audio scene** from one
 prompt: speech, sound effects, ambience and music in one track, up to 120 s, and a
 **`[start s:end s]` timecode in front of an element pins it to that interval** (100 ms
-precision). So every segment gets a soundtrack node next to its video node, and the prompt is
-derived — not written — from `segment.sound` and the cut timing:
+precision). In `bed` / `full` every segment gets a soundtrack node next to its video node and the
+music goes into the clip prompt too; the prompt is derived — not written — from `segment.sound` and
+the cut timing:
 
 ```bash
 node {baseDir}/scripts/short-drama.mjs prompt <project> audio E01-05            # bed (default)
@@ -427,13 +450,13 @@ Music, instrumental: the low cello note.
 No dialogue, no vocals, no narration.
 ```
 
-`params.audioTrack` decides what the node makes: **`bed`** (default) — ambience + sfx + timed
+`params.audioTrack` decides what the node makes: **`bed`** — ambience + sfx + timed
 music, no voices: the editor's music-and-effects track, and what the video model's same-pass sound
 is checked against; **`full`** — the bed plus every line at its cut (start = cut start +
 `speechLeadSeconds`, length from the reading rate), each read in the connected voice sample
 (`--from` the voice display nodes in speaking order → `@Audio1..3`; Seed Audio takes at most 3, so a
-segment with four speakers stays `bed`); **`none`** — no node. Music dynamics are written as cues on
-`sound.music`:
+segment with four speakers stays `bed`); **`none`** / **`post`** — no node. Music dynamics are
+written as cues on `sound.music` in every mode:
 
 ```json
 "music": { "intent": "the low cello note",

package/skill/short-drama/reference/workspace-layout.md

@@ -109,11 +109,11 @@ By default the video node references the sheets directly — **no storyboard ima
 video node, and the video node then references those instead of the sheets.
 
 - Display nodes: `canvas add image --asset <assetId> -n "C01 sheet"` and `canvas add audio --asset <voiceAsset> -n "C01 voice"` — record both in the episode's `wji.displayNodes` (`"C01:sheet"`, `"C01:voice"`) so every generate node can `--from` them. Pictures become `@imageN`, voices `@audioN`, numbered per type in connection order.
-- Per segment, beside the video node, a **soundtrack audio node** (`canvas add audio -n "E01-01 track" -m seed-audio-1.0 --prompt-file …`, prompt from `prompt audio E01-01`) → `storyboard.segments.E01-01.wji.nodes.track`; its asset joins the shot dataset's `images[]`.
+- Soundtrack: in `audioTrack: post` (default) no per-segment node — after the edit, the episode canvas gets one **score node per ≤ 120 s section** (`canvas add audio -n "E01 score 1" -m seed-audio-1.0 --prompt-file …`, from the conformed `prompt score E01` sheet), and the episode binds `wji.assets.edit` / `wji.assets.mix`. In `bed` / `full`, per segment, beside the video node, a **soundtrack audio node** (`canvas add audio -n "E01-01 track" -m seed-audio-1.0 --prompt-file …`, prompt from `prompt audio E01-01`) → `storyboard.segments.E01-01.wji.nodes.track`; its asset joins the shot dataset's `images[]`.
 - Video: `canvas add video -n "E01-01" --from <sheet display nodes in legend order> <voice display nodes of everyone who speaks> -m <the mode's model> -r <params.ratio> [-d <Σ cut seconds>] --prompt-file prompt.txt`. The model and its resolution spelling come from `prompt segment` (`video.model`, `video.resolution`). `-d` **only in `videoDuration: fixed`** — it switches auto-duration off, which pins the clip to the storyboard's timestamps but also fixes the price; in `auto` (the default) leave it out and the model matches the clip to the prompt.
 - Prompt text node: `canvas add text -n "E01-01 prompt" --text-file prompt.txt` beside the video node, for people reading the canvas.
 - Keyframes (opt-in only): `canvas add image -n "E01-01 f<k>" --from <sheet display nodes> -m <image model> -r <params.ratio> -p "<plan prefix + frame>"`, and the video node then `--from f1…fN` instead of the sheets.
-- Shot dataset: `dataset new <shot/E01 folder> "E01-01" -t shot -d "<one line>"`, `--set video_prompt=@prompt.txt`, images = the soundtrack (and keyframes, if any), `primaryVideo` = the clip.
+- Shot dataset: `dataset new <shot/E01 folder> "E01-01" -t shot -d "<one line>"`, `--set video_prompt=@prompt.txt`, images = the soundtrack in `bed` / `full` (and keyframes, if any), `primaryVideo` = the clip.
 
 Placement: `canvas add` puts a node right of its `--from` nodes; pass `--at x,y`
 for the row starts (title nodes) so blocks do not overlap. Take ids from the

package/skill/short-drama/scripts/selftest.mjs

@@ -3,10 +3,11 @@
 // case — a gate that cannot fail is not a gate.
 
 import assert from 'node:assert/strict';
-import { mkdtempSync, readFileSync, rmSync } from 'node:fs';
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { dirname, join } from 'node:path';
 import { fileURLToPath } from 'node:url';
+import { renderDashboard } from './short-drama-dashboard.mjs';
 import {
   CAMERAS, GATES, STAGES, STYLE_PRESETS, CINEMATOGRAPHY_PRESETS, cinematographyForGenre, ACTION_STYLES, ACTION_RULES, actionForGenre, IP_NAMES_RE, GRAMMARS, GRAMMAR_RULES, COMEDY_STYLES, FACESLAP, grammarsOf, grammarLines, grammarForGenre, comedyForGenre, COMPOSITION_STYLES, COMPOSITION_DEVICES, compositionOf, compositionForGenre, STORY_GENRES, STORY_RULES, storyOf, storyForGenre, FORMATS, span, runsAtLeast, formatForGenre, cinematographyOf, stopNumber,
   bindPath, claimedLines, containsName, cutStarts, dialogueSeconds, episodeSeconds,
@@ -17,7 +18,7 @@ import {
   speechDirection, cameraPhrase, lightPhrase, foregroundPhrase, focalOf, LIGHT_DIRECTIONS, identityOf, isEnglish,
   renderGuideline, renderDialogue, scriptSelfCheck, beatOffsets, turnPoints, parseAge, talkAndAction, normalizeRegister, REGISTERS, REGISTER_ACCENT,
   cutStart, segmentCharacters, keyScenes, shotTexts, motionSentence, cameraClause, PURPOSE_KINDS, ANGLES, HEIGHTS, NO_SUBTITLES, LIPS_CLOSED,
-  audioPrompt, musicOf, cueSeconds, AUDIO_MODEL, AUDIO_MAX_CHARS,
+  audioPrompt, musicOf, cueSeconds, AUDIO_MODEL, AUDIO_MAX_CHARS, AUDIO_MAX_SECONDS, scoreSheet, smpte,
   VIDEO_MODELS, VIDEO_MODES, DEFAULT_PARAMS, videoModel, videoResolutionFor, nearestResolution, segmentPlan, videoReferences,
 } from './short-drama.mjs';
 
@@ -45,6 +46,38 @@ const breaks = (gateId, project, episode, needle) => {
   if (needle) ok(f.some((m) => m.includes(needle)), `${gateId} names the problem (${needle}): ${f.join(' | ')}`);
 };
 
+// ── dashboard ──────────────────────────────────────────────────────────────
+{
+  const outputPath = join(tmpdir(), 'wji-short-drama-dashboard-test.html');
+  const html = renderDashboard({
+    project: PROJECT,
+    episodes: { 1: E01 },
+    gateResults: runGates(makeContext(PROJECT, { 1: E01 })),
+    projectDir: FIXTURE,
+    outputPath,
+  });
+  ok(html.startsWith('<!doctype html>'), 'dashboard is a standalone HTML document');
+  ok(html.includes('夜班便利店') && html.includes('人物设定'), 'dashboard renders project sections');
+  ok(html.includes('E01-01') && html.includes('制作总览'), 'dashboard renders episode storyboard');
+  ok(html.includes('图片占位'), 'dashboard keeps an image placeholder when no media is bound');
+
+  const mediaDir = mkdtempSync(join(tmpdir(), 'wji-dashboard-media-'));
+  const localImage = join(mediaDir, 'lin.jpg');
+  writeFileSync(localImage, '');
+  const withMedia = clone(PROJECT);
+  withMedia.characters[0].image.url = localImage;
+  withMedia.characters[1].video = { url: 'https://cdn.example.com/chen.mp4', type: 'video' };
+  const mediaHtml = renderDashboard({
+    project: withMedia,
+    episodes: { 1: E01 },
+    projectDir: mediaDir,
+    outputPath: join(mediaDir, 'dashboard.html'),
+  });
+  ok(mediaHtml.includes('lin.jpg') && mediaHtml.includes('<img'), 'dashboard uses a local image path');
+  ok(mediaHtml.includes('<video src="https://cdn.example.com/chen.mp4"'), 'dashboard uses a WJI video URL');
+  rmSync(mediaDir, { recursive: true, force: true });
+}
+
 // ── fixture is green ────────────────────────────────────────────────────────
 {
   const results = runGates(makeContext(PROJECT, { 1: E01 }));
@@ -1394,21 +1427,24 @@ const breaks = (gateId, project, episode, needle) => {
   eq(cueSeconds(3, seg5.cuts), 6, 'a cue at a cut number is that cut\'s start');
   eq(cueSeconds('10.5s', seg5.cuts), 10.5, 'a cue in seconds is read as seconds');
   eq(cueSeconds('later', seg5.cuts), null, 'a cue that is neither is null');
-  const bed = audioPrompt(PROJECT, E01, seg5);
-  eq(bed.mode, 'bed', 'the default track is a bed');
+  // A per-segment soundtrack node is the `bed` / `full` mode; the default is `post` (scored over the edit).
+  const PB = clone(PROJECT); PB.params.audioTrack = 'bed';
+  eq(paramsOf(PROJECT).audioTrack, 'post', 'the default soundtrack is made in post, over the edit');
+  const bed = audioPrompt(PB, E01, seg5);
+  eq(bed.mode, 'bed', 'bed mode makes a bed');
   eq(bed.model, AUDIO_MODEL, 'the soundtrack node is Seed Audio');
   ok(bed.prompt.startsWith('[0.0s:11.0s] Ambience, continuous: the low hum'), 'the bed opens with the timecoded ambience');
   ok(bed.prompt.includes('[6.0s:9.0s] SFX: a single ring of the brass wind chimes'), 'sfx are timecoded to their cut');
   ok(bed.prompt.includes('[6.0s:10.5s] the music swells — on the chime.') && bed.prompt.includes('[10.5s:11.0s] the music falls silent'), 'music cues become timecoded states');
   ok(bed.prompt.includes('No dialogue, no vocals, no narration.'), 'a bed carries no voices');
   eq(bed.references.length, 0, 'a bed references no voices');
-  const full = audioPrompt(PROJECT, E01, E01.storyboard.segments[1], { mode: 'full' });
+  const full = audioPrompt(PB, E01, E01.storyboard.segments[1], { mode: 'full' });
   eq(full.references.map((r) => r.label).join(','), '@Audio1,@Audio2', 'full mode references the speakers as @AudioN');
   ok(full.prompt.includes('[5.5s:7.1s] @Audio1 (the stocky man in his sixties with a grey crew cut), slow and gravelly'), 'a line is timecoded into its cut after the lead-in, in its voice and direction');
   ok(full.prompt.includes('says: 又是你值夜班。'), 'the line is verbatim');
-  const noMusic = audioPrompt(PROJECT, E01, E01.storyboard.segments[0]);
+  const noMusic = audioPrompt(PB, E01, E01.storyboard.segments[0]);
   ok(noMusic.prompt.includes('No music.'), 'N/A music is stated');
-  const md = renderSegment(PROJECT, E01, 'E01-05');
+  const md = renderSegment(PB, E01, 'E01-05');
   ok(md.includes('## 音轨提示词（seed-audio-1.0 · bed · 11 s）') && md.includes('配乐：the low cello note（镜1 enter，镜3 swell，10.5s silence）'), 'segment markdown carries the soundtrack prompt and the cues');
   // gates
   const e = clone(E01); e.storyboard.segments[4].sound.music = { intent: '', cues: [] };
@@ -1418,7 +1454,7 @@ const breaks = (gateId, project, episode, needle) => {
   const e3 = clone(E01); e3.storyboard.segments[4].sound.music.cues[0].at = '40s';
   breaks('sound', PROJECT, e3, 'inside the segment');
   const e4 = clone(E01); e4.storyboard.segments[4].sound.ambience = 'x'.repeat(AUDIO_MAX_CHARS);
-  breaks('audio-track', PROJECT, e4, 'max 2048');
+  breaks('audio-track', PB, e4, 'max 2048');
   const p = clone(PROJECT); p.params.audioTrack = 'full';
   p.characters.push({ id: 'C04', name: '路人', tier: 'functional', role: 'r', from: ['原创'], identityEn: 'a passer-by in a wet coat', voice: { videoDirection: 'flat' } });
   const e5 = clone(E01); e5.scenes[1].characters.push('C03', 'C04');
@@ -1428,9 +1464,51 @@ const breaks = (gateId, project, episode, needle) => {
   eq(failures('audio-track', p2, E01).length, 0, 'audioTrack none skips the gate');
   const p3 = clone(PROJECT); p3.params.audioTrack = 'stems';
   breaks('params', p3, null, 'audioTrack');
-  const items = manifest(PROJECT, { 1: E01 });
-  ok(items.some((i) => i.item === 'storyboard.segments.E01-01.wji.nodes.track'), 'manifest asks for the soundtrack node');
+  const items = manifest(PB, { 1: E01 });
+  ok(items.some((i) => i.item === 'storyboard.segments.E01-01.wji.nodes.track'), 'bed mode: manifest asks for the soundtrack node');
   ok(!manifest(p2, { 1: E01 }).some((i) => i.item.includes('.nodes.track')), 'audioTrack none asks for no soundtrack node');
+  // post (default): no music in any clip, no per-segment track, the episode is scored over the edit
+  ok(!manifest(PROJECT, { 1: E01 }).some((i) => i.item.includes('.nodes.track')), 'post asks for no per-segment soundtrack node');
+  eq(failures('audio-track', PROJECT, E01).length, 0, 'post skips the per-segment audio gate');
+  for (const video of ['h3', 'seedance2', 'wan']) {
+    for (const references of ['keyframes', 'sheets']) {
+      const pv = clone(PROJECT); pv.params.video = video; pv.params.references = references;
+      const vp = promptSkeleton(pv, E01, 'E01-05').video.prompt;
+      ok(!vp.includes('cello'), `post: the music intent stays out of the clip (${video}/${references})`);
+      ok(video === 'h3' ? vp.includes('non_diegetic_music: N/A') : video === 'wan' ? vp.includes('无配乐') : vp.includes('No music.'), `post: the clip is told there is no music (${video}/${references})`);
+      const pb = clone(pv); pb.params.audioTrack = 'bed';
+      ok(promptSkeleton(pb, E01, 'E01-05').video.prompt.includes('cello'), `bed: the music intent is still in the clip (${video}/${references})`);
+    }
+  }
+  const ePost = clone(E01); ePost.storyboard.segments[4].sound.sfx.push({ cut: 3, event: 'SPECIAL: a reversed-cymbal riser into a hard stop', post: true });
+  ok(!promptSkeleton(PROJECT, ePost, 'E01-05').video.prompt.includes('reversed-cymbal'), 'a post: true sfx stays out of the clip');
+  ok(audioPrompt(PB, ePost, ePost.storyboard.segments[4]).prompt.includes('reversed-cymbal'), 'a post: true sfx is still in a per-segment bed');
+  ok(renderSegment(PROJECT, E01, 'E01-05').includes('prompt score E01'), 'post: segment markdown points at the episode score sheet');
+  const sheet = scoreSheet(PROJECT, ePost);
+  eq(sheet.seconds, 68, 'the score sheet lays the segments end to end');
+  eq(sheet.fps, 24, 'the score sheet defaults to 24 fps');
+  eq(smpte(63.5, 24), '00:01:03:12', 'SMPTE is HH:MM:SS:FF');
+  eq(smpte(3725, 25), '01:02:05:00', 'SMPTE carries hours');
+  ok(sheet.rows.some((r) => r.layer === 'BGM' && r.at === 63 && r.what.startsWith('swell')), 'a music cue lands at its segment offset + cut start');
+  ok(sheet.rows.some((r) => r.layer === 'design' && r.at === 63 && r.what.includes('reversed-cymbal')), 'post sfx are sound design on the sheet');
+  ok(sheet.rows.some((r) => r.layer === 'sfx in clip' && r.what.includes('wind chimes')), 'in-clip sfx are listed so they are not doubled');
+  ok(sheet.rows.some((r) => r.layer === 'DX' && r.at === 3.5), 'lines are spotted after the lead-in, for ducking');
+  eq(sheet.rows.filter((r) => r.layer === 'BGE').length, 2, 'one ambience bed per scene, chained across its segments');
+  eq(sheet.rows.find((r) => r.layer === 'BGE' && r.segment === 'E01-02').end, 68, 'the scene bed runs to the scene end');
+  ok(sheet.rows.some((r) => r.layer === 'story' && r.what.startsWith('hook')) && sheet.rows.some((r) => r.layer === 'story' && r.what.startsWith('cliff')), 'the hook and the cliff anchor 起 / 合');
+  eq(sheet.sections.length, 1, 'a 68 s episode is one Seed Audio section');
+  const eLong = clone(E01);
+  for (const sg of eLong.storyboard.segments) for (const c of sg.cuts) c.seconds *= 2;   // 136 s
+  const long = scoreSheet(PROJECT, eLong, { fps: 25 });
+  ok(long.sections.length === 2 && long.sections.every((x) => x.seconds <= AUDIO_MAX_SECONDS), 'a long episode splits into ≤ 120 s sections');
+  eq(long.sections[0].segments.join(','), 'E01-01', 'the split falls on the scene change');
+  eq(long.fps, 25, '--fps overrides the frame rate');
+  const pp = clone(PROJECT); const eDone = clone(E01);
+  for (const sg of eDone.storyboard.segments) sg.wji = { ...(sg.wji || {}), assets: { ...(sg.wji?.assets || {}), video: 'a1' } };
+  ok(manifest(pp, { 1: eDone }).some((i) => i.item === 'wji.assets.edit'), 'post: once every clip exists the manifest asks for the edit');
+  eDone.wji = { ...(eDone.wji || {}), assets: { edit: 'e1' } };
+  ok(manifest(pp, { 1: eDone }).some((i) => i.item === 'wji.assets.mix'), 'post: after the edit it asks for the scored mix');
+  ok(!manifest(pp, { 1: E01 }).some((i) => i.item.startsWith('wji.assets.')), 'post: no edit item while clips are missing');
   ok(voiceSamplePrompt(PROJECT.characters[0], paramsOf(PROJECT)).includes('干声，无配乐无音效'), 'the voice sample is asked for dry');
 }