@koda-sl/baker-cli 0.90.1 → 0.91.0

package/README.md

@@ -3360,9 +3360,10 @@ Place and mix several audio clips onto one timeline — a music bed plus timed v
 
 | Name | Type | Required | Notes |
 |---|---|---|---|
-| `tracks` | array | yes | `[{slot, start_s, gain_db?}]` — `slot` matches a wired input; `start_s` is absolute seconds; `gain_db` ducks (e.g. `-12` for a music bed) |
+| `tracks` | array | yes | `[{slot, start_s, gain_db?}]` — `slot` matches a wired input; `start_s` is absolute seconds; `gain_db` sets the static level (e.g. `-20` for a music bed) |
 | `total_ms` | number | no | pins the final length (pad/trim) |
 | `output_format` | enum | no | `mp3` (default) / `wav` / `m4a` |
+| `duck` | object | no | sidechain-duck one track under others: `{ track, against: [...], threshold?, ratio?, attack?, release? }` — the `track` (music) drops while any `against` track (voice) carries signal and recovers in the gaps |
 
 **Outputs** — `audio`. **Cost** — free (local). Requires `ffmpeg`.
 
@@ -3619,11 +3620,11 @@ Turn a reference video into a **runnable, self-validated reproduction canvas** i
 1. **`video_deconstruct`** (`~google/gemini-pro-latest`, full mode) — reverse-engineers the video into a scene-by-scene blueprint + word-level transcript, written next to the canvas as **`prompt.json`**. Each scene's `start_frame_prompt`/`end_frame_prompt` are inlined into the frame nodes (see below); `prompt.json` then rides along as the shared **global style reference** (palette, cast cohesion) and as provenance.
 2. **recurring-element selection** (`~google/gemini-flash-latest`) — picks only the **recurring, identity-critical** elements (each `global.cast` person, a recurring animal, a showcased product, the brand logo) and the scene indices each appears in. One real reference image grounds each element across **every** frame it appears in, so the same actor stays consistent the whole video. This selection runs as a **second pass over a slimmed blueprint** (cast/branding + each scene's frame prompts only) — a long ad's full blueprint can exceed the engine's inline-prompt limit, so the heavy per-scene detail (dialogue, overlays, transcript) the selector never reads is dropped before the prompt.
 
-Before the deconstruct it runs a **local shot-cut pass** on the source file with **[PySceneDetect](https://www.scenedetect.com)** (`scenedetect` CLI, `detect-content` — the battle-tested HSV content detector, installed in the canvas sandbox) and passes the cut timestamps as `video_deconstruct`'s `shot_cuts`. The deconstruct snaps its scene boundaries onto those real cuts and **splits any scene that spans one**, so a scene's frames can never straddle a hard cut (the failure where a scene's start frame was the couch and its end frame the b-roll). Two knobs tuned for fast social ads: the content **threshold defaults to 18** (PySceneDetect's own default of 27 misses soft reframes) and the **minimum scene length is dropped to 0.25s** (its default ~0.6s merges away rapid montage flashes) — so super-fast cuts survive and become cheap still-holds downstream. Tune sensitivity per video with **`--shot-threshold N`** (lower = more cuts). If `scenedetect` is unavailable it warns loudly and degrades to LLM-only boundaries.
+Before the deconstruct it runs a **local shot-cut pass** on the source file with **[PySceneDetect](https://www.scenedetect.com)** (`scenedetect` CLI, `detect-content` — the battle-tested HSV content detector, installed in the canvas sandbox) and passes the cut timestamps as `video_deconstruct`'s `shot_cuts`. The deconstruct snaps its scene boundaries onto those real cuts and **splits any scene that spans one**, so a scene's frames can never straddle a hard cut (the failure where a scene's start frame was the couch and its end frame the b-roll). Two knobs tuned for fast social ads: the content **threshold defaults to 18** (PySceneDetect's own default of 27 misses soft reframes) and the **minimum scene length is dropped to 0.25s** (its default ~0.6s merges away rapid montage flashes) — so super-fast cuts survive and become cheap still-holds downstream. The threshold is **adaptive**: if the first pass looks like a continuous shot shredded into many close micro-cuts (a talking-head selfie's natural motion), it re-runs at PySceneDetect's own default of 27 and trusts that — real montage cuts survive it, motion artifacts don't. Pinning **`--shot-threshold N`** disables the re-check (lower = more cuts). The backend additionally coalesces residual same-shot slivers. If `scenedetect` is unavailable it warns loudly and degrades to LLM-only boundaries.
 
 A shot longer than the video model's per-clip ceiling (Seedance's 15s, passed as `video_deconstruct`'s `max_clip_s`) is split into equal **continuation sub-scenes** that share their splice boundary exactly — so a long shot is reproduced in **full** (no truncation) and joins seamlessly. Each sub-scene carries `continues_previous`.
 
-It then scaffolds the full pipeline like an **editing timeline**: each clip gets a **static-ad-grade start AND end keyframe** (`image_generate`, each with its **own self-contained `params.prompt`** — edit a frame node to change only that frame; `prompt.json` wired as the **authoritative shared `target_blueprint`**, plus a per-element reference legend). Each keyframe is **fully recast** to the dropped `el_*` reference images, and — exactly like a static ad — the **original extracted frame is kept LAST as a composition anchor** (the RECAST block + "ignore its brand/colors" legend stop it leaking identity). Both keyframes feed `video_generate` (`first_frame`+`last_frame`, so Seedance interpolates real in-shot motion; ultra-detailed motion brief; duration snapped to the nearest allowed clip length). Every keyframe grounds **only on its own extracted frame + `el_*` slots** — no reference to any other generated frame — so all images render **in parallel** (no cascade). Source-frame URLs are **deduped** (each ingested once). `--frames reuse` wires the real source frame straight in.
+It then scaffolds the full pipeline like an **editing timeline**: each clip gets a **static-ad-grade start AND end keyframe** (`image_generate`, each with its **own self-contained `params.prompt`** — edit a frame node to change only that frame; `prompt.json` wired as the **authoritative shared `target_blueprint`**, plus a per-element reference legend). Each keyframe is **fully recast** to the dropped `el_*` reference images. For a frame with **no person/animal** the original extracted frame is kept LAST as a pure composition anchor; for any frame **with a face it is dropped entirely** (it leaked the source person's identity — the hook used to render the reference woman, not our actor), so the recast `el_*`/actor-sheet is the sole identity reference. Both keyframes feed `video_generate` (`first_frame`+`last_frame`, so Seedance interpolates real in-shot motion; ultra-detailed motion brief; duration snapped to the nearest allowed clip length). Every keyframe grounds **only on its own extracted frame + `el_*` slots** — no reference to any other generated frame — so all images render **in parallel** (no cascade). Source-frame URLs are **deduped** (each ingested once). `--frames reuse` wires the real source frame straight in.
 
 **Composited scenes (split-screen / picture-in-picture / keyed presenter).** Real ads aren't always one full-frame shot — a frame can be **persistently divided** (b-roll on top, a presenter talking on the bottom) or **layer a presenter** over background footage (boxed in a corner, or green-screen keyed). The deconstruct now reports this per scene as `scene.composition` (`layout: split_screen | pip | keyed_overlay`, with one `region` per stream — each its own clean-plate frame + motion brief, the talking-head region flagged `is_presenter`). The scaffold reproduces a composited scene by building **one clip per region** (`s<i>_r0_*`, `s<i>_r1_*`, …) and compositing them with ffmpeg: a split-screen `vstack`/`hstack` (stack direction read from the region **panels**, so a top/bottom split always stacks vertically), or a picture-in-picture `overlay` of the presenter inset at its corner. A **keyed** presenter is first cut to transparency by `video_background_remove` (`s<i>_key`), then overlaid. The presenter region carries the native lip-synced voice; b-roll/render panels stay silent. To change a layout, edit `composition` in `prompt.json` and re-scaffold, or hand-edit the `s<i>_composite` ffmpeg args. Plain full-frame scenes (the default) are unaffected.
 
@@ -3664,7 +3665,6 @@ baker canvas run ./reference-ad.video.canvas.json
 | `--out <path>` | `<video-dir>/<name>.video.canvas.json` | Where to write the canvas (composition is copied alongside). |
 | `--frames <mode>` | `generate` | `generate` emits ONE recast keyframe per scene (the original frame is dropped so the dropped `el_*` assets drive identity); `reuse` wires the real extracted first+last frames straight into the clips (faithful, cheaper, no recast). |
 | `--ambient` | off | Give silent **b-roll** scenes native diegetic ambient (Seedance `generate_audio`), mixed deep under the music bed. Talking scenes already carry voice; check levels don't muddy the mix before keeping it. |
-| `--actor-sheets` | off | Lock a recast **person/animal that recurs across ≥2 scenes** to ONE multi-view turnaround (`image_reference_sheet`) that every frame grounds on — the strongest cross-scene identity lock. Costs extra credits per sheet; a fused sheet can over-polish, so eyeball it. |
 | `--max-scenes <n>` | all source scenes | **Cost lever that reduces fidelity** — caps the deconstruct, MERGING away every scene beyond the cap (fewer cuts, lost beats). Prints a warning when set; omit it to reproduce every scene. |
 | `--language <code>` | auto | Transcript/dialogue language hint (e.g. `fr`, `en`). |
 | `--focus <text>` | — | Known provenance/emphasis to ground the deconstruct. |
@@ -3675,6 +3675,8 @@ baker canvas run ./reference-ad.video.canvas.json
 
 Each scene is captured in a **shoot mode** — `ugc_selfie` (talking heads, the default look), `ugc_broll`, `studio_product` (pack shot), `lifestyle_cinematic`, or `screen_ui`. The scaffold derives one per scene (UGC by default; the cinematic and screen lanes are opt-in) and bakes its capture block into the frame and a camera default into the clip; override per scene with a `shoot_mode` field in `prompt.json`. Capture aesthetic + depth-of-field follow the mode (UGC stays flat; studio/lifestyle allow shallow DoF). Clips also carry **diegetic native audio** — the scene's own ambience described in the Seedance prompt, never music (the music bed is a separate, ducked track); set a scene's `ambient` field to steer it.
 
+**Automatic by default (no flags).** A recast **person/pet recurring across ≥2 scenes** is always locked to ONE multi-view turnaround (`image_reference_sheet`) every frame grounds on. An **app/website/chat screen** is never sent to the video model — the scaffold drops the scene to a clean talking-head and seeds a phone-mockup PIP stub to fill with a real `baker images screenshot` or brand HTML block (Seedance garbles UI and a split leaves a seam). The **music bed is instrumental** (the script is never fed to the music model — it would sing over the voice), enters only after the hook, and is **sidechain-ducked** under the voice. **Word-synced TikTok captions** are wired off the deconstruct transcript whenever the ad has speech. Seeded overlays are pushed **off the subject's face** (dead-center → bottom band).
+
 The two scaffold passes are billed (the full `video_deconstruct` is the heavy one); **running** the result then generates many image/video/audio assets and is not free. Defaults to vertical 1080×1920 overlays — copy + edit the composition for other aspect ratios. For on-brand overlay type, drop `brand-bold.otf`/`brand-regular.otf` into the copied `video-overlay-composition/` dir (wired via `@font-face`, with a system fallback). Richer transcription (punctuated words + paragraphs) is available via the deconstruct's `transcriber: "deepgram"` param when `DEEPGRAM_API_KEY` is set.
 
 #### `baker canvas scaffold-static-ad <image> [flags]`

package/canvas/video-overlay-composition/index.html

@@ -64,12 +64,33 @@
       }
       .ov.fe { font-size: 30px; font-weight: 600; opacity: 0.9; }
 
+      /* SAFE ZONES — a 9:16 talking-head's FACE fills the vertical CENTER band. Keep
+         graphics in the TOP band (≤360px) or BOTTOM band (≥1400px); never park a caption
+         dead-center over the face. The scaffold already pushes center placements to the
+         bottom, but if you hand-place, respect the bands. */
+
+      /* COLLISION-SAFE TOP BAR — wrap co-timed top items (logo + trust badge + …) in ONE
+         .top-bar so they pack side-by-side with a gap and never overlap. Example:
+           <div class="top-bar" data-start="0" data-dur="30">
+             <img class="brandmark" src="logo.svg"><span class="trust">★ 4,5/5</span>
+           </div> */
+      .top-bar {
+        position: absolute; top: 70px; left: 56px; right: 56px;
+        display: flex; align-items: center; gap: 20px; flex-wrap: wrap;
+      }
+
+      /* STAGGER — give a group `data-stagger="0.12"` and its direct children reveal one
+         after another (e.g. coverage chips appearing as each is spoken), not as a clump. */
+      .chips { display: flex; flex-wrap: wrap; gap: 14px; justify-content: center; max-width: 92%; }
+
       /* 9-grid position helpers (absolute). Tweak the insets or add your own. */
       .pos-top-left      { top: 90px; left: 56px; text-align: left; }
       .pos-top-center    { top: 90px; left: 50%; transform: translateX(-50%); }
       .pos-top-right     { top: 90px; right: 56px; text-align: right; }
       .pos-mid-left,
       .pos-center-left   { top: 50%; left: 56px; transform: translateY(-50%); text-align: left; }
+      /* Center = the face. Kept only for non-talking-head plates; the scaffold remaps
+         seeded center overlays to the bottom band so they never cover the subject. */
       .pos-center,
       .pos-mid-center    { top: 50%; left: 50%; transform: translate(-50%,-50%); }
       .pos-mid-right,
@@ -115,26 +136,31 @@
         const els = Array.from(document.querySelectorAll('#overlay-root [data-start]'));
 
         // Generic timeline: show each element at data-start, hide at start+data-dur,
-        // with an optional canned entrance from data-anim. No styling decisions here —
+        // with an optional canned entrance from data-anim. With data-stagger="<sec>" the
+        // element's DIRECT CHILDREN reveal one-by-one (e.g. coverage chips landing on the
+        // word each is spoken) instead of as a single clump. No styling decisions here —
         // the look lives entirely in the CSS/markup above.
         for (const el of els) {
           const at = parseFloat(el.getAttribute('data-start') || '0') || 0;
           const dur = parseFloat(el.getAttribute('data-dur') || '2.5') || 2.5;
           const anim = el.getAttribute('data-anim') || '';
+          const stagger = parseFloat(el.getAttribute('data-stagger') || '0') || 0;
           // Preserve any positioning transform the CSS set (translate(...)).
           const baseTransform = getComputedStyle(el).transform;
           const tx = baseTransform && baseTransform !== 'none' ? baseTransform : '';
 
+          // Stagger animates the children; otherwise the element itself enters.
+          const targets = stagger > 0 && el.children.length ? Array.from(el.children) : el;
           tl.set(el, { visibility: 'visible' }, at);
           if (anim === 'pop') {
-            tl.fromTo(el, { opacity: 0, scale: 0.7 }, { opacity: 1, scale: 1, duration: 0.3, ease: 'back.out(1.7)' }, at);
+            tl.fromTo(targets, { opacity: 0, scale: 0.7 }, { opacity: 1, scale: 1, duration: 0.3, ease: 'back.out(1.7)', stagger }, at);
           } else if (anim === 'slide_up') {
-            tl.fromTo(el, { opacity: 0, yPercent: 30 }, { opacity: 1, yPercent: 0, duration: 0.3, ease: 'power2.out' }, at);
+            tl.fromTo(targets, { opacity: 0, yPercent: 30 }, { opacity: 1, yPercent: 0, duration: 0.3, ease: 'power2.out', stagger }, at);
           } else if (anim === 'slide_down') {
-            tl.fromTo(el, { opacity: 0, yPercent: -30 }, { opacity: 1, yPercent: 0, duration: 0.3, ease: 'power2.out' }, at);
+            tl.fromTo(targets, { opacity: 0, yPercent: -30 }, { opacity: 1, yPercent: 0, duration: 0.3, ease: 'power2.out', stagger }, at);
           } else {
             // Default / any unrecognized data-anim value: a plain fade.
-            tl.fromTo(el, { opacity: 0 }, { opacity: 1, duration: 0.25, ease: 'power1.out' }, at);
+            tl.fromTo(targets, { opacity: 0 }, { opacity: 1, duration: 0.25, ease: 'power1.out', stagger }, at);
           }
           tl.to(el, { opacity: 0, duration: 0.2 }, Math.max(at + 0.2, at + dur));
           tl.set(el, { visibility: 'hidden' }, at + dur + 0.21);

package/dist/{chunk-2E4H2GIJ.js → chunk-LMVDA3EZ.js}

@@ -3853,19 +3853,56 @@ var Track = z6.object({
   /** Optional level adjustment in dB (negative ducks, e.g. a music bed at -12). */
   gain_db: z6.number().optional()
 }).strict();
+var DuckSpec = z6.object({
+  track: z6.string().min(1),
+  against: z6.array(z6.string().min(1)).min(1),
+  threshold: z6.number().min(0).max(1).optional(),
+  ratio: z6.number().min(1).max(20).optional(),
+  attack: z6.number().min(0.01).optional(),
+  release: z6.number().min(0.01).optional()
+}).strict();
 var AudioTimelineParams = z6.object({
   tracks: z6.array(Track).min(1),
   /** Final track length in ms — pads short / trims long. Defaults to the natural mix length. */
   total_ms: z6.number().int().positive().optional(),
-  output_format: z6.enum(["mp3", "wav", "m4a"]).optional()
+  output_format: z6.enum(["mp3", "wav", "m4a"]).optional(),
+  /** Optional sidechain duck of one track under others (e.g. music under voice). */
+  duck: DuckSpec.optional()
 }).strict();
 var AudioTimelineInputs = z6.record(z6.string(), z6.unknown());
 var AudioTimelineOutputs = z6.object({ audio: z6.custom() }).strict();
+function applyDuck(params, labelFor, filterChains) {
+  const duck = params.duck;
+  if (!duck) return;
+  const trackIdx = params.tracks.findIndex((t) => t.slot === duck.track);
+  const keyIdxs = duck.against.map((s) => params.tracks.findIndex((t) => t.slot === s)).filter((i) => i >= 0);
+  if (trackIdx < 0 || keyIdxs.length === 0) return;
+  const keyLabels = [];
+  for (const ki of keyIdxs) {
+    const base = labelFor[ki];
+    filterChains.push(`[${base}]asplit=2[${base}m][${base}k]`);
+    labelFor[ki] = `${base}m`;
+    keyLabels.push(`[${base}k]`);
+  }
+  let keyOut = keyLabels[0];
+  if (keyLabels.length > 1) {
+    filterChains.push(`${keyLabels.join("")}amix=inputs=${keyLabels.length}:normalize=0[duckkey]`);
+    keyOut = "[duckkey]";
+  }
+  const th = duck.threshold ?? 0.03;
+  const ra = duck.ratio ?? 8;
+  const at = duck.attack ?? 5;
+  const re = duck.release ?? 300;
+  filterChains.push(
+    `[${labelFor[trackIdx]}]${keyOut}sidechaincompress=threshold=${th}:ratio=${ra}:attack=${at}:release=${re}[ducked]`
+  );
+  labelFor[trackIdx] = "ducked";
+}
 function buildAudioTimelineArgs(params) {
   const fmt = params.output_format ?? "mp3";
   const inputArgs = [];
   const filterChains = [];
-  const mixLabels = [];
+  const labelFor = [];
   params.tracks.forEach((track, i) => {
     inputArgs.push("-i", `{{in.${track.slot}}}`);
     const delayMs = Math.round(track.start_s * 1e3);
@@ -3873,8 +3910,10 @@ function buildAudioTimelineArgs(params) {
     if (track.gain_db !== void 0) steps.push(`volume=${track.gain_db}dB`);
     const label = `a${i}`;
     filterChains.push(`[${i}:a]${steps.join(",")}[${label}]`);
-    mixLabels.push(`[${label}]`);
+    labelFor[i] = label;
   });
+  applyDuck(params, labelFor, filterChains);
+  const mixLabels = labelFor.map((l) => `[${l}]`);
   let graph = `${filterChains.join(";")};${mixLabels.join("")}amix=inputs=${params.tracks.length}:normalize=0`;
   if (params.total_ms !== void 0) {
     const totalS = params.total_ms / 1e3;
@@ -3885,7 +3924,7 @@ function buildAudioTimelineArgs(params) {
 }
 var audioTimelineNode = defineNode({
   id: "audio_timeline",
-  version: "1.0.0",
+  version: "1.1.0",
   category: "audio",
   location: "local",
   summary: "Place and mix several audio clips onto one timeline: each track starts at a given second (optionally level-adjusted in dB), then they're combined into a single track. Built for laying a music bed plus timed voiceover lines and sound effects under a video.",
@@ -3910,6 +3949,21 @@ var audioTimelineNode = defineNode({
         });
       }
     });
+    const duck = parsed.data.duck;
+    if (duck) {
+      const slots = new Set(parsed.data.tracks.map((t) => t.slot));
+      if (!slots.has(duck.track)) {
+        issues.push({ path: "params.duck.track", message: `duck.track "${duck.track}" is not one of the tracks` });
+      }
+      duck.against.forEach((s, k) => {
+        if (!slots.has(s)) {
+          issues.push({ path: `params.duck.against[${k}]`, message: `duck.against "${s}" is not one of the tracks` });
+        }
+        if (s === duck.track) {
+          issues.push({ path: `params.duck.against[${k}]`, message: "a track cannot duck against itself" });
+        }
+      });
+    }
     return issues;
   },
   async cacheKeyExtras() {
@@ -6085,4 +6139,4 @@ export {
   defaultRegistry,
   createEngineFromEnv
 };
-//# sourceMappingURL=chunk-2E4H2GIJ.js.map
+//# sourceMappingURL=chunk-LMVDA3EZ.js.map