@koda-sl/baker-cli 0.116.0 → 0.118.0

package/README.md

@@ -3855,7 +3855,7 @@ Turn a reference video into a **runnable, self-validated reproduction canvas** i
 1. **`video_deconstruct`** (`~google/gemini-pro-latest`, full mode) — reverse-engineers the video into a scene-by-scene blueprint + word-level transcript, written next to the canvas as **`prompt.json`**. Each scene's `start_frame_prompt`/`end_frame_prompt` are inlined into the frame nodes (see below); `prompt.json` then rides along as the shared **global style reference** (palette, cast cohesion) and as provenance.
 2. **recurring-element selection** (`~google/gemini-flash-latest`) — picks only the **recurring, identity-critical** elements (each `global.cast` person, a recurring animal, a showcased product, the brand logo) and the scene indices each appears in. One real reference image grounds each element across **every** frame it appears in, so the same actor stays consistent the whole video. This selection runs as a **second pass over a slimmed blueprint** (cast/branding + each scene's frame prompts only) — a long ad's full blueprint can exceed the engine's inline-prompt limit, so the heavy per-scene detail (dialogue, overlays, transcript) the selector never reads is dropped before the prompt.
 
-Before the deconstruct it runs a **local shot-cut pass** on the source file with **[PySceneDetect](https://www.scenedetect.com)** (`scenedetect` CLI, `detect-content` — the battle-tested HSV content detector, installed in the canvas sandbox) and passes the cut timestamps as `video_deconstruct`'s `shot_cuts`. The deconstruct snaps its scene boundaries onto those real cuts and **splits any scene that spans one**, so a scene's frames can never straddle a hard cut (the failure where a scene's start frame was the couch and its end frame the b-roll). Two knobs tuned for fast social ads: the content **threshold defaults to 18** (PySceneDetect's own default of 27 misses soft reframes) and the **minimum scene length is dropped to 0.25s** (its default ~0.6s merges away rapid montage flashes) — so super-fast cuts survive and become cheap still-holds downstream. The threshold is **adaptive**: if the first pass looks like a continuous shot shredded into many close micro-cuts (a talking-head selfie's natural motion), it re-runs at PySceneDetect's own default of 27 and trusts that — real montage cuts survive it, motion artifacts don't. Pinning **`--shot-threshold N`** disables the re-check (lower = more cuts). The backend additionally coalesces residual same-shot slivers. If `scenedetect` is unavailable it warns loudly and degrades to LLM-only boundaries.
+Before the deconstruct it runs a **local shot-cut pass** on the source file with **[PySceneDetect](https://www.scenedetect.com)** (`scenedetect` CLI, `detect-content` — the battle-tested HSV content detector, installed in the canvas sandbox) and passes the cut timestamps as `video_deconstruct`'s `shot_cuts`. The deconstruct snaps its scene boundaries onto those real cuts and **splits any scene that spans one**, so a scene's frames can never straddle a hard cut (the failure where a scene's start frame was the couch and its end frame the b-roll). Two knobs tuned for fast social ads: the content **threshold defaults to 18** (PySceneDetect's own default of 27 misses soft reframes) and the **minimum scene length is dropped to 0.25s** (its default ~0.6s merges away rapid montage flashes) — so super-fast cuts survive and become cheap still-holds downstream. The threshold is **adaptive**: if the first pass looks like a continuous shot shredded into many close micro-cuts (a talking-head selfie's natural motion), it re-runs at PySceneDetect's own default of 27 and **merges the two passes** — the high-threshold set is the base, and the low pass's *isolated* extras (real soft blur-morph transitions that vanish at 27) are added back while clustered extras (motion shred) stay dropped. Pinning **`--shot-threshold N`** disables the re-check (lower = more cuts). The backend snap window is likewise **adaptive** (up to 1s onto an unambiguous nearest cut, shrinking around dense cut pairs so a boundary never jumps past the wrong cut), any scene spanning an interior cut is split, and the residual-sliver coalesce is **cut-aware**: a drift sliver folds backward across its non-cut edge and never re-merges across a real cut. If `scenedetect` is unavailable it warns loudly and degrades to LLM-only boundaries.
 
 A shot longer than the video model's per-clip ceiling (Seedance's 15s, passed as `video_deconstruct`'s `max_clip_s`) is split into equal **continuation sub-scenes** that share their splice boundary exactly — so a long shot is reproduced in **full** (no truncation) and joins seamlessly. Each sub-scene carries `continues_previous`.
 
@@ -3863,9 +3863,17 @@ It then scaffolds the full pipeline like an **editing timeline**: each clip gets
 
 **Composited scenes (split-screen / picture-in-picture / keyed presenter).** Real ads aren't always one full-frame shot — a frame can be **persistently divided** (b-roll on top, a presenter talking on the bottom) or **layer a presenter** over background footage (boxed in a corner, or green-screen keyed). The deconstruct now reports this per scene as `scene.composition` (`layout: split_screen | pip | keyed_overlay`, with one `region` per stream — each its own clean-plate frame + motion brief, the talking-head region flagged `is_presenter`). The scaffold reproduces a composited scene by building **one clip per region** (`s<i>_r0_*`, `s<i>_r1_*`, …) and compositing them with ffmpeg: a split-screen `vstack`/`hstack` (stack direction read from the region **panels**, so a top/bottom split always stacks vertically), or a picture-in-picture `overlay` of the presenter inset at its corner. A **keyed** presenter is first cut to transparency by `video_background_remove` (`s<i>_key`), then overlaid. The presenter region carries the native lip-synced voice; b-roll/render panels stay silent. To change a layout, edit `composition` in `prompt.json` and re-scaffold, or hand-edit the `s<i>_composite` ffmpeg args. Plain full-frame scenes (the default) are unaffected.
 
+**Typed region kinds & real screen surfaces.** Each composition region now carries a `kind` — `camera` (filmed footage, re-generated), `screen_capture` (app/site/document screen recording), `static_graphic` (designed text/graphic panel), or `generated` (3D/motion graphics) — plus an optional `nested` list for video-in-video (a Loom-style camera bubble inside a screen share). `kind` is authoritative for routing (prose keywords remain the fallback for older blueprints): `screen_capture`/`static_graphic` regions are **never generated by the video model** — the scene renders as a clean background plate (its clip prompt is scrubbed of all screen narration and forbids rendering UI) and the real surface is composited on the overlay layer. The route is decided **once per persistent layout run** (consecutive scenes sharing one composition signature), so a layout that runs unbroken across many scenes can't flip between pipelines on wording differences. A persistent surface seeds **ONE grouped stub** in `video-overlay-composition/index.html` spanning its whole window, with a per-scene **state timeline** — build one continuous screen recording/mockup, not one screenshot per scene. A `screen_capture` region also carries `surface_id`: a source video routinely **splices two unrelated screen recordings** under one persistent layout (a live app-processing capture, then an unrelated pre-made demo note) — the deconstruct assigns a stable id while the SAME recording continues and a new one when the on-screen content genuinely changes, so the run splits into **separate stubs** at the splice instead of asking for one screenshot that can't cover both. `baker canvas validate` additionally warns (`VIDEO_UI_IN_PROMPT`) if any clip prompt still narrates a screen surface, and (`VIDEO_BRANDMARK_IN_PROMPT`) if a generate prompt asks the model to paint a brand logo/wordmark (generation garbles marks; source the real one with `baker images logo` and composite it on the overlay layer).
+
+**Designed graphics are rebuilt, not generated.** A `static_graphic` surface (a newspaper-collage panel, a meme card, a marketing composition) seeds a **GRAPHIC PANEL** stub — rebuild it as brand HTML or drop the design asset; it never gets the "screenshot the live page" instruction (there is no live page). A **full-frame** designed-graphic scene (the deconstruct emits one full-frame `static_graphic` region for meme/collage/motion-graphic beats) routes to a real design plate the same way screens do — no `image_generate`/`video_generate` — and dialogue over an all-graphic scene is voiceover by definition (nobody is on screen to lip-sync). A region typed `generated` whose own prose reads like a UI/designed panel is treated as a surface candidate too (the frame-grounded continuity checker delivers the verdict and corrects the kind), so one mistyped kind can't re-open the Seedance-paints-UI hole. Floating FX elements (hearts, sparkles, badges) ride the overlay layer: their narration is **scrubbed from clip briefs** and a categorical no-decorations directive is added, so the model can't bake a second, uneditable copy under the real composited one.
+
+**Same-take continuity.** Scene boundaries that don't land on a detected shot cut are marked `continues_previous` (they're content beats inside one camera take — a genuinely cut-less video is ONE take), and a mid-take phrase clip **chains its start keyframe from the previous phrase's end keyframe**, so one continuous shot never jumps identity/pose at an invented boundary.
+
+**Speech budgets use the speaker's observed pace.** `metadata.video.talking_scenes` stamps `est_speech_s` measured from the deconstruct's word timings plus `speech_words`; the `VIDEO_SPEECH_OVERRUN` check budgets a (re-authored) line at that real rate instead of a fixed words-per-second guess, so a fast talker's line that demonstrably fits its clip no longer fails validation. Numeral spelling for native clips applies **only to the spoken dialogue/transcript**, never to visual descriptions (a UI progress bar's "4%" stays "4%"). And on a blocking validation issue the canvas is **still written** (with the issues under `metadata.todo.blocking_validation_issues`) — the paid deconstruct is never discarded; fix in place and re-`validate`.
+
 **True output aspect, even when the video model can't generate it.** The scaffold resolves TWO ratios: the ad's **output** ratio (the source video's detected `aspect_ratio`, or **`--aspect`**, e.g. `4:5` → a 1080×1350 canvas) and the **generation** ratio — the nearest ratio the video model supports (Seedance has no 4:5, so clips + conditioning keyframes generate at `3:4`). When they differ the scaffold warns loudly, records `metadata.todo.aspect_remap`, normalizes **every clip** onto the output canvas (scale-to-fill, center-crop — keep subjects centered), sizes every ffmpeg plate/still/composite to the output dims, and stamps the copied overlay/caption compositions' `meta.json` to match. Nothing is silently dropped: an unknown ratio snaps to the nearest known one and stays explicit on every node.
 
-**Montage flashes held as stills.** A rapid-cut beat shorter than ~2s with no spoken line is a **flash** — Seedance's shortest clip is 4s, so generating one (then trimming away most of it) burns credits for motion no viewer perceives. The scaffold instead **holds one keyframe as a still** for the scene length (a cheap ffmpeg loop, no billed `video_generate`), same look at a fraction of the cost. Talking/ambient beats keep a real clip (they need motion + native audio).
+**Montage flashes held as stills — unless the picture really moves.** A rapid-cut beat shorter than ~2s with no spoken line is a **flash** — Seedance's shortest clip is 4s, so generating one (then trimming away most of it) burns credits for motion no viewer perceives. The scaffold instead **holds one keyframe as a still** for the scene length (a cheap ffmpeg loop, no billed `video_generate`), same look at a fraction of the cost. The deconstruct now stamps each scene's **`motion_level`** (`static` / `subtle` / `dynamic`): a **dynamic** flash (pouring chocolate, hands working, walking) keeps a **real trimmed clip** — freezing a moving montage turns it into a slideshow — while genuinely static beats (a logo card, a pinned photo, a product still) keep the cheap hold. Talking/ambient beats always keep a real clip (they need motion + native audio). The deconstruct also stamps each dialogue line's **`on_camera`** flag — a voice playing over b-roll, a graphic, or a mere *photo* of the speaker stays voiceover, so the scaffold never lip-syncs a scene with no speaking face (the polaroid close-up failure).
 
 **The phrase model (voice cut at pauses, not at visual cuts).** The voice is grouped into **phrases** — runs of continuous speech with no real pause, which may span several visual scenes. A phrase is voiced ONCE (so a sentence the deconstruct split at a visual cut never breaks mid-word): if the speaker is **shown** anywhere in the phrase it's a single Seedance clip (`s<anchor>_clip`, native lip-sync + audio) re-voiced to the brand voice; if the speaker is **never shown** it's one ElevenLabs `tts` read. The picture is then assembled **scene by scene**: a scene that shows the speaker **slices its window** out of the phrase clip (`s<i>_seg`, an ffmpeg `-ss`/`-t` cut — video and audio come from the *same* clip, so lip-sync holds), and a **b-roll cutaway** gets its own silent clip while the phrase's voice plays underneath. "Shown" is decided by the **presenter element's per-scene presence**, not just who's speaking — a scene where a cast member narrates over b-roll (their element absent) is treated as a cutaway, so the talking head never appears where the original cut away. A presenter run longer than the **gateway-safe ~10s clip ceiling splits at a scene boundary** into contiguous takes (each its own clip + convert), so a sliced window never reads past its clip. (Seedance's *API* max is 15s, but the generation gateway frequently times out — **HTTP 524** — before it can deliver a clip longer than ~10s, so the scaffold never asks for one that long; 10s is a Seedance-allowed duration, so the split clip still snaps cleanly.) A b-roll cutaway *inside* a phrase lands at an **approximate** time (Seedance exposes no word timing) — nudge the scene boundary if it's off its beat.
 

package/dist/{chunk-ZWMMCLJ4.js → chunk-MF34WJ7M.js}

@@ -1644,7 +1644,11 @@ var VideoMeta = z.object({
         // Advisory: the scene's visual length vs the estimated spoken length, so
         // a reviewer can see a native line that may run past its cut. Not gated.
         scene_s: z.number().optional(),
-        est_speech_s: z.number().optional()
+        est_speech_s: z.number().optional(),
+        // Word count of the line est_speech_s was measured for. Together they carry
+        // the speaker's OBSERVED pace (from the deconstruct's word timings), so the
+        // overrun check budgets re-authored lines at the real rate, not a wps guess.
+        speech_words: z.number().optional()
       }),
       z.object({ scene: z.number(), lipsync_node: z.string() })
     ])
@@ -2174,7 +2178,9 @@ var STAGE_CODES = {
   SPEECH_OVERRUN: "VIDEO_SPEECH_OVERRUN",
   ASPECT_MISMATCH: "VIDEO_ASPECT_MISMATCH",
   REFERENCE_MISSING: "VIDEO_REFERENCE_MISSING",
-  SPAN_EXCEEDS_MODEL: "VIDEO_SPAN_EXCEEDS_MODEL"
+  SPAN_EXCEEDS_MODEL: "VIDEO_SPAN_EXCEEDS_MODEL",
+  UI_IN_PROMPT: "VIDEO_UI_IN_PROMPT",
+  BRANDMARK_IN_PROMPT: "VIDEO_BRANDMARK_IN_PROMPT"
 };
 var SPAN_MODEL_SLACK_S = 0.25;
 var VIDEO_TIME_SLACK_S = 0.75;
@@ -2553,6 +2559,8 @@ function checkVideoInvariants(ctx) {
   }
   checkSpeechOverrun(ctx, meta.talking_scenes);
   checkAspectConsistency(ctx);
+  checkUiInPrompt(ctx);
+  checkBrandmarkInPrompt(ctx);
   checkReferenceCompleteness(ctx, meta);
   checkClipSpanFitsModel(ctx, meta);
 }
@@ -2582,18 +2590,30 @@ function keywordTokens(text) {
   if (!text) return [];
   return text.toLowerCase().split(/[^a-z0-9]+/).filter((t) => t.length >= 3 && !KEYWORD_STOPWORDS.has(t));
 }
-function keywordsForElement(el) {
-  const type = el.type.toLowerCase();
-  const typeWords = ELEMENT_TYPE_KEYWORDS[type] ?? [];
+function elementMentionKeywords(el) {
+  const typeWords = ELEMENT_TYPE_KEYWORDS[el.type.toLowerCase()] ?? [];
   return [.../* @__PURE__ */ new Set([...typeWords, ...keywordTokens(el.label), ...keywordTokens(el.description)])];
 }
+function keywordsForElement(el) {
+  return elementMentionKeywords(el);
+}
 function containsWord(text, word) {
   const esc = word.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
   return new RegExp(`\\b${esc}\\b`, "i").test(text);
 }
+var FRAME_DESCRIPTION_START = "FRAME DESCRIPTION (this frame's editable prompt):";
+var FRAME_DESCRIPTION_END = "Render exactly what the FRAME DESCRIPTION";
+function frameDescriptionOf(prompt) {
+  const i = prompt.indexOf(FRAME_DESCRIPTION_START);
+  if (i < 0) return prompt;
+  const rest = prompt.slice(i + FRAME_DESCRIPTION_START.length);
+  const j = rest.indexOf(FRAME_DESCRIPTION_END);
+  return j < 0 ? rest : rest.slice(0, j);
+}
 function checkFrameReferences(ctx, node, index, keyworded) {
-  const prompt = node.params?.prompt;
-  if (typeof prompt !== "string" || prompt.length === 0) return;
+  const rawPrompt = node.params?.prompt;
+  if (typeof rawPrompt !== "string" || rawPrompt.length === 0) return;
+  const prompt = frameDescriptionOf(rawPrompt);
   const inputsBlob = JSON.stringify(node.inputs ?? {});
   for (const { el, keywords } of keyworded) {
     if (inputsBlob.includes(el.ref)) continue;
@@ -2643,13 +2663,24 @@ function checkClipSpanFitsModel(ctx, meta) {
     });
   }
 }
-function speechOverrunOf(node) {
+var OBSERVED_WPS_MIN = 1;
+var OBSERVED_WPS_MAX = 6;
+function secondsPerWord(stamped) {
+  const est = stamped?.est_speech_s;
+  const words = stamped?.speech_words;
+  if (est && words && est > 0 && words > 0) {
+    const wps = words / est;
+    if (wps >= OBSERVED_WPS_MIN && wps <= OBSERVED_WPS_MAX) return est / words;
+  }
+  return 1 / SPEECH_WORDS_PER_SECOND;
+}
+function speechOverrunOf(node, secPerWord) {
   const params = node.params;
   if (params?.generate_audio !== true) return null;
   const line = nativeDialogueOf(params.prompt);
   const duration = typeof params.duration === "number" ? params.duration : void 0;
   if (!line || !duration) return null;
-  const estSpeechS = line.split(/\s+/).filter(Boolean).length / SPEECH_WORDS_PER_SECOND;
+  const estSpeechS = line.split(/\s+/).filter(Boolean).length * secPerWord;
   return estSpeechS > duration * SPEECH_OVERRUN_RATIO ? { estSpeechS, duration } : null;
 }
 function checkSpeechOverrun(ctx, talkingScenes) {
@@ -2658,7 +2689,7 @@ function checkSpeechOverrun(ctx, talkingScenes) {
     const nativeClipRe = new RegExp(`^s${entry.scene}(_r\\d+)?_clip$`);
     for (const n of ctx.canvas.nodes) {
       if (!nativeClipRe.test(n.id) || n.type !== "video_generate") continue;
-      const overrun = speechOverrunOf(n);
+      const overrun = speechOverrunOf(n, secondsPerWord(entry));
       if (!overrun) continue;
       ctx.issues.push({
         path: `nodes[${ctx.idToIndex.get(n.id) ?? -1}].params.prompt`,
@@ -2668,6 +2699,38 @@ function checkSpeechOverrun(ctx, talkingScenes) {
     }
   }
 }
+var UI_IN_PROMPT_RE = /\bscreen[- ]?(?:recording|capture|grab|share)\b|\bapp (?:interface|screen)\b|\bphone screen overlay\b/i;
+function checkUiInPrompt(ctx) {
+  for (const n of ctx.canvas.nodes) {
+    if (n.type !== "video_generate") continue;
+    const prompt = n.params?.prompt;
+    if (typeof prompt !== "string" || !UI_IN_PROMPT_RE.test(prompt)) continue;
+    ctx.issues.push({
+      path: `nodes[${ctx.idToIndex.get(n.id) ?? -1}].params.prompt`,
+      code: STAGE_CODES.UI_IN_PROMPT,
+      severity: "warning",
+      message: `"${n.id}" asks the video model to render a screen/UI surface \u2014 generative video garbles UI text and chrome. Composite the real screen on the overlay layer (screenshot / brand HTML) and keep this prompt to the background plate`,
+      node_id: n.id,
+      node_type: "video_generate"
+    });
+  }
+}
+var BRANDMARK_IN_PROMPT_RE = /\b(?:logo|wordmark) (?:overlay|animation|sting|card|reveal)\b|\b(?:google|facebook|instagram|tiktok|youtube|amazon|apple|microsoft|whatsapp|netflix|spotify|excel|trustpilot) (?:logo|wordmark|branding)\b/i;
+function checkBrandmarkInPrompt(ctx) {
+  for (const n of ctx.canvas.nodes) {
+    if (n.type !== "video_generate" && n.type !== "image_generate") continue;
+    const prompt = n.params?.prompt;
+    if (typeof prompt !== "string" || !BRANDMARK_IN_PROMPT_RE.test(prompt)) continue;
+    ctx.issues.push({
+      path: `nodes[${ctx.idToIndex.get(n.id) ?? -1}].params.prompt`,
+      code: STAGE_CODES.BRANDMARK_IN_PROMPT,
+      severity: "warning",
+      message: `"${n.id}" asks the model to render a brand logo/wordmark \u2014 generation garbles marks and third-party logos carry IP exposure. Source the real mark (baker images logo <domain>) and composite it on the overlay layer`,
+      node_id: n.id,
+      node_type: n.type
+    });
+  }
+}
 function checkAspectConsistency(ctx) {
   const clips = ctx.canvas.nodes.filter((n) => n.type === "video_generate");
   if (clips.length < 2) return;
@@ -6489,6 +6552,7 @@ export {
   IMAGE_GENERATE_MODELS,
   MODEL_REGISTRY,
   resolveConcurrency,
+  elementMentionKeywords,
   BackendClient2 as BackendClient,
   Engine2 as Engine,
   LocalAssetStore2 as LocalAssetStore,
@@ -6499,4 +6563,4 @@ export {
   defaultRegistry,
   createEngineFromEnv
 };
-//# sourceMappingURL=chunk-ZWMMCLJ4.js.map
+//# sourceMappingURL=chunk-MF34WJ7M.js.map