@vanillaskyai/video 0.10.2 → 0.10.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/CHANGELOG.md +8 -0
  2. package/README.md +12 -7
  3. package/dist/check-runtime.js +2 -2
  4. package/dist/{chunk-6KZGF63O.js → chunk-3U2F7C7C.js} +53 -34
  5. package/dist/{chunk-5C6HZNHY.js → chunk-LJR5VLEI.js} +3 -2
  6. package/dist/{chunk-77VY4O7A.js → chunk-SI6CFFAA.js} +57 -10
  7. package/dist/{chunk-KHLO5OHT.js → chunk-UCXQORDZ.js} +1 -1
  8. package/dist/{chunk-VQH3JTQC.js → chunk-VVNZCY2U.js} +1 -1
  9. package/dist/{chunk-TVIU23OM.js → chunk-YZZSWGC3.js} +8 -3
  10. package/dist/{cinema-media-APDJS7SC.js → cinema-media-4JUYKPQI.js} +4 -4
  11. package/dist/{comparison-ZVE2DE7U.js → comparison-3QLMUC5V.js} +4 -4
  12. package/dist/{editorial-timeline-YIXXVJUV.js → editorial-timeline-IVE2P25N.js} +4 -4
  13. package/dist/{key-figure-I2C37FWV.js → key-figure-APD56LCO.js} +4 -4
  14. package/dist/{mobile-message-JSGBU6EX.js → mobile-message-WLK3WY5B.js} +4 -4
  15. package/dist/{quote-XH54G3FS.js → quote-Z6YET5BO.js} +4 -4
  16. package/dist/react.js +25 -13
  17. package/dist/{scene-video-backdrop-PCTARAAJ.js → scene-video-backdrop-OWUXPXHZ.js} +2 -2
  18. package/dist/server.js +302 -257
  19. package/docs/agent-integration.md +5 -4
  20. package/docs/getting-started.md +7 -5
  21. package/docs/media-and-audio.md +23 -17
  22. package/docs/production.md +7 -4
  23. package/docs/prompt-and-input.md +24 -15
  24. package/docs/provider-integration.md +2 -2
  25. package/docs/reference/protocol.md +7 -1
  26. package/docs/reference/provider-adapters.md +5 -3
  27. package/docs/testing.md +7 -2
  28. package/package.json +1 -1
  29. package/registry/items/backgrounds.json +2 -2
  30. package/registry/items/cinemaMedia.json +1 -1
  31. package/registry/items/comparison.json +1 -1
  32. package/registry/items/editorialTimeline.json +1 -1
  33. package/registry/items/keyFigure.json +1 -1
  34. package/registry/items/mobileMessage.json +1 -1
  35. package/registry/items/quote.json +1 -1
  36. package/starters/video-chat/README.md +18 -17
  37. package/starters/video-chat/package.json +1 -1
  38. package/starters/video-chat/providers/video.ts +5 -4
@@ -22,8 +22,9 @@ npm run dev
22
22
  ```
23
23
 
24
24
  Init generates the thin application-owned shell around the SDK's complete
25
- `VideoChat` experience and runs doctor automatically. The default uses packaged
26
- templates + browser voice and installs no optional provider packages.
25
+ `VideoChat` experience and runs doctor automatically. The baseline uses an immediate introduction and browser voice, and installs
26
+ no optional provider packages. Add the video adapter for moving video answers;
27
+ without a media provider, narration and subtitles remain available.
27
28
  `ANTHROPIC_API_KEY` is the only required key. Rerun init after an interrupted
28
29
  installation.
29
30
 
@@ -48,8 +49,8 @@ URL in a real browser. Browser automation should use normal motion
48
49
  its final frame, and returns to a usable composer.
49
50
  3. Ask one unrelated creative question in the same conversation and confirm it
50
51
  produces a distinct response.
51
- 4. If doctor reports generated video ready, choose that mode and verify one
52
- suitable prompt without editing client code.
52
+ 4. If doctor reports generated video ready, verify moving footage throughout
53
+ one complete answer, including its ending, without editing client code.
53
54
  5. Check failed network responses before reporting success.
54
55
 
55
56
  The handoff should name the localhost URL and ready capabilities. It should
@@ -3,9 +3,10 @@
3
3
  # Getting started
4
4
 
5
5
  The fastest VanillaSky integration is the complete, general-purpose video chat.
6
- It starts with packaged templates and browser voice, then turns on optional
7
- speech and generated video when you install their adapters and add server keys.
8
- Stock media needs only its server key.
6
+ It starts with a template introduction and browser voice. Add the video adapter
7
+ and its server key for generated footage, plus reviewed stock for fallback.
8
+ Without a media provider, the answer retains narration and subtitles with an
9
+ unavailable-visual state. Generated speech is optional.
9
10
 
10
11
  ## Create the app
11
12
 
@@ -96,8 +97,9 @@ spoken hook from the response stream before the full plan is complete, holds its
96
97
  scene is ready, reaches its final frame, and leaves the composer ready for
97
98
  another turn. With stock media enabled, click a welcome or follow-up card and
98
99
  confirm its footage carries directly into that opening. With generated video
99
- enabled, confirm that relevant generated shots and editorial templates form one
100
- cinematic response, and the first shot continues the hook without repeating it.
100
+ enabled, confirm that moving shots develop a complete answer with narration
101
+ and subtitles, and that the first shot continues the hook without repeating it.
102
+ Check that no finished clip freezes while narration continues.
101
103
 
102
104
  The generated local authorization accepts localhost only. Replace it with your
103
105
  real session check, rate limits, and usage policy before deploying.
@@ -6,22 +6,23 @@ VanillaSky keeps provider choice in the application. The SDK defines small
6
6
  server callbacks, advertises only the capabilities you configure, and keeps
7
7
  all credentials out of React and the browser bundle.
8
8
 
9
- ## One cinematic mode
9
+ ## AI-first video answers
10
10
 
11
- The director combines relevant footage and seven editorial templates. Graphics
12
- use black and white; footage carries atmosphere and concrete action. Each media
13
- scene declares `mediaSource: "stock"` or `"generate"`. Provider availability and
14
- host budgets constrain those choices. There is no all-stock or all-AI switch.
11
+ The default chat shows an immediate template introduction while footage prepares,
12
+ then moving video with narration and subtitles. The planner describes visible
13
+ subjects and actions, adapting its direction to the request's intent. The runtime
14
+ attempts generated footage first and uses relevant stock when generation is
15
+ unavailable, denied by the host budget, or fails. It does not select body cards.
15
16
 
16
- The canonical templates are `cinemaMedia`, `chapterTitle`,
17
- `editorialTimeline`, `mobileMessage`, `comparison`, `quote`, and `keyFigure`.
18
- All except `chapterTitle` accept optional media backgrounds. Editorial overlays use soft text shadows and a dark contrast scrim over relevant media, and fall back to black. Use at most one explanatory overlay per typical thirty-second answer. Full-bleed
19
- footage has no headline; narration and subtitles carry the explanation.
17
+ The seven packaged templates remain available to custom compositions. Default
18
+ chat body shots use `cinemaMedia` as the footage renderer, without extra headlines.
19
+ If both media providers miss, the answer retains its narration and subtitles
20
+ with an explicit unavailable-visual state rather than a text-slide replacement.
20
21
 
21
22
  ## Reviewed stock
22
23
 
23
- Add `searchMedia` when template answers, the welcome screen, and follow-up
24
- cards should use approved photography or footage:
24
+ Add `searchMedia` for relevant fallback footage, welcome covers, and follow-up
25
+ cards:
25
26
 
26
27
  ```ts
27
28
  createVideoChatHandler({
@@ -45,14 +46,14 @@ createVideoChatHandler({
45
46
  The planner emits a short semantic keyword, not a URL. The callback returns an
46
47
  application-approved image or video URL, and the SDK validates it before it
47
48
  reaches a scene. Return `null` when no licensed, safe, relevant asset exists;
48
- `cinemaMedia` becomes a chapter using its grounded `fallbackText`; a message can
49
- keep its content on black. Missing fallback copy produces an explicit error.
49
+ default chat retains the spoken answer when footage is unavailable. An explicit custom `templates` registry retains its structured planner,
50
+ validation, and fallback contracts.
50
51
 
51
52
  For Pexels, keep `PEXELS_API_KEY` on the server, enforce a deadline, filter for
52
53
  orientation, and return only validated Pexels asset domains. Licensing,
53
54
  attribution, caching, MIME checks, and byte limits remain application-owned.
54
55
 
55
- ## Planned generated shots
56
+ ## Generated shots
56
57
 
57
58
  Add `generateVideo` to enable generated shots within cinematic responses. It receives the planned visual
58
59
  subject plus the generated look so every clip can follow the same direction:
@@ -75,6 +76,8 @@ createVideoChatHandler({
75
76
  orientation,
76
77
  requestId,
77
78
  sceneId: scene?.id,
79
+ shotDirection: typeof scene?.variables.shotDirection === "string"
80
+ ? scene.variables.shotDirection : undefined,
78
81
  signal,
79
82
  maxRetries: 0,
80
83
  });
@@ -83,8 +86,9 @@ createVideoChatHandler({
83
86
  });
84
87
  ```
85
88
 
86
- The first streamed object reserves the first generated shot while the same
87
- model call continues planning later scenes. Use `requestId` and `scene.id` as
89
+ One model stream supplies an answer brief and shot descriptions. The runtime
90
+ prepares ordered shots while the model continues planning; the first and later
91
+ body shots follow the same path. Use `requestId` and `scene.id` as
88
92
  an idempotency key, because generated clips are billable. Keep `maxRetries: 0`
89
93
  inside provider calls, honour `signal`, and make retries an explicit product
90
94
  decision with a known budget.
@@ -129,7 +133,9 @@ interaction before audible playback on many devices.
129
133
  - Keep every provider key in server-only environment variables.
130
134
  - Never let a planner return arbitrary final media URLs.
131
135
  - Bound query length, response size, duration, concurrency, and generated spend.
132
- - Preload the next asset and keep the current visual when media is late.
136
+ - Preload upcoming footage and pause narration honestly when usable media is late.
137
+ - Plan enough moving footage for the spoken beat; do not hold a finished frame
138
+ while narration continues.
133
139
  - Return a safe fallback instead of leaving the response waiting forever.
134
140
 
135
141
  [← Documentation home](../README.md) · [Previous: Customization](customization.md) · [Next: Custom templates →](custom-templates.md)
@@ -22,10 +22,13 @@ player. Read the [security guide](security.md) for the complete controls.
22
22
 
23
23
  ## Cinematic direction and providers
24
24
 
25
- One cinematic mode combines footage and editorial graphics according to the
26
- story. Keep templates available as grounded fallbacks. Add `searchMedia` for
27
- reviewed stock and `generateVideo` for selected illustrative shots. The planner
28
- sets `mediaSource` per media scene; the host enforces the generation budget.
25
+ Default chat uses a template introduction during preparation, then generated
26
+ footage with narration and subtitles. Add `generateVideo` for these shots and
27
+ `searchMedia` for relevant stock fallback. The runtime selects media within the
28
+ host generation budget; the model supplies narration and visible actions.
29
+ If both sources miss, valid narration remains available with an explicit visual
30
+ unavailability state. Explicit custom template registries keep their existing
31
+ composition and fallback contracts.
29
32
  A stock candidate must match the subject, action and permitted crop. Return
30
33
  `null` for uncertainty rather than broadening an essential detail.
31
34
 
@@ -55,7 +55,7 @@ viewer prompt separate from these trusted server-side instructions.
55
55
 
56
56
  ## What reaches the model
57
57
 
58
- `createVideoChatHandler` builds the trusted template catalog, video rules,
58
+ `createVideoChatHandler` builds the shot-planning instructions, video rules,
59
59
  conversation context, and application guidance. Your provider adapter receives
60
60
  two complete strings:
61
61
 
@@ -68,9 +68,9 @@ streamText: ({ systemPrompt, userPrompt, signal }) => streamText({
68
68
  });
69
69
  ```
70
70
 
71
- Pass both strings unchanged. The system prompt describes the installed
72
- templates, schema limits, pacing, narration, opening contract, and safe media
73
- fields. The user prompt contains the current request, bounded prior turns,
71
+ Pass both strings unchanged. The system prompt describes the internal answer
72
+ brief and shot format, pacing, narration, opening contract, and safe media
73
+ directions. The user prompt contains the current request, bounded prior turns,
74
74
  orientation, visual mode, and whether an opening was already spoken.
75
75
 
76
76
  Provider credentials and raw media URLs never belong in either prompt. Media
@@ -79,19 +79,27 @@ output has been parsed.
79
79
 
80
80
  ## One stream, one answer
81
81
 
82
- The planner first emits a 6–9 word spoken hook and a media keyword, then keeps
83
- streaming complete scenes. It is not a separate hook call followed by a second
84
- planning call. This keeps the opening consistent with the scenes that follow
85
- and lets the first playable scene arrive without waiting for the complete plan.
82
+ The planner streams an answer brief containing the opening, creative direction,
83
+ and ending, followed by narrated shot descriptions. One model request serves
84
+ both suggested and typed prompts; a suggestion may already supply its opening.
85
+ There is no separate classification or first-shot planning request.
86
+
87
+ The runtime assigns scene IDs, uses the footage renderer, resolves media, and
88
+ finalizes the response when planning ends. The model does not choose body
89
+ layouts, media providers, or lifecycle events. Narration and visible action are
90
+ planned together: explanations show mechanisms, stories develop consequences,
91
+ comedy times its reveal, imaginative requests depict their invented world, and
92
+ practical answers demonstrate usable steps. These are directions, not fixed
93
+ scene counts or one universal story structure.
86
94
 
87
95
  Every `scene.add` is validated before the browser receives it. The model never
88
- returns React, HTML, CSS, or executable JavaScript. Accepted scenes are
89
- immutable; invalid scenes are reported through safe warnings and omitted.
96
+ returns React, HTML, CSS, or executable JavaScript. Invalid planning content
97
+ produces safe diagnostics. A media failure does not delete valid narration.
90
98
 
91
- In template mode, the planner can choose any trusted template and request stock
92
- media through `searchMedia`. In full mode, it plans a complete generated-video
93
- answer and `generateVideo` resolves every visual beat. There is no mixed
94
- "generate a few clips" mode.
99
+ Generated footage is preferred within the host's allowance. Relevant stock is
100
+ fallback. If neither is available, retain narration and subtitles and report
101
+ unavailable visuals. Handlers configured with an explicit custom `templates` registry continue to
102
+ support the trusted catalog and its existing structured planner contract.
95
103
 
96
104
  ## Grounding
97
105
 
@@ -110,7 +118,8 @@ Check these boundaries in order:
110
118
  3. Does the provider pass `systemPrompt` and `userPrompt` unchanged?
111
119
  4. Is extended reasoning delaying the first streamed object?
112
120
  5. Do `onWarning` and `onComplete` show rejected scenes or a length limit?
113
- 6. Does the trusted registry contain a suitable template for the requested answer?
121
+ 6. Do the planned actions develop the answer, and does its ending resolve the request?
122
+ 7. Does resolved footage actually play for the spoken duration?
114
123
 
115
124
  Log request IDs, safe warning codes, provider finish reasons, model IDs, and
116
125
  token usage. Never log credentials or expose raw provider errors in the video.
@@ -12,7 +12,7 @@ npm run dev
12
12
  ```
13
13
 
14
14
  Init runs doctor automatically. The generated `server.ts` starts with one
15
- `ANTHROPIC_API_KEY`, packaged templates, and browser voice; it installs no
15
+ `ANTHROPIC_API_KEY`, a template introduction, and browser voice; it installs no
16
16
  optional speech or video packages.
17
17
 
18
18
  Use `npx vanillasky providers add speech` to install xAI speech, or
@@ -180,7 +180,7 @@ is intentionally substantial. It is stable for the same SDK version, template
180
180
  kit, media policy, and base prompt. Record input-token usage, keep the selected
181
181
  kit no broader than the product needs, and enable provider-side prompt caching
182
182
  where the chosen provider/model supports it. VanillaSky does not assume one
183
- provider's cache controls in its provider-neutral adapter. The cinematic catalog contains seven templates; use provider-reported token
183
+ provider's cache controls in its provider-neutral adapter. Default chat streams an answer brief and shot directions rather than the template catalog; use provider-reported token
184
184
  usage as the authoritative measurement rather than a character estimate.
185
185
 
186
186
  Provider finish reasons `error` and `tool-calls` are terminal failures.
@@ -70,7 +70,13 @@ change core video state.
70
70
 
71
71
  ## Planning boundary
72
72
 
73
- LLMs do not emit public protocol envelopes. A server-only planner emits
73
+ LLMs do not emit public protocol envelopes. Default `createVideoChatHandler`
74
+ planning uses an internal answer brief and shot descriptions. The runtime
75
+ translates them into footage scenes and completes the answer at stream end,
76
+ without asking the model for lifecycle commands.
77
+
78
+ Handlers with an explicit custom `templates` registry keep the existing
79
+ composition planner contract. A server-only planner emits
74
80
  validated `scene.add` or `plan.complete` parts. The runtime assigns sequences,
75
81
  IDs, terminal snapshots, and checksums. Generated HTML, React, JavaScript, CSS, component
76
82
  source, audio events, protocol envelopes, and unknown part types are rejected.
@@ -69,7 +69,9 @@ VanillaSky or its protocol. The same boundary also accepts a self-hosted model
69
69
  or a provider-native async text stream when it is not represented in the AI
70
70
  SDK. VanillaSky has no model allowlist.
71
71
 
72
- The provider must emit NDJSON text: one complete VanillaSky plan part per line.
72
+ The provider must emit NDJSON text matching the supplied prompt: one complete
73
+ object per line. Default chat planning uses an answer brief and shot descriptions;
74
+ the SDK translates them into validated scene events.
73
75
  The SDK buffers arbitrary text chunks until a newline, parses the completed
74
76
  object, validates it, and only then forwards it to the motion runtime. Do not
75
77
  replace that per-line validator with whole-response structured output: motion
@@ -93,8 +95,8 @@ stream-reconnect contract.
93
95
  ## Generated-video budget
94
96
 
95
97
  Set `maxGeneratedVideos` on `createVideoChatHandler` to a nonnegative safe integer
96
- (default `5`). This is a per-response attempt limit for full mode, including the
97
- reserved first shot and failures. Use `0` to keep the configured full mode on
98
+ (default `5`). This is a per-response generation attempt limit, including
99
+ failures. Default chat attempts generated footage before relevant stock. Use `0` to keep the configured full mode on
98
100
  stock footage. Stock lookup is independent of this limit. Retries inside your
99
101
  provider callback can incur additional charges; bound those separately. Never
100
102
  copy an untrusted request value into this application-owned option.
package/docs/testing.md CHANGED
@@ -7,12 +7,15 @@ no network request, and needs no model key.
7
7
  ## Test the chat route with Vitest
8
8
 
9
9
  Pass `createMockVideoPlanner()` to the same `createVideoChatHandler` used by the
10
- application. A standard `Request` exercises parsing, validation, pacing, the
10
+ application, using an explicit template registry for the structured composition
11
+ fixtures. Default AI-first chat tests should instead return an authored answer
12
+ brief and shot descriptions matching the supplied planning prompt. A standard
13
+ `Request` exercises parsing, validation, pacing, the
11
14
  opening extension, and SSE without starting an HTTP server.
12
15
 
13
16
  ```ts
14
17
  import { describe, expect, it } from "vitest";
15
- import { createVideoChatHandler } from "@vanillaskyai/video/server";
18
+ import { createVideoChatHandler, createServerTemplateRegistry } from "@vanillaskyai/video/server";
16
19
  import { createMockVideoPlanner } from "@vanillaskyai/video/test";
17
20
 
18
21
  describe("POST /api/video-chat", () => {
@@ -20,6 +23,7 @@ describe("POST /api/video-chat", () => {
20
23
  const handle = createVideoChatHandler({
21
24
  authorize: "none", // Only acceptable because this handler stays in process.
22
25
  heartbeatMs: false,
26
+ templates: createServerTemplateRegistry({ templates: [] }),
23
27
  streamText: createMockVideoPlanner(),
24
28
  generateText: async ({ task }) => task === "suggestions"
25
29
  ? JSON.stringify({ suggestions: [] })
@@ -43,6 +47,7 @@ describe("POST /api/video-chat", () => {
43
47
 
44
48
  expect(response.status).toBe(200);
45
49
  expect(body).toContain('"type":"data.video-chat-opening"');
50
+ expect(body).toContain('"type":"scene.add"');
46
51
  expect(body).toContain('"type":"response.complete"');
47
52
  });
48
53
  });
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@vanillaskyai/video",
3
- "version": "0.10.2",
3
+ "version": "0.10.3",
4
4
  "description": "Open-source voice-and-video chat SDK for AI applications.",
5
5
  "keywords": [
6
6
  "video-chat",
@@ -28,7 +28,7 @@
28
28
  "path": "src/visual-system/scene-templates/external-video-backdrop.tsx",
29
29
  "type": "registry:lib",
30
30
  "target": "vanillasky/scene-templates/external-video-backdrop.tsx",
31
- "content": "import React from \"react\";\n\n// Source-owned templates may live in a consumer's tree while VideoFrame comes\n// from the package. Both copies must observe the same internal context or the\n// consumer template would mount a second video over the player-owned plane,\n// and would never inherit the player's native media audio state.\nexport type ExternalVideoBackdropMode = false | \"pending\" | \"ready\" | \"fallback\";\n\ninterface BackdropContextValue {\n mode: ExternalVideoBackdropMode;\n audioMuted: boolean;\n audioVolume: number;\n}\n\nconst DEFAULT: BackdropContextValue = { mode: false, audioMuted: true, audioVolume: 1 };\n\nconst sharedContext = globalThis as typeof globalThis & {\n __vanillaskyVideoBackdropContext?: React.Context<BackdropContextValue>;\n};\nconst BackdropContext = sharedContext.__vanillaskyVideoBackdropContext\n ??= React.createContext<BackdropContextValue>(DEFAULT);\n\nexport function ExternalVideoBackdropProvider({\n mode,\n audioMuted = true,\n audioVolume = 1,\n children,\n}: {\n mode: ExternalVideoBackdropMode;\n audioMuted?: boolean;\n audioVolume?: number;\n children: React.ReactNode;\n}) {\n const value = React.useMemo(\n () => ({ mode, audioMuted, audioVolume }),\n [mode, audioMuted, audioVolume],\n );\n return (\n <BackdropContext.Provider value={value}>\n {children}\n </BackdropContext.Provider>\n );\n}\n\nexport function useExternalVideoBackdrop(): ExternalVideoBackdropMode {\n return React.useContext(BackdropContext).mode;\n}\n\nexport function useMediaAudio(): { muted: boolean; volume: number } {\n const { audioMuted, audioVolume } = React.useContext(BackdropContext);\n return { muted: audioMuted, volume: audioVolume };\n}\n"
31
+ "content": "import React from \"react\";\n\n// Source-owned templates may live in a consumer's tree while VideoFrame comes\n// from the package. Both copies must observe the same internal context or the\n// consumer template would mount a second video over the player-owned plane,\n// and would never inherit the player's native media audio state.\nexport type ExternalVideoBackdropMode = false | \"pending\" | \"ready\" | \"fallback\";\n\ninterface BackdropContextValue {\n mode: ExternalVideoBackdropMode;\n audioMuted: boolean;\n audioVolume: number;\n preparingNarration?: boolean;\n}\n\nconst DEFAULT: BackdropContextValue = { mode: false, audioMuted: true, audioVolume: 1 };\n\nconst sharedContext = globalThis as typeof globalThis & {\n __vanillaskyVideoBackdropContext?: React.Context<BackdropContextValue>;\n};\nconst BackdropContext = sharedContext.__vanillaskyVideoBackdropContext\n ??= React.createContext<BackdropContextValue>(DEFAULT);\n\nexport function ExternalVideoBackdropProvider({\n mode,\n audioMuted = true,\n audioVolume = 1,\n preparingNarration = false,\n children,\n}: {\n mode: ExternalVideoBackdropMode;\n audioMuted?: boolean;\n audioVolume?: number;\n preparingNarration?: boolean;\n children: React.ReactNode;\n}) {\n const value = React.useMemo(\n () => ({ mode, audioMuted, audioVolume, preparingNarration }),\n [mode, audioMuted, audioVolume, preparingNarration],\n );\n return (\n <BackdropContext.Provider value={value}>\n {children}\n </BackdropContext.Provider>\n );\n}\n\nexport function useExternalVideoBackdrop(): ExternalVideoBackdropMode {\n return React.useContext(BackdropContext).mode;\n}\n\nexport function useMediaAudio(): { muted: boolean; volume: number } {\n const { audioMuted, audioVolume } = React.useContext(BackdropContext);\n return { muted: audioMuted, volume: audioVolume };\n}\n\n/** Internal first-frame priming state; an explicit viewer pause never sets it. */\nexport function useNarrationPreroll(): boolean {\n return React.useContext(BackdropContext).preparingNarration === true;\n}\n"
32
32
  },
33
33
  {
34
34
  "path": "src/visual-system/scene-templates/media-position.ts",
@@ -40,7 +40,7 @@
40
40
  "path": "src/visual-system/scene-templates/scene-video-backdrop.tsx",
41
41
  "type": "registry:lib",
42
42
  "target": "vanillasky/scene-templates/scene-video-backdrop.tsx",
43
- "content": "import React, { useEffect, useRef, useState } from \"react\";\nimport { getBackgroundTransform } from \"../backgrounds\";\nimport { useMediaAudio } from \"./external-video-backdrop\";\nimport { resolveMediaPosition } from \"./media-position\";\n\nexport interface SceneVideoBackdropProps {\n mediaUrl: string;\n mediaPoster?: string;\n mediaPosition?: string;\n backgroundEffect?: string;\n progress: number;\n beatIntensity?: number;\n isPlaying: boolean;\n muted?: boolean;\n volume?: number;\n playbackId?: string;\n retainPoster?: boolean;\n persistent?: boolean;\n preparedPoster?: {\n presentationKey: string;\n mediaPoster: string;\n mediaPosition: string;\n backgroundEffect?: string;\n /** Existing global transition progress. On decoder-constrained Safari,\n * this fades the decoded incoming still above the outgoing video before\n * the single video element changes source. */\n opacity?: number;\n };\n onReady?: () => void;\n onError?: () => void;\n}\n\nexport const SceneVideoBackdrop: React.FC<SceneVideoBackdropProps> = ({\n mediaUrl,\n mediaPoster,\n mediaPosition = \"center\",\n backgroundEffect,\n progress,\n beatIntensity = 0,\n isPlaying,\n muted,\n volume,\n playbackId = mediaUrl,\n retainPoster = false,\n persistent = false,\n preparedPoster,\n onReady,\n onError,\n}) => {\n const inheritedAudio = useMediaAudio();\n const resolvedMuted = muted ?? inheritedAudio.muted;\n const resolvedVolume = volume ?? inheritedAudio.volume;\n const resolvedPosition = resolveMediaPosition(mediaPosition);\n const bgTransform = getBackgroundTransform(backgroundEffect, progress, beatIntensity);\n const [decodedVideoUrl, setDecodedVideoUrl] = useState<string>();\n const videoRef = useRef<HTMLVideoElement>(null);\n const startedVideoUrl = useRef<string | undefined>(undefined);\n const startedPlaybackId = useRef<string | undefined>(undefined);\n const videoPresentationKey = `${playbackId}\\0${mediaUrl}`;\n\n useEffect(() => {\n const video = videoRef.current;\n if (!video) return;\n // React Strict Mode rehearses setup → cleanup → setup in development.\n // The cleanup deliberately releases the decoder, so the repeated setup\n // must restore the declarative source before the playback effect runs.\n if (video.getAttribute(\"src\") !== mediaUrl) {\n video.setAttribute(\"src\", mediaUrl);\n video.load();\n }\n return () => {\n video.pause();\n video.removeAttribute(\"src\");\n video.load();\n startedVideoUrl.current = undefined;\n startedPlaybackId.current = undefined;\n };\n }, [mediaUrl]);\n\n useEffect(() => {\n const video = videoRef.current;\n if (video) video.volume = resolvedVolume;\n }, [resolvedVolume]);\n\n useEffect(() => {\n const video = videoRef.current;\n if (!video) return;\n if (!isPlaying) {\n video.pause();\n return;\n }\n if (startedPlaybackId.current === playbackId) {\n if (!video.ended) void video.play().catch(() => {});\n return;\n }\n const changingSource = startedVideoUrl.current !== undefined && startedVideoUrl.current !== mediaUrl;\n video.playbackRate = 1;\n if (!changingSource && video.currentTime > 0) video.currentTime = 0;\n video.play().catch(() => {});\n startedVideoUrl.current = mediaUrl;\n startedPlaybackId.current = playbackId;\n }, [isPlaying, mediaUrl, playbackId]);\n\n const mediaStyle: React.CSSProperties = {\n position: \"absolute\",\n inset: 0,\n width: \"100%\",\n height: \"100%\",\n objectFit: \"cover\",\n objectPosition: resolvedPosition,\n transform: bgTransform.transform,\n transformOrigin: bgTransform.transformOrigin,\n zIndex: persistent ? 1 : undefined,\n };\n const preparedPosition = preparedPoster\n ? resolveMediaPosition(preparedPoster.mediaPosition)\n : resolvedPosition;\n const preparedTransform = getBackgroundTransform(preparedPoster?.backgroundEffect, 0, 0);\n const posterPlanes = [\n ...(persistent && mediaPoster ? [{\n presentationKey: videoPresentationKey,\n mediaPoster,\n mediaPosition: resolvedPosition,\n transform: bgTransform.transform,\n transformOrigin: bgTransform.transformOrigin,\n opacity: 1,\n zIndex: 0,\n role: \"current\",\n }] : []),\n ...(preparedPoster && preparedPoster.presentationKey !== videoPresentationKey ? [{\n presentationKey: preparedPoster.presentationKey,\n mediaPoster: preparedPoster.mediaPoster,\n mediaPosition: preparedPosition,\n transform: preparedTransform.transform,\n transformOrigin: preparedTransform.transformOrigin,\n opacity: preparedPoster.opacity ?? 0,\n zIndex: 2,\n role: \"prepared\",\n }] : []),\n ];\n\n return (\n <>\n {posterPlanes.map((posterPlane) => (\n <img\n key={posterPlane.presentationKey}\n src={posterPlane.mediaPoster}\n alt=\"\"\n aria-hidden=\"true\"\n draggable={false}\n data-video-poster-plane={posterPlane.role}\n data-video-poster-visible={posterPlane.opacity > 0 ? \"true\" : \"false\"}\n style={{\n position: \"absolute\",\n inset: 0,\n width: \"100%\",\n height: \"100%\",\n objectFit: \"cover\",\n objectPosition: posterPlane.mediaPosition,\n transform: posterPlane.transform,\n transformOrigin: posterPlane.transformOrigin,\n zIndex: posterPlane.zIndex,\n opacity: posterPlane.opacity,\n pointerEvents: \"none\",\n }}\n />\n ))}\n <video\n ref={videoRef}\n src={mediaUrl}\n poster={retainPoster || decodedVideoUrl !== mediaUrl ? mediaPoster || undefined : undefined}\n muted={resolvedMuted}\n loop={false}\n playsInline\n preload=\"auto\"\n onLoadedData={(event) => {\n const video = event.currentTarget;\n const markPresented = () => {\n if (!video.isConnected) return;\n onReady?.();\n if (!retainPoster) setDecodedVideoUrl(mediaUrl);\n };\n if (video.requestVideoFrameCallback) {\n video.requestVideoFrameCallback(markPresented);\n return;\n }\n markPresented();\n }}\n onError={onError}\n data-media-position={mediaPosition}\n data-video-backdrop={persistent ? \"persistent\" : \"scene\"}\n style={mediaStyle}\n />\n </>\n );\n};\n"
43
+ "content": "import React, { useCallback, useEffect, useRef, useState } from \"react\";\nimport { getBackgroundTransform } from \"../backgrounds\";\nimport { useMediaAudio, useNarrationPreroll } from \"./external-video-backdrop\";\nimport { resolveMediaPosition } from \"./media-position\";\n\nexport interface SceneVideoBackdropProps {\n mediaUrl: string;\n mediaPoster?: string;\n mediaPosition?: string;\n backgroundEffect?: string;\n progress: number;\n /** Narration-led visible duration; muted or pitch-preserving footage may be gently retimed. */\n sceneDuration?: number;\n /** Internal player-owned decoder priming, distinct from viewer pause. */\n preparingNarration?: boolean;\n beatIntensity?: number;\n isPlaying: boolean;\n muted?: boolean;\n volume?: number;\n playbackId?: string;\n retainPoster?: boolean;\n persistent?: boolean;\n preparedPoster?: {\n presentationKey: string;\n mediaPoster: string;\n mediaPosition: string;\n backgroundEffect?: string;\n /** Existing global transition progress. On decoder-constrained Safari,\n * this fades the decoded incoming still above the outgoing video before\n * the single video element changes source. */\n opacity?: number;\n };\n onReady?: () => void;\n onError?: () => void;\n}\n\nexport const SceneVideoBackdrop: React.FC<SceneVideoBackdropProps> = ({\n mediaUrl,\n mediaPoster,\n mediaPosition = \"center\",\n backgroundEffect,\n progress,\n sceneDuration,\n preparingNarration = false,\n beatIntensity = 0,\n isPlaying,\n muted,\n volume,\n playbackId = mediaUrl,\n retainPoster = false,\n persistent = false,\n preparedPoster,\n onReady,\n onError,\n}) => {\n const inheritedAudio = useMediaAudio();\n const inheritedPreroll = useNarrationPreroll();\n const rewindPreroll = preparingNarration || inheritedPreroll;\n const resolvedMuted = muted ?? inheritedAudio.muted;\n const resolvedVolume = volume ?? inheritedAudio.volume;\n const resolvedPosition = resolveMediaPosition(mediaPosition);\n const bgTransform = getBackgroundTransform(backgroundEffect, progress, beatIntensity);\n const [decodedVideoUrl, setDecodedVideoUrl] = useState<string>();\n const [waitingKey, setWaitingKey] = useState<string>();\n const [exhaustedKey, setExhaustedKey] = useState<string>();\n const continuityReplay = useRef<string | undefined>(undefined);\n const videoRef = useRef<HTMLVideoElement>(null);\n const startedVideoUrl = useRef<string | undefined>(undefined);\n const startedPlaybackId = useRef<string | undefined>(undefined);\n const videoPresentationKey = `${playbackId}\\0${mediaUrl}`;\n\n const presentationRef = useRef({ key: videoPresentationKey, playing: isPlaying });\n presentationRef.current = { key: videoPresentationKey, playing: isPlaying };\n const unavailable = () => {\n if (presentationRef.current.key === videoPresentationKey && presentationRef.current.playing) {\n setExhaustedKey(videoPresentationKey);\n onError?.();\n }\n };\n const fitDuration = useCallback((video: HTMLVideoElement) => {\n // Allow a small decode-to-speech onset margin without changing narration.\n video.playbackRate = (resolvedMuted || video.preservesPitch === true) && sceneDuration && Number.isFinite(video.duration) && video.duration > 0\n ? Math.max(.75, Math.min(1, video.duration / (sceneDuration + .2))) : 1;\n }, [resolvedMuted, sceneDuration]);\n useEffect(() => {\n if (videoRef.current) fitDuration(videoRef.current);\n }, [fitDuration]);\n const continueMotion = (video: HTMLVideoElement) => {\n if (!isPlaying) return;\n // The planner supplies short shots as the normal coverage. A single replay\n // bridges exceptional speech overrun/late delivery; never loop indefinitely.\n if (!resolvedMuted || continuityReplay.current === videoPresentationKey) {\n setExhaustedKey(videoPresentationKey);\n return;\n }\n continuityReplay.current = videoPresentationKey;\n video.currentTime = 0;\n void video.play().catch(unavailable);\n };\n\n useEffect(() => {\n const video = videoRef.current;\n if (!video) return;\n // React Strict Mode rehearses setup → cleanup → setup in development.\n // The cleanup deliberately releases the decoder, so the repeated setup\n // must restore the declarative source before the playback effect runs.\n if (video.getAttribute(\"src\") !== mediaUrl) {\n video.setAttribute(\"src\", mediaUrl);\n video.load();\n }\n return () => {\n video.pause();\n video.removeAttribute(\"src\");\n video.load();\n startedVideoUrl.current = undefined;\n startedPlaybackId.current = undefined;\n };\n }, [mediaUrl]);\n\n useEffect(() => {\n const video = videoRef.current;\n if (video) video.volume = resolvedVolume;\n }, [resolvedVolume]);\n\n useEffect(() => {\n const video = videoRef.current;\n if (!video) return;\n if (!isPlaying) {\n video.pause();\n if (rewindPreroll && video.currentTime > 0) video.currentTime = 0;\n return;\n }\n if (startedPlaybackId.current === playbackId) {\n if (video.ended) continueMotion(video);\n else void video.play().catch(unavailable);\n return;\n }\n const changingSource = startedVideoUrl.current !== undefined && startedVideoUrl.current !== mediaUrl;\n fitDuration(video);\n if (!changingSource && video.currentTime > 0) video.currentTime = 0;\n video.play().catch(unavailable);\n startedVideoUrl.current = mediaUrl;\n startedPlaybackId.current = playbackId;\n }, [isPlaying, mediaUrl, playbackId, rewindPreroll]);\n\n const mediaStyle: React.CSSProperties = {\n position: \"absolute\",\n inset: 0,\n width: \"100%\",\n height: \"100%\",\n objectFit: \"cover\",\n objectPosition: resolvedPosition,\n transform: bgTransform.transform,\n transformOrigin: bgTransform.transformOrigin,\n zIndex: persistent ? 1 : undefined,\n };\n const preparedPosition = preparedPoster\n ? resolveMediaPosition(preparedPoster.mediaPosition)\n : resolvedPosition;\n const preparedTransform = getBackgroundTransform(preparedPoster?.backgroundEffect, 0, 0);\n const posterPlanes = [\n ...(persistent && mediaPoster ? [{\n presentationKey: videoPresentationKey,\n mediaPoster,\n mediaPosition: resolvedPosition,\n transform: bgTransform.transform,\n transformOrigin: bgTransform.transformOrigin,\n opacity: 1,\n zIndex: 0,\n role: \"current\",\n }] : []),\n ...(preparedPoster && preparedPoster.presentationKey !== videoPresentationKey ? [{\n presentationKey: preparedPoster.presentationKey,\n mediaPoster: preparedPoster.mediaPoster,\n mediaPosition: preparedPosition,\n transform: preparedTransform.transform,\n transformOrigin: preparedTransform.transformOrigin,\n opacity: preparedPoster.opacity ?? 0,\n zIndex: 2,\n role: \"prepared\",\n }] : []),\n ];\n\n return (\n <>\n {posterPlanes.map((posterPlane) => (\n <img\n key={posterPlane.presentationKey}\n src={posterPlane.mediaPoster}\n alt=\"\"\n aria-hidden=\"true\"\n draggable={false}\n data-video-poster-plane={posterPlane.role}\n data-video-poster-visible={posterPlane.opacity > 0 ? \"true\" : \"false\"}\n style={{\n position: \"absolute\",\n inset: 0,\n width: \"100%\",\n height: \"100%\",\n objectFit: \"cover\",\n objectPosition: posterPlane.mediaPosition,\n transform: posterPlane.transform,\n transformOrigin: posterPlane.transformOrigin,\n zIndex: posterPlane.zIndex,\n opacity: posterPlane.opacity,\n pointerEvents: \"none\",\n }}\n />\n ))}\n {(exhaustedKey === videoPresentationKey || waitingKey === videoPresentationKey) && <div\n role=\"status\" data-media-continuity={exhaustedKey === videoPresentationKey ? \"exhausted\" : \"waiting\"}\n style={{ position: \"absolute\", inset: 0, zIndex: 3, background: \"#000\", color: \"#bbb\", display: \"grid\", placeContent: \"center\", font: \"14px system-ui\" }}\n >{exhaustedKey === videoPresentationKey ? \"Visual unavailable\" : \"Loading visual\"}</div>}\n <video\n ref={videoRef}\n src={mediaUrl}\n poster={retainPoster || decodedVideoUrl !== mediaUrl ? mediaPoster || undefined : undefined}\n muted={resolvedMuted}\n loop={false}\n playsInline\n preload=\"auto\"\n onLoadedMetadata={event => fitDuration(event.currentTarget)}\n onEnded={event => continueMotion(event.currentTarget)}\n onWaiting={() => { if (isPlaying) setWaitingKey(videoPresentationKey); }}\n onPlaying={() => setWaitingKey(undefined)}\n onLoadedData={(event) => {\n const video = event.currentTarget;\n const markPresented = () => {\n if (!video.isConnected) return;\n onReady?.();\n if (!retainPoster) setDecodedVideoUrl(mediaUrl);\n };\n if (video.requestVideoFrameCallback) {\n video.requestVideoFrameCallback(markPresented);\n return;\n }\n markPresented();\n }}\n onError={onError}\n data-media-position={mediaPosition}\n data-video-backdrop={persistent ? \"persistent\" : \"scene\"}\n style={{ ...mediaStyle, visibility: exhaustedKey === videoPresentationKey || waitingKey === videoPresentationKey ? \"hidden\" : undefined }}\n />\n </>\n );\n};\n"
44
44
  },
45
45
  {
46
46
  "path": "src/visual-system/scene-templates/color-utils.ts",
@@ -29,7 +29,7 @@
29
29
  "path": "src/visual-system/scene-templates/scene-background.tsx",
30
30
  "type": "registry:component",
31
31
  "target": "vanillasky/scene-templates/scene-background.tsx",
32
- "content": "/**\n * Shared photo/video backdrop for built-in and customer-owned templates.\n * Media is host-resolved; this component never searches or generates assets.\n * The base is fixed black. Photos and videos cover it when available, and\n * optional scrims appear only once the media can paint. The cinematic built-ins\n * request no scrim; custom templates can choose a treatment and text anchor.\n * The retained internal \"gradient\" media sentinel selects the black base and\n * ignores mediaUrl. It is not a built-in authoring mode or brand-color control.\n * Video playback can be owned by the player's persistent external backdrop.\n */\n\nimport React, { useEffect, useState } from \"react\";\nimport {\n hasSceneMedia,\n resolveMediaType,\n type ResolvedMediaType,\n} from \"./media-source\";\nimport type { TemplateStyle } from \"../template-context\";\nimport { BrandGradientOverlay } from \"../backgrounds\";\nimport { getBackgroundTransform } from \"../backgrounds\";\nimport { useExternalVideoBackdrop } from \"./external-video-backdrop\";\nimport { resolveMediaPosition, type MediaPosition } from \"./media-position\";\nimport { SceneVideoBackdrop } from \"./scene-video-backdrop\";\n\nexport { hasSceneMedia, resolveMediaType };\nexport { resolveMediaPosition } from \"./media-position\";\nexport { SceneVideoBackdrop } from \"./scene-video-backdrop\";\nexport type { ResolvedMediaType };\n\nexport type { MediaPosition };\nexport type MediaTreatment = \"none\" | \"subtle\" | \"cinematic\" | \"text-safe\";\n\nexport function resolveMediaTreatment(value: string): MediaTreatment {\n return value === \"none\" || value === \"subtle\" || value === \"text-safe\" ? value : \"cinematic\";\n}\n\nexport interface MediaTreatmentLayer {\n id: \"vignette\" | \"center-scrim\" | \"bottom-scrim\";\n background: string;\n style?: React.CSSProperties;\n}\n\n/**\n * Where the template puts its type. The scrim is shaped to the copy, not to\n * the frame: darkening picture the type never touches costs contrast in the\n * photo and buys no legibility. \"full\" is the conservative default for\n * templates that have not declared an anchor.\n */\nexport type MediaTextAnchor = \"center\" | \"bottom\" | \"full\";\n\n/**\n * Smoothstep-sampled alpha stops between `start`% and `end`% of the gradient\n * box, held at full strength before `start` and after `end`.\n *\n * A two-stop `rgba(0,0,0,a) → transparent` scrim ramps alpha linearly, so it\n * ends with a constant slope. Lateral inhibition in the eye amplifies that\n * slope discontinuity into a visible band — the grey bar cutting across the\n * frame that makes an overlay read as an overlay. Smoothstep flattens the\n * curve at both ends, so the scrim holds where the type sits and then leaves\n * without an edge: the same peak coverage over the copy, noticeably less of\n * the picture spent getting there.\n */\nconst SCRIM_STOP_COUNT = 7;\n\nfunction smoothstep(t: number): number {\n return t * t * (3 - 2 * t);\n}\n\nfunction easedStops(\n peakAlpha: number,\n start: number,\n end: number,\n direction: \"fade-out\" | \"fade-in\",\n): string {\n const alphaAt = (t: number): string => {\n const eased = direction === \"fade-out\" ? 1 - smoothstep(t) : smoothstep(t);\n return `rgba(0,0,0,${Number((peakAlpha * eased).toFixed(3))})`;\n };\n const stops: string[] = [];\n if (start > 0) stops.push(`${alphaAt(0)} 0%`);\n for (let i = 0; i < SCRIM_STOP_COUNT; i += 1) {\n const t = i / (SCRIM_STOP_COUNT - 1);\n const position = Number((start + (end - start) * t).toFixed(2));\n stops.push(`${alphaAt(t)} ${position}%`);\n }\n if (end < 100) stops.push(`${alphaAt(1)} 100%`);\n return stops.join(\", \");\n}\n\n/**\n * Export-safe contrast recipes. Overlays only: SVG capture cannot rely on CSS\n * filters, so a blur-behind-text plate is off the table.\n *\n * The scrims deliberately stop short of solving legibility on their own. A\n * uniform darkening strong enough to carry white type over a blown-out sky\n * needs roughly 0.8 alpha — at that point the photo is a texture, not a\n * picture. The cheaper half of the job belongs to the type: a per-glyph halo\n * (MEDIA_TEXT_SHADOW) buys local contrast exactly where it is needed and\n * costs the image nothing. Scrim for the plate, halo for the glyph.\n */\nexport function getMediaTreatmentLayers(\n value: string,\n anchor: MediaTextAnchor = \"full\",\n): MediaTreatmentLayer[] {\n const treatment = resolveMediaTreatment(value);\n // Nothing over the picture at all. Every other treatment exists to carry\n // type across a photograph; where the scene has no type - a generated clip\n // that is the whole point of the beat - even a vignette is something the\n // viewer did not ask to look through.\n if (treatment === \"none\") return [];\n const vignette: MediaTreatmentLayer = {\n id: \"vignette\",\n background:\n treatment === \"subtle\"\n ? `radial-gradient(ellipse at center, ${easedStops(0.28, 45, 100, \"fade-in\")})`\n : `radial-gradient(ellipse at center, ${easedStops(0.72, 32, 100, \"fade-in\")})`,\n };\n if (treatment === \"subtle\") return [vignette];\n\n const textSafe = treatment === \"text-safe\";\n const layers: MediaTreatmentLayer[] = [vignette];\n\n if (anchor !== \"bottom\") {\n layers.push({\n id: \"center-scrim\",\n background: textSafe\n ? `radial-gradient(ellipse 92% 58% at 50% 50%, ${easedStops(0.46, 34, 90, \"fade-out\")})`\n : `radial-gradient(ellipse 88% 52% at 50% 50%, ${easedStops(0.26, 30, 88, \"fade-out\")})`,\n });\n }\n\n if (anchor !== \"center\") {\n layers.push({\n id: \"bottom-scrim\",\n background: `linear-gradient(to top, ${easedStops(textSafe ? 0.64 : 0.5, 8, 100, \"fade-out\")})`,\n style: { top: \"55%\" },\n });\n }\n\n return layers;\n}\n\n/**\n * Whether the backdrop is actually painting, which is what decides if a scrim\n * is earned. \"pending\" is a browser-only state: static and export renders\n * never run effects and never wait on a network, so they start (and stay)\n * ready and their output is unchanged.\n */\ntype MediaPaintState = \"pending\" | \"ready\" | \"failed\";\n\nfunction initialMediaPaint(\n wantsMedia: boolean,\n resolved: ResolvedMediaType,\n mediaUrl: string,\n mediaPoster: string | undefined,\n): MediaPaintState {\n if (typeof window === \"undefined\") return \"ready\";\n if (!wantsMedia) return \"ready\";\n // A poster paints the video's frame immediately, so the scene is already\n // showing footage even though the stream is still decoding.\n if (resolved === \"video\") return mediaPoster ? \"ready\" : \"pending\";\n if (typeof Image === \"undefined\") return \"ready\";\n // Preloaded or browser-cached media decodes synchronously. Reporting it\n // ready on the first render keeps the common mid-playback case free of a\n // black-then-photo flicker.\n const cached = new Image();\n cached.src = mediaUrl;\n return cached.complete && cached.naturalWidth > 0 ? \"ready\" : \"pending\";\n}\n\nexport function getMediaBackgroundProps(variables: Record<string, unknown>) {\n return {\n mediaUrl: String(variables.mediaUrl || \"\"),\n mediaType: String(variables.mediaType || \"auto\"),\n mediaPoster: String(variables.mediaPoster || \"\"),\n mediaPosition: String(variables.mediaPosition || \"center\"),\n mediaTreatment: String(variables.mediaTreatment || \"cinematic\"),\n };\n}\n\nexport interface SceneBackgroundProps {\n style: TemplateStyle;\n progress: number;\n sceneDuration?: number;\n width: number;\n height: number;\n mediaUrl?: string;\n mediaType?: string;\n /** Still image URL shown while the <video> backdrop decodes its first\n * frame. Without it the element renders transparent during the\n * decode window and the black base shows through. */\n mediaPoster?: string;\n /** Cover-crop focal anchor. Keeps the important edge/subject visible. */\n mediaPosition?: string;\n /** Overlay recipe: subtle, cinematic, or stronger text-safe contrast. */\n mediaTreatment?: string;\n /** Where this template's copy sits, so the scrim is shaped to the type\n * instead of to the frame. Defaults to \"full\" (scrim both the middle and\n * the lower third) for templates that have not declared an anchor. */\n textAnchor?: MediaTextAnchor;\n /** Background motion effect (drift / pulse / Ken Burns). Applied to the photo/video. */\n backgroundEffect?: string;\n /** Retained seed input for customer-owned backdrop compositions. */\n seed?: number | string;\n /** Pause video when preview is paused. Defaults to true (export path). */\n isPlaying?: boolean;\n beatIntensity?: number;\n}\n\nexport const SceneBackground: React.FC<SceneBackgroundProps> = ({\n style,\n progress,\n sceneDuration,\n width: _width, // accepted for symmetry; not currently used in render\n height: _height,\n mediaUrl = \"\",\n mediaType = \"auto\",\n mediaPoster,\n mediaPosition = \"center\",\n mediaTreatment = \"cinematic\",\n textAnchor = \"full\",\n backgroundEffect,\n seed,\n isPlaying = true,\n beatIntensity = 0,\n}) => {\n void _width;\n void _height;\n const resolved = resolveMediaType(mediaType, mediaUrl);\n const wantsMedia = resolved !== \"gradient\" && !!mediaUrl;\n const externalVideoBackdrop = useExternalVideoBackdrop();\n const hasExternalVideoBackdrop = externalVideoBackdrop !== false && resolved === \"video\";\n const externalVideoFailed = externalVideoBackdrop === \"fallback\" && resolved === \"video\";\n const externalVideoReady = externalVideoBackdrop === \"ready\" && resolved === \"video\";\n\n // Apply picture and scrim together. A loading or failed asset keeps the\n // fixed black base instead of painting contrast treatment over empty media.\n const [mediaPaint, setMediaPaint] = useState<MediaPaintState>(() =>\n initialMediaPaint(wantsMedia, resolved, mediaUrl, mediaPoster),\n );\n useEffect(() => {\n setMediaPaint(initialMediaPaint(wantsMedia, resolved, mediaUrl, mediaPoster));\n // Video reports its own paint through onLoadedData / onError below.\n if (!wantsMedia || resolved !== \"photo\") return;\n if (typeof Image === \"undefined\") return;\n let cancelled = false;\n const probe = new Image();\n probe.onload = () => {\n if (!cancelled) setMediaPaint(\"ready\");\n };\n probe.onerror = () => {\n if (!cancelled) setMediaPaint(\"failed\");\n };\n probe.src = mediaUrl;\n if (probe.complete) setMediaPaint(probe.naturalWidth > 0 ? \"ready\" : \"failed\");\n return () => {\n cancelled = true;\n probe.onload = null;\n probe.onerror = null;\n };\n }, [mediaUrl, mediaPoster, resolved, wantsMedia]);\n\n // The element stays mounted while pending — that is what loads it. Only a\n // confirmed failure takes it back out.\n const showMedia = wantsMedia && mediaPaint !== \"failed\";\n const showTreatment = wantsMedia && mediaPaint === \"ready\";\n const resolvedPosition = resolveMediaPosition(mediaPosition);\n const resolvedTreatment = resolveMediaTreatment(mediaTreatment);\n const treatmentLayers = getMediaTreatmentLayers(resolvedTreatment, textAnchor);\n\n const gradSeed =\n typeof seed === \"number\"\n ? seed\n : typeof seed === \"string\"\n ? seed.split(\"\").reduce((acc, c) => acc + c.charCodeAt(0), 0)\n : 0;\n\n const bgTransform = getBackgroundTransform(\n backgroundEffect,\n progress,\n beatIntensity,\n );\n\n return (\n <>\n {(!hasExternalVideoBackdrop || externalVideoFailed) && (\n <BrandGradientOverlay\n style={style}\n progress={progress}\n sceneDuration={sceneDuration}\n seed={gradSeed}\n />\n )}\n\n {showMedia && !hasExternalVideoBackdrop &&\n (resolved === \"video\" ? (\n <SceneVideoBackdrop\n mediaUrl={mediaUrl}\n mediaPoster={mediaPoster}\n mediaPosition={mediaPosition}\n backgroundEffect={backgroundEffect}\n progress={progress}\n beatIntensity={beatIntensity}\n isPlaying={isPlaying}\n onReady={() => setMediaPaint(\"ready\")}\n onError={() => setMediaPaint(\"failed\")}\n />\n ) : (\n <img\n src={mediaUrl}\n alt=\"\"\n aria-hidden=\"true\"\n draggable={false}\n data-media-position={mediaPosition}\n style={{\n position: \"absolute\",\n inset: 0,\n transform: bgTransform.transform,\n transformOrigin: bgTransform.transformOrigin,\n width: \"100%\",\n height: \"100%\",\n objectFit: \"cover\",\n objectPosition: resolvedPosition,\n }}\n />\n ))}\n\n {(hasExternalVideoBackdrop ? externalVideoReady : showTreatment) && !externalVideoFailed &&\n treatmentLayers.map((layer) => (\n <div\n key={layer.id}\n data-media-treatment={resolvedTreatment}\n data-media-overlay={layer.id}\n style={{\n position: \"absolute\",\n inset: 0,\n background: layer.background,\n pointerEvents: \"none\",\n ...layer.style,\n }}\n />\n ))}\n </>\n );\n};\n"
32
+ "content": "/**\n * Shared photo/video backdrop for built-in and customer-owned templates.\n * Media is host-resolved; this component never searches or generates assets.\n * The base is fixed black. Photos and videos cover it when available, and\n * optional scrims appear only once the media can paint. The cinematic built-ins\n * request no scrim; custom templates can choose a treatment and text anchor.\n * The retained internal \"gradient\" media sentinel selects the black base and\n * ignores mediaUrl. It is not a built-in authoring mode or brand-color control.\n * Video playback can be owned by the player's persistent external backdrop.\n */\n\nimport React, { useEffect, useState } from \"react\";\nimport {\n hasSceneMedia,\n resolveMediaType,\n type ResolvedMediaType,\n} from \"./media-source\";\nimport type { TemplateStyle } from \"../template-context\";\nimport { BrandGradientOverlay } from \"../backgrounds\";\nimport { getBackgroundTransform } from \"../backgrounds\";\nimport { useExternalVideoBackdrop } from \"./external-video-backdrop\";\nimport { resolveMediaPosition, type MediaPosition } from \"./media-position\";\nimport { SceneVideoBackdrop } from \"./scene-video-backdrop\";\n\nexport { hasSceneMedia, resolveMediaType };\nexport { resolveMediaPosition } from \"./media-position\";\nexport { SceneVideoBackdrop } from \"./scene-video-backdrop\";\nexport type { ResolvedMediaType };\n\nexport type { MediaPosition };\nexport type MediaTreatment = \"none\" | \"subtle\" | \"cinematic\" | \"text-safe\";\n\nexport function resolveMediaTreatment(value: string): MediaTreatment {\n return value === \"none\" || value === \"subtle\" || value === \"text-safe\" ? value : \"cinematic\";\n}\n\nexport interface MediaTreatmentLayer {\n id: \"vignette\" | \"center-scrim\" | \"bottom-scrim\";\n background: string;\n style?: React.CSSProperties;\n}\n\n/**\n * Where the template puts its type. The scrim is shaped to the copy, not to\n * the frame: darkening picture the type never touches costs contrast in the\n * photo and buys no legibility. \"full\" is the conservative default for\n * templates that have not declared an anchor.\n */\nexport type MediaTextAnchor = \"center\" | \"bottom\" | \"full\";\n\n/**\n * Smoothstep-sampled alpha stops between `start`% and `end`% of the gradient\n * box, held at full strength before `start` and after `end`.\n *\n * A two-stop `rgba(0,0,0,a) → transparent` scrim ramps alpha linearly, so it\n * ends with a constant slope. Lateral inhibition in the eye amplifies that\n * slope discontinuity into a visible band — the grey bar cutting across the\n * frame that makes an overlay read as an overlay. Smoothstep flattens the\n * curve at both ends, so the scrim holds where the type sits and then leaves\n * without an edge: the same peak coverage over the copy, noticeably less of\n * the picture spent getting there.\n */\nconst SCRIM_STOP_COUNT = 7;\n\nfunction smoothstep(t: number): number {\n return t * t * (3 - 2 * t);\n}\n\nfunction easedStops(\n peakAlpha: number,\n start: number,\n end: number,\n direction: \"fade-out\" | \"fade-in\",\n): string {\n const alphaAt = (t: number): string => {\n const eased = direction === \"fade-out\" ? 1 - smoothstep(t) : smoothstep(t);\n return `rgba(0,0,0,${Number((peakAlpha * eased).toFixed(3))})`;\n };\n const stops: string[] = [];\n if (start > 0) stops.push(`${alphaAt(0)} 0%`);\n for (let i = 0; i < SCRIM_STOP_COUNT; i += 1) {\n const t = i / (SCRIM_STOP_COUNT - 1);\n const position = Number((start + (end - start) * t).toFixed(2));\n stops.push(`${alphaAt(t)} ${position}%`);\n }\n if (end < 100) stops.push(`${alphaAt(1)} 100%`);\n return stops.join(\", \");\n}\n\n/**\n * Export-safe contrast recipes. Overlays only: SVG capture cannot rely on CSS\n * filters, so a blur-behind-text plate is off the table.\n *\n * The scrims deliberately stop short of solving legibility on their own. A\n * uniform darkening strong enough to carry white type over a blown-out sky\n * needs roughly 0.8 alpha — at that point the photo is a texture, not a\n * picture. The cheaper half of the job belongs to the type: a per-glyph halo\n * (MEDIA_TEXT_SHADOW) buys local contrast exactly where it is needed and\n * costs the image nothing. Scrim for the plate, halo for the glyph.\n */\nexport function getMediaTreatmentLayers(\n value: string,\n anchor: MediaTextAnchor = \"full\",\n): MediaTreatmentLayer[] {\n const treatment = resolveMediaTreatment(value);\n // Nothing over the picture at all. Every other treatment exists to carry\n // type across a photograph; where the scene has no type - a generated clip\n // that is the whole point of the beat - even a vignette is something the\n // viewer did not ask to look through.\n if (treatment === \"none\") return [];\n const vignette: MediaTreatmentLayer = {\n id: \"vignette\",\n background:\n treatment === \"subtle\"\n ? `radial-gradient(ellipse at center, ${easedStops(0.28, 45, 100, \"fade-in\")})`\n : `radial-gradient(ellipse at center, ${easedStops(0.72, 32, 100, \"fade-in\")})`,\n };\n if (treatment === \"subtle\") return [vignette];\n\n const textSafe = treatment === \"text-safe\";\n const layers: MediaTreatmentLayer[] = [vignette];\n\n if (anchor !== \"bottom\") {\n layers.push({\n id: \"center-scrim\",\n background: textSafe\n ? `radial-gradient(ellipse 92% 58% at 50% 50%, ${easedStops(0.46, 34, 90, \"fade-out\")})`\n : `radial-gradient(ellipse 88% 52% at 50% 50%, ${easedStops(0.26, 30, 88, \"fade-out\")})`,\n });\n }\n\n if (anchor !== \"center\") {\n layers.push({\n id: \"bottom-scrim\",\n background: `linear-gradient(to top, ${easedStops(textSafe ? 0.64 : 0.5, 8, 100, \"fade-out\")})`,\n style: { top: \"55%\" },\n });\n }\n\n return layers;\n}\n\n/**\n * Whether the backdrop is actually painting, which is what decides if a scrim\n * is earned. \"pending\" is a browser-only state: static and export renders\n * never run effects and never wait on a network, so they start (and stay)\n * ready and their output is unchanged.\n */\ntype MediaPaintState = \"pending\" | \"ready\" | \"failed\";\n\nfunction initialMediaPaint(\n wantsMedia: boolean,\n resolved: ResolvedMediaType,\n mediaUrl: string,\n mediaPoster: string | undefined,\n): MediaPaintState {\n if (typeof window === \"undefined\") return \"ready\";\n if (!wantsMedia) return \"ready\";\n // A poster paints the video's frame immediately, so the scene is already\n // showing footage even though the stream is still decoding.\n if (resolved === \"video\") return mediaPoster ? \"ready\" : \"pending\";\n if (typeof Image === \"undefined\") return \"ready\";\n // Preloaded or browser-cached media decodes synchronously. Reporting it\n // ready on the first render keeps the common mid-playback case free of a\n // black-then-photo flicker.\n const cached = new Image();\n cached.src = mediaUrl;\n return cached.complete && cached.naturalWidth > 0 ? \"ready\" : \"pending\";\n}\n\nexport function getMediaBackgroundProps(variables: Record<string, unknown>) {\n return {\n mediaUrl: String(variables.mediaUrl || \"\"),\n mediaType: String(variables.mediaType || \"auto\"),\n mediaPoster: String(variables.mediaPoster || \"\"),\n mediaPosition: String(variables.mediaPosition || \"center\"),\n mediaTreatment: String(variables.mediaTreatment || \"cinematic\"),\n };\n}\n\nexport interface SceneBackgroundProps {\n style: TemplateStyle;\n progress: number;\n sceneDuration?: number;\n width: number;\n height: number;\n mediaUrl?: string;\n mediaType?: string;\n /** Still image URL shown while the <video> backdrop decodes its first\n * frame. Without it the element renders transparent during the\n * decode window and the black base shows through. */\n mediaPoster?: string;\n /** Cover-crop focal anchor. Keeps the important edge/subject visible. */\n mediaPosition?: string;\n /** Overlay recipe: subtle, cinematic, or stronger text-safe contrast. */\n mediaTreatment?: string;\n /** Where this template's copy sits, so the scrim is shaped to the type\n * instead of to the frame. Defaults to \"full\" (scrim both the middle and\n * the lower third) for templates that have not declared an anchor. */\n textAnchor?: MediaTextAnchor;\n /** Background motion effect (drift / pulse / Ken Burns). Applied to the photo/video. */\n backgroundEffect?: string;\n /** Retained seed input for customer-owned backdrop compositions. */\n seed?: number | string;\n /** Pause video when preview is paused. Defaults to true (export path). */\n isPlaying?: boolean;\n beatIntensity?: number;\n}\n\nexport const SceneBackground: React.FC<SceneBackgroundProps> = ({\n style,\n progress,\n sceneDuration,\n width: _width, // accepted for symmetry; not currently used in render\n height: _height,\n mediaUrl = \"\",\n mediaType = \"auto\",\n mediaPoster,\n mediaPosition = \"center\",\n mediaTreatment = \"cinematic\",\n textAnchor = \"full\",\n backgroundEffect,\n seed,\n isPlaying = true,\n beatIntensity = 0,\n}) => {\n void _width;\n void _height;\n const resolved = resolveMediaType(mediaType, mediaUrl);\n const wantsMedia = resolved !== \"gradient\" && !!mediaUrl;\n const externalVideoBackdrop = useExternalVideoBackdrop();\n const hasExternalVideoBackdrop = externalVideoBackdrop !== false && resolved === \"video\";\n const externalVideoFailed = externalVideoBackdrop === \"fallback\" && resolved === \"video\";\n const externalVideoReady = externalVideoBackdrop === \"ready\" && resolved === \"video\";\n\n // Apply picture and scrim together. A loading or failed asset keeps the\n // fixed black base instead of painting contrast treatment over empty media.\n const [mediaPaint, setMediaPaint] = useState<MediaPaintState>(() =>\n initialMediaPaint(wantsMedia, resolved, mediaUrl, mediaPoster),\n );\n useEffect(() => {\n setMediaPaint(initialMediaPaint(wantsMedia, resolved, mediaUrl, mediaPoster));\n // Video reports its own paint through onLoadedData / onError below.\n if (!wantsMedia || resolved !== \"photo\") return;\n if (typeof Image === \"undefined\") return;\n let cancelled = false;\n const probe = new Image();\n probe.onload = () => {\n if (!cancelled) setMediaPaint(\"ready\");\n };\n probe.onerror = () => {\n if (!cancelled) setMediaPaint(\"failed\");\n };\n probe.src = mediaUrl;\n if (probe.complete) setMediaPaint(probe.naturalWidth > 0 ? \"ready\" : \"failed\");\n return () => {\n cancelled = true;\n probe.onload = null;\n probe.onerror = null;\n };\n }, [mediaUrl, mediaPoster, resolved, wantsMedia]);\n\n // The element stays mounted while pending — that is what loads it. Only a\n // confirmed failure takes it back out.\n const showMedia = wantsMedia && mediaPaint !== \"failed\";\n const showTreatment = wantsMedia && mediaPaint === \"ready\";\n const resolvedPosition = resolveMediaPosition(mediaPosition);\n const resolvedTreatment = resolveMediaTreatment(mediaTreatment);\n const treatmentLayers = getMediaTreatmentLayers(resolvedTreatment, textAnchor);\n\n const gradSeed =\n typeof seed === \"number\"\n ? seed\n : typeof seed === \"string\"\n ? seed.split(\"\").reduce((acc, c) => acc + c.charCodeAt(0), 0)\n : 0;\n\n const bgTransform = getBackgroundTransform(\n backgroundEffect,\n progress,\n beatIntensity,\n );\n\n return (\n <>\n {(!hasExternalVideoBackdrop || externalVideoFailed) && (\n <BrandGradientOverlay\n style={style}\n progress={progress}\n sceneDuration={sceneDuration}\n seed={gradSeed}\n />\n )}\n\n {showMedia && !hasExternalVideoBackdrop &&\n (resolved === \"video\" ? (\n <SceneVideoBackdrop\n mediaUrl={mediaUrl}\n mediaPoster={mediaPoster}\n mediaPosition={mediaPosition}\n backgroundEffect={backgroundEffect}\n progress={progress}\n sceneDuration={sceneDuration}\n beatIntensity={beatIntensity}\n isPlaying={isPlaying}\n onReady={() => setMediaPaint(\"ready\")}\n onError={() => setMediaPaint(\"failed\")}\n />\n ) : (\n <img\n src={mediaUrl}\n alt=\"\"\n aria-hidden=\"true\"\n draggable={false}\n data-media-position={mediaPosition}\n style={{\n position: \"absolute\",\n inset: 0,\n transform: bgTransform.transform,\n transformOrigin: bgTransform.transformOrigin,\n width: \"100%\",\n height: \"100%\",\n objectFit: \"cover\",\n objectPosition: resolvedPosition,\n }}\n />\n ))}\n\n {(hasExternalVideoBackdrop ? externalVideoReady : showTreatment) && !externalVideoFailed &&\n treatmentLayers.map((layer) => (\n <div\n key={layer.id}\n data-media-treatment={resolvedTreatment}\n data-media-overlay={layer.id}\n style={{\n position: \"absolute\",\n inset: 0,\n background: layer.background,\n pointerEvents: \"none\",\n ...layer.style,\n }}\n />\n ))}\n </>\n );\n};\n"
33
33
  },
34
34
  {
35
35
  "path": "src/visual-system/scene-templates/media-source.ts",
@@ -42,7 +42,7 @@
42
42
  "path": "src/visual-system/scene-templates/scene-background.tsx",
43
43
  "type": "registry:component",
44
44
  "target": "vanillasky/scene-templates/scene-background.tsx",
45
- "content": "/**\n * Shared photo/video backdrop for built-in and customer-owned templates.\n * Media is host-resolved; this component never searches or generates assets.\n * The base is fixed black. Photos and videos cover it when available, and\n * optional scrims appear only once the media can paint. The cinematic built-ins\n * request no scrim; custom templates can choose a treatment and text anchor.\n * The retained internal \"gradient\" media sentinel selects the black base and\n * ignores mediaUrl. It is not a built-in authoring mode or brand-color control.\n * Video playback can be owned by the player's persistent external backdrop.\n */\n\nimport React, { useEffect, useState } from \"react\";\nimport {\n hasSceneMedia,\n resolveMediaType,\n type ResolvedMediaType,\n} from \"./media-source\";\nimport type { TemplateStyle } from \"../template-context\";\nimport { BrandGradientOverlay } from \"../backgrounds\";\nimport { getBackgroundTransform } from \"../backgrounds\";\nimport { useExternalVideoBackdrop } from \"./external-video-backdrop\";\nimport { resolveMediaPosition, type MediaPosition } from \"./media-position\";\nimport { SceneVideoBackdrop } from \"./scene-video-backdrop\";\n\nexport { hasSceneMedia, resolveMediaType };\nexport { resolveMediaPosition } from \"./media-position\";\nexport { SceneVideoBackdrop } from \"./scene-video-backdrop\";\nexport type { ResolvedMediaType };\n\nexport type { MediaPosition };\nexport type MediaTreatment = \"none\" | \"subtle\" | \"cinematic\" | \"text-safe\";\n\nexport function resolveMediaTreatment(value: string): MediaTreatment {\n return value === \"none\" || value === \"subtle\" || value === \"text-safe\" ? value : \"cinematic\";\n}\n\nexport interface MediaTreatmentLayer {\n id: \"vignette\" | \"center-scrim\" | \"bottom-scrim\";\n background: string;\n style?: React.CSSProperties;\n}\n\n/**\n * Where the template puts its type. The scrim is shaped to the copy, not to\n * the frame: darkening picture the type never touches costs contrast in the\n * photo and buys no legibility. \"full\" is the conservative default for\n * templates that have not declared an anchor.\n */\nexport type MediaTextAnchor = \"center\" | \"bottom\" | \"full\";\n\n/**\n * Smoothstep-sampled alpha stops between `start`% and `end`% of the gradient\n * box, held at full strength before `start` and after `end`.\n *\n * A two-stop `rgba(0,0,0,a) → transparent` scrim ramps alpha linearly, so it\n * ends with a constant slope. Lateral inhibition in the eye amplifies that\n * slope discontinuity into a visible band — the grey bar cutting across the\n * frame that makes an overlay read as an overlay. Smoothstep flattens the\n * curve at both ends, so the scrim holds where the type sits and then leaves\n * without an edge: the same peak coverage over the copy, noticeably less of\n * the picture spent getting there.\n */\nconst SCRIM_STOP_COUNT = 7;\n\nfunction smoothstep(t: number): number {\n return t * t * (3 - 2 * t);\n}\n\nfunction easedStops(\n peakAlpha: number,\n start: number,\n end: number,\n direction: \"fade-out\" | \"fade-in\",\n): string {\n const alphaAt = (t: number): string => {\n const eased = direction === \"fade-out\" ? 1 - smoothstep(t) : smoothstep(t);\n return `rgba(0,0,0,${Number((peakAlpha * eased).toFixed(3))})`;\n };\n const stops: string[] = [];\n if (start > 0) stops.push(`${alphaAt(0)} 0%`);\n for (let i = 0; i < SCRIM_STOP_COUNT; i += 1) {\n const t = i / (SCRIM_STOP_COUNT - 1);\n const position = Number((start + (end - start) * t).toFixed(2));\n stops.push(`${alphaAt(t)} ${position}%`);\n }\n if (end < 100) stops.push(`${alphaAt(1)} 100%`);\n return stops.join(\", \");\n}\n\n/**\n * Export-safe contrast recipes. Overlays only: SVG capture cannot rely on CSS\n * filters, so a blur-behind-text plate is off the table.\n *\n * The scrims deliberately stop short of solving legibility on their own. A\n * uniform darkening strong enough to carry white type over a blown-out sky\n * needs roughly 0.8 alpha — at that point the photo is a texture, not a\n * picture. The cheaper half of the job belongs to the type: a per-glyph halo\n * (MEDIA_TEXT_SHADOW) buys local contrast exactly where it is needed and\n * costs the image nothing. Scrim for the plate, halo for the glyph.\n */\nexport function getMediaTreatmentLayers(\n value: string,\n anchor: MediaTextAnchor = \"full\",\n): MediaTreatmentLayer[] {\n const treatment = resolveMediaTreatment(value);\n // Nothing over the picture at all. Every other treatment exists to carry\n // type across a photograph; where the scene has no type - a generated clip\n // that is the whole point of the beat - even a vignette is something the\n // viewer did not ask to look through.\n if (treatment === \"none\") return [];\n const vignette: MediaTreatmentLayer = {\n id: \"vignette\",\n background:\n treatment === \"subtle\"\n ? `radial-gradient(ellipse at center, ${easedStops(0.28, 45, 100, \"fade-in\")})`\n : `radial-gradient(ellipse at center, ${easedStops(0.72, 32, 100, \"fade-in\")})`,\n };\n if (treatment === \"subtle\") return [vignette];\n\n const textSafe = treatment === \"text-safe\";\n const layers: MediaTreatmentLayer[] = [vignette];\n\n if (anchor !== \"bottom\") {\n layers.push({\n id: \"center-scrim\",\n background: textSafe\n ? `radial-gradient(ellipse 92% 58% at 50% 50%, ${easedStops(0.46, 34, 90, \"fade-out\")})`\n : `radial-gradient(ellipse 88% 52% at 50% 50%, ${easedStops(0.26, 30, 88, \"fade-out\")})`,\n });\n }\n\n if (anchor !== \"center\") {\n layers.push({\n id: \"bottom-scrim\",\n background: `linear-gradient(to top, ${easedStops(textSafe ? 0.64 : 0.5, 8, 100, \"fade-out\")})`,\n style: { top: \"55%\" },\n });\n }\n\n return layers;\n}\n\n/**\n * Whether the backdrop is actually painting, which is what decides if a scrim\n * is earned. \"pending\" is a browser-only state: static and export renders\n * never run effects and never wait on a network, so they start (and stay)\n * ready and their output is unchanged.\n */\ntype MediaPaintState = \"pending\" | \"ready\" | \"failed\";\n\nfunction initialMediaPaint(\n wantsMedia: boolean,\n resolved: ResolvedMediaType,\n mediaUrl: string,\n mediaPoster: string | undefined,\n): MediaPaintState {\n if (typeof window === \"undefined\") return \"ready\";\n if (!wantsMedia) return \"ready\";\n // A poster paints the video's frame immediately, so the scene is already\n // showing footage even though the stream is still decoding.\n if (resolved === \"video\") return mediaPoster ? \"ready\" : \"pending\";\n if (typeof Image === \"undefined\") return \"ready\";\n // Preloaded or browser-cached media decodes synchronously. Reporting it\n // ready on the first render keeps the common mid-playback case free of a\n // black-then-photo flicker.\n const cached = new Image();\n cached.src = mediaUrl;\n return cached.complete && cached.naturalWidth > 0 ? \"ready\" : \"pending\";\n}\n\nexport function getMediaBackgroundProps(variables: Record<string, unknown>) {\n return {\n mediaUrl: String(variables.mediaUrl || \"\"),\n mediaType: String(variables.mediaType || \"auto\"),\n mediaPoster: String(variables.mediaPoster || \"\"),\n mediaPosition: String(variables.mediaPosition || \"center\"),\n mediaTreatment: String(variables.mediaTreatment || \"cinematic\"),\n };\n}\n\nexport interface SceneBackgroundProps {\n style: TemplateStyle;\n progress: number;\n sceneDuration?: number;\n width: number;\n height: number;\n mediaUrl?: string;\n mediaType?: string;\n /** Still image URL shown while the <video> backdrop decodes its first\n * frame. Without it the element renders transparent during the\n * decode window and the black base shows through. */\n mediaPoster?: string;\n /** Cover-crop focal anchor. Keeps the important edge/subject visible. */\n mediaPosition?: string;\n /** Overlay recipe: subtle, cinematic, or stronger text-safe contrast. */\n mediaTreatment?: string;\n /** Where this template's copy sits, so the scrim is shaped to the type\n * instead of to the frame. Defaults to \"full\" (scrim both the middle and\n * the lower third) for templates that have not declared an anchor. */\n textAnchor?: MediaTextAnchor;\n /** Background motion effect (drift / pulse / Ken Burns). Applied to the photo/video. */\n backgroundEffect?: string;\n /** Retained seed input for customer-owned backdrop compositions. */\n seed?: number | string;\n /** Pause video when preview is paused. Defaults to true (export path). */\n isPlaying?: boolean;\n beatIntensity?: number;\n}\n\nexport const SceneBackground: React.FC<SceneBackgroundProps> = ({\n style,\n progress,\n sceneDuration,\n width: _width, // accepted for symmetry; not currently used in render\n height: _height,\n mediaUrl = \"\",\n mediaType = \"auto\",\n mediaPoster,\n mediaPosition = \"center\",\n mediaTreatment = \"cinematic\",\n textAnchor = \"full\",\n backgroundEffect,\n seed,\n isPlaying = true,\n beatIntensity = 0,\n}) => {\n void _width;\n void _height;\n const resolved = resolveMediaType(mediaType, mediaUrl);\n const wantsMedia = resolved !== \"gradient\" && !!mediaUrl;\n const externalVideoBackdrop = useExternalVideoBackdrop();\n const hasExternalVideoBackdrop = externalVideoBackdrop !== false && resolved === \"video\";\n const externalVideoFailed = externalVideoBackdrop === \"fallback\" && resolved === \"video\";\n const externalVideoReady = externalVideoBackdrop === \"ready\" && resolved === \"video\";\n\n // Apply picture and scrim together. A loading or failed asset keeps the\n // fixed black base instead of painting contrast treatment over empty media.\n const [mediaPaint, setMediaPaint] = useState<MediaPaintState>(() =>\n initialMediaPaint(wantsMedia, resolved, mediaUrl, mediaPoster),\n );\n useEffect(() => {\n setMediaPaint(initialMediaPaint(wantsMedia, resolved, mediaUrl, mediaPoster));\n // Video reports its own paint through onLoadedData / onError below.\n if (!wantsMedia || resolved !== \"photo\") return;\n if (typeof Image === \"undefined\") return;\n let cancelled = false;\n const probe = new Image();\n probe.onload = () => {\n if (!cancelled) setMediaPaint(\"ready\");\n };\n probe.onerror = () => {\n if (!cancelled) setMediaPaint(\"failed\");\n };\n probe.src = mediaUrl;\n if (probe.complete) setMediaPaint(probe.naturalWidth > 0 ? \"ready\" : \"failed\");\n return () => {\n cancelled = true;\n probe.onload = null;\n probe.onerror = null;\n };\n }, [mediaUrl, mediaPoster, resolved, wantsMedia]);\n\n // The element stays mounted while pending — that is what loads it. Only a\n // confirmed failure takes it back out.\n const showMedia = wantsMedia && mediaPaint !== \"failed\";\n const showTreatment = wantsMedia && mediaPaint === \"ready\";\n const resolvedPosition = resolveMediaPosition(mediaPosition);\n const resolvedTreatment = resolveMediaTreatment(mediaTreatment);\n const treatmentLayers = getMediaTreatmentLayers(resolvedTreatment, textAnchor);\n\n const gradSeed =\n typeof seed === \"number\"\n ? seed\n : typeof seed === \"string\"\n ? seed.split(\"\").reduce((acc, c) => acc + c.charCodeAt(0), 0)\n : 0;\n\n const bgTransform = getBackgroundTransform(\n backgroundEffect,\n progress,\n beatIntensity,\n );\n\n return (\n <>\n {(!hasExternalVideoBackdrop || externalVideoFailed) && (\n <BrandGradientOverlay\n style={style}\n progress={progress}\n sceneDuration={sceneDuration}\n seed={gradSeed}\n />\n )}\n\n {showMedia && !hasExternalVideoBackdrop &&\n (resolved === \"video\" ? (\n <SceneVideoBackdrop\n mediaUrl={mediaUrl}\n mediaPoster={mediaPoster}\n mediaPosition={mediaPosition}\n backgroundEffect={backgroundEffect}\n progress={progress}\n beatIntensity={beatIntensity}\n isPlaying={isPlaying}\n onReady={() => setMediaPaint(\"ready\")}\n onError={() => setMediaPaint(\"failed\")}\n />\n ) : (\n <img\n src={mediaUrl}\n alt=\"\"\n aria-hidden=\"true\"\n draggable={false}\n data-media-position={mediaPosition}\n style={{\n position: \"absolute\",\n inset: 0,\n transform: bgTransform.transform,\n transformOrigin: bgTransform.transformOrigin,\n width: \"100%\",\n height: \"100%\",\n objectFit: \"cover\",\n objectPosition: resolvedPosition,\n }}\n />\n ))}\n\n {(hasExternalVideoBackdrop ? externalVideoReady : showTreatment) && !externalVideoFailed &&\n treatmentLayers.map((layer) => (\n <div\n key={layer.id}\n data-media-treatment={resolvedTreatment}\n data-media-overlay={layer.id}\n style={{\n position: \"absolute\",\n inset: 0,\n background: layer.background,\n pointerEvents: \"none\",\n ...layer.style,\n }}\n />\n ))}\n </>\n );\n};\n"
45
+ "content": "/**\n * Shared photo/video backdrop for built-in and customer-owned templates.\n * Media is host-resolved; this component never searches or generates assets.\n * The base is fixed black. Photos and videos cover it when available, and\n * optional scrims appear only once the media can paint. The cinematic built-ins\n * request no scrim; custom templates can choose a treatment and text anchor.\n * The retained internal \"gradient\" media sentinel selects the black base and\n * ignores mediaUrl. It is not a built-in authoring mode or brand-color control.\n * Video playback can be owned by the player's persistent external backdrop.\n */\n\nimport React, { useEffect, useState } from \"react\";\nimport {\n hasSceneMedia,\n resolveMediaType,\n type ResolvedMediaType,\n} from \"./media-source\";\nimport type { TemplateStyle } from \"../template-context\";\nimport { BrandGradientOverlay } from \"../backgrounds\";\nimport { getBackgroundTransform } from \"../backgrounds\";\nimport { useExternalVideoBackdrop } from \"./external-video-backdrop\";\nimport { resolveMediaPosition, type MediaPosition } from \"./media-position\";\nimport { SceneVideoBackdrop } from \"./scene-video-backdrop\";\n\nexport { hasSceneMedia, resolveMediaType };\nexport { resolveMediaPosition } from \"./media-position\";\nexport { SceneVideoBackdrop } from \"./scene-video-backdrop\";\nexport type { ResolvedMediaType };\n\nexport type { MediaPosition };\nexport type MediaTreatment = \"none\" | \"subtle\" | \"cinematic\" | \"text-safe\";\n\nexport function resolveMediaTreatment(value: string): MediaTreatment {\n return value === \"none\" || value === \"subtle\" || value === \"text-safe\" ? value : \"cinematic\";\n}\n\nexport interface MediaTreatmentLayer {\n id: \"vignette\" | \"center-scrim\" | \"bottom-scrim\";\n background: string;\n style?: React.CSSProperties;\n}\n\n/**\n * Where the template puts its type. The scrim is shaped to the copy, not to\n * the frame: darkening picture the type never touches costs contrast in the\n * photo and buys no legibility. \"full\" is the conservative default for\n * templates that have not declared an anchor.\n */\nexport type MediaTextAnchor = \"center\" | \"bottom\" | \"full\";\n\n/**\n * Smoothstep-sampled alpha stops between `start`% and `end`% of the gradient\n * box, held at full strength before `start` and after `end`.\n *\n * A two-stop `rgba(0,0,0,a) → transparent` scrim ramps alpha linearly, so it\n * ends with a constant slope. Lateral inhibition in the eye amplifies that\n * slope discontinuity into a visible band — the grey bar cutting across the\n * frame that makes an overlay read as an overlay. Smoothstep flattens the\n * curve at both ends, so the scrim holds where the type sits and then leaves\n * without an edge: the same peak coverage over the copy, noticeably less of\n * the picture spent getting there.\n */\nconst SCRIM_STOP_COUNT = 7;\n\nfunction smoothstep(t: number): number {\n return t * t * (3 - 2 * t);\n}\n\nfunction easedStops(\n peakAlpha: number,\n start: number,\n end: number,\n direction: \"fade-out\" | \"fade-in\",\n): string {\n const alphaAt = (t: number): string => {\n const eased = direction === \"fade-out\" ? 1 - smoothstep(t) : smoothstep(t);\n return `rgba(0,0,0,${Number((peakAlpha * eased).toFixed(3))})`;\n };\n const stops: string[] = [];\n if (start > 0) stops.push(`${alphaAt(0)} 0%`);\n for (let i = 0; i < SCRIM_STOP_COUNT; i += 1) {\n const t = i / (SCRIM_STOP_COUNT - 1);\n const position = Number((start + (end - start) * t).toFixed(2));\n stops.push(`${alphaAt(t)} ${position}%`);\n }\n if (end < 100) stops.push(`${alphaAt(1)} 100%`);\n return stops.join(\", \");\n}\n\n/**\n * Export-safe contrast recipes. Overlays only: SVG capture cannot rely on CSS\n * filters, so a blur-behind-text plate is off the table.\n *\n * The scrims deliberately stop short of solving legibility on their own. A\n * uniform darkening strong enough to carry white type over a blown-out sky\n * needs roughly 0.8 alpha — at that point the photo is a texture, not a\n * picture. The cheaper half of the job belongs to the type: a per-glyph halo\n * (MEDIA_TEXT_SHADOW) buys local contrast exactly where it is needed and\n * costs the image nothing. Scrim for the plate, halo for the glyph.\n */\nexport function getMediaTreatmentLayers(\n value: string,\n anchor: MediaTextAnchor = \"full\",\n): MediaTreatmentLayer[] {\n const treatment = resolveMediaTreatment(value);\n // Nothing over the picture at all. Every other treatment exists to carry\n // type across a photograph; where the scene has no type - a generated clip\n // that is the whole point of the beat - even a vignette is something the\n // viewer did not ask to look through.\n if (treatment === \"none\") return [];\n const vignette: MediaTreatmentLayer = {\n id: \"vignette\",\n background:\n treatment === \"subtle\"\n ? `radial-gradient(ellipse at center, ${easedStops(0.28, 45, 100, \"fade-in\")})`\n : `radial-gradient(ellipse at center, ${easedStops(0.72, 32, 100, \"fade-in\")})`,\n };\n if (treatment === \"subtle\") return [vignette];\n\n const textSafe = treatment === \"text-safe\";\n const layers: MediaTreatmentLayer[] = [vignette];\n\n if (anchor !== \"bottom\") {\n layers.push({\n id: \"center-scrim\",\n background: textSafe\n ? `radial-gradient(ellipse 92% 58% at 50% 50%, ${easedStops(0.46, 34, 90, \"fade-out\")})`\n : `radial-gradient(ellipse 88% 52% at 50% 50%, ${easedStops(0.26, 30, 88, \"fade-out\")})`,\n });\n }\n\n if (anchor !== \"center\") {\n layers.push({\n id: \"bottom-scrim\",\n background: `linear-gradient(to top, ${easedStops(textSafe ? 0.64 : 0.5, 8, 100, \"fade-out\")})`,\n style: { top: \"55%\" },\n });\n }\n\n return layers;\n}\n\n/**\n * Whether the backdrop is actually painting, which is what decides if a scrim\n * is earned. \"pending\" is a browser-only state: static and export renders\n * never run effects and never wait on a network, so they start (and stay)\n * ready and their output is unchanged.\n */\ntype MediaPaintState = \"pending\" | \"ready\" | \"failed\";\n\nfunction initialMediaPaint(\n wantsMedia: boolean,\n resolved: ResolvedMediaType,\n mediaUrl: string,\n mediaPoster: string | undefined,\n): MediaPaintState {\n if (typeof window === \"undefined\") return \"ready\";\n if (!wantsMedia) return \"ready\";\n // A poster paints the video's frame immediately, so the scene is already\n // showing footage even though the stream is still decoding.\n if (resolved === \"video\") return mediaPoster ? \"ready\" : \"pending\";\n if (typeof Image === \"undefined\") return \"ready\";\n // Preloaded or browser-cached media decodes synchronously. Reporting it\n // ready on the first render keeps the common mid-playback case free of a\n // black-then-photo flicker.\n const cached = new Image();\n cached.src = mediaUrl;\n return cached.complete && cached.naturalWidth > 0 ? \"ready\" : \"pending\";\n}\n\nexport function getMediaBackgroundProps(variables: Record<string, unknown>) {\n return {\n mediaUrl: String(variables.mediaUrl || \"\"),\n mediaType: String(variables.mediaType || \"auto\"),\n mediaPoster: String(variables.mediaPoster || \"\"),\n mediaPosition: String(variables.mediaPosition || \"center\"),\n mediaTreatment: String(variables.mediaTreatment || \"cinematic\"),\n };\n}\n\nexport interface SceneBackgroundProps {\n style: TemplateStyle;\n progress: number;\n sceneDuration?: number;\n width: number;\n height: number;\n mediaUrl?: string;\n mediaType?: string;\n /** Still image URL shown while the <video> backdrop decodes its first\n * frame. Without it the element renders transparent during the\n * decode window and the black base shows through. */\n mediaPoster?: string;\n /** Cover-crop focal anchor. Keeps the important edge/subject visible. */\n mediaPosition?: string;\n /** Overlay recipe: subtle, cinematic, or stronger text-safe contrast. */\n mediaTreatment?: string;\n /** Where this template's copy sits, so the scrim is shaped to the type\n * instead of to the frame. Defaults to \"full\" (scrim both the middle and\n * the lower third) for templates that have not declared an anchor. */\n textAnchor?: MediaTextAnchor;\n /** Background motion effect (drift / pulse / Ken Burns). Applied to the photo/video. */\n backgroundEffect?: string;\n /** Retained seed input for customer-owned backdrop compositions. */\n seed?: number | string;\n /** Pause video when preview is paused. Defaults to true (export path). */\n isPlaying?: boolean;\n beatIntensity?: number;\n}\n\nexport const SceneBackground: React.FC<SceneBackgroundProps> = ({\n style,\n progress,\n sceneDuration,\n width: _width, // accepted for symmetry; not currently used in render\n height: _height,\n mediaUrl = \"\",\n mediaType = \"auto\",\n mediaPoster,\n mediaPosition = \"center\",\n mediaTreatment = \"cinematic\",\n textAnchor = \"full\",\n backgroundEffect,\n seed,\n isPlaying = true,\n beatIntensity = 0,\n}) => {\n void _width;\n void _height;\n const resolved = resolveMediaType(mediaType, mediaUrl);\n const wantsMedia = resolved !== \"gradient\" && !!mediaUrl;\n const externalVideoBackdrop = useExternalVideoBackdrop();\n const hasExternalVideoBackdrop = externalVideoBackdrop !== false && resolved === \"video\";\n const externalVideoFailed = externalVideoBackdrop === \"fallback\" && resolved === \"video\";\n const externalVideoReady = externalVideoBackdrop === \"ready\" && resolved === \"video\";\n\n // Apply picture and scrim together. A loading or failed asset keeps the\n // fixed black base instead of painting contrast treatment over empty media.\n const [mediaPaint, setMediaPaint] = useState<MediaPaintState>(() =>\n initialMediaPaint(wantsMedia, resolved, mediaUrl, mediaPoster),\n );\n useEffect(() => {\n setMediaPaint(initialMediaPaint(wantsMedia, resolved, mediaUrl, mediaPoster));\n // Video reports its own paint through onLoadedData / onError below.\n if (!wantsMedia || resolved !== \"photo\") return;\n if (typeof Image === \"undefined\") return;\n let cancelled = false;\n const probe = new Image();\n probe.onload = () => {\n if (!cancelled) setMediaPaint(\"ready\");\n };\n probe.onerror = () => {\n if (!cancelled) setMediaPaint(\"failed\");\n };\n probe.src = mediaUrl;\n if (probe.complete) setMediaPaint(probe.naturalWidth > 0 ? \"ready\" : \"failed\");\n return () => {\n cancelled = true;\n probe.onload = null;\n probe.onerror = null;\n };\n }, [mediaUrl, mediaPoster, resolved, wantsMedia]);\n\n // The element stays mounted while pending — that is what loads it. Only a\n // confirmed failure takes it back out.\n const showMedia = wantsMedia && mediaPaint !== \"failed\";\n const showTreatment = wantsMedia && mediaPaint === \"ready\";\n const resolvedPosition = resolveMediaPosition(mediaPosition);\n const resolvedTreatment = resolveMediaTreatment(mediaTreatment);\n const treatmentLayers = getMediaTreatmentLayers(resolvedTreatment, textAnchor);\n\n const gradSeed =\n typeof seed === \"number\"\n ? seed\n : typeof seed === \"string\"\n ? seed.split(\"\").reduce((acc, c) => acc + c.charCodeAt(0), 0)\n : 0;\n\n const bgTransform = getBackgroundTransform(\n backgroundEffect,\n progress,\n beatIntensity,\n );\n\n return (\n <>\n {(!hasExternalVideoBackdrop || externalVideoFailed) && (\n <BrandGradientOverlay\n style={style}\n progress={progress}\n sceneDuration={sceneDuration}\n seed={gradSeed}\n />\n )}\n\n {showMedia && !hasExternalVideoBackdrop &&\n (resolved === \"video\" ? (\n <SceneVideoBackdrop\n mediaUrl={mediaUrl}\n mediaPoster={mediaPoster}\n mediaPosition={mediaPosition}\n backgroundEffect={backgroundEffect}\n progress={progress}\n sceneDuration={sceneDuration}\n beatIntensity={beatIntensity}\n isPlaying={isPlaying}\n onReady={() => setMediaPaint(\"ready\")}\n onError={() => setMediaPaint(\"failed\")}\n />\n ) : (\n <img\n src={mediaUrl}\n alt=\"\"\n aria-hidden=\"true\"\n draggable={false}\n data-media-position={mediaPosition}\n style={{\n position: \"absolute\",\n inset: 0,\n transform: bgTransform.transform,\n transformOrigin: bgTransform.transformOrigin,\n width: \"100%\",\n height: \"100%\",\n objectFit: \"cover\",\n objectPosition: resolvedPosition,\n }}\n />\n ))}\n\n {(hasExternalVideoBackdrop ? externalVideoReady : showTreatment) && !externalVideoFailed &&\n treatmentLayers.map((layer) => (\n <div\n key={layer.id}\n data-media-treatment={resolvedTreatment}\n data-media-overlay={layer.id}\n style={{\n position: \"absolute\",\n inset: 0,\n background: layer.background,\n pointerEvents: \"none\",\n ...layer.style,\n }}\n />\n ))}\n </>\n );\n};\n"
46
46
  },
47
47
  {
48
48
  "path": "src/visual-system/scene-templates/media-source.ts",