@vanillaskyai/video 0.10.22 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +74 -1
- package/PUBLIC-API.md +45 -438
- package/README.md +57 -96
- package/dist/builtin-metadata-OT6V7TB4.js +8 -0
- package/dist/{chapter-title-2JVDU62E.js → chapter-title-2RCXX7SN.js} +1 -2
- package/dist/{chunk-PDQFQIQW.js → chunk-3EQ6PVWL.js} +1 -1
- package/dist/chunk-4G4JBMCM.js +37 -0
- package/dist/{chunk-7AA2JWHZ.js → chunk-5DOQTIMD.js} +6 -6
- package/dist/chunk-666HGTVZ.js +41 -0
- package/dist/{chunk-7M56IUUX.js → chunk-6Z3ID54H.js} +0 -18
- package/dist/{chunk-5BSJLT6H.js → chunk-E7FHBLAC.js} +16 -29
- package/dist/chunk-HIQM4J3U.js +30 -0
- package/dist/chunk-ISWQ6T5X.js +36 -0
- package/dist/{chunk-224QNWRA.js → chunk-K5J7ESRO.js} +1 -10
- package/dist/chunk-KMVRBUL5.js +50 -0
- package/dist/chunk-NM4CXXZY.js +25 -0
- package/dist/{chunk-44WND2VP.js → chunk-V2CP7PVY.js} +19 -63
- package/dist/{chunk-3O7OMMMF.js → chunk-WZESNEPT.js} +0 -1
- package/dist/{chunk-RXTN2CW6.js → chunk-Z3DLSLAJ.js} +1 -1
- package/dist/cinema-media-YZ2UANJG.js +365 -0
- package/dist/cli.js +154 -1753
- package/dist/{compose-video-BH5K6XWT.js → compose-video-H2YUQDUY.js} +4 -4
- package/dist/{events-B4YCc4vc.d.ts → events-BZAfl0Dm.d.ts} +1 -1
- package/dist/index.d.ts +2 -2
- package/dist/index.js +4 -6
- package/dist/preload-media-23JXUFCF.js +77 -0
- package/dist/react.d.ts +39 -15
- package/dist/react.js +1350 -739
- package/dist/scene-validation-SGLLLFY5.js +9 -0
- package/dist/server.d.ts +67 -117
- package/dist/server.js +262 -614
- package/dist/test.d.ts +2 -2
- package/dist/test.js +21 -21
- package/dist/{text-stream-LD364KBI.js → text-stream-SOVLYR2L.js} +2 -2
- package/dist/{types-BqB8zC9u.d.ts → types-CG-kPI81.d.ts} +3 -1
- package/dist/{types-DdZw4GRQ.d.ts → types-ht-Zw3Wv.d.ts} +1 -1
- package/docs/agent-integration.md +19 -64
- package/docs/architecture.md +82 -87
- package/docs/customization.md +49 -66
- package/docs/development.md +18 -17
- package/docs/errors.md +4 -4
- package/docs/getting-started.md +74 -93
- package/docs/media-and-audio.md +9 -7
- package/docs/performance.md +11 -6
- package/docs/persistence.md +3 -7
- package/docs/production.md +4 -15
- package/docs/prompt-and-input.md +1 -2
- package/docs/provider-integration.md +134 -181
- package/docs/reference/protocol.md +5 -14
- package/docs/reference/provider-adapters.md +21 -13
- package/docs/security.md +1 -14
- package/docs/testing.md +56 -123
- package/package.json +5 -27
- package/starters/video-chat/.env.example +12 -4
- package/starters/video-chat/.env.native.example +13 -0
- package/starters/video-chat/README.md +72 -11
- package/starters/video-chat/package.json +1 -1
- package/starters/video-chat/providers/text-native.ts +94 -0
- package/starters/video-chat/providers/text.ts +19 -0
- package/starters/video-chat/providers/transcription.ts +32 -0
- package/starters/video-chat/providers/video-custom.ts +45 -0
- package/starters/video-chat/providers/video-delivery.ts +57 -0
- package/starters/video-chat/providers/video-google.ts +48 -0
- package/starters/video-chat/providers/video-job.ts +103 -0
- package/starters/video-chat/providers/video-runway.ts +45 -0
- package/starters/video-chat/providers/video.ts +39 -63
- package/starters/video-chat/server.ts +3 -29
- package/starters/video-chat/vite.config.ts +2 -8
- package/styles/video-chat.css +8 -105
- package/dist/builtin-server-W4PLJUZ5.js +0 -8
- package/dist/catalog-types-WTbLP6Jh.d.ts +0 -78
- package/dist/check-runtime.d.ts +0 -15
- package/dist/check-runtime.js +0 -96
- package/dist/chunk-2E6T633S.js +0 -27
- package/dist/chunk-4YM2M62S.js +0 -13
- package/dist/chunk-4ZJLPHBV.js +0 -684
- package/dist/chunk-5JBMYQP6.js +0 -156
- package/dist/chunk-73NTSFFI.js +0 -81
- package/dist/chunk-EGVQODKU.js +0 -83
- package/dist/chunk-HFVNAPHZ.js +0 -35
- package/dist/chunk-IIN5M5HW.js +0 -697
- package/dist/chunk-IQMYK5DX.js +0 -133
- package/dist/chunk-IR44XKBI.js +0 -46
- package/dist/chunk-JKVOBTRO.js +0 -145
- package/dist/chunk-LVM5Q2DL.js +0 -68
- package/dist/chunk-M4QTEJTK.js +0 -709
- package/dist/chunk-QSBDB4J2.js +0 -16
- package/dist/chunk-R3XAOMKP.js +0 -29
- package/dist/chunk-SPVTJH3F.js +0 -24
- package/dist/chunk-YAHT3LST.js +0 -663
- package/dist/chunk-ZD2VTUYR.js +0 -28
- package/dist/cinema-media-HYCDG65Z.js +0 -11
- package/dist/comparison-XXAP4S4J.js +0 -36
- package/dist/editorial-timeline-VUQC6KPA.js +0 -42
- package/dist/key-figure-4TJK7HLT.js +0 -29
- package/dist/kit-DrRpdn0p.d.ts +0 -79
- package/dist/mobile-message-FIHAO6XC.js +0 -49
- package/dist/preload-media-LJXWKTWG.js +0 -57
- package/dist/quote-GRPIYKQJ.js +0 -32
- package/dist/system-prompt-AG26KJAA.js +0 -12
- package/dist/template-catalog.d.ts +0 -400
- package/dist/template-catalog.js +0 -6
- package/dist/templates.d.ts +0 -25
- package/dist/templates.js +0 -102
- package/dist/validate-4OUD2CLX.js +0 -10
- package/docs/concepts.md +0 -107
- package/docs/custom-templates.md +0 -346
- package/docs/immersive-interface.md +0 -86
- package/docs/maintainers/cinematic-migration.md +0 -38
- package/docs/motion-and-effects.md +0 -109
- package/docs/reference/design-system.html +0 -125
- package/docs/responsive-orientation.md +0 -37
- package/docs/streaming-protocol.md +0 -15
- package/examples/custom-template/README.md +0 -19
- package/examples/custom-template/minimal-text.tsx +0 -82
- package/examples/custom-template/structured-data.tsx +0 -104
- package/registry/items/backgrounds.json +0 -70
- package/registry/items/chapterTitle.json +0 -80
- package/registry/items/cinemaMedia.json +0 -136
- package/registry/items/comparison.json +0 -168
- package/registry/items/editorialTimeline.json +0 -172
- package/registry/items/keyFigure.json +0 -162
- package/registry/items/mobileMessage.json +0 -153
- package/registry/items/motion.json +0 -45
- package/registry/items/quote.json +0 -161
- package/registry/items/template-context.json +0 -31
- package/registry/items/theme.json +0 -47
- package/registry/items/typography.json +0 -47
package/dist/test.d.ts
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { e as VideoScene, f as VIDEO_PROTOCOL_VERSION,
|
|
2
|
-
import { V as VideoWarning, a as VideoFinishReason } from './events-
|
|
1
|
+
import { e as VideoScene, f as VIDEO_PROTOCOL_VERSION, V as VideoOrientation, a as VideoStyle, b as Video, d as VideoAudio, j as VideoInput } from './types-ht-Zw3Wv.js';
|
|
2
|
+
import { V as VideoWarning, a as VideoFinishReason } from './events-BZAfl0Dm.js';
|
|
3
3
|
|
|
4
4
|
type MockVideoStreamPart = {
|
|
5
5
|
type: "scene.add";
|
package/dist/test.js
CHANGED
|
@@ -25,8 +25,8 @@ var portrait = {
|
|
|
25
25
|
type: "scene.add",
|
|
26
26
|
scene: {
|
|
27
27
|
id: "portrait-summary",
|
|
28
|
-
templateId: "
|
|
29
|
-
variables: {
|
|
28
|
+
templateId: "chapterTitle",
|
|
29
|
+
variables: { title: "Grounded product update" },
|
|
30
30
|
timing: { fixedDuration: 4 }
|
|
31
31
|
}
|
|
32
32
|
},
|
|
@@ -43,8 +43,8 @@ var landscape = {
|
|
|
43
43
|
type: "scene.add",
|
|
44
44
|
scene: {
|
|
45
45
|
id: "landscape-metric",
|
|
46
|
-
templateId: "
|
|
47
|
-
variables: {
|
|
46
|
+
templateId: "chapterTitle",
|
|
47
|
+
variables: { title: "Revenue reached 42 million" },
|
|
48
48
|
timing: { fixedDuration: 4 }
|
|
49
49
|
}
|
|
50
50
|
},
|
|
@@ -60,8 +60,8 @@ var truncated = [
|
|
|
60
60
|
type: "scene.add",
|
|
61
61
|
scene: {
|
|
62
62
|
id: "truncated-partial",
|
|
63
|
-
templateId: "
|
|
64
|
-
variables: {
|
|
63
|
+
templateId: "chapterTitle",
|
|
64
|
+
variables: { title: "Playable partial result" },
|
|
65
65
|
timing: { fixedDuration: 4 }
|
|
66
66
|
}
|
|
67
67
|
},
|
|
@@ -81,8 +81,8 @@ var invalidScene = [
|
|
|
81
81
|
type: "scene.add",
|
|
82
82
|
scene: {
|
|
83
83
|
id: "valid-after-invalid",
|
|
84
|
-
templateId: "
|
|
85
|
-
variables: {
|
|
84
|
+
templateId: "chapterTitle",
|
|
85
|
+
variables: { title: "Recovered after invalid content" },
|
|
86
86
|
timing: { fixedDuration: 4 }
|
|
87
87
|
}
|
|
88
88
|
},
|
|
@@ -99,8 +99,8 @@ var contentFilter = [
|
|
|
99
99
|
type: "scene.add",
|
|
100
100
|
scene: {
|
|
101
101
|
id: "content-filter-partial",
|
|
102
|
-
templateId: "
|
|
103
|
-
variables: {
|
|
102
|
+
templateId: "chapterTitle",
|
|
103
|
+
variables: { title: "Safe partial result" },
|
|
104
104
|
timing: { fixedDuration: 4 }
|
|
105
105
|
}
|
|
106
106
|
},
|
|
@@ -111,8 +111,8 @@ var abort = [
|
|
|
111
111
|
type: "scene.add",
|
|
112
112
|
scene: {
|
|
113
113
|
id: "abort-partial",
|
|
114
|
-
templateId: "
|
|
115
|
-
variables: {
|
|
114
|
+
templateId: "chapterTitle",
|
|
115
|
+
variables: { title: "Partial result before cancellation" },
|
|
116
116
|
timing: { fixedDuration: 4 }
|
|
117
117
|
}
|
|
118
118
|
},
|
|
@@ -123,8 +123,8 @@ var timeout = [
|
|
|
123
123
|
type: "scene.add",
|
|
124
124
|
scene: {
|
|
125
125
|
id: "timeout-partial",
|
|
126
|
-
templateId: "
|
|
127
|
-
variables: {
|
|
126
|
+
templateId: "chapterTitle",
|
|
127
|
+
variables: { title: "Partial result before timeout" },
|
|
128
128
|
timing: { fixedDuration: 4 }
|
|
129
129
|
}
|
|
130
130
|
},
|
|
@@ -233,10 +233,10 @@ async function* simulateVideoStream(parts, options = {}) {
|
|
|
233
233
|
if (timeoutMs != null && (!Number.isFinite(timeoutMs) || timeoutMs < 0)) {
|
|
234
234
|
throw new Error("Simulation timeoutMs must be a non-negative finite number");
|
|
235
235
|
}
|
|
236
|
-
const { createVideo } = await import("./compose-video-
|
|
237
|
-
const { createTextDeltaVideoPlanner } = await import("./text-stream-
|
|
238
|
-
const {
|
|
239
|
-
const {
|
|
236
|
+
const { createVideo } = await import("./compose-video-H2YUQDUY.js");
|
|
237
|
+
const { createTextDeltaVideoPlanner } = await import("./text-stream-SOVLYR2L.js");
|
|
238
|
+
const { SCENE_DEFINITIONS, getBuiltinSceneDefinition } = await import("./builtin-metadata-OT6V7TB4.js");
|
|
239
|
+
const { validateBuiltinScene } = await import("./scene-validation-SGLLLFY5.js");
|
|
240
240
|
const controller = new AbortController();
|
|
241
241
|
const forwardAbort = () => controller.abort(options.signal?.reason);
|
|
242
242
|
if (options.signal?.aborted) forwardAbort();
|
|
@@ -248,9 +248,9 @@ async function* simulateVideoStream(parts, options = {}) {
|
|
|
248
248
|
const run = createVideo(input, {
|
|
249
249
|
requestId: options.requestId ?? DEFAULT_REQUEST_ID,
|
|
250
250
|
runId: options.runId ?? DEFAULT_RUN_ID,
|
|
251
|
-
capabilities:
|
|
252
|
-
validateScene:
|
|
253
|
-
getTemplatePacing:
|
|
251
|
+
capabilities: { templates: SCENE_DEFINITIONS.map((scene) => scene.id) },
|
|
252
|
+
validateScene: validateBuiltinScene,
|
|
253
|
+
getTemplatePacing: getBuiltinSceneDefinition,
|
|
254
254
|
invalidPartBehavior: options.invalidPartBehavior ?? "drop",
|
|
255
255
|
signal: controller.signal,
|
|
256
256
|
generate
|
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
import {
|
|
2
2
|
createTextDeltaVideoPlanner
|
|
3
|
-
} from "./chunk-
|
|
3
|
+
} from "./chunk-3EQ6PVWL.js";
|
|
4
4
|
import "./chunk-E7CL7UPB.js";
|
|
5
|
-
import "./chunk-
|
|
5
|
+
import "./chunk-6Z3ID54H.js";
|
|
6
6
|
import "./chunk-7NZOMTAL.js";
|
|
7
7
|
export {
|
|
8
8
|
createTextDeltaVideoPlanner
|
|
@@ -15,6 +15,8 @@ interface VideoChatMedia {
|
|
|
15
15
|
url: string;
|
|
16
16
|
type: "image" | "video";
|
|
17
17
|
posterUrl?: string;
|
|
18
|
+
/** Actual media duration, not a narration estimate. */
|
|
19
|
+
durationSec?: number;
|
|
18
20
|
}
|
|
19
21
|
interface VideoChatSuggestion {
|
|
20
22
|
prompt: string;
|
|
@@ -43,4 +45,4 @@ interface VideoChatWelcomeOptions {
|
|
|
43
45
|
prompts?: readonly VideoChatWelcomePrompt[];
|
|
44
46
|
}
|
|
45
47
|
|
|
46
|
-
export type {
|
|
48
|
+
export type { VideoChatConversationTurn as V, VideoChatMode as a, VideoChatWelcomeOptions as b, VideoChatCapabilities as c, VideoChatWelcomePrompt as d, VideoChatAskOptions as e, VideoChatMedia as f, VideoChatSuggestion as g, VideoChatWelcome as h };
|
|
@@ -191,4 +191,4 @@ interface VideoSnapshotRetention {
|
|
|
191
191
|
suppliedMediaUrls?: boolean;
|
|
192
192
|
}
|
|
193
193
|
|
|
194
|
-
export { type
|
|
194
|
+
export { type VideoOrientation as V, type VideoStyle as a, type Video as b, type VideoCapabilities as c, type VideoAudio as d, type VideoScene as e, VIDEO_PROTOCOL_VERSION as f, type VideoTemplatePacing as g, type VideoNarrationGroup as h, type VideoStyleOptions as i, type VideoInput as j, type VideoPlanner as k, type VideoRequest as l, type VideoSceneValidator as m, type VideoSnapshotRetention as n, type VideoResumeCursor as o, type VideoGenerationContext as p };
|
|
@@ -1,71 +1,26 @@
|
|
|
1
|
-
[← Documentation home](../README.md) · [Next: Getting started →](getting-started.md)
|
|
2
|
-
|
|
3
1
|
# Agent integration
|
|
4
2
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
```bash
|
|
9
|
-
npx skills add VanillaSkyAi/video@vanillasky
|
|
10
|
-
```
|
|
11
|
-
|
|
12
|
-
Then prompt: `Use $vanillasky to set up and verify a general-purpose video chat in this project.`
|
|
13
|
-
|
|
14
|
-
## Set up the canonical chat
|
|
15
|
-
|
|
16
|
-
The skill follows this public path in an empty npm project:
|
|
17
|
-
|
|
18
|
-
```bash
|
|
19
|
-
npx @vanillaskyai/video init
|
|
20
|
-
npx vanillasky doctor
|
|
21
|
-
npm run dev
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
Init generates the thin application-owned shell around the SDK's complete
|
|
25
|
-
`VideoChat` experience and runs doctor automatically. The baseline uses an immediate introduction and browser voice, and installs
|
|
26
|
-
no optional provider packages. Add the video adapter for moving video answers;
|
|
27
|
-
without a media provider, narration and subtitles remain available.
|
|
28
|
-
`ANTHROPIC_API_KEY` is the only required key. Rerun init after an interrupted
|
|
29
|
-
installation.
|
|
30
|
-
|
|
31
|
-
If doctor reports a missing key, add the named key to the ignored `.env.local`
|
|
32
|
-
yourself. Never read or print a secret value, and never paste one into the chat.
|
|
33
|
-
Doctor is the safe capability interface: it reports names and readiness only.
|
|
34
|
-
|
|
35
|
-
For requested upgrades, run `npx vanillasky providers add speech` for xAI speech
|
|
36
|
-
or `npx vanillasky providers add video` for FAL video and transcription. Have the
|
|
37
|
-
developer add `XAI_API_KEY` or `FAL_KEY` locally, then restart the server. Stock
|
|
38
|
-
media needs only `PEXELS_API_KEY`. The client does not change.
|
|
39
|
-
|
|
40
|
-
## Verify, do not merely scaffold
|
|
41
|
-
|
|
42
|
-
The agent should leave the dev server running and open its reported localhost
|
|
43
|
-
URL in a real browser. Browser automation should use normal motion
|
|
44
|
-
(`reducedMotion: "no-preference"`) so playback advances.
|
|
45
|
-
|
|
46
|
-
1. Confirm the welcome screen, suggestions, composer, and voice control load
|
|
47
|
-
without console or page errors.
|
|
48
|
-
2. Ask one explanatory question and confirm the answer streams, speaks, reaches
|
|
49
|
-
its final frame, and returns to a usable composer.
|
|
50
|
-
3. Ask one unrelated creative question in the same conversation and confirm it
|
|
51
|
-
produces a distinct response.
|
|
52
|
-
4. If doctor reports generated video ready, verify moving footage throughout
|
|
53
|
-
one complete answer, including its ending, without editing client code.
|
|
54
|
-
5. Check failed network responses before reporting success.
|
|
3
|
+
Build the supported chat path first. The optional integration skill is installed
|
|
4
|
+
with `npx skills add VanillaSkyAi/video@vanillasky`.
|
|
55
5
|
|
|
56
|
-
|
|
57
|
-
|
|
6
|
+
In an empty app, run `npx @vanillaskyai/video init`, use
|
|
7
|
+
`npx vanillasky doctor` to identify missing setup, and start `npm run dev`.
|
|
8
|
+
For an existing app, keep its framework and connect `VideoChat` to one
|
|
9
|
+
`createVideoChatHandler` using the [provider guide](provider-integration.md).
|
|
10
|
+
Do not copy SDK internals or invent a second generation client.
|
|
58
11
|
|
|
59
|
-
|
|
12
|
+
The host adds credentials to ignored server-only environment files. Never read,
|
|
13
|
+
print, screenshot, or send secret values. Doctor exposes key names only.
|
|
60
14
|
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
15
|
+
Verify in a real browser: a complete answer, its ending, a follow-up, pause/mute,
|
|
16
|
+
and console/network failures. Test moving footage when a video adapter is
|
|
17
|
+
configured. Report the localhost URL and ready capabilities, not just scaffold
|
|
18
|
+
completion.
|
|
64
19
|
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
`
|
|
20
|
+
The SDK owns chat, shot planning, streaming, voice timing, and the two renderers.
|
|
21
|
+
The application owns providers, keys, auth, persistence, copy, and spending.
|
|
22
|
+
Use `useVideoChat` only when the host needs a custom UI; use `parseVideo` and
|
|
23
|
+
`VideoPlayer` for saved responses. Renderer plugins and template authoring are
|
|
24
|
+
not supported.
|
|
69
25
|
|
|
70
|
-
|
|
71
|
-
human references. Evaluation-only package checks remain in the maintainer docs.
|
|
26
|
+
[Getting started](getting-started.md) · [Documentation home](../README.md)
|
package/docs/architecture.md
CHANGED
|
@@ -1,97 +1,92 @@
|
|
|
1
1
|
# Architecture
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
3
|
+
The product is one video-chat pipeline: a chapter introduction, then narrated
|
|
4
|
+
footage, with chapter fallback when footage is unavailable. The application owns
|
|
5
|
+
providers and policy. There is no public template builder or template registry.
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## Follow one response
|
|
8
8
|
|
|
9
9
|
```text
|
|
10
|
-
|
|
11
|
-
→
|
|
12
|
-
|
|
13
|
-
→
|
|
14
|
-
→
|
|
10
|
+
VideoChat / useVideoChat
|
|
11
|
+
→ createVideoChatHandler (host policy, capabilities)
|
|
12
|
+
→ optional resolveAnswer (application's completed answer)
|
|
13
|
+
→ createChatShotPlanner (answer brief → ordered shots)
|
|
14
|
+
→ concurrent footage jobs + early preparation announcements
|
|
15
|
+
→ createVideo (validated protocol events)
|
|
16
|
+
→ browser speech/media preparation → ordered player + narration
|
|
15
17
|
```
|
|
16
18
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
19
|
+
Without `resolveAnswer`, shot planning begins directly from the request and
|
|
20
|
+
conversation. With it, the handler waits for the application's completed answer
|
|
21
|
+
before planning; that answer becomes the sole factual source. The hook is not
|
|
22
|
+
a stream of partially written assistant tokens.
|
|
21
23
|
|
|
22
|
-
|
|
24
|
+
Scenes can start before the full video plan is ready. Preparation overlaps;
|
|
25
|
+
early media announcements let the browser warm upcoming assets even while an
|
|
26
|
+
earlier shot is pending. Public scene order stays deterministic. The final Video is replayable data;
|
|
27
|
+
saved replay does not request another generation.
|
|
23
28
|
|
|
24
|
-
|
|
29
|
+
Narration is estimated against each clip with a 0.8-second tail. An oversized
|
|
30
|
+
beat gets at most one short rewrite before footage is requested. A failed or
|
|
31
|
+
still-oversized rewrite preserves the original speech in a chapter instead of
|
|
32
|
+
spending on unusable footage. Measured speech remains authoritative during
|
|
33
|
+
playback; video plays once and unexpected overruns recover to a chapter. Estimation is
|
|
34
|
+
not a guarantee that every voice/language finishes inside its clip.
|
|
35
|
+
|
|
36
|
+
## Where to work
|
|
37
|
+
|
|
38
|
+
| Change | Source |
|
|
25
39
|
| --- | --- |
|
|
26
|
-
| `src/
|
|
27
|
-
| `src/server/
|
|
28
|
-
|
|
|
29
|
-
| `src/server/
|
|
30
|
-
|
|
|
31
|
-
|
|
|
32
|
-
|
|
|
33
|
-
|
|
|
34
|
-
|
|
|
35
|
-
| `src/
|
|
36
|
-
| `src/visual-system/
|
|
37
|
-
|
|
|
38
|
-
|
|
|
39
|
-
|
|
|
40
|
-
|
|
|
41
|
-
|
|
|
42
|
-
|
|
43
|
-
The
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
## Where to change common behavior
|
|
81
|
-
|
|
82
|
-
- Model connection: `src/server/create-video-handler.ts`
|
|
83
|
-
- Base planning rules: `src/server/prompts/system-prompt.ts`
|
|
84
|
-
- Per-request context formatting: `src/server/prompts/user-prompt.ts`
|
|
85
|
-
- Template-specific prompt catalog: `src/visual-system/catalog/prompt.ts`
|
|
86
|
-
- Wire contract: `src/protocol/types.ts` and `src/protocol/events.ts`
|
|
87
|
-
- Scene rendering: `src/visual-system/scene-templates/`
|
|
88
|
-
- Background effects: `src/visual-system/backgrounds/` and
|
|
89
|
-
`src/visual-system/scene-templates/background-effect.ts`
|
|
90
|
-
- Text effects: `src/visual-system/scene-templates/text-archetypes.ts`
|
|
91
|
-
- Gradients and design tokens: `src/visual-system/theme/`
|
|
92
|
-
|
|
93
|
-
## Public vocabulary
|
|
94
|
-
|
|
95
|
-
Use `VideoChat`, `useVideoChat`, `createVideoChatHandler`, `Video`, and `VideoPlayer` for the product API. Use “video response” when
|
|
96
|
-
describing the lifecycle or output category. Use “motion” only for animation
|
|
97
|
-
behavior inside the visual system.
|
|
40
|
+
| Chat interface and lifecycle | `src/video-chat/` |
|
|
41
|
+
| HTTP admission, methods, CORS and bounded body reading | `src/server/video-chat-http.ts` |
|
|
42
|
+
| Response orchestration, host answer and provider callbacks | `src/server/create-video-chat-handler.ts` |
|
|
43
|
+
| Default planning instructions | `src/server/video-chat-prompts.ts` |
|
|
44
|
+
| Answer brief and shot planning | `src/server/chat-shot-planner.ts` |
|
|
45
|
+
| Validated composition and completion | `src/server/compose-video.ts` |
|
|
46
|
+
| Wire contract and reduction | `src/protocol/` |
|
|
47
|
+
| Turn-owned speech queue and media preparation | `src/video-chat/scene-preparation.ts` |
|
|
48
|
+
| Media readiness, timeline and narration | `src/player/` |
|
|
49
|
+
| Shared speech/clip budget | `src/protocol/clip-budget.ts` |
|
|
50
|
+
| Footage and chapter rendering | `src/visual-system/scene-templates/` |
|
|
51
|
+
| CLI setup and doctor | `src/cli/` |
|
|
52
|
+
| Application-owned provider examples | `starters/video-chat/` |
|
|
53
|
+
| Local real-chat harness | `dev/chat/` |
|
|
54
|
+
| Public entry points | `src/index.ts`, `src/server.ts`, `src/react.ts`, `src/test.ts` |
|
|
55
|
+
| Scoped UI styles and packaged fonts | `styles/` |
|
|
56
|
+
|
|
57
|
+
The root and test entry points are React-free. Server imports must not pull in
|
|
58
|
+
React; browser entries must not pull in Node or provider libraries.
|
|
59
|
+
[PUBLIC-API.md](../PUBLIC-API.md) and the packed API checks enforce that boundary.
|
|
60
|
+
|
|
61
|
+
## Prompts and ownership
|
|
62
|
+
|
|
63
|
+
The handler creates the real chat prompt from the opening, shot, pacing, and
|
|
64
|
+
visual-direction rules. Application `instructions` add product guidance;
|
|
65
|
+
the separate user prompt carries the request and bounded conversation.
|
|
66
|
+
The model emits structured directions, never React, HTML, or executable code.
|
|
67
|
+
Approved media URLs enter only through server callbacks.
|
|
68
|
+
|
|
69
|
+
Providers can return an async text iterable directly or an AI SDK-shaped result.
|
|
70
|
+
No particular host, model SDK, database, or storage vendor is required.
|
|
71
|
+
Authentication, spending limits, licensing, and persistent media delivery belong
|
|
72
|
+
to the application.
|
|
73
|
+
|
|
74
|
+
The optional `resolveAnswer({ prompt, conversation, signal })` accepts one
|
|
75
|
+
completed, nonempty string of at most 32,000 characters. It has a fixed,
|
|
76
|
+
abortable 30-second bound; invalid output or failure returns an explicit error,
|
|
77
|
+
not a newly invented answer. The video planner still needs a text callback to
|
|
78
|
+
turn that source into shots. See [integration](provider-integration.md).
|
|
79
|
+
|
|
80
|
+
The generated-video callback receives `requestedDurationSec`, `shotDirection`
|
|
81
|
+
and an absolute `deadlineAt`, alongside orientation, look and cancellation.
|
|
82
|
+
Its result can report `durationSec`. Model selection, supported duration and
|
|
83
|
+
resolution, concurrency, submission/polling and durable delivery stay in the
|
|
84
|
+
adapter. The SDK does not own provider jobs or a storage service.
|
|
85
|
+
|
|
86
|
+
Progressive scene delivery cannot remove a vendor's generation delay. A
|
|
87
|
+
minutes-long job API remains minutes-long even when the next scene is prepared
|
|
88
|
+
early. Scope is the beta npm SDK, its documented React/browser boundaries and
|
|
89
|
+
best-effort integration support—not a hosted video service or vendor uptime SLA.
|
|
90
|
+
|
|
91
|
+
See [development](development.md) for the fast edit loop and
|
|
92
|
+
[testing](testing.md) for deterministic public helpers.
|
package/docs/customization.md
CHANGED
|
@@ -1,25 +1,24 @@
|
|
|
1
|
-
[← Documentation home](../README.md)
|
|
2
|
-
|
|
3
1
|
# Customization
|
|
4
2
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
conversation field, on-video subtitles, and a single dark settings/history
|
|
9
|
-
treatment. See [interface behavior](immersive-interface.md) and the
|
|
10
|
-
[component reference](reference/design-system.html). Pass a custom welcome
|
|
11
|
-
heading and a root class when the application needs its own copy or chrome colors:
|
|
3
|
+
The default interface places footage behind a floating conversation field,
|
|
4
|
+
with on-video subtitles, pause/mute controls, suggestions, settings, and history.
|
|
5
|
+
Keep that interface unless your application needs to own it.
|
|
12
6
|
|
|
13
7
|
```tsx
|
|
14
8
|
<VideoChat
|
|
15
9
|
className="acme-chat"
|
|
16
10
|
welcomeTitle={<>Ask Acme<br />See the answer</>}
|
|
11
|
+
branding={{
|
|
12
|
+
name: "Acme",
|
|
13
|
+
logo: <img src="/acme-logo.svg" alt="" width={120} height={30} />,
|
|
14
|
+
homeUrl: "/app",
|
|
15
|
+
showDeveloperLinks: false,
|
|
16
|
+
}}
|
|
17
17
|
/>
|
|
18
18
|
```
|
|
19
19
|
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
that instance:
|
|
20
|
+
Import `@vanillaskyai/video/video-chat.css` once. Override its scoped variables
|
|
21
|
+
on the instance class, not global element selectors:
|
|
23
22
|
|
|
24
23
|
```css
|
|
25
24
|
.acme-chat {
|
|
@@ -30,72 +29,56 @@ that instance:
|
|
|
30
29
|
}
|
|
31
30
|
```
|
|
32
31
|
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
32
|
+
`welcomeTitle` changes the heading. `branding` changes only the navigation
|
|
33
|
+
identity: `name` supplies the accessible home-link label and becomes the visible
|
|
34
|
+
wordmark when `logo` is omitted. Size an app-owned image or React logo explicitly
|
|
35
|
+
to fit the header (about 120–144px wide and 30–36px high). Keep it non-interactive:
|
|
36
|
+
the surrounding home link already handles navigation.
|
|
37
|
+
|
|
38
|
+
Omit `branding` for the unchanged VanillaSky logo and interface. Omit `homeUrl`
|
|
39
|
+
even with custom branding to keep the existing Home behavior: at `/`, an ordinary
|
|
40
|
+
click starts a new session; elsewhere it navigates to `/`. An explicit `homeUrl`
|
|
41
|
+
navigates normally and never resets the current session first. Only root-relative
|
|
42
|
+
paths and HTTP(S) links without embedded credentials are accepted; other values
|
|
43
|
+
fall back to the existing Home behavior.
|
|
36
44
|
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
entire interface.
|
|
45
|
+
`showDeveloperLinks` defaults to true. It controls the SDK's Docs/About/GitHub
|
|
46
|
+
section in Settings, not an About page for your application. This is a small UI
|
|
47
|
+
option, not a theme system; it does not affect narration, providers or history.
|
|
41
48
|
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
49
|
+
The chapter
|
|
50
|
+
introduction uses black with neutral system typography. The supported scenes
|
|
51
|
+
are chapters and footage; there is no renderer-extension or brand-kit API.
|
|
45
52
|
|
|
46
|
-
|
|
53
|
+
Use `options` for endpoint, request headers, orientation, and voice.
|
|
54
|
+
Capabilities are discovered from the server. `useVideoChat` provides the same
|
|
55
|
+
conversation/playback lifecycle for application-owned controls.
|
|
47
56
|
|
|
48
|
-
|
|
49
|
-
and Reach out can show naturally colored footage; the other six templates
|
|
50
|
-
explain with their own composition and motion. There is no brand-kit option.
|
|
57
|
+
## Visual direction
|
|
51
58
|
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
the
|
|
59
|
+
Use server `instructions` for trusted audience, tone, or domain guidance.
|
|
60
|
+
A shared `style.generatedLook` can guide generated footage; the planner carries
|
|
61
|
+
consistent response-specific subjects and setting into each shot.
|
|
62
|
+
[Media and voice](media-and-audio.md) explains the look and adapter contract.
|
|
63
|
+
Style prompts cannot restyle stock assets.
|
|
56
64
|
|
|
57
|
-
## Opening
|
|
65
|
+
## Opening and layout
|
|
58
66
|
|
|
59
|
-
|
|
60
|
-
opening until its speech finishes and the first scene is ready. A selected
|
|
61
|
-
suggestion can start with its prewritten opening and already-loaded media:
|
|
67
|
+
A selected suggestion can start its prepared opening immediately:
|
|
62
68
|
|
|
63
69
|
```ts
|
|
64
70
|
await chat.ask(card.prompt, { opening: card.opening, openingMedia: card.media });
|
|
65
71
|
```
|
|
66
72
|
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
## Aspect ratio and responsive layout
|
|
70
|
-
|
|
71
|
-
The player is responsive by default: it fills its container width. Templates
|
|
72
|
-
and copy must work at either aspect ratio; orientation is not an AI-planning
|
|
73
|
-
input and must not influence the selected templates or wording.
|
|
74
|
-
|
|
75
|
-
`portrait` reserves a 9:16 response/export frame and `landscape` reserves 16:9.
|
|
76
|
-
This input setting remains stable in the completed config. For an embed that
|
|
77
|
-
should display landscape on desktop and portrait on mobile without changing the
|
|
78
|
-
saved response, pass `orientation="auto"` to `VideoPlayer`; it responds
|
|
79
|
-
to its container width. See [responsive orientation](responsive-orientation.md).
|
|
80
|
-
|
|
81
|
-
## Media and voice
|
|
82
|
-
|
|
83
|
-
Configure `searchMedia`, `generateVideo`, and `generateSpeech` on the server
|
|
84
|
-
handler. They progressively enhance the same chat; failed optional providers
|
|
85
|
-
fall back to templates or browser voice. See [Media and voice](media-and-audio.md).
|
|
86
|
-
|
|
87
|
-
## Custom templates
|
|
73
|
+
Typed prompts receive their opening from the same model stream as the answer.
|
|
74
|
+
The opening holds until its narration completes and the first scene is ready.
|
|
88
75
|
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
run `npx vanillasky templates sync`, then run `npx vanillasky templates check` before committing.
|
|
94
|
-
Pass the generated registry to the server and browser; project-owned IDs
|
|
95
|
-
replace matching built-ins and new IDs extend the catalog.
|
|
76
|
+
`portrait` reserves a 9:16 response frame; `landscape` reserves 16:9.
|
|
77
|
+
The saved orientation stays stable. For responsive display without changing the
|
|
78
|
+
saved response, `<VideoPlayer orientation="auto" />` follows container width.
|
|
79
|
+
Keep the player in a container with a usable width and height.
|
|
96
80
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
React, HTML, CSS, or JavaScript on the live path.
|
|
81
|
+
History in `VideoChat` is in memory. Durable storage belongs to the host;
|
|
82
|
+
see [persistence](persistence.md).
|
|
100
83
|
|
|
101
|
-
|
|
84
|
+
[Documentation home](../README.md)
|