@vanillaskyai/video 0.10.22 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (128) hide show
  1. package/CHANGELOG.md +74 -1
  2. package/PUBLIC-API.md +45 -438
  3. package/README.md +57 -96
  4. package/dist/builtin-metadata-OT6V7TB4.js +8 -0
  5. package/dist/{chapter-title-2JVDU62E.js → chapter-title-2RCXX7SN.js} +1 -2
  6. package/dist/{chunk-PDQFQIQW.js → chunk-3EQ6PVWL.js} +1 -1
  7. package/dist/chunk-4G4JBMCM.js +37 -0
  8. package/dist/{chunk-7AA2JWHZ.js → chunk-5DOQTIMD.js} +6 -6
  9. package/dist/chunk-666HGTVZ.js +41 -0
  10. package/dist/{chunk-7M56IUUX.js → chunk-6Z3ID54H.js} +0 -18
  11. package/dist/{chunk-5BSJLT6H.js → chunk-E7FHBLAC.js} +16 -29
  12. package/dist/chunk-HIQM4J3U.js +30 -0
  13. package/dist/chunk-ISWQ6T5X.js +36 -0
  14. package/dist/{chunk-224QNWRA.js → chunk-K5J7ESRO.js} +1 -10
  15. package/dist/chunk-KMVRBUL5.js +50 -0
  16. package/dist/chunk-NM4CXXZY.js +25 -0
  17. package/dist/{chunk-44WND2VP.js → chunk-V2CP7PVY.js} +19 -63
  18. package/dist/{chunk-3O7OMMMF.js → chunk-WZESNEPT.js} +0 -1
  19. package/dist/{chunk-RXTN2CW6.js → chunk-Z3DLSLAJ.js} +1 -1
  20. package/dist/cinema-media-YZ2UANJG.js +365 -0
  21. package/dist/cli.js +154 -1753
  22. package/dist/{compose-video-BH5K6XWT.js → compose-video-H2YUQDUY.js} +4 -4
  23. package/dist/{events-B4YCc4vc.d.ts → events-BZAfl0Dm.d.ts} +1 -1
  24. package/dist/index.d.ts +2 -2
  25. package/dist/index.js +4 -6
  26. package/dist/preload-media-23JXUFCF.js +77 -0
  27. package/dist/react.d.ts +39 -15
  28. package/dist/react.js +1350 -739
  29. package/dist/scene-validation-SGLLLFY5.js +9 -0
  30. package/dist/server.d.ts +67 -117
  31. package/dist/server.js +262 -614
  32. package/dist/test.d.ts +2 -2
  33. package/dist/test.js +21 -21
  34. package/dist/{text-stream-LD364KBI.js → text-stream-SOVLYR2L.js} +2 -2
  35. package/dist/{types-BqB8zC9u.d.ts → types-CG-kPI81.d.ts} +3 -1
  36. package/dist/{types-DdZw4GRQ.d.ts → types-ht-Zw3Wv.d.ts} +1 -1
  37. package/docs/agent-integration.md +19 -64
  38. package/docs/architecture.md +82 -87
  39. package/docs/customization.md +49 -66
  40. package/docs/development.md +18 -17
  41. package/docs/errors.md +4 -4
  42. package/docs/getting-started.md +74 -93
  43. package/docs/media-and-audio.md +9 -7
  44. package/docs/performance.md +11 -6
  45. package/docs/persistence.md +3 -7
  46. package/docs/production.md +4 -15
  47. package/docs/prompt-and-input.md +1 -2
  48. package/docs/provider-integration.md +134 -181
  49. package/docs/reference/protocol.md +5 -14
  50. package/docs/reference/provider-adapters.md +21 -13
  51. package/docs/security.md +1 -14
  52. package/docs/testing.md +56 -123
  53. package/package.json +5 -27
  54. package/starters/video-chat/.env.example +12 -4
  55. package/starters/video-chat/.env.native.example +13 -0
  56. package/starters/video-chat/README.md +72 -11
  57. package/starters/video-chat/package.json +1 -1
  58. package/starters/video-chat/providers/text-native.ts +94 -0
  59. package/starters/video-chat/providers/text.ts +19 -0
  60. package/starters/video-chat/providers/transcription.ts +32 -0
  61. package/starters/video-chat/providers/video-custom.ts +45 -0
  62. package/starters/video-chat/providers/video-delivery.ts +57 -0
  63. package/starters/video-chat/providers/video-google.ts +48 -0
  64. package/starters/video-chat/providers/video-job.ts +103 -0
  65. package/starters/video-chat/providers/video-runway.ts +45 -0
  66. package/starters/video-chat/providers/video.ts +39 -63
  67. package/starters/video-chat/server.ts +3 -29
  68. package/starters/video-chat/vite.config.ts +2 -8
  69. package/styles/video-chat.css +8 -105
  70. package/dist/builtin-server-W4PLJUZ5.js +0 -8
  71. package/dist/catalog-types-WTbLP6Jh.d.ts +0 -78
  72. package/dist/check-runtime.d.ts +0 -15
  73. package/dist/check-runtime.js +0 -96
  74. package/dist/chunk-2E6T633S.js +0 -27
  75. package/dist/chunk-4YM2M62S.js +0 -13
  76. package/dist/chunk-4ZJLPHBV.js +0 -684
  77. package/dist/chunk-5JBMYQP6.js +0 -156
  78. package/dist/chunk-73NTSFFI.js +0 -81
  79. package/dist/chunk-EGVQODKU.js +0 -83
  80. package/dist/chunk-HFVNAPHZ.js +0 -35
  81. package/dist/chunk-IIN5M5HW.js +0 -697
  82. package/dist/chunk-IQMYK5DX.js +0 -133
  83. package/dist/chunk-IR44XKBI.js +0 -46
  84. package/dist/chunk-JKVOBTRO.js +0 -145
  85. package/dist/chunk-LVM5Q2DL.js +0 -68
  86. package/dist/chunk-M4QTEJTK.js +0 -709
  87. package/dist/chunk-QSBDB4J2.js +0 -16
  88. package/dist/chunk-R3XAOMKP.js +0 -29
  89. package/dist/chunk-SPVTJH3F.js +0 -24
  90. package/dist/chunk-YAHT3LST.js +0 -663
  91. package/dist/chunk-ZD2VTUYR.js +0 -28
  92. package/dist/cinema-media-HYCDG65Z.js +0 -11
  93. package/dist/comparison-XXAP4S4J.js +0 -36
  94. package/dist/editorial-timeline-VUQC6KPA.js +0 -42
  95. package/dist/key-figure-4TJK7HLT.js +0 -29
  96. package/dist/kit-DrRpdn0p.d.ts +0 -79
  97. package/dist/mobile-message-FIHAO6XC.js +0 -49
  98. package/dist/preload-media-LJXWKTWG.js +0 -57
  99. package/dist/quote-GRPIYKQJ.js +0 -32
  100. package/dist/system-prompt-AG26KJAA.js +0 -12
  101. package/dist/template-catalog.d.ts +0 -400
  102. package/dist/template-catalog.js +0 -6
  103. package/dist/templates.d.ts +0 -25
  104. package/dist/templates.js +0 -102
  105. package/dist/validate-4OUD2CLX.js +0 -10
  106. package/docs/concepts.md +0 -107
  107. package/docs/custom-templates.md +0 -346
  108. package/docs/immersive-interface.md +0 -86
  109. package/docs/maintainers/cinematic-migration.md +0 -38
  110. package/docs/motion-and-effects.md +0 -109
  111. package/docs/reference/design-system.html +0 -125
  112. package/docs/responsive-orientation.md +0 -37
  113. package/docs/streaming-protocol.md +0 -15
  114. package/examples/custom-template/README.md +0 -19
  115. package/examples/custom-template/minimal-text.tsx +0 -82
  116. package/examples/custom-template/structured-data.tsx +0 -104
  117. package/registry/items/backgrounds.json +0 -70
  118. package/registry/items/chapterTitle.json +0 -80
  119. package/registry/items/cinemaMedia.json +0 -136
  120. package/registry/items/comparison.json +0 -168
  121. package/registry/items/editorialTimeline.json +0 -172
  122. package/registry/items/keyFigure.json +0 -162
  123. package/registry/items/mobileMessage.json +0 -153
  124. package/registry/items/motion.json +0 -45
  125. package/registry/items/quote.json +0 -161
  126. package/registry/items/template-context.json +0 -31
  127. package/registry/items/theme.json +0 -47
  128. package/registry/items/typography.json +0 -47
package/CHANGELOG.md CHANGED
@@ -4,6 +4,79 @@ VanillaSky follows semantic versioning. This changelog begins with the 0.1 beta.
4
4
 
5
5
  ## Unreleased
6
6
 
7
+ ## 0.11.0
8
+
9
+ ### Integration and developer experience
10
+
11
+ - Add optional `resolveAnswer({ prompt, conversation, signal })` for completed existing-assistant answers. Validate input before invoking it; preserve cancellation and fail explicitly on unusable output.
12
+ - Separate bounded HTTP admission from response policy; stop oversized streamed bodies without waiting for unresponsive cleanup. Keep inbound limits separate from the SDK-owned answer envelope.
13
+ - Add small optional application branding controls with unchanged default UI.
14
+ - Update the architecture, quickstart and provider/UI recipes around footage and chapters. Remove retired internal prompt guidance and duplicate prose/source-structure assertions.
15
+ - Verify native existing-assistant integration through the installed npm artifact, with no React or provider frameworks required by the server entry.
16
+
17
+ ### Provider setup
18
+
19
+ - Keep providers application-owned: editable fal, Google, Runway and arbitrary-provider examples share the same client contract, not a fixed vendor list.
20
+ - Add `init --native` with plain HTTP text callbacks and no Vercel/Anthropic dependency. Keep the optional Vercel AI SDK starter.
21
+ - Separate video, speech and transcription setup; make `doctor` inspect the selected configuration. Bundle the native starter and all adapter helpers in npm.
22
+ - Submit video jobs once, retain identity, bound polling/ledger/cleanup callbacks and cancel unfinished work best-effort. Preserve terminal job records and native text finish reasons.
23
+ - Require application-owned video storage/delivery in direct-provider examples. No paid generation runs in CI.
24
+
25
+ ### Timing and preparation
26
+
27
+ - Reset an unexpected native video start during narration preparation so paused WebKit decoders cannot drift ahead and freeze when speech begins.
28
+ - Share a clip budget between planning and playback, with a 0.8-second narration tail and one bounded rewrite before any paid clip. If it cannot fit, keep the complete original narration on a chapter.
29
+ - Check measured speech and actual decoded footage; play normal footage at native speed without looping. Respect browser speech completion with bounded failure handling.
30
+ - Announce completed media ahead of ordered scene delivery. Limit actual decoding to active and next scenes on every browser; isolate speech/media preparation from session state.
31
+ - Add content-free duration, buffering, repeat and wait-reason diagnostics. Stop automatically retrying ambiguous cinematic responses.
32
+ - Pass requested duration, shot direction and an absolute deadline to application video callbacks; accept actual media duration in results. Allow long-running provider deadlines without client edits.
33
+
34
+ ### Breaking changes
35
+
36
+ The beta now focuses on footage and chapter opening/recovery. The default chat
37
+ experience and saved `cinemaMedia` / `chapterTitle` playback remain supported.
38
+ Custom renderers, the five other built-ins, all registry bundles, template CLI
39
+ commands, and `/templates` and `/templates/catalog` exports are removed.
40
+ Server template registries and custom-registry options are removed as well.
41
+ There is no compatibility loader for retired custom-template videos.
42
+
43
+ Before, applications could install and pass a custom registry:
44
+
45
+ ```tsx
46
+ import { createTemplateRegistry } from "@vanillaskyai/video/templates";
47
+ const templates = createTemplateRegistry({ templates: customScenes });
48
+ <VideoChat options={{ endpoint: "/api/video", templates }} />;
49
+ ```
50
+
51
+ ### Adoption
52
+
53
+ Use the built-in video-and-chapter experience. Remove registry imports, custom
54
+ template files, authoring scripts and their configuration from your application.
55
+ Use visual direction for the footage and the headless hook for custom chat UI:
56
+
57
+ ```tsx
58
+ import { VideoChat } from "@vanillaskyai/video/react";
59
+ import "@vanillaskyai/video/video-chat.css";
60
+ <VideoChat options={{ endpoint: "/api/video" }} />;
61
+ ```
62
+
63
+ Keep provider integrations application-owned. npm distribution and optional
64
+ Vercel AI SDK integration remain; the SDK does not require `ai`, Anthropic or
65
+ any video-provider package. Support is best-effort beta support for the focused
66
+ chat flow, not an expanding model compatibility matrix.
67
+
68
+ ## 0.10.23
69
+
70
+ - Stop shipping the maintainer-only cinematic migration note in the published
71
+ package. The changelog now links it on GitHub.
72
+ - Remove the unreferenced text-rendering and emoji modules from the source tree.
73
+ The bundle is unchanged; this code was already excluded from it.
74
+ - Correct the repository map in the architecture guide, and drop the stale
75
+ comments that described a text component the templates no longer use. Those
76
+ comments also shipped inside the installable `theme` registry item.
77
+ - Remove the CLI redirect for template command names that were renamed before
78
+ the package had users. Use `vanillasky templates <command>`.
79
+
7
80
  ## 0.10.22
8
81
 
9
82
  - Offer eight diverse homepage prompts with curated footage, a balanced fresh-page shuffle, and stable ordering when returning Home.
@@ -175,7 +248,7 @@ Before:
175
248
 
176
249
  ### Adoption
177
250
 
178
- Remove brand options and regenerate source-owned templates from the new catalog. Re-author or regenerate saved videos from retained source material; do not rename old IDs or change their version field blindly. See [cinematic migration](docs/maintainers/cinematic-migration.md).
251
+ Remove brand options and regenerate source-owned templates from the new catalog. Re-author or regenerate saved videos from retained source material; do not rename old IDs or change their version field blindly. See [cinematic migration](https://github.com/VanillaSkyAi/video/blob/main/docs/maintainers/cinematic-migration.md).
179
252
 
180
253
  ```tsx
181
254
  <VideoChat options={{ endpoint: "/api/video-chat" }} />
package/PUBLIC-API.md CHANGED
@@ -1,474 +1,81 @@
1
1
  # VanillaSky Video public API
2
2
 
3
- Status: current public beta contract.
4
-
5
- This document defines the API that may enter the fresh
6
- `@vanillaskyai/video` package. An export not listed here is internal. Tests and
7
- the packed-package verifier must fail if the final package adds or removes an
8
- export without changing this contract intentionally.
9
-
10
- `tests/fixtures/public-api-signatures.json` is the reviewed normalized
11
- declaration report for this surface. `npm run verify:api` checks the local build's
12
- public names, complete reachable signatures, and runtime and declaration
13
- dependency boundaries. `npm run verify:package` applies the same contract to the
14
- exact packed artifact. Regenerate the report only as part of an intentional
15
- public API review.
16
-
17
- ## Compatibility promise
18
-
19
- - Patch releases do not make breaking changes to documented APIs or the
20
- serialized `Video` schema.
21
- - A compatible later `0.x` minor needs no migration ceremony. A breaking
22
- pre-1.0 minor must document both `### Breaking changes` and `### Adoption` in
23
- its changelog section, with concrete fenced before and after code examples
24
- respectively.
25
- - An API prefixed with `experimental_` may change in a patch. Canonical examples
26
- must pin an exact package version when they use one.
27
- - Deprecated APIs remain usable until the next minor release. The `0.1`
28
- package starts without undocumented compatibility aliases.
29
- - The package is ESM-only, targets ES2022, and supports Node.js 22 or newer.
30
- - React is an optional peer dependency. Only `/react` and renderer definitions
31
- under `/templates` may depend on React.
32
- - Framework adapters are examples, not separate public APIs. The release suite
33
- verifies current Next.js and Vite production builds.
34
-
35
- The frozen public API surface report is compared on every pull request. It is
36
- intentionally conservative: every existing normalized declaration and reachable
37
- support declaration must remain exactly equal because the report cannot safely
38
- distinguish input and output positions.
39
- As a result, even optional field additions to an existing public
40
- type fail a patch gate; use a documented pre-1.0 minor unless a separately
41
- reviewed, direction-aware compatibility check can prove the change safe. New
42
- exports, wider supported peer ranges, and new optional peers remain additive.
43
-
44
- ## Environment boundaries
45
-
46
- | Entry point | Environment | May import React | May import Node built-ins |
47
- |---|---|---:|---:|
48
- | `@vanillaskyai/video` | Universal | No | No |
49
- | `@vanillaskyai/video/server` | Server | No | No required runtime built-ins |
50
- | `@vanillaskyai/video/react` | Browser/React | Yes | No |
51
- | `@vanillaskyai/video/templates` | Browser/React authoring | Yes | No |
52
- | `@vanillaskyai/video/templates/catalog` | Universal JSON metadata | No | No |
53
- | `@vanillaskyai/video/test` | Node test runners | No | Node test helpers allowed |
54
-
55
- The CLI is exposed separately as the `vanillasky` binary.
56
-
57
- ## Root
58
-
59
- The root contains serializable protocol types and pure helpers. It never starts
60
- a request, renders React, imports a provider, or accesses browser globals.
61
-
62
- ### Values
63
-
64
- - `getVideoDuration(video: Video): number`
65
- - `getSceneDuration(scene, metadata)`
66
- - `getSceneDurationBounds(scene, metadata)`
67
- - `getSpokenDuration(text)`
68
- - `parseVideo(value: unknown): Video`
69
- - `resolveVideoBrand(input?: VideoBrandInput): VideoBrand`
70
- - `VideoValidationError`
71
-
72
- ### Types
73
-
74
- - `Video`
75
- - `VideoAudio`
76
- - `VideoBackground`
77
- - `VideoBrand`
78
- - `VideoBrandInput`
79
- - `VideoOrientation`
80
- - `VideoScene`
81
- - `VideoStyle`
82
- - `VideoStyleOptions`
83
- - `VideoStatus`
84
- - `VideoValidationErrorCode`
85
- - `SceneDurationBounds`
86
-
87
- `VideoState` remains internal protocol reducer state. Browser consumers use the
88
- normalized fields returned by `useVideoChat` instead.
89
-
90
- ## Server
91
-
92
- The server entry point creates a customer-owned authenticated route. It accepts
93
- the result of Vercel AI SDK `streamText()` directly while retaining a small
94
- provider-neutral text-delta escape hatch.
95
-
96
- ### Values
97
-
98
- - `createVideoChatHandler(options)`
99
- - `createServerTemplateRegistry(options)`
100
-
101
- ### Types
102
-
103
- - `VideoChatCapabilities`
104
- - `VideoChatConversationTurn`
105
- - `VideoChatHandlerOptions`
106
- - `VideoChatMode`
107
- - `VideoChatWelcomeOptions`
108
- - `VideoChatWelcomePrompt`
109
- - `ServerTemplateRegistry`
110
- - `ServerTemplateMetadata`
111
- - `VideoFinishReason`
112
- - `VideoGenerationSummary`
113
- - `VideoProviderUsage`
114
- - `VideoWarning`
115
- - `VideoWarningCategory`
116
-
117
- ### Handler contract
118
-
119
- The chat contract includes two footage modes: `"cinematic"` (labelled AI video)
120
- and `"pexels"`. AI mode does not fall back to stock; either mode recovers missing
121
- footage to an authored chapter. Existing `"cinematic"` requests remain valid.
122
- `generatedClipDurationSec?: number` on the handler defaults to 5 and accepts 2–20;
123
- it must match the host adapter's requested duration. `VideoChatTurn.mode` is
124
- optional so older in-memory saved turns remain readable.
125
-
126
- A declared `data.video-chat-preparation` extension carries
127
- `{ sceneId: string, narration: string }` before media resolves. It permits bounded
128
- speech preparation, not playback of an unvalidated scene. The client limits
129
- speech preparation to two concurrent jobs and reuses it when `scene.add` arrives.
130
- The six public exports, persisted video schema and protocol version are unchanged.
131
-
132
- `createVideoChatHandler` is the opinionated general-purpose video-chat route.
133
- Mount it once and use its bounded `action` query parameter for capabilities,
134
- responses, opening media, narration, suggestions, speech,
135
- transcription, and the welcome screen. It owns the general response prompts,
136
- application-configured visual-mode attempt limits, capability fallbacks, and auxiliary response
137
- shapes. The application supplies provider-neutral `streamText`, `generateText`,
138
- `generateSpeech`, `transcribe`, `searchMedia`, and `generateVideo` callbacks.
139
- Only the two text callbacks are required. Missing optional callbacks remove
140
- their capability; templates and browser speech remain available. Failed or late
141
- footage becomes an authored chapter with complete narration. The default handler
142
- skips invalid planner parts and preserves playable scenes on an interrupted plan,
143
- emitting non-fatal warnings. Explicit `invalidPartBehavior: "fail"` retains strict
144
- generation semantics.
145
-
146
- - A response accepts `prompt`, `mode`, `orientation`, optional bounded
147
- `conversation`, `opening`, and `style`. `opening` is an optional
148
- prewritten hook from a selected suggestion. The response returns protocol
149
- `0.6` SSE and negotiates `data.video-chat-opening` and
150
- `data.video-chat-preparation` before the associated scene is ready.
151
- - The planner produces the opening and shots in one model stream, reserving the
152
- intended ending. Each shot supplies an authored recovery title, complete
153
- narration and visual direction. The default UI uses a chapter opening; the
154
- separate opening-media action remains available to custom consumers.
155
- - `cinematic` uses the application-owned `maxGeneratedVideos` limit (default
156
- five, nonnegative safe integer). It counts generation attempts, including
157
- failures. Zero skips generation and keeps authored chapters. Expired scene
158
- deadlines also skip generation without consuming an attempt. This limit is
159
- neither a currency nor duration cap. Pexels mode uses only `searchMedia` and
160
- never consumes this allowance. Capabilities expose Pexels when stock search
161
- is configured; AI mode remains usable with chapter recovery without video.
162
- - The response planner writes narration on each scene. The narration action is
163
- retained as a compatibility fallback for missing lines, not used in the
164
- normal path.
165
- - Every action applies the same authorization, origin, request-size,
166
- cancellation, safe-error, and server-only-provider boundaries.
167
-
168
- The chat handler requires `authorize`; `authorize: "none"` is only for an
169
- intentionally non-public in-process/test handler. All provider callbacks receive
170
- cancellation signals. The application owns authentication, origins, request
171
- limits, provider credentials, deadlines, and retry budgets.
172
-
173
- `onWarning` receives safe typed diagnostics; `onComplete` receives a server-only
174
- `VideoGenerationSummary` after an actual `response.complete`, including recovered
175
- playable responses. `onError` receives private internal errors. Observer failures
176
- do not change playback. Provider metadata remains server-side, with raw usage
177
- and metadata requiring the bounded `includeRawProviderData` opt-in.
178
-
179
- The handler exposes chat-relevant policy, template, and provider options only.
180
- Standalone soundtrack selection, snapshot-retention overrides, and durable
181
- stream replay are not public chat options.
182
-
183
- ## React
184
-
185
- ### Values
186
-
187
- - `VideoChat(props)`
188
- - `useVideoChat(options?)`
189
- - `createVideoChatVoice(options?)`
190
- - `VideoPlayer`
191
- - `VideoError`
192
-
193
- ### Types
194
-
195
- - `VideoChatProps`
196
- - `UseVideoChatOptions`
197
- - `UseVideoChatResult`
198
- - `VideoChatAskOptions`
199
- - `VideoChatFirstFrameMetric`
200
- - `VideoChatTurn`
201
- - `VideoChatStatus`
202
- - `VideoChatMode`
203
- - `VideoChatCapabilities`
204
- - `VideoChatWelcome`
205
- - `VideoChatSuggestion`
206
- - `VideoChatMedia`
207
- - `VideoChatVoice`
208
- - `VideoChatPreparedSpeech`
209
- - `CreateVideoChatVoiceOptions`
210
- - `VideoPlaybackMode`
211
- - `VideoPlayerProps`
212
- - `VideoErrorOptions`
213
-
214
- `VideoChat` is the complete default interface. Import its scoped stylesheet
215
- explicitly so the host application keeps control over CSS loading:
3
+ The 0.11 beta is a focused video-chat SDK. It supports footage (`cinemaMedia`) and chapter opening/recovery (`chapterTitle`). Custom renderers, registries, source installation and template authoring are no longer part of the product.
216
4
 
217
- ```tsx
218
- import { VideoChat } from "@vanillaskyai/video/react";
219
- import "@vanillaskyai/video/video-chat.css";
5
+ ## Supported entry points
220
6
 
221
- export function App() {
222
- return <VideoChat />;
223
- }
224
- ```
7
+ - `@vanillaskyai/video`: serializable video types, validation and duration helpers; no React, browser globals or providers.
8
+ - `@vanillaskyai/video/server`: `createVideoChatHandler` and its configuration/diagnostic types; no required provider or Node runtime dependencies.
9
+ - `@vanillaskyai/video/react`: `VideoChat`, `useVideoChat`, `VideoPlayer`, `createVideoChatVoice`, `VideoError` and their types.
10
+ - `@vanillaskyai/video/test`: deterministic `createMockVideoPlanner`, `simulateVideoStream`, `videoFixtures`.
11
+ - `@vanillaskyai/video/video-chat.css`: explicitly imported, scoped default UI styles.
12
+ - `vanillasky` executable: starter initialization, configuration diagnosis and application-owned provider setup.
225
13
 
226
- Pass the same session configuration through `options`, for example
227
- `<VideoChat options={{ endpoint, headers, templates }} />`. `className`
228
- and `welcomeTitle` are the only shell-level customizations. The stylesheet is
229
- scoped under `.vanillasky-video-chat` and does not style the host document.
14
+ The exact named exports are recorded in `tests/fixtures/public-api-surface.json`; normalized declarations and all reachable types are recorded in `public-api-signatures.json` beside it. Regenerate these only for an intentional contract change. `verify:api` checks a local build; `verify:package` checks the actual packed artifact and environment boundaries.
230
15
 
231
- `useVideoChat` is the headless client for `createVideoChatHandler`. It owns
232
- the conversation lifecycle and returns UI-neutral state plus a spread-ready
233
- player binding:
16
+ ## Beta compatibility and support
234
17
 
235
- ```tsx
236
- const chat = useVideoChat();
18
+ The package is ESM-only, targets ES2022 and supports Node.js 22+. React 18/19 are optional peers needed only by the React entry. No provider SDK is a runtime or peer dependency. npm distribution remains supported with best-effort beta maintenance, not an unlimited vendor/model compatibility promise.
237
19
 
238
- await chat.ask("Tell me a tiny mystery set on a night train");
20
+ Patch releases preserve documented APIs and saved video data. Breaking pre-1.0 minor releases explain removal and adoption in the changelog; 0.11 intentionally removes `/templates`, `/templates/catalog`, server template-registry exports and custom-registry options. No retired-feature aliases or execution machinery remain. Saved footage/chapter identifiers and replay remain supported; saved custom-template scenes are outside this release's supported contract.
239
21
 
240
- return chat.playerProps
241
- ? <VideoPlayer key={chat.playerKey} {...chat.playerProps} />
242
- : null;
243
- ```
22
+ ## Default server flow
244
23
 
245
- The hook loads capabilities and welcome suggestions, sends completed turns as
246
- bounded context, cancels replaced prompts, retries once only before playback,
247
- paces each scene to its prepared speech, and keeps pause, mute, replay, history,
248
- captions, and actual playback completion synchronized. The default voice tries
249
- the handler's generated-speech action and falls back to browser speech. Pass a
250
- `VideoChatVoice` to replace it without rebuilding session orchestration.
251
- `createVideoChatVoice({ onFallback })` optionally observes a generated-speech
252
- failure that selects browser speech. Observer exceptions and rejected promises
253
- are isolated from playback; no provider diagnostics are passed to this callback.
254
-
255
- `chat.warnings` exposes concise notices for the displayed turn, also retained
256
- as optional `VideoChatTurn.warnings`. The default interface recovers silently; these diagnostics are
257
- for application developers and are not rendered to viewers. Optional scene, narration, speech, or stream failures preserve
258
- playable output; an error is shown only when no playable response remains.
259
- `ask(prompt, { opening, openingMedia })` lets a custom interface start a
260
- prewritten suggestion hook immediately and reuse its image or video without
261
- another model or media lookup. The hook otherwise reads the opening from the
262
- response stream and starts the planned timeline only when opening speech and
263
- the contiguous prepared scenes are ready. It no longer requests opening stock
264
- automatically. `UseVideoChatOptions.onFirstFrame` receives a
265
- `VideoChatFirstFrameMetric` once after its first active scene commits and reaches
266
- an animation-frame opportunity. This measures presentation readiness, not physical
267
- screen paint or media decoding. Each `VideoChatTurn` exposes `openingMedia` and `completed`, so
268
- custom interfaces can render the same handoff while partial or cancelled
269
- responses remain visible without being mistaken for conversation context.
270
-
271
- `VideoPlayer` accepts either streaming player props or a completed saved video:
24
+ `createVideoChatHandler` mounts one endpoint. The `action` query selects capabilities, response, opening media, narration, suggestions, speech, transcription or welcome. Every action applies authorization, origin restrictions, bounded request bodies, cancellation and safe public errors.
272
25
 
273
- ```tsx
274
- <VideoPlayer {...chat.playerProps} />
275
- <VideoPlayer video={savedVideo} />
276
- ```
26
+ Applications supply required `streamText` and `generateText` callbacks, plus optional `generateSpeech`, `transcribe`, `searchMedia` and `generateVideo`. Text streams can be plain async iterables or structurally compatible AI SDK results. Vercel AI SDK is an optional application integration, not the core abstraction or a requirement.
277
27
 
278
- Built-in renderers are always available, so a streaming `VideoPlayer` does not
279
- require a template registry. `playerProps` is a spread-ready player binding; it
280
- contains the customer registry supplied to `useVideoChat` when one exists, but it
281
- does not expose the built-in planning catalog. Import `builtinTemplates` from
282
- `@vanillaskyai/video/templates/catalog` for labels, schemas, selection guidance,
283
- and other React-free metadata.
28
+ Optional `resolveAnswer({ prompt, conversation, signal })` returns an existing assistant's completed answer as a nonempty string, bounded to 32,000 characters after trimming and 30 seconds. It runs only after authorization and full input validation. Its output is the video planner's sole factual source, not a second assistant answer. Empty, oversized, failed or timed-out output returns `502 answer_unavailable`; cancellation returns `499 aborted`. This source constraint is model guidance, not automatic fact verification.
284
29
 
285
- The chat owns playback startup, speech, and synchronization. Custom saved-video
286
- players may choose a `VideoPlaybackMode` to match browser autoplay rules.
30
+ The response action accepts `prompt`, `mode`, `orientation`, optional bounded `conversation`, `opening` and `style`. The model produces a single creative brief followed by shots, reserving an authored ending. The server converts these into validated footage scenes or chapters; models do not choose renderer IDs or author lifecycle events.
287
31
 
288
- Set `nativeMediaAudio={{ volume: 0.85 }}` when scene video files contain an
289
- embedded audio track. The active clip's audio becomes a second layer alongside
290
- the video's continuous `audio` soundtrack. Both follow the player's master
291
- mute control; `nativeMediaAudio.volume` and serialized `audio.volume` set their
292
- independent mix levels. Incoming preroll videos remain muted until active.
32
+ Two visual modes remain: `cinematic` (AI video) and `pexels` (application-owned stock search). AI mode never silently substitutes stock. Missing/failed footage becomes an authored chapter without losing narration. The default UI opens on a chapter; the opening-media action is available to custom interfaces.
293
33
 
294
- Saved-video playback performs no generation request. `VideoPlayerBinding` and
295
- the internal reducer state are not public types.
34
+ `maxGeneratedVideos` bounds attempted generated clips, including failures (default five). Zero keeps chapter recovery. `mediaConcurrency` overlaps bounded jobs while preserving narrative order. `generatedClipDurationSec` must match the application adapter. `generateVideoTimeoutMs` bounds waiting, not provider charges. The application owns credentials, authentication, storage, delivery, billing and provider cancellation.
296
35
 
297
- `loop` and `onSceneChange` apply to saved-video playback. `loop` restarts a
298
- completed video from the beginning instead of showing the replay affordance;
299
- `onSceneChange(scene, index)` fires whenever the scene under the playhead
300
- changes, including when a loop wraps back to the first scene. Streaming
301
- playback is unaffected by either. `onComplete(video)` reports that a streamed
302
- response finished composing; `onPlaybackEnd(video)` reports that the visible
303
- playhead actually reached the end for either a stream or saved replay.
36
+ The video callback receives `requestedDurationSec`, `shotDirection` and an absolute `deadlineAt`, alongside its abort signal. Returned media may report `durationSec`. Configure the model, clip duration and timeout together in the application adapter; the server accepts timeouts up to ten minutes, while the client defaults to an eleven-minute response deadline. These are safety ceilings, not latency promises.
304
37
 
305
- Graphic scenes use fixed black backgrounds, white/neutral typography, and the system font stack. `VideoInput.brand`, `VideoStyle.brand`, `VideoBrand`, `VideoBrandInput`, `VideoBackground`, and `resolveVideoBrand` are removed. Media retains natural color. Persisted schema `0.2` and event protocol `0.6` reject earlier versions; changing a version field alone is not a migration.
38
+ Planning and playback share one conservative narration budget, reserving at least 0.8 seconds of quiet footage. Before buying a clip, the handler allows one bounded `generateText` call with task `narration-rewrite`. Oversized or failed rewrites retain the original narration on a chapter and do not buy a clip. Measured audio and decoded footage are checked again before playback; normal video runs at its native speed without repeats. Rewriting is model-assisted, not automatic fact verification.
306
39
 
307
- ## Template authoring
40
+ `authorize` is required; use `authorize: "none"` only for intentionally local/test handlers. `invalidPartBehavior: "drop"` preserves valid scenes after malformed planner output; `"fail"` opts into strict failure. Interrupted plans preserve playable partial answers and emit a safe warning. The narration action is a fallback for missing narration, not a second model call in the normal path.
308
41
 
309
- The template entry point owns React render definitions only. React-free server
310
- metadata registries are created through `/server`.
42
+ ## Streaming, replay and diagnostics
311
43
 
312
- ### Values
44
+ Responses use protocol 0.6 SSE. A declared `data.video-chat-opening` event announces the opening and `data.video-chat-preparation` allows bounded preparation ahead of ordered `scene.add`. Preparation alone never authorizes scene playback. Saved videos retain schema 0.2; parsing/checksums do not authenticate ownership or tenancy.
313
45
 
314
- - `defineTemplate(definition)`
315
- - `createTemplateRegistry(options)`
46
+ Completed media is announced immediately, even when an earlier shot is still generating. Byte warming uses a bounded queue; only the active and next scenes may allocate playback decoders. An ambiguous interrupted cinematic request is not automatically resubmitted.
316
47
 
317
- ### Types
48
+ `onError` receives private errors server-side. `onWarning` receives bounded safe warnings. `onComplete` receives a server-only generation summary after an actual completion, including recovered partial output. Raw provider usage/metadata requires explicit `includeRawProviderData`. Observer failures never affect generation.
318
49
 
319
- - `TemplateDefinition`
320
- - `TemplateExample`
321
- - `TemplateJsonSchema`
322
- - `TemplateFamily`
323
- - `TemplateTimingMetadata`
324
- - `TemplateTransitionTiming`
325
- - `TemplateRegistry`
326
- - `SceneTemplate`
327
- - `SceneTemplateMetadata`
328
- - `SceneTemplateProps`
329
-
330
- `SceneTemplateProps.progress` is always the raw `0 → 1` scene clock.
331
- Transition-enabled templates may use the optional `motionProgress` clock for
332
- presentation motion; the renderer-owned overlap never prevents that clock from
333
- reaching `1` across the complete template lifecycle. Their metadata must provide
334
- `transitionTiming.entryReadyProgress` and `transitionTiming.holdProgress`.
50
+ `onDiagnostic` and client `onPlaybackMetric` provide opt-in, local callbacks with opaque IDs, fixed reason codes and timing measurements. They contain no prompts, narration, asset URLs or credentials and perform no automatic reporting. First-frame measures a presentation opportunity; first-media-frame requires actual decoder proof. Speech onset uses playback events, not preparation estimates.
335
51
 
336
- Template timing uses `preferredDuration`; `duration` is not part of the
337
- package. `AuthoringTemplate` is inferred and internal.
52
+ Duration diagnostics distinguish estimated/rewritten narration from prepared speech and actual clip duration. Playback reports buffered seconds, native media/scene durations and repeat count. Stall reasons distinguish generation, speech and media decoding.
338
53
 
339
- ## Built-in catalog
54
+ ## React integration
340
55
 
341
- The catalog entry point is JSON-safe metadata. It does not contain renderers.
56
+ ```tsx
57
+ import { VideoChat } from "@vanillaskyai/video/react";
58
+ import "@vanillaskyai/video/video-chat.css";
342
59
 
343
- ### Values
60
+ export function App() {
61
+ return <VideoChat options={{ endpoint: "/api/video" }} />;
62
+ }
63
+ ```
344
64
 
345
- - `builtinTemplates`
65
+ The default experience includes loading states, captions, pause/mute, interruption, complete-turn history and replay. The real `npm run dev:chat` interface is the UI reference.
346
66
 
347
- ### Types
348
-
349
- - `BuiltinTemplateId`
350
- - `BuiltinTemplateMetadata`
67
+ `VideoChat` accepts optional `branding: { name, logo?, homeUrl?, showDeveloperLinks? }`. It changes application identity only; omitting it preserves the default UI. Home URLs must be HTTP(S) or root-relative; no external theme or template system is introduced.
351
68
 
352
- Template family and timing types have one canonical home under `/templates`.
353
-
354
- ## Test utilities
69
+ For an application-owned interface, call `useVideoChat(options)` and spread `chat.playerProps` onto `VideoPlayer`, keyed by `chat.playerKey`. A custom `VideoChatVoice` replaces voice output without replacing session orchestration. Its optional `onStart` callback must signal actual speech onset.
355
70
 
356
- The test entry point allows deterministic consumer tests without provider
357
- credentials or model spend.
358
-
359
- ### Values
360
-
361
- - `createMockVideoPlanner(options?)`
362
- - `simulateVideoStream(parts, options?)`
363
- - `videoFixtures`
364
-
365
- ### Types
71
+ Unclocked browser/custom speech owns its completion, with a bounded watchdog and quiet tail before advancing. Direct player integrations can supply `narrationActive(scene)` to preserve that behavior; the standard hook wires it automatically.
366
72
 
367
- - `MockVideoPlannerOptions`
368
- - `SimulatedVideoStreamOptions`
73
+ When wiring `onStallChange(stalled, reason)` manually, do not pause the voice for a `speech` wait: the player is waiting for speech to start or finish. Pause voice only for generation or decoding waits. The standard hook already applies this policy.
369
74
 
370
- The fixtures cover successful generation, delayed streaming, truncation,
371
- invalid scenes, provider failure, content filtering, abort, and timeout.
372
-
373
- ## CLI
374
-
375
- The `vanillasky` binary supports:
376
-
377
- - `init`
378
- - `doctor`
379
- - `templates list`
380
- - `templates describe`
381
- - `templates create`
382
- - `templates add`
383
- - `templates sync`
384
- - `templates check`
385
- - `templates add --dry-run`
386
- - `templates add --diff`
387
-
388
- `init` writes the canonical app-owned provider and configuration shell while
389
- using the SDK's packaged interface, session, handler, and built-in templates.
390
- `doctor` is read-only and offline. The old top-level template commands are not
391
- aliases; this is an intentional pre-1.0 namespace change.
75
+ `createVideoChatVoice` uses generated speech when configured and browser speech otherwise. Preparation duration may be estimated; measured generated audio supports offsets for narration groups. Cancellation releases stale voice/media work. Missing optional footage must not discard the spoken answer.
392
76
 
393
- Generated customer files import only the public entry points in this document.
394
- Browser registries import `createTemplateRegistry` from `/templates`. Server
395
- registries import `createServerTemplateRegistry` and `ServerTemplateMetadata`
396
- from `/server` without crossing a React type boundary.
397
-
398
- ## Serialized video
399
-
400
- - A completed `Video` is JSON-serializable and may be stored by the host.
401
- - Every completed value carries the required storage field
402
- `schemaVersion: "0.2"`; it is independent from streaming protocol `0.6`.
403
- - `parseVideo(value: unknown)` is the strict universal storage boundary. It
404
- validates the full document and returns a detached, deeply frozen `Video`.
405
- - JSON serialization remains platform-native; the SDK has no redundant public
406
- serializer.
407
- - Patch releases preserve round-trip compatibility.
408
- - The 0.1 contract supports the current schema only. It has no compatibility
409
- aliases or implicit coercions.
410
- - Invalid documents throw `VideoValidationError` with `invalid_video`.
411
- Unsupported future or unknown versions use `unsupported_video_version` and
412
- fail before any renderer runs; they are never rendered partially.
413
- - Raw prompts, provider payloads, and credentials are never retained by
414
- default.
415
- - Chat snapshots omit raw source, instructions, and the supplied-media URL
416
- index. Hosts own storage, tenant policy, deletion, and media URL expiry.
417
- - Replay through `<VideoPlayer video={savedVideo} />` never calls an LLM.
418
- - Completion checksums detect accidental drift only; they do not provide
419
- authenticity, authorization, or tenancy security.
420
-
421
- ## Intentionally excluded from 0.1
422
-
423
- - Provider-specific OpenAI or Anthropic wrapper clients.
424
- - Rendering/export infrastructure.
425
- - Hosted persistence.
426
- - OpenTelemetry integration.
427
- - Automatic factual verification or scene repair.
428
- - Standalone video generation hooks, handlers, narration, and timeline factories.
429
- - Undocumented API aliases.
430
-
431
-
432
- ## Playback measurements
77
+ `chat.warnings` are developer notices; the unchanged default shell stays quiet. `showRecoveryNotice` opts into a fixed dismissible recovery notice without exposing arbitrary provider messages.
433
78
 
434
- `UseVideoChatOptions.onPlaybackMetric` receives the exported
435
- `VideoChatPlaybackMetric` union. Every event contains an opaque `turnId`, `mode`,
436
- and nonnegative `elapsedMs` since the prompt was submitted:
79
+ ## Deliberately outside the maintenance scope
437
80
 
438
- - `first-frame`: first committed active scene at an animation-frame opportunity;
439
- - `first-media-frame`: first decoded frame reported by the mounted footage surface;
440
- - `first-speech`: actual speech playback onset, with `source` equal to `browser`,
441
- `generated`, or `custom`;
442
- - `stall`: a finished wait for the next prepared scene, with `durationMs` and
443
- `reason: "scene-generation"`.
444
-
445
- Metrics contain no prompt, narration, scene, URL, or provider diagnostics. There
446
- is no automatic network reporting. Use opaque identifiers when supplying
447
- `createTurnId`. Observer throws and rejected promises cannot affect playback.
448
- Replay and stale/cancelled callbacks do not create fresh response measurements.
449
- Pauses end a stall interval; paused time is excluded from stall duration.
450
-
451
- Custom `VideoChatVoice.speak` implementations can call their optional
452
- `onStart()` argument when audio actually begins. Without that signal, speech
453
- onset is unavailable, not zero. Built-in voices use browser utterance `start`
454
- and audio `playing` events, not request completion or preparation estimates.
455
-
456
- `VideoPlayerProps.onFramePresented` and `onStallChange` expose the underlying
457
- presentation and stream-starvation signals for custom players. They do not
458
- measure media-decoder buffering. See [Performance](docs/performance.md).
459
-
460
- ### Bounded generated-video previews
461
-
462
- `VideoChatHandlerOptions.generateVideoTimeoutMs` optionally sets the generated media
463
- resolver deadline (integer 1–120000 milliseconds, default 15000). The host still owns
464
- provider cancellation, authorization and spend limits; a timeout does not prove the
465
- provider avoided a charge. `maxGeneratedVideos` continues to count attempted clips.
466
-
467
- `VideoChatProps.showRecoveryNotice` opts into a dismissible, fixed SDK
468
- media-recovery notice; it never displays arbitrary provider messages.
469
-
470
- `VideoChatHandlerOptions.onDiagnostic` is an optional host-only observer for
471
- request, authoring and media phase timings with fixed recovery reason codes.
472
- It does not retain prompt, narration, query, asset URL or provider error text,
473
- and cannot affect response delivery. No new entry points or named exports are
474
- introduced; its additive signature is checked in the packed consumer contract.
81
+ No provider SDK wrappers in core, custom-template product, rendering/export service, hosted persistence, automatic fact verification, provider billing management or model compatibility matrix. Applications own delivery and provider-specific configuration. See the quickstart and provider recipes for supported reference integrations.