@vanillaskyai/video 0.5.8 → 0.7.0

package/docs/custom-templates.md

@@ -24,28 +24,28 @@ List the effective catalog, including project-owned templates, before choosing
 what to build or copy:
 
 ```bash
-npx vanillasky list
+npx vanillasky templates list
 ```
 
 Create an original template:
 
 ```bash
-npx vanillasky create customer-health
-npx vanillasky describe customer-health
+npx vanillasky templates create customer-health
+npx vanillasky templates describe customer-health
 ```
 
 Or copy a built-in when its behavior is already close:
 
 ```bash
-npx vanillasky add bigNumber
+npx vanillasky templates add bigNumber
 ```
 
 Then edit the owned `.tsx` file, regenerate the two small registries, and check
 the complete contract:
 
 ```bash
-npx vanillasky sync
-npx vanillasky check
+npx vanillasky templates sync
+npx vanillasky templates check
 ```
 
 For an original template, the source is
@@ -62,8 +62,8 @@ registry parity.
 Preview either operation without applying the proposed file writes:
 
 ```bash
-npx vanillasky add bigNumber --dry-run
-npx vanillasky add bigNumber --diff
+npx vanillasky templates add bigNumber --dry-run
+npx vanillasky templates add bigNumber --diff
 ```
 
 `--dry-run` lists every proposed file and `--diff` shows its content changes,
@@ -79,7 +79,7 @@ An edited file is never replaced unless you explicitly pass `--overwrite`.
 
 ## One file is the contract
 
-The file created by `vanillasky create` is a complete working template. Keep
+The file created by `vanillasky templates create` is a complete working template. Keep
 these concerns together:
 
 - `useWhen` and `avoidWhen` tell the AI when the visual is appropriate;

package/docs/customization.md

@@ -2,6 +2,42 @@
 
 # Customization
 
+## Video chat interface
+
+The default `VideoChat` interface is already responsive and accessible. Pass a
+custom welcome heading and a root class when the application needs its own copy
+or chrome colors:
+
+```tsx
+<VideoChat
+  className="acme-chat"
+  welcomeTitle={<>Ask Acme<br />See the answer</>}
+  options={{ brand: acmeBrand }}
+/>
+```
+
+Override the scoped custom properties after importing
+`@vanillaskyai/video/video-chat.css`; the selectors and values stay local to
+that instance:
+
+```css
+.acme-chat {
+  --vs-accent: #0057ff;
+  --vs-accent-deep: #003fb8;
+  --vs-voice: #e11d74;
+  --vs-font: "Inter", sans-serif;
+  --vs-radius-lg: 1rem;
+}
+```
+
+Use `options` for the endpoint, templates, orientation, visual brand, request
+headers, and an optional custom voice. Provider capabilities are discovered
+from the server. Use `useVideoChat()` only when the application needs to own the
+entire interface.
+
+The sections below customize lower-level, one-shot video composition. They are
+not required for the default chat.
+
 ## Background and semantic brand
 
 Omit brand configuration to use the standard `cosmic` background. Prefer a
@@ -125,16 +161,16 @@ Pass `audio: { src }` for a specific track, omit `audio` to let the server choos
 synchronously from a preloaded catalog, or pass `audio: false` for silence.
 VanillaSky infers deterministic duration, volume, beat, and fade-out metadata.
 The soundtrack should continue across visual generation gaps and finish with
-the final scene. Narration, TTS, and speech synchronization are not part of the
-0.1 SDK contract.
+the final scene. `VideoChat` separately owns narration, TTS integration, browser
+voice fallback, and speech synchronization for conversational responses.
 
 ## Custom templates
 
 The built-in catalog needs no setup. Only source-owned templates need the
 optional local TSX compiler; install it once with `npm install --save-dev tsx`.
-Then use `npx vanillasky create <id>` for an original one-file template or
-`npx vanillasky add <builtin>` to copy a close built-in. Edit the owned file,
-run `npx vanillasky sync`, then run `npx vanillasky check` before committing.
+Then use `npx vanillasky templates create <id>` for an original one-file template or
+`npx vanillasky templates add <builtin>` to copy a close built-in. Edit the owned file,
+run `npx vanillasky templates sync`, then run `npx vanillasky templates check` before committing.
 Pass the generated registry to the server and browser; project-owned IDs
 replace matching built-ins and new IDs extend the catalog.
 

package/docs/getting-started.md

@@ -1,109 +1,106 @@
-[← Documentation home](../README.md) · [Previous: Prompt and input](prompt-and-input.md) · [Next: Next.js →](integrate-nextjs.md)
+[← Documentation home](../README.md) · [Next: Provider integration →](provider-integration.md)
 
 # Getting started
 
-Install VanillaSky:
+The fastest VanillaSky integration is the complete, general-purpose video chat.
+It starts with packaged templates and browser voice, then turns on optional
+speech, stock, transcription, and generated video when their server keys exist.
 
-```bash
-npm install @vanillaskyai/video
-```
+## Create the app
 
-VanillaSky does not select a provider or model during installation. Configure
-that separately on the server. The route below assumes your application exports
-an AI SDK `LanguageModel` as `videoModel`; if it does not, follow
-[Provider integration](provider-integration.md) first.
-
-Create one authenticated server route with `createVideoHandler`. Connect the
-app-owned model through `streamText`; keep credentials, authentication, rate
-limits, and media policy on the server:
-
-```ts
-import { streamText } from "ai";
-import { createVideoHandler } from "@vanillaskyai/video/server";
-import { videoModel } from "@/lib/video-model";
-
-const handle = createVideoHandler({
-  // Local development only. Replace with your session check before deploying.
-  authorize: (request) => {
-    if (process.env.VANILLASKY_LOCAL_DEMO !== "1") return false;
-    const hostname = new URL(request.url).hostname;
-    return hostname === "localhost" || hostname === "127.0.0.1";
-  },
-  streamText: ({ systemPrompt, userPrompt, signal }) => streamText({
-    model: videoModel,
-    system: systemPrompt,
-    prompt: userPrompt,
-    abortSignal: signal,
-  }),
-});
-
-export const POST = handle;
-export const OPTIONS = handle;
+Start in an empty folder:
+
+```bash
+npx @vanillaskyai/video init
 ```
 
-The local bypass is intentionally fail-closed: the packaged development command
-sets its marker only for `next dev`, and it accepts only localhost. Every
-production request is denied. Replace it with your real session validation
-before deploying. For literal files and commands,
-use the tested [`examples/nextjs-quickstart` directory](../examples/nextjs-quickstart).
+Init installs the exact SDK version that ran it, creates a small application
+shell, and installs its provider dependencies.
+It does not copy VanillaSky's template tree. The important generated files are:
 
-`videoModel` can come from any AI SDK provider, registry, gateway, compatible
-API, or custom implementation. The application can choose a cheaper, faster,
-or higher-quality model per request without changing VanillaSky.
+| File | Your application owns |
+| --- | --- |
+| `src/main.tsx` | The mount point for the SDK-owned chat |
+| `server.ts` | Provider choices and callbacks |
+| `stock.ts` | Optional stock search policy |
+| `vite.config.ts` | Local UI and `/api/video-chat` endpoint |
+| `.env.local` | Ignored server-only credentials |
 
-Call that route from React and render the player:
+The generated browser entry is intentionally tiny:
 
 ```tsx
-"use client";
+import { StrictMode } from "react";
+import { createRoot } from "react-dom/client";
+import { VideoChat } from "@vanillaskyai/video/react";
+import "@vanillaskyai/video/video-chat.css";
+
+createRoot(document.getElementById("root")!).render(
+  <StrictMode><VideoChat /></StrictMode>,
+);
+```
 
-import { VideoPlayer, useVideo } from "@vanillaskyai/video/react";
+The SDK owns the responsive interface, conversation state, suggestions, voice
+input, narration pacing, streaming player, and packaged templates. Your shell
+stays responsible for providers, keys, authorization, limits, storage,
+branding, and product copy.
 
-export function GeneratedVideo({ input }: { input: string }) {
-  const video = useVideo();
+## Add the one required key
 
-  return <>
-    <button onClick={() => { void video.generate({ input }); }}>Generate video</button>
-    {video.error && <p role="alert">Video generation failed.</p>}
-    <VideoPlayer {...video.playerProps} />
-  </>;
-}
-```
+Add the text-provider key to the generated, ignored `.env.local`:
 
-No template setup is required. VanillaSky advertises the trusted built-in
-catalog to your LLM, validates its selected scenes, and lazy-loads only the
-renderers the video uses.
+```dotenv
+ANTHROPIC_API_KEY=
+```
 
-`useVideo()` uses `/api/video` by default. Pass `endpoint` to use another route.
-`generate()` returns a `Promise<Video>` if you need the completed config directly:
+Fill the value locally; do not expose it through a client-prefixed environment
+variable. Then inspect the setup without calling any provider:
 
-```ts
-const completedVideo = await video.generate({ input });
+```bash
+npx vanillasky doctor
 ```
 
-The promise resolves only after successful completion. It rejects on terminal
-generation errors and aborts; `video.error` contains the same typed error for
-reactive UI.
+The base experience reports `templates + browser voice`. A ready text key makes
+the chat answer. Optional keys progressively add capabilities:
 
-`video.status` is `idle`, `streaming`, `complete`, `error`, or `aborted`.
-`video.video` is the latest deterministic video, and `video.warnings` contains
-bounded typed diagnostics safe to show or branch on. A playable response that
-stops at a planner length limit includes a `plan_incomplete` warning because
-requested scenes or the ending may be missing. Provider finish reasons and
-content-filter details remain available to the server through the `onComplete`
-summary; surface that server-owned state separately when completeness matters
-to your product.
+```dotenv
+# Optional generated speech
+XAI_API_KEY=
 
-Persist a completed `Video` as JSON and play it later without another model
-request:
+# Optional generated video and voice transcription
+FAL_KEY=
 
-```tsx
-<VideoPlayer video={savedVideo} />
+# Optional stock media
+PEXELS_API_KEY=
 ```
 
-Built-in templates work without additional props. Pass `templates` when the
-saved video uses customer-owned templates.
+Doctor reports only key names and readiness, never values. Adding or removing
+an optional key changes the available modes after a server restart; the client
+does not need to change.
+
+## Run and verify
+
+```bash
+npm run dev
+```
 
-All other input controls are optional. Continue with [input and opening
-scenes](input-and-first-scene.md), [branding and personalization](branding-and-personalization.md),
-or [media and soundtrack audio](media-and-audio.md). To edit or create visual
-building blocks, see [custom templates](custom-templates.md).
+Open the reported localhost URL. Try one explanatory question and one unrelated
+creative request in the same conversation. Confirm each response starts with a
+spoken hook from the response stream before the full plan is complete, holds its opening until the first
+scene is ready, reaches its final frame, and leaves the composer ready for
+another turn. With stock media enabled, click a welcome or follow-up card and
+confirm its footage carries directly into that opening. With generated video
+enabled, confirm the only choices are Templates and Full AI video and that the
+first generated shot continues the spoken hook without repeating it.
+
+The generated local authorization accepts localhost only. Replace it with your
+real session check, rate limits, and usage policy before deploying.
+
+## Continue
+
+- Change providers or add media capabilities in [Provider integration](provider-integration.md).
+- Brand or reshape the default experience in [Customization](customization.md).
+- Use a fully custom UI with the headless chat hook described in
+  [Provider integration](provider-integration.md#custom-interface).
+- Copy and edit a visual only when needed in [Custom templates](custom-templates.md).
+- Apply production authorization and key handling from
+  [Production](production.md) and [Security](security.md).

package/docs/media-and-audio.md

@@ -1,139 +1,135 @@
 [← Documentation home](../README.md) · [Previous: Branding and personalization](branding-and-personalization.md) · [Next: Custom templates →](custom-templates.md)
 
-# Media and soundtrack audio
+# Media, voice, and audio
 
-Applications own media and soundtrack audio. VanillaSky accepts ordinary
-authorized URLs in the deterministic video model; it does not bundle tracks or
-couple your app to a stock-media provider.
+VanillaSky keeps provider choice in the application. The SDK defines small
+server callbacks, advertises only the capabilities you configure, and keeps
+all credentials out of React and the browser bundle.
 
-The 0.1 SDK does not provide narration, TTS, or speech synchronization.
-If a product needs spoken audio, the application must create and synchronize
-that experience outside this contract.
+## Two visual modes
 
-Only send source URLs you trust. Pass known approved assets through
-`suppliedMedia`, or configure the server-only `resolveMedia` callback for
-planner-selected backgrounds. Preload the next scene's asset before it becomes
-active. Keep provider credentials on the server.
+The chat exposes only two clear choices:
 
-Supplied URLs and data URIs are not copied into the LLM prompt. The model sees
-an optional pool of opaque HTTPS-shaped references plus safe descriptive
-metadata, selects only relevant assets, and the SDK restores their original
-addresses on the server before scene validation. Supplying an asset does not
-require the completed video to use it.
+| Mode | Visual source | Required callback |
+| --- | --- | --- |
+| `templates` | Trusted rendered templates, optionally with stock footage | none; `searchMedia` is optional |
+| `full` | A generated clip for every visual beat | `generateVideo` |
 
-Audio timing, volume, beat markers, and fade-out remain part of the serialized
-output video, so replay and export stay deterministic. The input stays at the
-intent level: provide only the source URL and the SDK infers those playback
-defaults. Hosting and licensing are the application's responsibility.
+Templates are always available and are the fast, inexpensive fallback. The
+`full` mode is advertised only when `generateVideo` exists. VanillaSky does not
+offer a mixed mode that generates only some scenes.
 
-Host tracks in your application's public assets, object storage, or CDN, then
-pass their URL through the normal video input. For example, a file at
-`public/audio/calm.mp3` can be used without an SDK audio package:
+## Stock media for templates
+
+Add `searchMedia` when template answers, the welcome screen, and follow-up
+cards should use approved photography or footage:
 
 ```ts
-const input = {
-  input: "Grounded source material",
-  audio: { src: "/audio/calm.mp3" },
-  maxDurationSec: 24,
-};
+createVideoChatHandler({
+  authorize: verifySession,
+  streamText: planWithYourModel,
+  generateText: runSmallTextTask,
+  searchMedia: async (query, { purpose, orientation, signal }) => {
+    const asset = await searchApprovedCatalog({
+      query,
+      purpose,
+      orientation,
+      signal,
+    });
+    return asset
+      ? { url: asset.url, type: asset.type, posterUrl: asset.posterUrl }
+      : null;
+  },
+});
 ```
 
-The normalized output uses `trackId: "soundtrack"`, the video duration,
-default beat detection, an empty beat-marker list, full volume, and a
-three-second fade-out. Omit `audio` to let the server's synchronous
-`selectAudio` callback choose from an app-owned catalog. Pass `audio: false`
-to guarantee a silent video.
+The planner emits a short semantic keyword, not a URL. The callback returns an
+application-approved image or video URL, and the SDK validates it before it
+reaches a scene. Return `null` when no licensed, safe, relevant asset exists;
+the template falls back to its built-in treatment.
 
-Keep the catalog and files in your application so you control caching,
-licensing, and deployment. The SDK continues to handle playback, timing,
-serialization, replay, and export from the supplied URL.
+For Pexels, keep `PEXELS_API_KEY` on the server, enforce a deadline, filter for
+orientation, and return only validated Pexels asset domains. Licensing,
+attribution, caching, MIME checks, and byte limits remain application-owned.
 
-## Start with sound
+## Full generated video
 
-Browsers block audible autoplay unless the viewer has already interacted with
-the page. For a sound-first experience, keep the branded generation intro
-visible and let the player's start control provide that interaction:
-
-```tsx
-<VideoPlayer
-  {...video.playerProps}
-  playbackMode="autoplay-after-interaction"
-/>
-```
-
-The generation cover immediately includes a centered **Play with sound**
-button. A click starts the soundtrack and holds that branded cover as a
-three-second generation intro while planning continues. Once both the intro
-and the first validated scene are ready, the generated timeline begins from
-time zero. The generation intro remains visible indefinitely if the viewer has
-not clicked yet, even when the complete generated video is already ready. The
-server keeps the generated first scene on screen for at least three more
-seconds when the duration budget permits. After that first successful sound start,
-replacement streams on the same mounted player autoplay the same generation
-intro with sound and fall back to a start control if the browser blocks them.
-
-When `VideoInput.opening` is supplied, its asset-free gradient `media` scene
-replaces the generic generation cover as soon as it is available. It remains
-as the static start poster until the viewer clicks, then begins the actual
-timeline with sound; there is no additional generic pre-roll before it.
-
-Use `playbackMode="manual"` to require the button on every run,
-`playbackMode="muted-autoplay"` for browser-safe muted autoplay, or
-`playbackMode="autoplay-with-sound"` to try audible autoplay immediately. The
-lower-level `autoPlay` and `startMuted` props remain available when no playback
-mode is set. For a chat response that should try audible autoplay without the
-SDK generation intro, pass `opening: false`, wait to mount the player until a
-generated scene exists, and render it with `autoPlay` and `startMuted={false}`.
-If the browser blocks the audible start, the player returns to the first frame
-and exposes its sound-start control.
-
-For a saved video rendered with `<VideoPlayer video={video} loop />`, the
-soundtrack loops too. A track shorter than the visual timeline repeats without
-a silent gap, and the player restarts it from the beginning when the visual
-timeline wraps. This does not bypass browser autoplay policy: use muted autoplay
-for unattended playback and let a viewer unmute, or start audible playback from
-a user interaction.
-
-## Media providers
-
-VanillaSky is provider-independent. When `resolveMedia` is configured, the
-built-in planner may emit a bounded semantic query for later media-capable
-scenes. The SDK calls the application-owned resolver on the server, replaces
-the query with the approved URL, type, and optional poster, then validates the
-scene before emitting it. Without the callback, media intent stays hidden from
-the planner. The first accepted generated scene remains asset-free.
+Add `generateVideo` to enable full AI video. It receives the planned visual
+subject plus the generated look so every clip can follow the same direction:
 
 ```ts
-import { createVideoHandler } from "@vanillaskyai/video/server";
-
-createVideoHandler({
-  authorize: checkSession,
+createVideoChatHandler({
+  authorize: verifySession,
   streamText: planWithYourModel,
-  resolveMedia: async (query, { preferredType, signal }) => {
-    const asset = await searchYourApprovedCatalog({ query, preferredType, signal });
-    return asset
-      ? { url: asset.url, type: asset.type, posterUrl: asset.posterUrl }
-      : null;
+  generateText: runSmallTextTask,
+  generateVideo: async (subject, {
+    generatedLook,
+    orientation,
+    requestId,
+    scene,
+    signal,
+  }) => {
+    const asset = await generateAndStore({
+      subject,
+      generatedLook,
+      orientation,
+      requestId,
+      sceneId: scene?.id,
+      signal,
+      maxRetries: 0,
+    });
+    return asset ? { url: asset.url, type: "video" } : null;
   },
 });
 ```
 
-The resolver query is 2–80 characters and at most eight words. Return `null`
-when no licensed, safe, relevant asset exists; media-capable templates fall
-back to the brand gradient when their schema permits it. The browser never
-receives `mediaKeyword`, provider keys, or raw provider metadata.
-
-`allowMediaUrl` is an authorization hook for applications with their own custom
-stream adapter. It validates a final URL; it does not search for, fetch, or
-resolve media. The default 0.1 path needs no callback because every planner URL
-must already be present in `suppliedMedia`.
-
-Do not expose provider keys to React or allow arbitrary planner URLs. Templates
-describe visual building blocks; the application owns media retrieval,
-caching, licensing, and delivery.
-
-For Pexels, keep `PEXELS_API_KEY` on the server and implement `resolveMedia`
-with the Pexels API. Return only validated `images.pexels.com` or
-`videos.pexels.com` results. The application remains responsible for
-attribution, search, orientation filtering, MIME checks, timeouts, caching,
-and fallback behavior.
+The first streamed object reserves the first generated shot while the same
+model call continues planning later scenes. Use `requestId` and `scene.id` as
+an idempotency key, because generated clips are billable. Keep `maxRetries: 0`
+inside provider calls, honour `signal`, and make retries an explicit product
+decision with a known budget.
+
+VanillaSky does not depend on a video model or storage service. The application
+owns the provider key, model, spend, generated bytes, retention, and delivery.
+
+## Voice and transcription
+
+Without `generateSpeech`, `VideoChat` uses the browser voice. Add a speech
+callback for a consistent generated voice:
+
+```ts
+generateSpeech: async ({ text, signal }) => {
+  const speech = await synthesize(text, { signal });
+  return { audio: speech.bytes, mediaType: speech.mediaType };
+},
+```
+
+The SDK measures or estimates each line, keeps narration synchronized with the
+picture, and prevents a new scene from replacing speech that is still playing.
+If generated speech fails, the interface can fall back to browser speech.
+
+Add `transcribe` for server-side microphone transcription when browser speech
+recognition is unavailable. Set `maxAudioBytes`, validate the media type, and
+apply a provider deadline.
+
+## Native clip audio and soundtrack
+
+Generated clips can contain diegetic audio. The chat player keeps that audio
+separate from narration and uses one master mute control. Avoid generated
+voiceover or music inside clips so it does not compete with the answer voice.
+
+A serialized `Video` can also contain an application-owned soundtrack for
+replay or custom playback. Soundtrack files, licenses, beat markers, volume,
+and fade-out remain host-owned; narration and speech synchronization remain
+the chat layer's responsibility. Browser autoplay rules still require a viewer
+interaction before audible playback on many devices.
+
+## Safety rules
+
+- Keep every provider key in server-only environment variables.
+- Never let a planner return arbitrary final media URLs.
+- Bound query length, response size, duration, concurrency, and generated spend.
+- Preload the next asset and keep the current visual when media is late.
+- Return a safe fallback instead of leaving the response waiting forever.
+
+[← Documentation home](../README.md) · [Previous: Branding and personalization](branding-and-personalization.md) · [Next: Custom templates →](custom-templates.md)

package/docs/persistence.md

@@ -1,4 +1,4 @@
-[← Documentation home](../README.md) · [Previous: Core concepts](concepts.md) · [Next: Live channels →](live-channels.md)
+[← Documentation home](../README.md) · [Previous: Core concepts](concepts.md) · [Next: Streaming protocol →](streaming-protocol.md)
 
 # Persistence and replay