@vanillaskyai/video 0.10.22 → 0.11.0

package/docs/custom-templates.md

@@ -1,346 +0,0 @@
-[← Documentation home](../README.md) · [Previous: Customization](customization.md) · [Next: Motion and effects →](motion-and-effects.md)
-
-# Custom templates
-
-VanillaSky turns context into a visual response. Your AI determines what matters
-from the available context; a template defines how one kind of grounded answer
-appears in motion. Custom templates let your application own that visual
-vocabulary without letting the model generate React, HTML, or CSS.
-
-You edit one source file. VanillaSky derives the model-facing description,
-server validation, browser registry, and TypeScript variable shape from it.
-
-## The shortest path
-
-The playback and server package does not install a TypeScript compiler. Install
-the optional compiler only for source-owned templates and the template
-ownership commands below:
-
-```bash
-npm install --save-dev tsx
-```
-
-List the effective catalog, including project-owned templates, before choosing
-what to build or copy:
-
-```bash
-npx vanillasky templates list
-```
-
-Create an original template:
-
-```bash
-npx vanillasky templates create customer-health
-npx vanillasky templates describe customer-health
-```
-
-Or copy a built-in when its behavior is already close:
-
-```bash
-npx vanillasky templates add keyFigure
-```
-
-Then edit the owned `.tsx` file, regenerate the two small registries, and check
-the complete contract:
-
-```bash
-npx vanillasky templates sync
-npx vanillasky templates check
-```
-
-For an original template, the source is
-`vanillasky/templates/customer-health.tsx`. For the copied built-in, it is
-`vanillasky/templates/keyFigure.tsx`. These are application source: commit them,
-review them, and change them like any other React component.
-
-`create` and `add` run `sync` once, so the generated entry points exist
-immediately. Run `sync` again after every source edit. Run `check` before every
-commit. `check` validates metadata, schema/default validity, named examples, deterministic
-renders at progress boundaries in portrait and landscape, and browser/server
-registry parity.
-
-Preview either operation without applying the proposed file writes:
-
-```bash
-npx vanillasky templates add keyFigure --dry-run
-npx vanillasky templates add keyFigure --diff
-```
-
-`--dry-run` lists every proposed file and `--diff` shows its content changes,
-including the generated browser and server registries. The CLI does not apply
-any proposed file write. When project templates already exist, these previews
-execute that trusted application source to derive the complete registry plan.
-Because trusted project source can have its own side effects, preview execution is
-resource-bounded but is not a sandbox and does not guarantee that the entire
-project remains byte-identical. Review project template code before running
-preview commands.
-
-An edited file is never replaced unless you explicitly pass `--overwrite`.
-
-## One file is the contract
-
-The file created by `vanillasky templates create` is a complete working template. Keep
-these concerns together:
-
-- `useWhen` and `avoidWhen` tell the AI when the visual is appropriate;
-- `jobs` declares the narrative roles the template can fill, and decides
-  whether it may close a video;
-- `schema` defines allowed variables, validation, defaults, labels, and
-  grounding formats;
-- named `examples` provide complete deterministic preview values;
-- `component` receives only validated variables and render context;
-- raw `progress`, dimensions, and `safeZone` make semantic state deterministic
-  and layout safe in portrait and landscape.
-
-### Which templates may close a video
-
-Every complete plan ends on exactly one closer, and the runtime only accepts a
-closer whose `jobs` include `"ask"` or `"payoff"`. Marking any other template as
-the closer rejects that scene with `Scene template <id> cannot be used as a
-closer`, and the reserved closing time is given to a template that qualifies.
-
-Declare `jobs: ["ask"]` on a call to action and `jobs: ["payoff"]` on a
-celebration or resolution beat. A template that only presents evidence keeps
-`jobs: ["proof"]` and is never asked to close.
-
-The same rule reads the built-in catalog, so an application that wants to
-constrain how its videos end can compute the eligible IDs instead of guessing:
-
-```ts
-import { builtinTemplates } from "@vanillaskyai/video/templates/catalog";
-
-const closerTemplateIds = builtinTemplates
-  .filter((template) => template.jobs.some((job) => job === "ask" || job === "payoff"))
-  .map((template) => template.id);
-```
-
-Pacing uses the same signal: the planner reserves time for the closer whenever
-the selected catalog contains a template with one of those two jobs.
-
-Customer templates hard-cut by default. Opt into renderer-owned fades only
-for scenes that use the standard media-background variables, and only after
-both timing points are visually tested in portrait and landscape. Shared render
-black backgrounds and unchanged media do not crossfade:
-
-```tsx
-export default defineTemplate({
-  // ...the normal one-file contract
-  usesGlobalTransition: true,
-  transitionTiming: { entryReadyProgress: 0.2, holdProgress: 0.7 },
-  component: ({ progress, motionProgress = progress, ...props }) => {
-    // Both clocks cover the complete declared 0→1 scene duration.
-    // Use progress for facts/media/screens; motionProgress for presentation.
-    return <CustomerScene {...props} dataProgress={progress} motionProgress={motionProgress} />;
-  },
-});
-```
-
-`entryReadyProgress` must show recognizable content in verification captures.
-`holdProgress` is the audited complete-readable checkpoint before the
-template's own exit begins. These values document and test the choreography;
-they never advance or shorten the runtime clock. The SDK validates
-`0 <= entryReadyProgress < holdProgress <= 1`.
-
-When changed media crossfades, the incoming layer can be partly visible while
-raw `progress` is exactly zero. A custom template that counts or types a value
-from a synthetic placeholder must hide only that transient wrapper during the
-crossfade. Compare the formatted display value to the formatted final value so
-a sourced zero is not mistaken for a placeholder:
-
-<!-- verify:transition-semantic-value:start -->
-```tsx
-import type { CSSProperties } from "react";
-
-interface TransitionSemanticValueProps {
-  displayValue: string;
-  finalValue: string;
-}
-
-export function TransitionSemanticValue({
-  displayValue,
-  finalValue,
-}: TransitionSemanticValueProps) {
-  const isTransient = displayValue !== finalValue;
-
-  return <span
-    data-transition-semantic={isTransient ? "transient" : undefined}
-    style={isTransient ? {
-      visibility: "var(--vanillasky-transition-semantic-visibility, visible)" as CSSProperties["visibility"],
-    } : undefined}
-  >
-    {displayValue}
-  </span>;
-}
-```
-<!-- verify:transition-semantic-value:end -->
-
-The variable defaults to `visible`, so direct rendering and hard cuts are
-unchanged. Do not apply it to grounded headings, CTA copy, supplied media, or
-the whole scene: those must remain recognizable at `entryReadyProgress`.
-
-The only SDK import an original one-file template needs is public:
-
-```tsx
-import { defineTemplate } from "@vanillaskyai/video/templates";
-```
-
-Start from the generated file or one of the packaged references:
-
-| Content shape | Reference |
-| --- | --- |
-| One headline and supporting line | [Minimal text](../examples/custom-template/minimal-text.tsx) |
-| One exact metric and its change | [Structured data](../examples/custom-template/structured-data.tsx) |
-
-The [reference comparison](../examples/custom-template/README.md) explains when
-to choose each. Copy a file into `vanillasky/templates/`, change its ID and
-content contract, then run `sync` and `check`.
-
-## Connect the generated registries
-
-`sync` generates two files from the owned templates:
-
-```text
-vanillasky/
-├── templates/
-│   └── customer-health.tsx  # source you edit
-├── index.ts                 # browser registry, generated
-└── server.ts                # React-free model/validation registry, generated
-```
-
-The server registry connects custom template metadata to the prompt, model
-selection, and validator. Import it in the route that connects your model:
-
-```ts
-// src/video-route.ts
-import { generateText, streamText } from "ai";
-import { createVideoChatHandler } from "@vanillaskyai/video/server";
-import { videoModel } from "./video-model";
-import { templates } from "../vanillasky/server";
-
-export const handleVideoChat = createVideoChatHandler({
-  templates,
-  authorize: (request) => {
-    if (process.env.VANILLASKY_LOCAL_DEMO !== "1") return false;
-    const hostname = new URL(request.url).hostname;
-    return hostname === "localhost" || hostname === "127.0.0.1";
-  },
-  generateText: async ({ systemPrompt, userPrompt, signal }) => {
-    const result = await generateText({
-      model: videoModel,
-      system: systemPrompt,
-      prompt: userPrompt,
-      abortSignal: signal,
-    });
-    return result.text;
-  },
-  streamText: ({ systemPrompt, userPrompt, signal }) => streamText({
-    model: videoModel,
-    system: systemPrompt,
-    prompt: userPrompt,
-    abortSignal: signal,
-  }),
-});
-```
-
-The local-only authorization above is intentionally narrow. Supply its marker
-only from the development command; replace it with your application's session
-check before deployment, as shown in
-[Getting started](getting-started.md).
-
-Use the browser registry for generation and playback:
-
-```tsx
-// src/video-chat.tsx
-import { VideoChat } from "@vanillaskyai/video/react";
-import "@vanillaskyai/video/video-chat.css";
-import { templates } from "../vanillasky";
-
-export function App() {
-  return <VideoChat options={{ templates }} />;
-}
-```
-
-Adjust relative paths for your application layout. Do not edit `index.ts` or
-`server.ts`; `sync` replaces them. A matching ID replaces that built-in on both
-sides, a new ID extends the catalog, and every other built-in remains available.
-
-## Preview without an LLM
-
-Template rendering is deterministic. Preview a saved `Video` directly in your
-application so iteration uses your actual CSS, fonts, container, React version,
-and browser. This path calls no model and needs no server route:
-
-<!-- verify:custom-template-preview:start -->
-```tsx
-import type { Video } from "@vanillaskyai/video";
-import { VideoPlayer } from "@vanillaskyai/video/react";
-import { templates } from "../vanillasky";
-
-const savedVideo: Video = {
-  schemaVersion: "0.2",
-  orientation: "portrait",
-  scenes: [{
-    id: "customer-health-preview",
-    templateId: "customer-health",
-    variables: {
-      title: "Customer health is improving",
-      subtitle: "Activation increased after guided onboarding.",
-    },
-    timing: { fixedDuration: 5 },
-  }],
-  style: {},
-};
-
-export function TemplatePreview() {
-  return <VideoPlayer video={savedVideo} templates={templates} autoPlay={false} />;
-}
-```
-<!-- verify:custom-template-preview:end -->
-
-Change the saved variables to each named example, inspect both orientations,
-and play the scene to inspect its full progress range. This is also the
-production replay path for a completed chat turn saved from `chat.turns`.
-
-## Schema and grounding
-
-Use JSON Schema as the single source of truth.
-
-Property defaults are optional authoring and renderer smoke values; they do not
-need to form a complete scene. Named examples must resolve to complete valid
-scenes. They may inherit omitted values from property defaults. Never invent
-media URLs or actions just to supply a default. Put complete scene examples in
-the top-level `examples` array, and do not maintain parallel variable or default
-maps.
-
-Useful formats add grounding behavior:
-
-- `grounded-stat` accepts numeric or string quantities and requires an exact
-  quantity in raw input, preserving its unit;
-- `grounded-quote` requires the quote to exist verbatim in the input;
-- `uri` validates approved media URLs restored by the server;
-- `stock-media-keyword` is only for hosts that resolve stock media before a
-  scene is committed.
-
-Templates whose core proof needs a real statistic can add
-`"x-vanillasky": { "requiresStat": true }`. See the structured-data reference for a complete example.
-
-For media-backed chat scenes, copy the built-in `cinemaMedia` template and keep its
-`mediaKeyword`, `mediaUrl`, and `mediaType` contract. Configure `searchMedia` on
-the chat handler so the server resolves semantic searches into approved assets.
-The chat request does not accept a separate supplied-media input list.
-
-The automatic `opening` uses the built-in `chapterTitle` variable `title`.
-If you replace that template while using automatic openings, keep this contract.
-
-## What belongs where
-
-Your application owns the model, credentials, live context, custom template
-source, authentication, media authorization, and saved `Video` values.
-VanillaSky owns the generated planning instructions, streaming protocol,
-runtime validation, built-in template catalog, and player. The model selects a
-trusted template and fills its declared schema; it never writes executable UI
-code.
-
-For the exact request path and source locations, see
-[Architecture](architecture.md). For media policies and provider callbacks,
-see [Media and audio](media-and-audio.md).

package/docs/immersive-interface.md

@@ -1,86 +0,0 @@
-[← Documentation home](../README.md)
-
-# Immersive video chat
-
-`VideoChat` provides the complete conversation interface over an adaptive video
-canvas. Import its stylesheet once; no copied example UI or component setup is
-required.
-
-```tsx
-import { VideoChat } from "@vanillaskyai/video/react";
-import "@vanillaskyai/video/video-chat.css";
-
-export function App() {
-  return <VideoChat options={{ endpoint: "/api/video-chat" }} />;
-}
-```
-
-The [interactive component reference](reference/design-system.html) uses the
-same packaged stylesheet. It demonstrates navigation, settings, history,
-conversation input, subtitles, suggestions and interaction states without
-requesting a microphone or calling a model.
-
-## Watching and asking
-
-The video owns the canvas. The navigation contains the VanillaSky logo, new
-session, history, voice and settings. One floating field supports speech and
-text; playback, microphone and send controls stay beside the question.
-
-Beginning a question pauses the picture and narration together. Speech becomes
-a draft to review. Sending begins the next answer; canceling returns to the
-previous playback state. Canceling voice input also discards pending
-transcription so a late result cannot replace a new draft.
-
-The first subtitle hides the input unless editing, listening or keyboard focus
-requires it. Pointer activity or touch reveals the controls again. They hide
-after two seconds of inactivity during playback; paused playback and active
-controls remain accessible. A stationary pointer left over Send does not hold
-the input open. Settings can keep the input visible.
-
-## Subtitles and transcripts
-
-Subtitles sit on a dark backing over the picture. The complete current cue
-remains available without a line clamp or ellipsis. When the input appears,
-subtitles move upward; when it disappears, they return to the bottom baseline.
-The transition preserves space for both rows instead of abruptly repositioning
-the subtitle.
-
-Subtitle actions fade while inactive and return on hover, touch or keyboard
-focus. Expand opens the answer transcript in the same surface. Hide removes
-subtitles; the Subtitles switch in Settings restores them. The transcript is
-part of the video interface rather than a separate modal.
-
-## Settings and history
-
-Settings and history share one dark glass treatment. Watching preferences
-control subtitles and input visibility. Video creation and style preferences
-apply to the next question; provider capabilities determine available options.
-History replays saved answers without generating them again. New session
-retains up to ten sessions in memory, preserving completed answers for replay.
-This history does not provide server or durable persistence; storage remains
-application-owned. See [persistence](persistence.md).
-
-Suggestions appear in a horizontal media-card rail. Pointer hover and keyboard
-focus select a card; choosing it begins a follow-up answer. Focus outlines and
-hover movement remain inside the rail's available space.
-
-## Layout and access
-
-The foreground video preserves its scene ratio, so diagrams and text remain
-visible without cropping or distortion. The surrounding viewport remains black during playback.
-Display layout does not rewrite the saved response's orientation. See
-[responsive orientation](responsive-orientation.md) for saved videos and custom
-players.
-
-Primary control targets are at least 44px. Bottom spacing includes the device
-safe area. Keyboard focus stays visible, settings retain native switch and
-radio behavior, and popovers support Escape and outside-click dismissal.
-Reduced motion removes movement and fades; increased contrast strengthens
-surface opacity and borders.
-
-## Customization
-
-Use the existing `welcomeTitle`, `className` and `options` props to customize
-copy, scoped CSS properties and generated footage direction. There is no separate
-light/dark appearance picker. See [customization](customization.md) for examples.
-Use `useVideoChat` when the application needs to own the whole interface.

package/docs/maintainers/cinematic-migration.md

@@ -1,38 +0,0 @@
-# Cinematic contract migration
-
-The eight-template replacement is an approved breaking pre-1.0 change. Persisted Video schema is now `0.2`; event protocol is `0.6`.
-
-## Breaking changes
-
-- Removed `VideoInput.brand`, `VideoStyle.brand`, `VideoBrand`, `VideoBrandInput`, `VideoBackground`, and `resolveVideoBrand`. Graphics use black, white/neutral text and system typography. Naturally colored media is unaffected.
-- Removed all 28 old built-in IDs. Eight new IDs describe different schemas: `cinemaMedia`, `chapterTitle`, `focusCards`, `editorialTimeline`, `mobileMessage`, `comparison`, `quote`, `keyFigure`.
-- `parseVideo` rejects the old persisted `0.1` fixture with `unsupported_video_version`. The parser also rejects `style.brand` if a caller merely stamps the new version onto an old object.
-- Scene source intent uses `mediaKeyword` (up to 80 characters) and optional `mediaSource`. The host owns `mediaUrl`, `mediaPoster` and resolved `mediaType`. No gradient fallback mode. Full-bleed requires a resolvable intent or asset; Reach out can remain on black when media is unavailable.
-
-## Adoption
-
-Regenerate videos from the retained source and narration using the new catalog. Do not automatically relabel old IDs: before/after emojis, count-ups and factual quote/stat schemas have different meanings. Retain an older published SDK in a separate legacy playback boundary if historical exports must continue to render; do not pass those payloads into the new parser.
-
-Before (old persisted format):
-
-```json
-{"schemaVersion":"0.1","scenes":[{"id":"one","templateId":"media","variables":{"texts":"A new perspective","mediaType":"gradient"},"timing":{"fixedDuration":4}}],"style":{"brand":{"font":"Inter"}}}
-```
-
-After re-authoring that title as a chapter:
-
-```json
-{"schemaVersion":"0.2","scenes":[{"id":"one","templateId":"chapterTitle","variables":{"title":"A new perspective"},"timing":{"fixedDuration":4}}],"style":{}}
-```
-
-Evidence: persistence tests preserve the untouched old release fixture and assert rejection, accept current-style round trips, reject hidden/non-JSON input as before, and pin new checksums. Template tests cover both orientations, deterministic seeking, exact evidence text and single-decoder backdrop ownership. Final packed consumer and mobile playback checks remain release gates.
-
-## Render fonts
-
-The cinematic templates use `-apple-system, BlinkMacSystemFont, "Helvetica Neue", Roboto, Arial, sans-serif` at regular and medium weights. Apple devices keep native system typography. The existing `@vanillaskyai/video/video-chat.css` entry registers packaged Roboto v51 WOFF2 subsets as the fallback for environments without those fonts. Standalone player and source-owned template integrations should also import that stylesheet. No font is fetched from Google at runtime; browsers fetch local packaged subsets only when Roboto is selected for the rendered glyphs. The font assets include the SIL Open Font License and a source/hash manifest.
-
-Live native fonts and a Linux renderer’s Roboto have slightly different metrics. For repeatable exports, keep the browser, installed fonts, viewport and package version fixed, and await `document.fonts.ready` after mounting the final scene before capturing frames. This is a host export responsibility; the SDK does not add a separate export API or force downloaded fonts onto Apple devices.
-
-## Media-led follow-up
-
-The owner requested removing `focusCards`. Regenerate persisted videos using that ID; migrate parallel explanations to narration over footage rather than another bullet layout. The remaining seven templates retain their IDs. Comparison, quote, key figure and timeline accept optional standard media variables and remain readable on black when assets are absent. No provider callback signatures or host limits change.

package/docs/motion-and-effects.md

@@ -1,109 +0,0 @@
-[← Documentation home](../README.md) · [Previous: Custom templates](custom-templates.md) · [Next: Production →](production.md)
-
-# Motion and effects
-
-VanillaSky motion is deterministic. The model selects from controlled names;
-the SDK calculates the same frame from the same video JSON and playback time.
-No generated CSS, animation code, or arbitrary transition is executed.
-
-## Text archetypes
-
-Text archetypes own a complete entrance, hold, and exit lifecycle:
-
-- `subtle` — quiet supporting copy;
-- `typewriter` — character-by-character reveal;
-- `wordStagger` — sequential word build;
-- `slam` — short, high-energy impact;
-- `cinematic` — depth-led trailer movement;
-- `heroWord` — one dominant word at a time.
-
-Templates declare whether they have a tight or open text canvas. VanillaSky
-normalizes incompatible or unknown values to a safe default.
-
-## Background effects
-
-Templates that support background motion can use:
-
-- `static`;
-- `slow-zoom-in`;
-- `slow-zoom-out`;
-- `ken-burns`;
-- `drift`;
-- `pulse`;
-- `breathe`;
-- `slow-tilt`;
-- `camera-shake`.
-
-`static` is the default, so a media change cannot restart an implicit camera
-transform. Apply another effect explicitly when a still image or authored scene
-benefits from it; footage keeps its own native motion by default.
-
-## Scene continuity
-
-Generated videos use `crossfade` by default. Persisted or application-authored
-`Video` values can set `style.defaultTransition` to `crossfade` or `fade`; an
-undefined or unknown value keeps the hard-cut behavior. The player applies a fade
-only when two ranges are contiguous (allowing floating-point arithmetic noise)
-and both templates declare `usesGlobalTransition: true` with valid
-`transitionTiming` metadata. It is also conditional on the effective backdrop:
-the 300 ms outer crossfade runs only when the resolved background media changes.
-Scenes that share the black base, or the same resolved media backdrop, do
-not crossfade. This keeps one stable background visible while each template
-plays its own entrance, hold, and exit choreography.
-
-The player owns a fixed black backdrop beneath the scenes. Built-in renderers
-preload through the same component state used for playback. A cold custom
-renderer reveals this black base while loading. Resolved scene media covers it
-only while the media scene is active.
-
-During a changed-media overlap, the current scene continues to its exact end.
-The incoming component may be pre-mounted for media readiness, but remains
-frozen at its true initial frame (`progress === motionProgress === 0`) until its
-declared range begins. The current scene remains interactive and exposed to
-assistive technology until the exact timeline boundary; the preview layer
-remains inert throughout the overlap.
-
-Templates always receive raw semantic scene time as `progress`. Grounded
-numbers, media time, screen sequences, and other content state must use that
-clock. An opted-in template also receives `motionProgress`; throughout active
-playback it is the same complete `0→1` clock. The player never pre-advances,
-caps, rewinds, or skips a template's entrance, internal motion, exit, or
-terminal frame. `transitionTiming` records audited entry-ready and readable
-checkpoints for verification, but does not remap runtime time. Use
-`motionProgress` only for presentation and fall back to `progress` when it is
-absent. Templates opt out by default.
-
-An incoming changed-media fade can expose the template's initial frame while
-its raw semantic clock is still zero.
-Do not show a synthetic `0%`, `0x`, empty total, or another value that could be
-mistaken for sourced content. Keep the grounded frame and CTA visible, but mark
-only a transient value wrapper with
-`visibility: var(--vanillasky-transition-semantic-visibility, visible)`. The
-player hides that wrapper for the incoming preview and reveals it as soon as
-the same mounted scene becomes active. This guard does not change `progress`,
-media time, the component lifecycle, or final values.
-
-As soon as a streamed scene arrives, the player starts warming its template and
-backdrop. A changed-media scene is also mounted invisibly before its cut so the
-browser can attach and decode the real element in advance. This media preroll
-also protects intentional hard cuts; it does not pause or otherwise alter the
-visual or soundtrack clocks.
-
-Undefined or unknown transition names preserve a hard cut and unmodified local
-motion. Overlapping ranges also hard-cut. A timeline gap renders the fixed black
-background instead of replaying an earlier scene.
-
-## Reduced motion
-
-The player respects `prefers-reduced-motion`. Applications should keep a
-visible playback control and must not rely on motion alone to communicate a
-fact or state.
-
-## Preview the catalog
-
-The public [motion and effects gallery](https://vanillasky.ai/motion/) renders
-the real SDK effects with their exact configuration. Use it to choose a
-controlled effect, then keep the initial integration on the defaults unless a
-specific editorial need calls for an override.
-
-[← Documentation home](../README.md) · [Previous: Custom templates](custom-templates.md) · [Next: Production →](production.md)