@vanillaskyai/video 0.1.0

package/docs/branding-and-personalization.md

@@ -0,0 +1,87 @@
+[← Documentation home](../README.md) · [Previous: Input and first scene](input-and-first-scene.md) · [Next: Media and audio →](media-and-audio.md)
+
+# Branding and personalization
+
+Brand values style the whole response. Personalization shapes who the story is for.
+
+## Background: standard, preset, or custom
+
+Do nothing to use the standard `cosmic` background:
+
+```ts
+video.generate({ input });
+```
+
+Choose a curated gradient or solid by name when the standard does not fit:
+
+```ts
+video.generate({
+  input,
+  brand: { background: "twilight" },
+});
+```
+
+Curated gradients are `cosmic`, `horizon`, `twilight`, `meadow`, `velvet`,
+`flamingo`, `peach`, and `saffron`. Curated solids are `black`, `midnight`,
+`aubergine`, `coal`, and `navy`.
+
+Use exact colors only when a real brand or campaign requires them:
+
+```ts
+brand: { background: { colors: ["#112233", "#334455"] } } // gradient
+brand: { background: { color: "#070B20" } }                // solid
+```
+
+These choices set the background colors used by every trusted gradient-backed
+template and by the automatic generation cover. Templates own their
+composition and type treatment, so there is no separate visual-preset choice.
+
+## Semantic brand tokens
+
+Most applications can stop at `brand.background` and `brand.font`. Supply
+semantic colors only when a real design system requires exact values:
+
+```ts
+const brand = {
+  name: "Acme Cloud",
+  logoUrl: approvedLogoUrl,
+  font: "Inter",
+  scriptFont: "Caveat",
+  background: "twilight",
+  colors: {
+    primary: "#7C5CFC",
+    secondary: "#3D2A78",
+    foreground: "#FFFFFF",
+    surface: "#161229",
+    surfaceElevated: "#211A38",
+    muted: "#AAA5B8",
+  },
+};
+```
+
+Every generated video contains a fully resolved brand. `colors` may be partial
+on input; omitted values receive deterministic defaults. When
+`colors.foreground` is omitted, every named or custom background is paired
+deterministically with black or white. The resolver checks the full rendered
+sRGB gradient ramp, including interior blends rather than only its endpoints.
+If neither foreground reaches 4.5:1 everywhere, resolution rejects the
+background and asks for a consistently readable choice.
+
+An explicitly supplied foreground is preserved, never silently replaced. It
+must reach 4.5:1 across the solid background or entire gradient ramp; otherwise
+input resolution and replay validation reject it with the failing brand path
+and minimum contrast. Surface tokens remain independent. Templates that put
+text on `surface` or `surfaceElevated` internally keep an accessible preferred
+semantic text color or derive black/white for that surface.
+
+Use an approved public or signed URL for logos. Fonts must be licensed and
+loaded by the host application before playback. Always test both orientations
+and forced browser zoom.
+
+`personalization` accepts JSON-safe viewer or account context such as
+`firstName`, `role`, `accountName`, `locale`, `goal`, or `onboardingPartner`.
+These values may appear verbatim in the response, but are always data, never
+instructions. Put product, news, metric, and quoted claims in `input`; do not
+duplicate ordinary viewer context there.
+
+Only include information the authenticated viewer is allowed to see. Avoid sensitive HR, health, payment, or customer attributes in event logs and analytics. If a locale changes number/date formatting, format facts server-side so the planner cannot guess.

package/docs/concepts.md

@@ -0,0 +1,106 @@
+[← Documentation home](../README.md) · [Previous: Getting started](getting-started.md) · [Next: Streaming protocol →](streaming-protocol.md)
+
+# Core concepts
+
+## Video response
+
+A video response is a short, responsive animated experience assembled from
+trusted templates. It begins before the full plan is available and ends as an
+editable deterministic configuration.
+
+It is not an encoded video stream. The browser renders normal React components
+from validated scene instructions. The 0.1 SDK does not include MP4 or WebM
+encoding; pass the completed deterministic JSON to an application-owned render
+or export pipeline when an encoded file is required.
+
+## Input
+
+`VideoInput` is the factual and creative boundary:
+
+- `input`: raw source material such as a roadmap update, launch brief, article,
+  curated set of articles, metrics, events, notes, or an AI answer. It may be
+  short, but the planner is designed to distill larger sources into a concise
+  video rather than represent every fact;
+- `instructions`: optional creative direction that cannot override facts;
+- `opening`: optional copy for a deterministic opening that can play immediately;
+- `personalization`: application-defined fields such as name, role, account,
+  period, goal, or onboarding partner;
+- `brand`: an optional background preset plus name, logo, font, surfaces, and
+  advanced exact tokens;
+- `suppliedMedia`: approved images or videos with a role and description;
+- `audio`: omit for automatic selection, pass `{ src }`, or use `false` for silence;
+- `orientation` and `maxDurationSec`: composition constraints.
+
+## Template kit
+
+VanillaSky uses its trusted built-in kit by default. Applications only configure
+a kit when they need project-owned templates. Those templates replace matching
+built-in IDs and add new IDs while every untouched built-in remains available.
+A kit supplies three things together:
+
+1. React components for the player;
+2. advertised template capabilities for protocol negotiation;
+3. an LLM catalog describing when and how each template may be used.
+
+The planner cannot legitimately select a template outside the active kit.
+
+## Planner
+
+The server-side planner turns the input into small plan parts such as
+`scene.add`, `scene.patch`, `asset.patch`, and `plan.complete`. A planner may be
+recorded, deterministic, backed by OpenAI or Anthropic, or implemented with
+another streaming text model.
+
+The planner does not create public event IDs, checksums, revisions, or final
+snapshots. The runtime owns those guarantees.
+
+| Boundary | Type | Owner |
+| --- | --- | --- |
+| Planner parts | Internal validated plan data | Server/provider adapter |
+| Values read from `response.stream` | `VideoEvent` | SDK runtime |
+| Terminal editable result | `Video` via `response.result` | SDK reducer |
+
+A planner must never yield `VideoEvent` envelopes. It yields plan
+parts; the runtime validates them and creates the public event metadata.
+
+## Event stream
+
+The runtime emits ordered protocol events:
+
+```text
+response.start
+audio.set             optional, before any scene
+scene.add             supplied opening
+scene.add             generated body
+scene.add             generated body
+response.complete     exact terminal snapshot
+```
+
+Network transport uses SSE. Direct in-process integrations expose the same
+events as an async iterable.
+
+## Playback and buffering
+
+The player renders the first committed scene and continues through the known
+timeline. Audio can loop during a generation gap. A media-bearing scene should
+not be committed until its asset is ready; start generated content with a
+typography-led scene so useful playback does not wait on media lookup.
+
+Played scenes cannot be patched. This keeps playback deterministic and avoids a
+scene changing after the viewer already saw it.
+
+The same player accepts a completed value directly. After persisting the JSON,
+load it through `parseVideo(JSON.parse(storedJson))`, then render
+`<VideoPlayer video={savedVideo} />` to replay it without opening a stream or
+calling the model again. Supply `templates` only when that video uses
+customer-owned templates. See [Persistence and replay](persistence.md) for the
+complete storage contract.
+
+## Completion snapshot
+
+`response.complete` contains the exact reduced `Video` plus a
+checksum. Persist the snapshot when you need replay, resume, export, analytics,
+or an editing handoff. The event log can also be persisted and revalidated.
+
+Read the complete [Protocol 0.4](reference/protocol.md) and the separate
+[persisted Video 0.1 contract](persistence.md).

package/docs/custom-templates.md

@@ -0,0 +1,329 @@
+[← Documentation home](../README.md) · [Previous: Branding and personalization](branding-and-personalization.md) · [Next: Motion and effects →](motion-and-effects.md)
+
+# Custom templates
+
+VanillaSky turns context into a visual response. Your AI determines what matters
+from the available context; a template defines how one kind of grounded answer
+appears in motion. Custom templates let your application own that visual
+vocabulary without letting the model generate React, HTML, or CSS.
+
+You edit one source file. VanillaSky derives the model-facing description,
+server validation, browser registry, and TypeScript variable shape from it.
+
+## The shortest path
+
+The playback and server package does not install a TypeScript compiler. Install
+the optional compiler only for source-owned templates and the template
+ownership commands below:
+
+```bash
+npm install --save-dev tsx
+```
+
+List the effective catalog, including project-owned templates, before choosing
+what to build or copy:
+
+```bash
+npx vanillasky list
+```
+
+Create an original template:
+
+```bash
+npx vanillasky create customer-health
+npx vanillasky describe customer-health
+```
+
+Or copy a built-in when its behavior is already close:
+
+```bash
+npx vanillasky add bigNumber
+```
+
+Then edit the owned `.tsx` file, regenerate the two small registries, and check
+the complete contract:
+
+```bash
+npx vanillasky sync
+npx vanillasky check
+```
+
+For an original template, the source is
+`vanillasky/templates/customer-health.tsx`. For the copied built-in, it is
+`vanillasky/templates/bigNumber.tsx`. These are application source: commit them,
+review them, and change them like any other React component.
+
+`create` and `add` run `sync` once, so the generated entry points exist
+immediately. Run `sync` again after every source edit. Run `check` before every
+commit. `check` validates metadata, schema/default validity, named examples, deterministic
+renders at progress boundaries in portrait and landscape, and browser/server
+registry parity.
+
+Preview either operation without applying the proposed file writes:
+
+```bash
+npx vanillasky add bigNumber --dry-run
+npx vanillasky add bigNumber --diff
+```
+
+`--dry-run` lists every proposed file and `--diff` shows its content changes,
+including the generated browser and server registries. The CLI does not apply
+any proposed file write. When project templates already exist, these previews
+execute that trusted application source to derive the complete registry plan.
+Because trusted project source can have its own side effects, preview execution is
+resource-bounded but is not a sandbox and does not guarantee that the entire
+project remains byte-identical. Review project template code before running
+preview commands.
+
+An edited file is never replaced unless you explicitly pass `--overwrite`.
+
+## One file is the contract
+
+The file created by `vanillasky create` is a complete working template. Keep
+these concerns together:
+
+- `useWhen` and `avoidWhen` tell the AI when the visual is appropriate;
+- `schema` defines allowed variables, validation, defaults, labels, and
+  grounding formats;
+- named `examples` provide complete deterministic preview values;
+- `component` receives only validated variables and render context;
+- raw `progress`, dimensions, and `safeZone` make semantic state deterministic
+  and layout safe in portrait and landscape.
+
+Customer templates hard-cut by default. Opt into renderer-owned fades only
+for scenes that use the standard media-background variables, and only after
+both timing points are visually tested in portrait and landscape. Shared brand
+gradients and unchanged media do not crossfade:
+
+```tsx
+export default defineTemplate({
+  // ...the normal one-file contract
+  usesGlobalTransition: true,
+  transitionTiming: { entryReadyProgress: 0.2, holdProgress: 0.7 },
+  component: ({ progress, motionProgress = progress, ...props }) => {
+    // Both clocks cover the complete declared 0→1 scene duration.
+    // Use progress for facts/media/screens; motionProgress for presentation.
+    return <CustomerScene {...props} dataProgress={progress} motionProgress={motionProgress} />;
+  },
+});
+```
+
+`entryReadyProgress` must show recognizable content in verification captures.
+`holdProgress` is the audited complete-readable checkpoint before the
+template's own exit begins. These values document and test the choreography;
+they never advance or shorten the runtime clock. The SDK validates
+`0 <= entryReadyProgress < holdProgress <= 1`.
+
+When changed media crossfades, the incoming layer can be partly visible while
+raw `progress` is exactly zero. A custom template that counts or types a value
+from a synthetic placeholder must hide only that transient wrapper during the
+crossfade. Compare the formatted display value to the formatted final value so
+a sourced zero is not mistaken for a placeholder:
+
+<!-- verify:transition-semantic-value:start -->
+```tsx
+import type { CSSProperties } from "react";
+
+interface TransitionSemanticValueProps {
+  displayValue: string;
+  finalValue: string;
+}
+
+export function TransitionSemanticValue({
+  displayValue,
+  finalValue,
+}: TransitionSemanticValueProps) {
+  const isTransient = displayValue !== finalValue;
+
+  return <span
+    data-transition-semantic={isTransient ? "transient" : undefined}
+    style={isTransient ? {
+      visibility: "var(--vanillasky-transition-semantic-visibility, visible)" as CSSProperties["visibility"],
+    } : undefined}
+  >
+    {displayValue}
+  </span>;
+}
+```
+<!-- verify:transition-semantic-value:end -->
+
+The variable defaults to `visible`, so direct rendering and hard cuts are
+unchanged. Do not apply it to grounded headings, CTA copy, supplied media, or
+the whole scene: those must remain recognizable at `entryReadyProgress`.
+
+The only SDK import an original one-file template needs is public:
+
+```tsx
+import { defineTemplate } from "@vanillaskyai/video/templates";
+```
+
+Start from the generated file or one of the packaged references:
+
+| Content shape | Reference |
+| --- | --- |
+| One headline and supporting line | [Minimal text](../examples/custom-template/minimal-text.tsx) |
+| One exact metric and its change | [Structured data](../examples/custom-template/structured-data.tsx) |
+| An application-supplied image | [Supplied media](../examples/custom-template/supplied-media.tsx) |
+
+The [reference comparison](../examples/custom-template/README.md) explains when
+to choose each. Copy a file into `vanillasky/templates/`, change its ID and
+content contract, then run `sync` and `check`.
+
+## Connect the generated registries
+
+`sync` generates two files from the owned templates:
+
+```text
+vanillasky/
+├── templates/
+│   └── customer-health.tsx  # source you edit
+├── index.ts                 # browser registry, generated
+└── server.ts                # React-free model/validation registry, generated
+```
+
+The server registry connects custom template metadata to the prompt, model
+selection, and validator. Import it in the route that connects your model:
+
+```ts
+// src/video-route.ts
+import { openai } from "@ai-sdk/openai";
+import { streamText } from "ai";
+import { createVideoHandler } from "@vanillaskyai/video/server";
+import { templates } from "../vanillasky/server";
+
+export const handleVideo = createVideoHandler({
+  templates,
+  authorize: (request) => {
+    if (process.env.NODE_ENV !== "development") return false;
+    const hostname = new URL(request.url).hostname;
+    return hostname === "localhost" || hostname === "127.0.0.1";
+  },
+  streamText: ({ systemPrompt, userPrompt, signal }) => streamText({
+    model: openai(process.env.OPENAI_MODEL ?? "gpt-4.1"),
+    system: systemPrompt,
+    prompt: userPrompt,
+    abortSignal: signal,
+  }),
+});
+```
+
+The local-only authorization above is intentionally narrow. Replace it with
+your application's session check before deployment, as shown in
+[Getting started](getting-started.md).
+
+Use the browser registry for generation and playback:
+
+```tsx
+// src/video-composer.tsx
+import { VideoPlayer, useVideo } from "@vanillaskyai/video/react";
+import { templates } from "../vanillasky";
+
+export function VideoComposer() {
+  const video = useVideo({ templates });
+
+  return <>
+    <button onClick={() => void video.generate({ input: "Grounded answer" })}>
+      Generate
+    </button>
+    <VideoPlayer {...video.playerProps} />
+  </>;
+}
+```
+
+Adjust relative paths for your application layout. Do not edit `index.ts` or
+`server.ts`; `sync` replaces them. A matching ID replaces that built-in on both
+sides, a new ID extends the catalog, and every other built-in remains available.
+
+## Preview without an LLM
+
+Template rendering is deterministic. Preview a saved `Video` directly in your
+application so iteration uses your actual CSS, fonts, container, React version,
+and browser. This path calls no model and needs no server route:
+
+<!-- verify:custom-template-preview:start -->
+```tsx
+import type { Video } from "@vanillaskyai/video";
+import { VideoPlayer } from "@vanillaskyai/video/react";
+import { templates } from "../vanillasky";
+
+const savedVideo: Video = {
+  schemaVersion: "0.1",
+  orientation: "portrait",
+  scenes: [{
+    id: "customer-health-preview",
+    templateId: "customer-health",
+    variables: {
+      title: "Customer health is improving",
+      subtitle: "Activation increased after guided onboarding.",
+    },
+    timing: { fixedDuration: 5 },
+  }],
+  style: {
+    brand: {
+      font: "Inter",
+      scriptFont: "Caveat",
+      background: { type: "gradient", colors: ["#8711C1", "#2167E3"] },
+      colors: {
+        primary: "#00E5A0",
+        secondary: "#006BE5",
+        foreground: "#FFFFFF",
+        surface: "#0A0A14",
+        surfaceElevated: "#14152A",
+        muted: "#A7A6B0",
+      },
+    },
+  },
+};
+
+export function TemplatePreview() {
+  return <VideoPlayer video={savedVideo} templates={templates} autoPlay={false} />;
+}
+```
+<!-- verify:custom-template-preview:end -->
+
+Change the saved variables to each named example, inspect both orientations,
+and play the scene to inspect its full progress range. This is also the
+production replay path for a completed video saved from
+`await video.generate(...)`.
+
+## Schema and grounding
+
+Use JSON Schema as the single source of truth.
+
+Property defaults are optional authoring and renderer smoke values; they do not
+need to form a complete scene. Named examples must resolve to complete valid
+scenes. They may inherit omitted values from property defaults. Never invent
+media URLs or actions just to supply a default. Put complete scene examples in
+the top-level `examples` array, and do not maintain parallel variable or default
+maps.
+
+Useful formats add grounding behavior:
+
+- `grounded-stat` marks numeric statistical evidence for the planner; it does
+  not compare the value against raw input at runtime;
+- `grounded-quote` requires the quote to exist verbatim in the input;
+- `supplied-image` requires an image URL listed in `VideoInput.suppliedMedia`;
+- `uri` applies the supplied-media or server URL policy to an approved URL;
+- `stock-media-keyword` is only for hosts that resolve stock media before a
+  scene is committed.
+
+Templates whose core proof needs a real statistic can add
+`"x-vanillasky": { "requiresStat": true }`. See the structured-data and
+supplied-media references for complete examples.
+
+The automatic `opening` uses the built-in `notification` variables (`appName`
+and `message`). If you replace `notification` while using automatic openings,
+keep that variable contract.
+
+## What belongs where
+
+Your application owns the model, credentials, live context, custom template
+source, authentication, media authorization, and saved `Video` values.
+VanillaSky owns the generated planning instructions, streaming protocol,
+runtime validation, built-in template catalog, and player. The model selects a
+trusted template and fills its declared schema; it never writes executable UI
+code.
+
+For the exact request path and source locations, see
+[Architecture](architecture.md). For media policies and supplied-media input,
+see [Media and audio](media-and-audio.md).

package/docs/customization.md

@@ -0,0 +1,141 @@
+[← Documentation home](../README.md)
+
+# Customization
+
+## Background and semantic brand
+
+Omit brand configuration to use the standard `cosmic` background. Prefer a
+named curated choice over raw color work:
+
+```ts
+const brand = {
+  name: "Acme",
+  logoUrl: "https://cdn.acme.com/logo.svg",
+  font: "Inter",
+  scriptFont: "Caveat",
+  background: "twilight",
+  colors: {
+    primary: "#6D5EF5",
+    secondary: "#3D2A78",
+    foreground: "#FFFFFF",
+    surface: "#17122F",
+    surfaceElevated: "#231B42",
+    muted: "#A7A6B0",
+  },
+};
+```
+
+Gradient presets: `cosmic`, `horizon`, `twilight`, `meadow`, `velvet`,
+`flamingo`, `peach`, `saffron`. Solid presets: `black`, `midnight`,
+`aubergine`, `coal`, `navy`. When a named choice genuinely cannot express the
+brand, use `{ colors: ["#112233", "#334455"] }` for a custom gradient or
+`{ color: "#070B20" }` for a custom solid.
+
+`colors` may be partial; the resolver fills every semantic token before the
+video is emitted. With no foreground, named and custom backgrounds
+deterministically select black or white for at least 4.5:1 contrast across the
+full rendered sRGB ramp, including gradient interiors. A ramp that neither can
+cover is rejected. An explicit foreground is preserved and validated by the
+same invariant during input resolution and replay; low-contrast values are
+rejected with the failing path and minimum ratio. Elevated surfaces derive an
+accessible internal text treatment without changing the semantic foreground.
+Use an approved public or signed URL for logos, and never put a private storage
+credential in the config.
+
+## Global visual direction
+
+Leave visual direction unset to use VanillaSky's defaults, or set one coherent
+look for the completed video:
+
+```ts
+style: {
+  density: "airy",           // airy | normal | packed
+  motion: "calm",            // calm | normal | punchy
+  textArchetype: "cinematic",
+  backgroundEffect: "slow-zoom-out",
+}
+```
+
+These are defaults, not generated CSS. A validated scene may still select a
+more appropriate text or background treatment when its trusted template allows
+it.
+
+## Personalization
+
+`personalization` accepts JSON-safe application fields. Include only values the
+viewer is allowed to see. The system prompt treats them as context, not as
+instructions.
+
+Good fields include `firstName`, `role`, `accountName`, `period`, `goal`,
+`locale`, and `onboardingPartner`. Keep facts in `input` as well when they must
+appear in the story.
+
+## Opening
+
+`VideoPlayer` automatically shows a brand-colored generation cover until the
+first validated scene arrives. The cover is player state, not video content: it
+is never written to the event log, replay, or export.
+
+An optional opening is deterministic and should not wait for an LLM or remote media:
+
+```ts
+opening: "Joris, your Q2 recap is ready."
+```
+
+Use `opening` only for a genuine opening that should remain in the completed
+response. VanillaSky infers the scene ID, `notification` template, variables,
+and five-second timing. You do not need to create a generic loading scene.
+
+## Aspect ratio and responsive layout
+
+The player is responsive by default: it fills its container width. Templates
+and copy must work at either aspect ratio; orientation is not an AI-planning
+input and must not influence the selected templates or wording.
+
+`portrait` reserves a 9:16 response/export frame and `landscape` reserves 16:9.
+This input setting remains stable in the completed config. For an embed that
+should display landscape on desktop and portrait on mobile without changing the
+saved response, pass `orientation="auto"` to `VideoPlayer`; it responds
+to its container width. See [responsive orientation](responsive-orientation.md).
+
+## Supplied media
+
+Provide approved media with semantic descriptions:
+
+```ts
+suppliedMedia: [{
+  id: "product-dashboard",
+  url: "https://cdn.example.com/dashboard.png",
+  type: "image",
+  description: "Activation dashboard after the Q2 release",
+  role: "product",
+}]
+```
+
+Resolve searched media before generation and pass the approved result through
+`suppliedMedia`. See [media providers](media-and-audio.md#media-providers).
+
+## Soundtrack audio
+
+Pass `audio: { src }` for a specific track, omit `audio` to let the server choose
+synchronously from a preloaded catalog, or pass `audio: false` for silence.
+VanillaSky infers deterministic duration, volume, beat, and fade-out metadata.
+The soundtrack should continue across visual generation gaps and finish with
+the final scene. Narration, TTS, and speech synchronization are not part of the
+0.1 SDK contract.
+
+## Custom templates
+
+The built-in catalog needs no setup. Only source-owned templates need the
+optional local TSX compiler; install it once with `npm install --save-dev tsx`.
+Then use `npx vanillasky create <id>` for an original one-file template or
+`npx vanillasky add <builtin>` to copy a close built-in. Edit the owned file,
+run `npx vanillasky sync`, then run `npx vanillasky check` before committing.
+Pass the generated registry to the server and browser; project-owned IDs
+replace matching built-ins and new IDs extend the catalog.
+
+The model sees selection guidance and a schema, not component source. It chooses
+a trusted template and fills validated variables. Never evaluate model-authored
+React, HTML, CSS, or JavaScript on the live path.
+
+See [custom templates](custom-templates.md).