@vanillaskyai/video 0.6.0 → 0.7.0

package/docs/provider-integration.md

@@ -1,49 +1,96 @@
-[← Documentation home](../README.md) · [Previous: Next.js](integrate-nextjs.md) · [Next: Input and first scene →](input-and-first-scene.md)
+[← Documentation home](../README.md) · [Previous: Getting started](getting-started.md) · [Next: Branding and personalization →](branding-and-personalization.md)
 
 # Provider integration
 
-The model connection enters the SDK through `streamText` in
-`src/server/create-video-handler.ts`. The SDK does not instantiate or choose a
-model; the application passes the generated `systemPrompt` and `userPrompt` to
-its provider here.
+Start from the generated chat so provider work stays confined to the
+application-owned server:
 
-Use the AI SDK as the normal app-owned provider adapter. Any AI SDK
-`LanguageModel` works. Keep selection in one server-only module so OpenAI and
-Anthropic use the same route and React component:
+```bash
+npx @vanillaskyai/video init
+npx vanillasky doctor
+npm run dev
+```
+
+The generated `server.ts` is the canonical provider example. It starts with one
+text key, packaged templates, and browser voice. Adding the optional generated
+speech, stock, transcription, or generated-video key enables its callback and
+advertises the capability automatically; `src/main.tsx` does not change.
+
+For the full chat experience, mount one `createVideoChatHandler` and keep every
+provider choice in its callbacks:
 
 ```ts
 import "server-only";
 import { anthropic } from "@ai-sdk/anthropic";
-import { openai } from "@ai-sdk/openai";
-import { streamText } from "ai";
-import { createVideoHandler } from "@vanillaskyai/video/server";
-
-const provider = process.env.VIDEO_PROVIDER;
-const modelId = process.env.VIDEO_MODEL;
-if (!modelId) throw new Error("Set VIDEO_MODEL in the server environment");
-const model = provider === "openai" ? openai(modelId)
-  : provider === "anthropic" ? anthropic(modelId)
-    : (() => { throw new Error("Set VIDEO_PROVIDER to openai or anthropic"); })();
-
-export const POST = createVideoHandler({
+import { generateText, streamText } from "ai";
+import { createVideoChatHandler } from "@vanillaskyai/video/server";
+
+export const handler = createVideoChatHandler({
   authorize: verifySession,
   streamText: ({ systemPrompt, userPrompt, signal }) => streamText({
-    model,
+    model: anthropic(process.env.TEXT_MODEL ?? "claude-sonnet-5"),
     system: systemPrompt,
     prompt: userPrompt,
     abortSignal: signal,
   }),
+  generateText: async ({ systemPrompt, userPrompt, maxOutputTokens, signal }) => {
+    const result = await generateText({
+      model: anthropic(process.env.TEXT_MODEL ?? "claude-sonnet-5"),
+      system: systemPrompt,
+      prompt: userPrompt,
+      maxOutputTokens,
+      abortSignal: signal,
+    });
+    return result.text;
+  },
 });
 ```
 
-The AI SDK result can be returned directly: its `textStream`, `finishReason`,
-`rawFinishReason`, usage, provider metadata, warnings, and response metadata
-match VanillaSky's structural callback contract. Keep the
-provider, model, and credentials on the server. The callback runs per request,
-so an application can route cheap/fast and high-capability models through the
-same handler without adding a VanillaSky abstraction. See
-[provider adapter reference](reference/provider-adapters.md) for model alternatives and advanced native
-provider loops.
+That one text provider gives the browser templated video responses and local
+browser speech. Supplying `generateSpeech`, `transcribe`, `searchMedia`, or
+`generateVideo` enables those capabilities automatically. The callbacks are
+structural and provider-neutral; their SDKs and credentials remain application
+dependencies and never enter the browser bundle.
+
+The planner emits a 6-9 word hook first, then continues into the scenes in the
+same stream. The stock lookup is a separate cancellable request, so it cannot
+delay speech or planning. A welcome card can carry a prewritten `opening`, which
+starts immediately with its already-loaded media. For a full AI response, the
+same first streamed object reserves the exact first body scene. Its clip starts
+generating while the planner continues with scenes two through five. Template
+responses stream their first resolved scene without waiting for the rest of the
+plan, and normal narration is written in that same planner call rather than
+through a second model round trip.
+
+The matching complete React interface is one component and one scoped style
+import:
+
+```tsx
+import { VideoChat } from "@vanillaskyai/video/react";
+import "@vanillaskyai/video/video-chat.css";
+
+export function App() {
+  return <VideoChat />;
+}
+```
+
+## Custom interface
+
+For a custom interface, use `useVideoChat` and render its `turns`, `welcome`,
+`suggestions`, `caption`, and `status`; the hook owns their network and playback
+lifecycle. Pass a selected card through
+`chat.ask(card.prompt, { opening: card.opening, openingMedia: card.media })` to
+start its hook and reuse its footage immediately. For typed prompts, the hook
+and media keyword arrive through the response stream automatically.
+
+Any AI SDK `LanguageModel` works in both `streamText` and `generateText`. Keep
+selection in one server-only module when an application supports several text
+providers. The chat route and React component stay unchanged; only the model
+passed to those callbacks changes. The AI SDK result can be returned directly:
+its text stream, finish reason, usage, warnings, and response metadata match the
+structural callback contract. See the
+[provider adapter reference](reference/provider-adapters.md) for native provider
+alternatives.
 
 ## Planning effort and reasoning modes
 
@@ -85,7 +132,7 @@ and credentials stay with the application.
 Use `onComplete` for server-side cost and completion measurement:
 
 ```ts
-createVideoHandler({
+createVideoChatHandler({
   authorize: verifySession,
   streamText: ({ systemPrompt, userPrompt, signal }) => streamText({
     model,
@@ -93,6 +140,7 @@ createVideoHandler({
     prompt: userPrompt,
     abortSignal: signal,
   }),
+  generateText: runSmallTextTask,
   onWarning: (warning) => logSafeWarning(warning.code, warning.category),
   onComplete: (summary) => recordGeneration({
     finishReason: summary.finishReason,
@@ -113,23 +161,18 @@ true` only when the host deliberately needs bounded provider-native usage and
 metadata and has an appropriate retention policy.
 
 `acceptedSceneCount`, `rejectedSceneCount`, and `timeToFirstSceneMs` describe
-model-generated scene additions; a deterministic supplied opening is not
-counted. Their sum is the proposed scene count. `videoDurationSec` is the
-duration actually committed; compare it with the `maxDurationSec` supplied by
-your application when applying a retry policy. These fields provide a
-server-side quality signal without exposing model metadata in the browser.
+model-generated scene additions; the streamed opening hook is not counted.
+Their sum is the proposed scene count. `videoDurationSec` is the duration
+actually committed. These fields provide a server-side quality signal without
+exposing model metadata in the browser.
 Warnings include the same bounded typed warnings emitted to the client.
 `plan_incomplete` identifies a playable partial response whose planner reported
 a length limit; applications should show that result as incomplete and may
 offer a bounded retry with a larger output or duration budget.
-`plan_missing_closer` identifies a playable plan that ended without the
-explicit final scene required by the standard handler. `createVideoHandler`
-sets `requireCloser: true` by default. Specialized deterministic integrations
-may set `requireCloser: false`; ordinary AI planners should keep the default.
-For non-interactive backfills, define an application threshold (for example,
-no rejected scenes and a useful committed-duration ratio) and retry a bounded
-number of times. Keep the best accepted result rather than treating
-`finishReason: "stop"` alone as a quality score.
+`plan_missing_closer` identifies a playable answer that ended without its
+explicit final scene. For non-interactive evaluation, define an application
+threshold and retry a bounded number of times. Keep the best accepted result
+rather than treating `finishReason: "stop"` alone as a quality score.
 
 The generated system prompt includes the selected trusted-template catalog and
 is intentionally substantial. It is stable for the same SDK version, template
@@ -150,9 +193,8 @@ same explicit request budget.
 
 ## Product-level planner guidance
 
-`createVideoHandler` constructs the planner prompt from the generated server
-template registry. Normal integrations do not build prompts or capabilities.
-Use the handler's `basePrompt` option only for durable product-level direction.
-`VideoInput.knowledgeMode` owns the knowledge boundary: `input-only` is strict
-and remains the default, while `general` permits stable model knowledge.
-Presentation guidance in `basePrompt` or `instructions` never changes it.
+`createVideoChatHandler` constructs the planner prompt from the trusted template
+registry. Normal integrations do not build prompts or capabilities. Use the
+handler's `instructions` option for durable product-level direction such as a
+character, audience, domain, or answer style. The current user prompt and
+bounded prior turns are supplied separately by the SDK.

package/docs/responsive-orientation.md

@@ -1,4 +1,4 @@
-[← Documentation home](../README.md) · [Previous: Input and first scene](input-and-first-scene.md)
+[← Documentation home](../README.md) · [Previous: Prompt and conversation input](prompt-and-input.md)
 
 # Responsive orientation
 

package/docs/security.md

@@ -7,7 +7,8 @@ The SDK validates protocol shape; your application still owns identity, authoriz
 ## Required server controls
 
 - Authenticate the user and tenant before reading the prompt body.
-- `createVideoHandler` requires an explicit `authorize` policy. The
+- `createVideoChatHandler` and the lower-level `createVideoHandler` require an
+  explicit `authorize` policy. The
   `authorize: "none"` escape hatch is for intentionally non-public in-process
   tests only; do not use it on a billable generation route.
 - Allowlist browser origins; CORS is not authentication.
@@ -15,10 +16,10 @@ The SDK validates protocol shape; your application still owns identity, authoriz
 - Keep provider keys, system prompts, tools, signed-URL credentials, and admin tokens server-side.
 - Use the same generated template registry on the server and in React; the handler infers validation.
 - Treat every project-owned template as trusted application build code and
-  review it before using the CLI. Normal `vanillasky list`,
-  `vanillasky describe`, `vanillasky add`, `vanillasky sync`, and
-  `vanillasky check` commands execute project template modules locally. This
-  includes `vanillasky add` previews with `--dry-run` or `--diff`, because the
+  review it before using the CLI. Normal `vanillasky templates list`,
+  `vanillasky templates describe`, `vanillasky templates add`, `vanillasky templates sync`, and
+  `vanillasky templates check` commands execute project template modules locally. This
+  includes `vanillasky templates add` previews with `--dry-run` or `--diff`, because the
   CLI must derive the proposed browser and server registries. Resource and
   environment boundaries reduce accidental damage but are not a portable
   JavaScript sandbox.

package/docs/streaming-protocol.md

@@ -1,4 +1,4 @@
-[← Documentation home](../README.md) · [Previous: Live channels](live-channels.md) · [Next: Security →](security.md)
+[← Documentation home](../README.md) · [Previous: Persistence and replay](persistence.md) · [Next: Security →](security.md)
 
 # Streaming protocol
 

package/docs/testing.md

@@ -1,56 +1,63 @@
-# Test integrations without a model
+# Test video chat without a model
 
-`@vanillaskyai/video/test` provides deterministic provider streams and in-process
-protocol events. It has no React or provider-SDK dependency, makes no network
-request, and does not require a model key.
+`@vanillaskyai/video/test` provides deterministic planner streams and
+in-process protocol events. It has no React or provider-SDK dependency, makes
+no network request, and needs no model key.
 
-## Test a route handler with Vitest
+## Test the chat route with Vitest
 
-Pass `createMockVideoPlanner()` directly to the same `createVideoHandler()`
-used by the application. Call the handler with a standard `Request`; no live
-HTTP server is involved.
+Pass `createMockVideoPlanner()` to the same `createVideoChatHandler` used by the
+application. A standard `Request` exercises parsing, validation, pacing, the
+opening extension, and SSE without starting an HTTP server.
 
 ```ts
 import { describe, expect, it } from "vitest";
-import { createVideoHandler } from "@vanillaskyai/video/server";
-import {
-  createMockVideoPlanner,
-  videoFixtures,
-} from "@vanillaskyai/video/test";
+import { createVideoChatHandler } from "@vanillaskyai/video/server";
+import { createMockVideoPlanner } from "@vanillaskyai/video/test";
 
-describe("POST /api/video", () => {
-  it("returns a completed video stream", async () => {
-    const handle = createVideoHandler({
-      authorize: "none", // Explicit because this in-process test route is never public.
+describe("POST /api/video-chat", () => {
+  it("returns a completed video answer", async () => {
+    const handle = createVideoChatHandler({
+      authorize: "none", // Only acceptable because this handler stays in process.
       heartbeatMs: false,
       streamText: createMockVideoPlanner(),
+      generateText: async ({ task }) => task === "suggestions"
+        ? JSON.stringify({ suggestions: [] })
+        : "A deterministic narration line.",
     });
-    const response = await handle(new Request("https://app.test/api/video", {
-      method: "POST",
-      body: JSON.stringify({
-        protocolVersion: "0.5",
-        requestId: "route-test",
-        input: videoFixtures.portrait.input,
-      }),
-    }));
+
+    const response = await handle(new Request(
+      "https://app.test/api/video-chat?action=response",
+      {
+        method: "POST",
+        headers: { "content-type": "application/json" },
+        body: JSON.stringify({
+          prompt: "Explain why the Moon shows one face",
+          opening: "The Moon turns, perfectly matching its orbit.",
+          mode: "templates",
+          orientation: "landscape",
+        }),
+      },
+    ));
     const body = await response.text();
 
     expect(response.status).toBe(200);
+    expect(body).toContain('"type":"data.video-chat-opening"');
     expect(body).toContain('"type":"response.complete"');
   });
 });
 ```
 
-The mock emits provider-style newline-delimited JSON. The route still parses,
-validates, paces, and converts those plan parts into SSE exactly as it does with
-a live provider.
+Also request `?action=capabilities` and `?action=welcome` in route tests. Assert
+that optional modes and controls appear only when their callbacks are present.
+For browser coverage, render `<VideoChat />`, submit through the visible prompt,
+and wait for a known template rather than reaching into hook internals.
 
 ## Test protocol events in process
 
-`simulateVideoStream(parts, options?)` runs the same composition and validation
-pipeline without an HTTP boundary. It yields structural, discriminated event
-objects. Default IDs are `test-request` and `test-run`; override `requestId` and
-`runId` when a test needs different fixed values.
+`simulateVideoStream(parts, options?)` runs the composition and validation
+pipeline without an HTTP boundary. It yields typed event objects and remains
+useful for focused planner and protocol cases beneath the chat handler.
 
 ```ts
 import { expect, it } from "vitest";
@@ -74,9 +81,8 @@ it("keeps a truncated result playable", async () => {
 });
 ```
 
-The portrait and landscape fixtures each contain `{ input, parts }`. All public
-fixture values are deeply frozen, and each helper clones parts before a run so
-one test cannot mutate another test's input.
+The portrait and landscape fixtures each contain `{ input, parts }`. Public
+fixture values are deeply frozen and every helper clones parts before a run.
 
 ## Delays, aborts, and timeouts
 
@@ -113,11 +119,11 @@ it("times out deterministically", async () => {
 });
 ```
 
-For a host cancellation, pass an `AbortController` signal and abort it after the
-desired partial event. Timeouts are opt-in and local to the simulator; at the
-route boundary the host remains responsible for aborting the request signal.
+For host cancellation, pass an `AbortController` signal and abort after the
+desired partial event. At the route boundary, the host remains responsible for
+aborting the request signal and each provider callback must honour it.
 
-## Fixed scenarios
+## Fixed planner scenarios
 
 `createMockVideoPlanner({ scenario })` accepts:
 
@@ -126,12 +132,14 @@ route boundary the host remains responsible for aborting the request signal.
 | `success` | Completed portrait fixture |
 | `delayed` | Success after a 25 ms fake-timer-safe delay |
 | `truncated` | Partial playable result completed with `length` |
-| `invalidScene` | Invalid scene dropped with a recoverable diagnostic, then completion |
+| `invalidScene` | Invalid scene dropped with a recoverable warning, then completion |
 | `providerFailure` | Redacted terminal generation failure |
 | `contentFilter` | Partial playable result completed with `content-filter` |
 | `abort` | Waits for the supplied request signal to abort |
 | `timeout` | Waits for the host signal or simulator `timeoutMs` |
 
-Use `parts` in `MockVideoPlannerOptions` to supply custom structural plan parts,
-and `delayMs` to delay every emitted provider chunk. Provider selection,
-credentials, retries, and real-provider acceptance remain application-owned.
+Use `parts` for custom structural plan parts and `delayMs` to delay each
+provider chunk. Keep provider selection, credentials, retries, and real-model
+acceptance outside deterministic CI.
+
+[← Documentation home](../README.md)

package/examples/custom-template/README.md

@@ -2,7 +2,7 @@
 
 These files are deliberately complete, one-file examples. Copy the closest
 shape into `vanillasky/templates/`, rename its ID, then run `npx vanillasky
-sync` and `npx vanillasky check`.
+templates sync` and `npx vanillasky templates check`.
 
 | Reference | Use it when |
 | --- | --- |

package/package.json

@@ -1,20 +1,23 @@
 {
   "name": "@vanillaskyai/video",
-  "version": "0.6.0",
-  "description": "Open-source video response SDK for personalized AI applications.",
+  "version": "0.7.0",
+  "description": "Open-source voice-and-video chat SDK for AI applications.",
   "keywords": [
+    "video-chat",
+    "voice-chat",
+    "ai-chat",
     "generative-video",
-    "personalized-video",
     "video-response",
     "ai-output",
-    "contextual-ai",
     "streaming",
     "react",
     "llm",
     "video"
   ],
   "type": "module",
-  "sideEffects": false,
+  "sideEffects": [
+    "./styles/video-chat.css"
+  ],
   "license": "Apache-2.0",
   "repository": {
     "type": "git",
@@ -27,15 +30,18 @@
   "files": [
     "bin",
     "dist",
+    "styles/video-chat.css",
+    "starters/video-chat/.env.example",
+    "starters/video-chat/README.md",
+    "starters/video-chat/index.html",
+    "starters/video-chat/package.json",
+    "starters/video-chat/server.ts",
+    "starters/video-chat/src/main.tsx",
+    "starters/video-chat/stock.ts",
+    "starters/video-chat/tsconfig.json",
+    "starters/video-chat/vite.config.ts",
     "registry/items",
     "examples/custom-template",
-    "examples/nextjs-quickstart/.env.example",
-    "examples/nextjs-quickstart/README.md",
-    "examples/nextjs-quickstart/next-env.d.ts",
-    "examples/nextjs-quickstart/package.json",
-    "examples/nextjs-quickstart/src",
-    "examples/nextjs-quickstart/tsconfig.json",
-    "examples/server-integrations/src/ai-sdk-media.ts",
     "README.md",
     "LICENSE",
     "SECURITY.md",
@@ -80,7 +86,8 @@
       "types": "./dist/test.d.ts",
       "import": "./dist/test.js",
       "default": "./dist/test.js"
-    }
+    },
+    "./video-chat.css": "./styles/video-chat.css"
   },
   "scripts": {
     "acceptance:live": "tsx scripts/acceptance/run-live.ts",
@@ -89,14 +96,6 @@
     "browser:install": "playwright install chromium firefox webkit",
     "browser:test": "playwright test",
     "build": "tsup",
-    "example:install": "npm run examples:install-current",
-    "example:build": "npm run build --prefix examples/react-vite",
-    "example:dev": "npm run dev --prefix examples/react-vite",
-    "example:preview": "npm run preview --prefix examples/react-vite -- --host 127.0.0.1 --port 4187 --strictPort",
-    "examples:verify-documented": "node scripts/verify-documented-examples.mjs",
-    "examples:install-current": "npm run build && node scripts/install-current-examples.mjs",
-    "server-examples:typecheck": "npm run typecheck --prefix examples/server-integrations",
-    "server-examples:compat": "npm run ai-sdk:compat --prefix examples/server-integrations",
     "lint": "eslint .",
     "catalog:sync": "tsx scripts/generate-builtin-catalog.ts",
     "catalog:check": "tsx scripts/generate-builtin-catalog.ts --check",
@@ -143,7 +142,6 @@
     "eslint": "^9.39.1",
     "globals": "^16.5.0",
     "jsdom": "^26.1.0",
-    "mdast-util-from-markdown": "2.0.3",
     "openai": "^7.5.0",
     "react": "^19.2.8",
     "react-dom": "^19.2.8",