@convai/web-sdk 1.4.0-beta.2 → 1.5.0-beta.0

package/README.md

@@ -1,1735 +1,140 @@
 # @convai/web-sdk
 
-`@convai/web-sdk` is a TypeScript-first SDK for building real-time conversational AI experiences with Convai characters on the web. It supports:
+**Real-time conversational AI characters for the web.**
 
-- React applications with ready-to-use hooks and widget components
-- Vanilla TypeScript/JavaScript applications with a framework-agnostic widget
-- Direct core client usage for custom UIs and advanced integrations
-- Optional lipsync data pipelines for ARKit and MetaHuman rigs
+[![npm](https://img.shields.io/npm/v/@convai/web-sdk)](https://www.npmjs.com/package/@convai/web-sdk)
+[![TypeScript](https://img.shields.io/badge/TypeScript-first-3178C6)](https://www.typescriptlang.org/)
 
-This document is written as a complete implementation reference, from first setup to production hardening.
+TypeScript-first SDK for embedding Convai AI characters into React and vanilla JS applications. Voice, text, lipsync, emotions, video, and screen share — all in one package.
 
-## Table of Contents
-
-- [1. Package Entry Points](#1-package-entry-points) — includes `@convai/web-sdk/vanilla/websocket` opt-in subpath
-- [2. Installation and Requirements](#2-installation-and-requirements)
-- [3. Credentials and Environment Setup](#3-credentials-and-environment-setup)
-- [4. Quick Start](#4-quick-start)
-- [5. Build a Chatbot from Scratch](#5-build-a-chatbot-from-scratch)
-- [6. Core Concepts and Lifecycle](#6-core-concepts-and-lifecycle)
-- [7. Configuration Reference (`ConvaiConfig`)](#7-configuration-reference-convaiconfig)
-  - [Transport Options](#transport-options)
-- [8. Core API Reference (`ConvaiClient`)](#8-core-api-reference-convaiclient)
-  - [Idle management and LLM silence handling](#idle-management-and-llm-silence-handling)
-  - [Server acknowledgments (`serverResponse`)](#server-acknowledgments-serverresponse)
-  - [Interaction lifecycle (`interactionCreated`)](#interaction-lifecycle-interactioncreated)
-- [9. Memory Management API](#9-memory-management-api)
-- [10. Message Semantics and Turn Completion](#10-message-semantics-and-turn-completion)
-- [11. React API Reference](#11-react-api-reference)
-- [12. Vanilla API Reference](#12-vanilla-api-reference)
-- [13. Audio Integration Best Practices (Vanilla TypeScript)](#13-audio-integration-best-practices-vanilla-typescript)
-- [14. Error Handling and Reliability Patterns](#14-error-handling-and-reliability-patterns)
-- [15. Troubleshooting](#15-troubleshooting)
-- [16. Lipsync Helpers Reference](#16-lipsync-helpers-reference)
-- [17. Examples](#17-examples)
-
-## 1. Package Entry Points
-
-The SDK is published with multiple entry points for different integration styles.
-
-### `@convai/web-sdk` (default)
-
-Primary exports:
-
-- `useConvaiClient`
-- `ConvaiWidget`
-- `useCharacterInfo`
-- `useLocalCameraTrack`
-- `ConvaiClient`
-- `AudioRenderer` (re-export of LiveKit `RoomAudioRenderer` for React usage)
-- `AudioContext` (re-export of LiveKit `RoomContext`)
-- Core types re-exported from `core/types`:
-  - `AudioSettings`
-  - `ConvaiConfig`
-  - `ChatMessage`
-  - `ConvaiClientState`
-  - `AudioControls`
-  - `VideoControls`
-  - `ScreenShareControls`
-  - `IConvaiClient`
-- All exports from `@convai/web-sdk/lipsync-helpers`
-
-### `@convai/web-sdk/react`
-
-React-focused entry point, equivalent to the default React API surface.
-
-### `@convai/web-sdk/vanilla`
-
-Vanilla/browser-focused exports:
-
-- `ConvaiClient`
-- `AudioRenderer` (vanilla audio playback manager)
-- `createConvaiWidget`
-- `destroyConvaiWidget`
-- Types:
-  - `VanillaWidget`
-  - `VanillaWidgetOptions`
-  - `IConvaiClient`
-  - `ConvaiConfig`
-  - `ConvaiClientState`
-  - `ChatMessage`
-
-### `@convai/web-sdk/core`
-
-Framework-agnostic low-level API:
-
-- `ConvaiClient`
-- `AudioManager`
-- `VideoManager`
-- `ScreenShareManager`
-- `MessageHandler`
-- `MemoryManager`
-- `BlendshapeQueue`
-- `EventEmitter`
-- Type alias: `ConvaiClientType`
-- All core types from `core/types`
-- `TurnStats` type
-
-### `@convai/web-sdk/vanilla/websocket`
-
-Optional subpath for the WebSocket (Pipecat) transport. Import this alongside `@convai/web-sdk/vanilla` when you use `transport: "websocket"` in `ConvaiConfig`. Importing it signals to your bundler that `@pipecat-ai/client-js` and `@pipecat-ai/websocket-transport` should be included.
-
-If you only use the default LiveKit transport you should **not** import this subpath — those pipecat packages will then be excluded from your bundle entirely.
-
-Exports:
-
-- `WebSocketSession`
+---
 
-### `@convai/web-sdk/lipsync-helpers`
+## Features
 
-Dedicated helpers for blendshape formats and queue creation. Full function list is in [Section 13](#13-lipsync-helpers-reference).
+- **React & vanilla JS** — `useConvaiClient` hook, `ConvaiWidget`, and a framework-agnostic core
+- **Real-time audio/video** — full-duplex WebRTC with echo cancellation, camera, and screen share
+- **Lipsync** — ARKit and MetaHuman blendshape streams for facial animation
+- **Emotions** — per-turn emotion detection with intensity scale
+- **Dynamic context** — inject game state, user preferences, or scene data mid-session
+- **Long-term memory** — persistent cross-session memories scoped to each end user
+- **File upload** — send images to the character during a live session
+- **WebSocket transport** — opt-in alternative to WebRTC for constrained networks
+- **Auth tokens** — server-side token exchange for production deployments
 
-## 2. Installation and Requirements
+---
 
-### Install
+## Installation
 
 ```bash
 npm install @convai/web-sdk
-```
-
-or
-
-```bash
+# or
 pnpm add @convai/web-sdk
-```
-
-or
-
-```bash
+# or
 yarn add @convai/web-sdk
 ```
 
-### Runtime requirements
-
-- Modern browser with WebRTC support
-- Secure context (`https://` or `http://localhost`) for microphone/camera/screen access
-
-### Peer dependencies
-
-If you are using React APIs:
+**React peer dependencies:** `react` and `react-dom` `^18 || ^19`
 
-- `react` `^18 || ^19`
-- `react-dom` `^18 || ^19`
+**Runtime requirement:** secure context (`https://` or `http://localhost`) for microphone/camera access.
 
-## 3. Credentials and Environment Setup
-
-### Obtain credentials
-
-1. Create/login to your Convai account.
-2. Create or select a character.
-3. Copy:
-   - API key
-   - Character ID
-
-### Store credentials in environment variables
-
-Do not hardcode credentials in source files.
-
-```bash
-# .env.local (example)
-VITE_CONVAI_API_KEY=<YOUR_CONVAI_API_KEY>
-VITE_CONVAI_CHARACTER_ID=<YOUR_CONVAI_CHARACTER_ID>
-VITE_CONVAI_API_URL=<OPTIONAL_CONVAI_BASE_URL>
-VITE_CONVAI_LOG_RTVI_MESSAGES=false # optional; incoming/outgoing RTVI logs are on by default
-```
-
-Use these values through your build system (`import.meta.env`, process env injection, or server-provided config).
+---
 
-## 4. Quick Start
+## Quick start
 
 ### React
 
 ```tsx
-import { ConvaiWidget, useConvaiClient } from "@convai/web-sdk";
+import { useConvaiClient, ConvaiWidget } from '@convai/web-sdk';
 
 export function App() {
-  const convaiClient = useConvaiClient({
+  const client = useConvaiClient({
     apiKey: import.meta.env.VITE_CONVAI_API_KEY,
     characterId: import.meta.env.VITE_CONVAI_CHARACTER_ID,
-    enableVideo: false,
-    startWithAudioOn: false,
   });
 
-  return <ConvaiWidget convaiClient={convaiClient} />;
+  return <ConvaiWidget convaiClient={client} />;
 }
 ```
 
 ### Vanilla TypeScript
 
 ```ts
-import { ConvaiClient, createConvaiWidget } from "@convai/web-sdk/vanilla";
+import { ConvaiClient, createConvaiWidget } from '@convai/web-sdk/vanilla';
 
 const client = new ConvaiClient({
   apiKey: import.meta.env.VITE_CONVAI_API_KEY,
   characterId: import.meta.env.VITE_CONVAI_CHARACTER_ID,
-  enableVideo: false,
-});
-
-const widget = createConvaiWidget(document.body, {
-  convaiClient: client,
-  defaultVoiceMode: true,
-  onConnect: () => console.log("Connected"),
-  onDisconnect: () => console.log("Disconnected"),
-});
-
-window.addEventListener("beforeunload", () => {
-  widget.destroy();
-  void client.disconnect().catch(() => undefined);
-});
-```
-
-## 5. Build a Chatbot from Scratch
-
-This section shows an end-to-end approach you can use in production.
-
-### A) React from scratch (custom connection flow)
-
-#### Step 1: Create the client
-
-```tsx
-import { useConvaiClient } from "@convai/web-sdk";
-
-const convaiClient = useConvaiClient({
-  apiKey: import.meta.env.VITE_CONVAI_API_KEY,
-  characterId: import.meta.env.VITE_CONVAI_CHARACTER_ID,
-  endUserId: "<UNIQUE_END_USER_ID>",
-  endUserMetadata: {
-    name: "John Doe",
-    age: "30",
-    // Add any additional metadata you want to send
-  },
-  enableVideo: true,
-  startWithVideoOn: false,
-  startWithAudioOn: false,
-  ttsEnabled: true,
-  enableLipsync: true,
-  blendshapeConfig: {
-    format: "arkit",
-    frames_buffer_duration: 0.5,
-  },
-});
-```
-
-#### Step 2: Connect from a user gesture with error handling
-
-```tsx
-async function handleConnect() {
-  try {
-    await convaiClient.connect();
-  } catch (error) {
-    console.error("Connection failed:", error);
-  }
-}
-```
-
-#### Step 3: Wait for readiness before sending text
-
-```tsx
-function sendMessage(text: string) {
-  if (!convaiClient.state.isConnected || !convaiClient.isBotReady) return;
-  convaiClient.sendUserTextMessage(text);
-}
-```
-
-#### Step 4: Render the widget or your own UI
-
-```tsx
-import { ConvaiWidget } from "@convai/web-sdk";
-
-<ConvaiWidget
-  convaiClient={convaiClient}
-  showVideo={true}
-  showScreenShare={true}
-  defaultVoiceMode={true}
-/>;
-```
-
-#### Step 5: Subscribe to lifecycle events
-
-```tsx
-useEffect(() => {
-  const unsubError = convaiClient.on("error", (error) => {
-    console.error("Convai error:", error);
-  });
-
-  const unsubState = convaiClient.on("stateChange", (state) => {
-    console.log("State:", state.agentState);
-    
-    // Access end user information from connection response
-    if (state.endUserId) {
-      console.log("End User ID:", state.endUserId);
-    }
-    
-    if (state.endUserMetadata) {
-      console.log("End User Metadata:", state.endUserMetadata);
-      // Example: { name: 'John', age: '30' }
-    }
-  });
-
-  const unsubMessages = convaiClient.on("messagesChange", (messages) => {
-    console.log("Messages:", messages.length);
-  });
-
-  return () => {
-    unsubError();
-    unsubState();
-    unsubMessages();
-  };
-}, [convaiClient]);
-```
-
-#### Step 6: Clean up on unmount
-
-```tsx
-useEffect(() => {
-  return () => {
-    void convaiClient.disconnect().catch(() => undefined);
-  };
-}, [convaiClient]);
-```
-
-### B) Vanilla TypeScript from scratch (widget + custom hooks)
-
-#### Step 1: Initialize client and widget
-
-```ts
-import { ConvaiClient, createConvaiWidget } from "@convai/web-sdk/vanilla";
-
-const client = new ConvaiClient({
-  apiKey: "<YOUR_CONVAI_API_KEY>",
-  characterId: "<YOUR_CHARACTER_ID>",
-  endUserId: "<UNIQUE_END_USER_ID>",
-  enableVideo: true,
-  startWithVideoOn: false,
-});
-
-const widget = createConvaiWidget(document.body, {
-  convaiClient: client,
-  showVideo: true,
-  showScreenShare: true,
-  defaultVoiceMode: true,
-  onConnect: () => console.log("Connected"),
-  onDisconnect: () => console.log("Disconnected"),
-  onMessage: (message) => console.log("Message:", message),
 });
-```
-
-#### Step 2: Add explicit error listeners
-
-```ts
-const unsubError = client.on("error", (error) => {
-  console.error("SDK error:", error);
-});
-```
-
-#### Step 3: Add guarded send utility
-
-```ts
-function safeSend(text: string) {
-  if (!text.trim()) return;
-  if (!client.state.isConnected) return;
-  if (!client.isBotReady) return;
-  client.sendUserTextMessage(text);
-}
-```
 
-#### Step 4: Cleanup
+const widget = createConvaiWidget(document.body, { convaiClient: client });
 
-```ts
-function destroy() {
-  unsubError();
+window.addEventListener('beforeunload', () => {
   widget.destroy();
-  void client.disconnect().catch(() => undefined);
-}
-```
-
-### C) Custom UI (framework-agnostic)
-
-If you are not using the built-in widget:
-
-- Use `ConvaiClient` from `@convai/web-sdk/core`
-- Use `AudioRenderer` from `@convai/web-sdk/vanilla` for remote audio playback
-- Render your own UI based on `stateChange`, `messagesChange`, and control manager events
-
-```ts
-import { ConvaiClient } from "@convai/web-sdk/core";
-import { AudioRenderer } from "@convai/web-sdk/vanilla";
-
-const client = new ConvaiClient({
-  apiKey: "<YOUR_CONVAI_API_KEY>",
-  characterId: "<YOUR_CHARACTER_ID>",
-});
-
-await client.connect();
-const audioRenderer = new AudioRenderer(client.room);
-
-// ... your custom UI logic
-
-audioRenderer.destroy();
-await client.disconnect();
-```
-
-## 6. Core Concepts and Lifecycle
-
-### Connection lifecycle
-
-1. `connect()` starts room and transport setup.
-2. `state.isConnected` becomes true when room connection is established.
-3. `botReady` event indicates the character is ready for interaction.
-4. Messages stream through data events into `chatMessages`.
-5. Audio/video/screen-share are managed through dedicated control managers.
-6. `disconnect()` tears down the session.
-
-### Activity lifecycle
-
-- `state.isThinking`: model is generating response
-- `state.isSpeaking`: model audio is currently speaking
-- `state.agentState`: combined high-level state (`disconnected | connected | listening | thinking | speaking`)
-- `state.endUserId`: end user ID returned from the connection response (if provided in config)
-- `state.endUserMetadata`: end user metadata returned from the connection response (if provided in config)
-- `state.metrics`: array of metrics events received during the current session (clears on `resetSession()`)
-
-### Widget lifecycle
-
-Both React and vanilla widgets:
-
-- auto-connect on first user interaction
-- expose optional callbacks/events
-- need explicit cleanup on app teardown
-
-## 7. Configuration Reference (`ConvaiConfig`)
-
-| Field                                     | Type               | Required | Default              | Description                                                                         |
-| ----------------------------------------- | ------------------ | -------- | -------------------- | ----------------------------------------------------------------------------------- |
-| `transport`                               | `"livekit" \| "websocket"` | No | `"livekit"`        | Transport layer for the session. See [Transport Options](#transport-options).       |
-| `apiKey`                                  | `string`           | Yes      | -                    | Convai API key.                                                                     |
-| `characterId`                             | `string`           | Yes      | -                    | Target character identifier.                                                        |
-| `endUserId`                               | `string`           | No       | `undefined`          | Stable end-user identity for memory/analytics continuity.                           |
-| `endUserMetadata`                         | `Record<string, unknown>` | No | `undefined`          | Additional end-user metadata (e.g., name, age) sent with the connection request.    |
-| `url`                                     | `string`           | No       | SDK internal default | Convai base URL. Set explicitly if your deployment requires a specific environment. |
-| `enableVideo`                             | `boolean`          | No       | `false`              | Enables video-capable connection type.                                              |
-| `startWithVideoOn`                        | `boolean`          | No       | `false`              | Auto-enable camera after connect.                                                   |
-| `startWithAudioOn`                        | `boolean`          | No       | `false`              | Auto-enable microphone after connect.                                               |
-| `ttsEnabled`                              | `boolean`          | No       | `true`               | Enables model text-to-speech output.                                                |
-| `enableLipsync`                           | `boolean`          | No       | `false`              | Requests blendshape payloads for facial animation.                                  |
-| `blendshapeConfig.format`                 | `"arkit" \| "mha"` | No       | `"mha"`              | Blendshape output format.                                                           |
-| `blendshapeConfig.frames_buffer_duration` | `number`           | No       | server-defined       | Buffering hint for audio/blendshape synchronization.                                |
-| `actionConfig`                            | object             | No       | `undefined`          | Action and scene-context metadata (actions, characters, objects, attention object). |
-| `dynamicInfo`                             | `string`           | No       | `undefined`          | Dynamic contextual information about the current situation sent to the LLM.         |
-| `keepInContext`                           | `boolean`          | No       | `false`              | Keep dynamic info in context as a static prompt. When true, persists throughout session; when false, allows updates via updateContext() or updateDynamicInfo(). |
-| `logRtviMessages`                         | `boolean`          | No       | `true`               | Log incoming and outgoing decoded RTVI data messages to the browser console. Set to `false` to disable. |
-
-### Transport Options
-
-The SDK supports two transport layers, selectable via the `transport` config field.
-
-#### `"livekit"` (default)
-
-Uses WebRTC via LiveKit. This is the default and is recommended for most use cases.
-
-- Full duplex audio with echo cancellation and noise suppression
-- Hardware-accelerated media pipeline in the browser
-- Supports video, camera, and screen share
-- Requires WebRTC support in the browser (all modern browsers)
-
-#### `"websocket"` _(new in v1.4.0)_
-
-Uses a WebSocket connection via the [Pipecat RTVI](https://github.com/pipecat-ai/pipecat) protocol. Useful when WebRTC is unavailable or undesirable (e.g. certain network environments, proxied deployments, or server-side rendering contexts).
-
-- Audio over WebSocket — no WebRTC peer connection required
-- Same event model and `ConvaiClient` API as the LiveKit transport
-- Lipsync (`enableLipsync`) is supported
-- Video, camera, and screen share are not available
-- Slightly higher latency than WebRTC in typical network conditions
-
-**Bundling — opt-in subpath import**
-
-The WebSocket transport depends on `@pipecat-ai/client-js` and `@pipecat-ai/websocket-transport`. These packages are loaded lazily inside `ConvaiClient` and will not appear in your bundle unless you explicitly import the `@convai/web-sdk/vanilla/websocket` subpath. This keeps the default LiveKit-only bundle free of pipecat code.
-
-When using the WebSocket transport, add one extra import alongside your normal SDK import:
-
-```ts
-// Opt in to the WebSocket transport (pulls pipecat into your bundle)
-import "@convai/web-sdk/vanilla/websocket";
-import { ConvaiClient } from "@convai/web-sdk/vanilla";
-
-const client = new ConvaiClient({
-  apiKey: "...",
-  characterId: "...",
-  transport: "websocket",
-});
-```
-
-If you only use LiveKit transport, omit the subpath import entirely:
-
-```ts
-// LiveKit only — pipecat packages are excluded from the bundle
-import { ConvaiClient } from "@convai/web-sdk/vanilla";
-
-const client = new ConvaiClient({
-  apiKey: "...",
-  characterId: "...",
-  // transport defaults to "livekit"
-});
-```
-
-**React usage:**
-
-```tsx
-import "@convai/web-sdk/vanilla/websocket"; // add once, e.g. in your entry file
-import { useConvaiClient } from "@convai/web-sdk";
-
-const client = useConvaiClient({
-  apiKey: "...",
-  characterId: "...",
-  transport: "websocket",
+  void client.disconnect();
 });
 ```
 
-Everything else — events, `sendUserTextMessage`, `updateContext`, `actionConfig`, lipsync — works identically regardless of transport. You can switch between `"livekit"` and `"websocket"` by changing one field (and adding or removing the subpath import).
-
 ---
 
-## 8. Core API Reference (`ConvaiClient`)
-
-Import:
-
-```ts
-import { ConvaiClient } from "@convai/web-sdk/core";
-```
+## Documentation
+
+Full documentation is at **[docs.convai.com](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk)**.
+
+| Guide | Description |
+|---|---|
+| [Quick Start](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | First working integration in under 5 minutes |
+| [Configuration](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | All `ConvaiConfig` options |
+| [React Integration](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | `useConvaiClient`, `ConvaiWidget`, and React-specific patterns |
+| [Vanilla JS](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | `ConvaiClient`, `createConvaiWidget`, and audio setup |
+| [Events](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | Full event reference — `botReady`, `stateChange`, `messagesChange`, `interactionCreated`, and more |
+| [Context Management](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | Dynamic context, `updateContext`, file upload, session management |
+| [Emotions](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | Per-turn emotion detection with provider options |
+| [Lipsync](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | ARKit / MetaHuman blendshape streams and `BlendshapeQueue` API |
+| [Actions](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | Trigger character behaviors and scene actions |
+| [Memory](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | Long-term memory scoped to end users |
+| [Audio & Video](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | Microphone, camera, screen share controls |
+| [Error Handling](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | `error`, `disconnect`, `serverResponse`, retry patterns |
+| [Auth Tokens](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | Server-side token exchange for production |
+| [WebSocket Transport](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) | Alternative transport for WebRTC-constrained environments |
 
-### ConvaiClient types
+---
 
-For TypeScript, use the types exported from the main package or core:
+## Package entry points
 
-- **`IConvaiClient`** — Interface implemented by `ConvaiClient`. Use this when you need to type variables, props, or function parameters that accept the client (e.g. `client: IConvaiClient`).
-- **`ConvaiClientState`** — Shape of `client.state` (connection and activity flags: `isConnected`, `isBotReady`, `agentState`, `emotion`, `endUserId`, `endUserMetadata`, `metrics`, etc.).
-- **`ConvaiConfig`** — Configuration object for `new ConvaiClient(config)` and `client.connect(config)`.
-- **`ChatMessage`** — Single message in `client.chatMessages`.
-- **`ConvaiMetrics`** — Metrics event structure containing data, timestamp, and unique identifier.
-- **`AudioControls`**, **`VideoControls`**, **`ScreenShareControls`** — Types for the control managers exposed as `client.audioControls`, `client.videoControls`, `client.screenShareControls`.
-- **`ServerResponse`** — Payload of the `serverResponse` event. Contains `event_type`, `status`, `message`, and `extras`. See [Server acknowledgments](#server-acknowledgments-serverresponse).
-- **`ServerResponseExtras`** — Token-tracking and metadata returned in `serverResponse.extras` for each event type (e.g. `token_count`, `remaining_tokens`, `content` for `context-update`).
-- **`InteractionCreated`** — Payload of the `interactionCreated` event. Contains `interactionId` and `characterSessionId`. See [Interaction lifecycle](#interaction-lifecycle-interactioncreated).
+| Import path | Contents |
+|---|---|
+| `@convai/web-sdk` | React hooks, components, and re-exported core types |
+| `@convai/web-sdk/react` | Same as default (React-explicit alias) |
+| `@convai/web-sdk/vanilla` | `ConvaiClient`, `createConvaiWidget`, `AudioRenderer` |
+| `@convai/web-sdk/core` | Framework-agnostic `ConvaiClient`, managers, and all types |
+| `@convai/web-sdk/lipsync-helpers` | Blendshape format utilities and queue helpers |
+| `@convai/web-sdk/vanilla/websocket` | **Opt-in.** Registers the WebSocket transport. Import alongside `/vanilla` when using `transport: "websocket"`. |
 
-Example:
+---
 
-```ts
-import type { IConvaiClient, ConvaiClientState, ConvaiConfig, ConvaiMetrics } from "@convai/web-sdk";
-// or from "@convai/web-sdk/core" when using core-only
-```
+## WebSocket transport
 
-### Constructor
+The default transport is WebRTC (LiveKit). A WebSocket-based transport is available for environments where WebRTC is unavailable or undesirable.
 
 ```ts
-new ConvaiClient(config?: ConvaiConfig)
-```
-
-### Properties
-
-| Property                | Type                         | Description                                              |
-| ----------------------- | ---------------------------- | -------------------------------------------------------- |
-| `state`                 | `ConvaiClientState`          | Real-time connection/activity state.                     |
-| `connectionType`        | `"audio" \| "video" \| null` | Active transport mode.                                   |
-| `apiKey`                | `string \| null`             | Active API key.                                          |
-| `characterId`           | `string \| null`             | Active character ID.                                     |
-| `speakerId`             | `string \| null`             | Resolved speaker identity.                               |
-| `room`                  | `Room`                       | Internal LiveKit room instance.                          |
-| `chatMessages`          | `ChatMessage[]`              | Conversation message store.                              |
-| `userTranscription`     | `string`                     | Current non-final voice transcription text.              |
-| `characterSessionId`    | `string \| null`             | Server conversation session identifier.                  |
-| `isBotReady`            | `boolean`                    | Character readiness flag.                                |
-| `audioControls`         | `AudioControls`              | Microphone controls.                                     |
-| `videoControls`         | `VideoControls`              | Camera controls.                                         |
-| `screenShareControls`   | `ScreenShareControls`        | Screen sharing controls.                                 |
-| `blendshapeQueue`       | `BlendshapeQueue`            | Buffer queue for lipsync frames.                         |
-| `conversationSessionId` | `number`                     | Incremental turn session ID used by conversation events. |
-| `memoryManager`         | `MemoryManager \| null`      | Long-term memory API manager. See [Memory API docs](./docs/MEMORY_API.md). |
-
-### Methods
-
-| Method                 | Signature                                                           | Description                                                |
-| ---------------------- | ------------------------------------------------------------------- | ---------------------------------------------------------- |
-| `connect`              | `(config?: ConvaiConfig) => Promise<void>`                          | Connect using passed config or stored config.              |
-| `disconnect`           | `() => Promise<void>`                                               | Disconnect and release session resources.                  |
-| `reconnect`            | `() => Promise<void>`                                               | Disconnect then connect with stored config.                |
-| `resetSession`         | `() => void`                                                        | Reset character session and clear conversation history.    |
-| `sendUserTextMessage`  | `(text: string) => void`                                            | Send text message to character.                            |
-| `sendTriggerMessage`   | `(triggerName?: string, triggerMessage?: string) => void`           | Send trigger/action message.                               |
-| `sendInterruptMessage` | `() => void`                                                        | Interrupt current bot response.                            |
-| `resetIdleTimer`       | `() => void`                                                        | Reset the server-side idle timer to prevent disconnection. |
-| `updateTemplateKeys`   | `(templateKeys: Record<string, string>) => void`                    | Update runtime template variables.                         |
-| `updateDynamicInfo`    | `(dynamicInfo: string) => void`                                     | Update dynamic context with a text description.            |
-| `toggleTts`            | `(enabled: boolean) => void`                                        | Enable/disable TTS for subsequent responses.               |
-| `on`                   | `(event: string, callback: (...args: any[]) => void) => () => void` | Subscribe to an event and receive an unsubscribe function. |
-| `off`                  | `(event: string, callback: (...args: any[]) => void) => void`       | Remove a specific listener.                                |
-
-### Common event names and payloads
-
-| Event                     | Payload                                 | Notes                                                                                                           |
-| ------------------------- | --------------------------------------- | --------------------------------------------------------------------------------------------------------------- |
-| `stateChange`             | `ConvaiClientState`                     | Any state transition.                                                                                           |
-| `message`                 | `ChatMessage`                           | Last message whenever `messagesChange` updates.                                                                 |
-| `messagesChange`          | `ChatMessage[]`                         | Full message array update.                                                                                      |
-| `userTranscriptionChange` | `string`                                | Live user speech text updates.                                                                                  |
-| `speakingChange`          | `boolean`                               | Bot speaking started/stopped.                                                                                   |
-| `botReady`                | `void`                                  | Bot can now receive interaction.                                                                                |
-| `connect`                 | `void`                                  | Client connected.                                                                                               |
-| `disconnect`              | `void`                                  | Client disconnected.                                                                                            |
-| `error`                   | `unknown`                               | Error surfaced by client.                                                                                       |
-| `conversationStart`       | `{ sessionId, userMessage, timestamp }` | Conversation turn started.                                                                                      |
-| `turnEnd`                 | `{ sessionId, duration, timestamp }`    | Server signaled end of turn (bot stopped speaking). Same semantics as `BlendshapeQueue.hasReceivedEndSignal()`. |
-| `blendshapes`             | `unknown`                               | Incoming blendshape chunk payload.                                                                              |
-| `blendshapeStatsReceived` | `unknown`                               | End-of-turn blendshape stats marker.                                                                            |
-| `metrics`                 | `Record<string, unknown>`               | Metrics data received from server. Multiple metrics events may occur per conversation.                          |
-| `idleWarning`             | `{ remainingSeconds: number \| null }`  | Server warns that the session will be disconnected due to inactivity. `remainingSeconds` is the countdown until disconnection. Call `resetIdleTimer()` on any user activity to keep the session alive. |
-| `llmNoResponse`           | `void`                                  | LLM explicitly chose not to respond (e.g. via an abstain tool call). Use this to clear any "thinking" indicator without expecting a reply. |
-| `serverResponse`          | `ServerResponse`                        | Server acknowledgment for every client-sent message. Contains `event_type` (which message triggered it), `status` (`"success"` \| `"error"` \| `"processing"` \| `"pending"`), `message` (human-readable description), and `extras` (event-specific data such as token counts for `context-update`). See [Server acknowledgments](#server-acknowledgments-serverresponse). |
-| `interactionCreated`      | `InteractionCreated`                    | Server assigned an interaction ID to this session. Contains `interactionId` and `characterSessionId`. Fire-once per session; useful for analytics and session tracking. See [Interaction lifecycle](#interaction-lifecycle-interactioncreated). |
-
-### Control manager APIs
-
-#### `audioControls`
-
-Properties:
-
-- `isAudioEnabled`
-- `isAudioMuted`
-- `audioLevel`
-
-Methods:
-
-- `enableAudio()`
-- `disableAudio()`
-- `muteAudio()`
-- `unmuteAudio()`
-- `toggleAudio()`
-- `setAudioDevice(deviceId)`
-- `getAudioDevices()`
-- `startAudioLevelMonitoring()`
-- `stopAudioLevelMonitoring()`
-- `on("audioStateChange", callback)`
-- `off("audioStateChange", callback)`
-
-#### `videoControls`
-
-Properties:
-
-- `isVideoEnabled`
-- `isVideoHidden`
-
-Methods:
-
-- `enableVideo()`
-- `disableVideo()`
-- `hideVideo()`
-- `showVideo()`
-- `toggleVideo()`
-- `setVideoDevice(deviceId)`
-- `getVideoDevices()`
-- `setVideoQuality("low" | "medium" | "high")`
-- `on("videoStateChange", callback)`
-- `off("videoStateChange", callback)`
-
-#### `screenShareControls`
-
-Properties:
-
-- `isScreenShareEnabled`
-- `isScreenShareActive`
-
-Methods:
-
-- `enableScreenShare()`
-- `disableScreenShare()`
-- `toggleScreenShare()`
-- `enableScreenShareWithAudio()`
-- `getScreenShareTracks()`
-- `on("screenShareStateChange", callback)`
-- `off("screenShareStateChange", callback)`
+// Add this import once (e.g. in your app entry file)
+import '@convai/web-sdk/vanilla/websocket';
+import { ConvaiClient } from '@convai/web-sdk/vanilla';
 
-### Using `updateDynamicInfo` for real-time context
-
-The `updateDynamicInfo` method allows you to send real-time context to the character using a simple text description.
-
-#### Basic usage
-
-```tsx
-// Simple text description
-convaiClient.updateDynamicInfo("Player health is low");
-```
-
-#### Advanced usage with detailed context
-
-```tsx
-// Game state example
-convaiClient.updateDynamicInfo(
-  "Player is in combat with 25% health and 40% stamina at battlefield location, 3 enemies nearby, equipped with sword"
-);
-```
-
-#### Using in connect config
-
-```tsx
-const config: ConvaiConfig = {
-  apiKey: 'your-api-key',
-  characterId: 'your-character-id',
-  dynamicInfo: "Initial game state: Player at spawn point, level 1, tutorial not complete",
-  keepInContext: false, // Allow updates via updateDynamicInfo() (default behavior)
-  // Set to true to make dynamicInfo behave as a static prompt for the session
-  // Other config fields...
-};
-
-await convaiClient.connect(config);
-```
-
-#### Real-time updates example
-
-```tsx
-// Update context as game state changes
-function updateCharacterContext(player: Player) {
-  const contextText = `Player health: ${player.health}%, mana: ${player.mana}%, location: ${player.location}, status: ${player.status}`;
-  
-  convaiClient.updateDynamicInfo(contextText);
-}
-
-// Call whenever game state changes
-player.on('stateChange', () => updateCharacterContext(player));
-```
-
-### Context management modes
-
-There are three ways to manage the LLM's runtime context, each suited to a different update pattern.
-
-#### Mode 1 — Static prompt via `keepInContext: true`
-
-When you set `keepInContext: true` in the connect config (or pass `keep_in_context: true` in the `dynamic_info` payload), the `text` field is treated as a **static system prompt** for the lifetime of that WebRTC connection. The LLM receives it on every turn without you needing to resend it.
-
-```tsx
-await convaiClient.connect({
-  apiKey: '...',
-  characterId: '...',
-  dynamicInfo: "This NPC is a blacksmith who only discusses weapons and armor.",
-  keepInContext: true,  // persists as a static prompt for this session
-});
-```
-
-> If you disconnect and reconnect, the static prompt is cleared — you must pass it again in the new `connect()` call.
-
-#### Mode 2 — Mutable context via `keepInContext: false` (default)
-
-When `keepInContext` is `false` (the default), the dynamic info is **not** kept as a standing prompt. You can overwrite or clear it at any time by calling `updateDynamicInfo()` again, or by sending a `context-update` / `update-dynamic-info` message directly.
-
-```tsx
-// Initial context at connect time (will be replaceable)
-await convaiClient.connect({
+const client = new ConvaiClient({
   apiKey: '...',
   characterId: '...',
-  dynamicInfo: "Player is at the starting zone.",
-  keepInContext: false,
-});
-
-// Later — overwrite with fresh state
-convaiClient.updateDynamicInfo("Player just entered the dungeon, health 80%.");
-
-// Or clear it entirely
-convaiClient.updateDynamicInfo("");
-```
-
-#### Mode 3 — Fine-grained context patches via `updateContext()`
-
-`updateContext()` gives you surgical control over the system instruction. It supports three modes and a `run_llm` flag that controls whether the LLM is triggered after the update.
-
-| Field                      | Values                                  | Description                                                  |
-|----------------------------|-----------------------------------------|--------------------------------------------------------------|
-| `text`                     | `string`                                | The context text to apply. Required unless `mode` is `"reset"` or only `current_attention_object` is being updated. |
-| `mode`                     | `"append"` \| `"replace"` \| `"reset"` | How the text is applied to the existing context. Defaults to `"append"`. |
-| `run_llm`                  | `"true"` \| `"false"` \| `"auto"`      | `"true"` runs the LLM immediately; `"false"` only updates the prompt; `"auto"` lets the LLM decide. Defaults to `"auto"`. |
-| `current_attention_object` | `string`                                | Updates the object the character is currently focused on. Must match one of the `objects[].name` values from `actionConfig`. Pass an empty string to clear. |
-| `remove_static`            | `boolean`                               | Only applies when `mode` is `"reset"`. If `true`, also clears the static context (the `dynamicInfo` set at connect time). Defaults to `false`. |
-
-```tsx
-// Append new information and run the LLM immediately
-convaiClient.updateContext({
-  text: "User just completed the dragon quest and received a golden sword.",
-  mode: "append",
-  run_llm: "true",
-});
-
-// Silently replace the full context without triggering a response
-convaiClient.updateContext({
-  text: "Game state: night-time, market district, raining.",
-  mode: "replace",
-  run_llm: "false",
-});
-
-// Clear runtime context only (preserves static context from connect-time dynamicInfo)
-convaiClient.updateContext({
-  mode: "reset",
-  run_llm: "auto",
-});
-
-// Clear everything — both runtime and static context
-convaiClient.updateContext({
-  mode: "reset",
-  remove_static: true,
-  run_llm: "false",
-});
-
-// Update only the attention object without touching context text
-convaiClient.updateContext({
-  current_attention_object: "golden_sword",
-});
-
-// Clear the attention object
-convaiClient.updateContext({
-  current_attention_object: "",
+  transport: 'websocket',
 });
 ```
 
-**When to use each `run_llm` value:**
-
-- `"true"` — the update is itself an event worth responding to (e.g. quest completion, NPC interaction).
-- `"false"` — background state update the player won't notice (e.g. syncing health/location every few seconds).
-- `"auto"` — the LLM reads the new context and decides on its own whether a response is appropriate.
+The WebSocket packages (`@pipecat-ai/client-js`, `@pipecat-ai/websocket-transport`) are **excluded from your bundle** unless you import the `/vanilla/websocket` subpath — so LiveKit-only apps pay no bundle cost.
 
-### Accessing metrics data
+See the [WebSocket Transport guide](https://docs.convai.com/api-docs/plugins-and-integrations/web-plugins/convai-web-sdk) for the full feature comparison.
 
-The SDK exposes metrics events received from the server through the client state. Multiple metrics events can be received during a single conversation.
-
-#### Accessing metrics from client state
-
-```tsx
-// Access metrics array from state
-const metrics = convaiClient.state.metrics;
-
-// Log all metrics
-console.log('Total metrics received:', metrics.length);
-metrics.forEach(metric => {
-  console.log('Metric ID:', metric.id);
-  console.log('Timestamp:', metric.timestamp);
-  console.log('Data:', metric.data);
-});
-
-// Get latest metric
-const latestMetric = metrics[metrics.length - 1];
-if (latestMetric) {
-  console.log('Latest metric data:', latestMetric.data);
-}
-```
-
-#### Listening to metrics events in real-time
-
-```tsx
-  // Subscribe to metrics events
-  const unsubMetrics = convaiClient.on('metrics', (metricsData) => {
-    console.log('New metrics received:', metricsData);
-    
-    // Process metrics data
-    if (metricsData.processingTime) {
-      console.log('Processing time:', metricsData.processingTime);
-    }
-  });
-
-// Cleanup
-unsubMetrics();
-```
-
-#### Accessing metrics in React
-
-```tsx
-import { useConvaiClient } from '@convai/web-sdk';
-import { useEffect, useState } from 'react';
-
-function MetricsDisplay() {
-  const convaiClient = useConvaiClient();
-  const [metrics, setMetrics] = useState<ConvaiMetrics[]>([]);
-
-  useEffect(() => {
-    // Subscribe to state changes
-    const unsubState = convaiClient.on('stateChange', (state) => {
-      setMetrics(state.metrics);
-    });
-
-    // Or subscribe to individual metrics events
-    const unsubMetrics = convaiClient.on('metrics', (metricsData) => {
-      console.log('New metrics:', metricsData);
-    });
-
-    return () => {
-      unsubState();
-      unsubMetrics();
-    };
-  }, [convaiClient]);
-
-  return (
-    <div>
-      <h3>Metrics ({metrics.length})</h3>
-      {metrics.map(metric => (
-        <div key={metric.id}>
-          <p>Time: {new Date(metric.timestamp).toLocaleTimeString()}</p>
-          <pre>{JSON.stringify(metric.data, null, 2)}</pre>
-        </div>
-      ))}
-    </div>
-  );
-}
-```
-
-#### Clearing metrics
-
-Metrics are automatically cleared when you call `resetSession()` or `disconnect()`:
-
-```tsx
-// Clear conversation history and metrics
-convaiClient.resetSession();
-
-// Or disconnect (also clears metrics)
-await convaiClient.disconnect();
-```
-
-#### Analyzing aggregated metrics
-
-```tsx
-function analyzeMetrics(client: IConvaiClient) {
-  const metrics = client.state.metrics;
-  
-  // Calculate average processing time if present
-  const processingTimes = metrics
-    .map(m => m.data.processingTime as number)
-    .filter(t => typeof t === 'number');
-  
-  if (processingTimes.length > 0) {
-    const avgTime = processingTimes.reduce((a, b) => a + b, 0) / processingTimes.length;
-    console.log('Average processing time:', avgTime.toFixed(2), 'ms');
-  }
-  
-  // Count metrics by type if available
-  const metricsByType = metrics.reduce((acc, metric) => {
-    const type = (metric.data.type as string) || 'unknown';
-    acc[type] = (acc[type] || 0) + 1;
-    return acc;
-  }, {} as Record<string, number>);
-  
-  console.log('Metrics by type:', metricsByType);
-}
-```
-
-### Idle management and LLM silence handling
-
-#### Idle warnings and LLM no-response as chat messages
-
-Both events automatically appear in `chatMessages` with distinct `type` values so you can handle them inline when rendering your message list — no extra event wiring required.
-
-| `type`           | `content`                         | Purpose                                      |
-|------------------|-----------------------------------|----------------------------------------------|
-| `idle-warning`   | Remaining seconds as string, e.g. `"45"` | Display a warning prompt in the chat        |
-| `llm-no-response`| `""` (always empty)               | Signal that no reply will come; hide or skip |
-
-```tsx
-// React — render chat messages with idle/no-response handling
-{chatMessages.map((msg) => {
-  if (msg.type === "idle-warning") {
-    const seconds = parseInt(msg.content, 10);
-    return (
-      <SystemNotice key={msg.id} variant="warning">
-        Session idle — disconnecting in {seconds}s.{" "}
-        <button onClick={() => convaiClient.resetIdleTimer()}>Stay connected</button>
-      </SystemNotice>
-    );
-  }
-
-  if (msg.type === "llm-no-response") {
-    // LLM chose not to reply — render nothing (or a subtle indicator)
-    return null;
-  }
-
-  return <ChatBubble key={msg.id} message={msg} />;
-})}
-```
-
-#### Receiving idle warnings via event
-
-If you prefer to react outside the message list (e.g. a toast or overlay):
-
-```ts
-const unsubIdle = client.on("idleWarning", ({ remainingSeconds }) => {
-  if (remainingSeconds !== null) {
-    showToast(`Session will end in ${remainingSeconds}s due to inactivity.`);
-  }
-});
-```
-
-#### Resetting the idle timer
-
-Call `resetIdleTimer()` whenever user activity is detected (clicks, keystrokes, UI interactions) to keep the session alive.
-
-```ts
-// Reset on any click anywhere on the page
-document.addEventListener("click", () => client.resetIdleTimer());
-
-// Or reset on specific UI interactions
-sendButton.addEventListener("click", () => {
-  client.resetIdleTimer();
-  client.sendUserTextMessage(inputField.value);
-});
-```
-
-#### React example — activity-aware idle management
-
-```tsx
-useEffect(() => {
-  const handleActivity = () => {
-    convaiClient.resetIdleTimer();
-  };
-
-  window.addEventListener("click", handleActivity);
-  window.addEventListener("keydown", handleActivity);
-
-  return () => {
-    window.removeEventListener("click", handleActivity);
-    window.removeEventListener("keydown", handleActivity);
-  };
-}, [convaiClient]);
-```
-
-#### Handling LLM no-response via event
-
-When the LLM deliberately abstains from replying (e.g. via an abstain tool call), the server emits `llmNoResponse`. If you are driving a custom "thinking" indicator outside of `chatMessages`, use this event to dismiss it.
-
-```ts
-const unsubNoResponse = client.on("llmNoResponse", () => {
-  setIsThinking(false);
-});
-```
-
-### Server acknowledgments (`serverResponse`)
-
-The server sends a `serverResponse` back for every message your client sends (`context-update`, `tts-toggle`, `trigger-message`, etc.). It tells you whether the server accepted or rejected the request, and for `context-update` it returns the current token budget so you can manage context size in real time.
-
-The dynamic context budget is **50,000 estimated tokens** total: up to 20,000 for static context (set at connect time via `dynamicInfo`) and up to 30,000 for runtime context (`updateContext` calls). When the runtime budget fills up the server keeps the newest updates and silently trims the oldest, so track `remaining_tokens` if your application sends context at high frequency.
-
-#### React
-
-```tsx
-import { useConvaiClient } from "@convai/web-sdk";
-import { useState, useEffect } from "react";
-import type { ServerResponse } from "@convai/web-sdk";
-
-function ContextMonitor() {
-  const convaiClient = useConvaiClient({ /* config */ });
-  const [tokenUsage, setTokenUsage] = useState<{ used: number; max: number } | null>(null);
-  const [contextError, setContextError] = useState<string | null>(null);
-
-  useEffect(() => {
-    const unsub = convaiClient.on("serverResponse", (response: ServerResponse) => {
-      // Track token budget after every context update
-      if (response.event_type === "context-update" && response.extras) {
-        const { token_count, max_tokens } = response.extras;
-        setTokenUsage({ used: token_count ?? 0, max: max_tokens ?? 50000 });
-      }
-
-      // Surface errors to the UI
-      if (response.status === "error") {
-        setContextError(`[${response.event_type}] ${response.message}`);
-      } else {
-        setContextError(null);
-      }
-    });
-
-    return unsub;
-  }, [convaiClient]);
-
-  const pushContext = () => {
-    convaiClient.updateContext({ text: "Player entered the dungeon.", mode: "append", run_llm: "false" });
-  };
-
-  const resetContext = () => {
-    convaiClient.updateContext({ mode: "reset", remove_static: false });
-  };
-
-  return (
-    <div>
-      <button onClick={pushContext}>Push context</button>
-      <button onClick={resetContext}>Reset context</button>
-      {tokenUsage && (
-        <p>
-          Context: {tokenUsage.used} / {tokenUsage.max} tokens
-          {tokenUsage.used / tokenUsage.max > 0.8 && " ⚠️ approaching limit"}
-        </p>
-      )}
-      {contextError && <p style={{ color: "red" }}>Error: {contextError}</p>}
-    </div>
-  );
-}
-```
-
-#### Vanilla TypeScript
-
-```ts
-import { ConvaiClient } from "@convai/web-sdk/core";
-import type { ServerResponse } from "@convai/web-sdk/core";
-
-const client = new ConvaiClient({ apiKey: "...", characterId: "..." });
-await client.connect();
-
-client.on("serverResponse", (response: ServerResponse) => {
-  if (response.event_type === "context-update" && response.extras) {
-    const { token_count, max_tokens, remaining_tokens } = response.extras;
-    updateTokenBar(token_count ?? 0, max_tokens ?? 50000);
-
-    if ((remaining_tokens ?? Infinity) < 5000) {
-      showWarning("Context nearing limit — oldest updates will be trimmed.");
-    }
-  }
-
-  if (response.status === "error") {
-    showError(`${response.event_type} failed: ${response.message}`);
-
-    // Context limit exceeded — reset runtime context and retry
-    if (response.event_type === "context-update") {
-      client.updateContext({ mode: "reset" });
-    }
-  }
-});
-
-// Push game state silently (no LLM trigger)
-function syncGameState(state: GameState) {
-  client.updateContext({
-    text: `Health: ${state.health}%, location: ${state.location}, enemies: ${state.enemies}`,
-    mode: "replace",
-    run_llm: "false",
-  });
-}
-
-// Notify the bot of a narrative event and trigger a response
-function onQuestComplete(questName: string) {
-  client.updateContext({
-    text: `Player just completed: ${questName}`,
-    mode: "append",
-    run_llm: "true",
-  });
-}
-
-// Clear everything at scene change
-function onSceneChange() {
-  client.updateContext({ mode: "reset", remove_static: false });
-}
-```
-
-### Interaction lifecycle (`interactionCreated`)
-
-The server fires `interactionCreated` once per session after connecting, assigning a stable `interactionId` to the conversation. Use it for analytics, crash reporting, or correlating client-side events with server-side logs.
-
-#### React
-
-```tsx
-import { useConvaiClient } from "@convai/web-sdk";
-import { useEffect, useRef } from "react";
-import type { InteractionCreated } from "@convai/web-sdk";
-
-function ConvaiSession() {
-  const convaiClient = useConvaiClient({ /* config */ });
-  const interactionIdRef = useRef<string | null>(null);
-
-  useEffect(() => {
-    const unsub = convaiClient.on("interactionCreated", ({ interactionId, characterSessionId }: InteractionCreated) => {
-      interactionIdRef.current = interactionId;
-
-      // Tag your analytics session
-      analytics.identify({ interactionId, characterSessionId });
-    });
-
-    return unsub;
-  }, [convaiClient]);
-
-  const handleSend = (text: string) => {
-    convaiClient.sendUserTextMessage(text);
-
-    // Attach the interaction ID to any events you fire
-    analytics.track("message_sent", { interactionId: interactionIdRef.current, text });
-  };
-
-  return <YourChatUI onSend={handleSend} />;
-}
-```
-
-#### Vanilla TypeScript
-
-```ts
-import { ConvaiClient } from "@convai/web-sdk/core";
-import type { InteractionCreated } from "@convai/web-sdk/core";
-
-const client = new ConvaiClient({ apiKey: "...", characterId: "..." });
-
-let interactionId: string | null = null;
-
-client.on("interactionCreated", ({ interactionId: id, characterSessionId }: InteractionCreated) => {
-  interactionId = id;
-  sessionStorage.setItem("convai_interaction_id", id);
-  sessionStorage.setItem("convai_session_id", characterSessionId);
-});
-
-await client.connect();
-
-// Later — include the ID when reporting errors or events
-window.onerror = (msg) => {
-  reportError({ msg, interactionId });
-};
-```
-
-### Advanced core classes (`@convai/web-sdk/core`)
-
-These are exported for advanced and custom pipeline use-cases.
-
-#### `BlendshapeQueue`
-
-Buffer for lipsync frames. Use `isConversationEnded()` for definitive end-of-conversation: it returns true only when the server has sent `blendshape-turn-stats` and either all expected frames have been consumed or the queue is empty (handles dropped frames). Use `hasReceivedEndSignal()` when you only need to know that the server signaled end (e.g. to keep playing remaining frames).
-
-Methods:
-
-- `addChunk(blendshapes)`
-- `getFrames()`
-- `getFrame(index)`
-- `getFrameWithAlpha(index)`
-- `consumeFrames(count)`
-- `hasFrames()`
-- `isConversationActive()`
-- `isConversationEnded()` — true when server signaled end and playback is complete (all frames consumed or queue empty)
-- `hasReceivedEndSignal()` — true when server sent `blendshape-turn-stats` (does not check frame consumption)
-- `startConversation()`
-- `startBotSpeaking()`
-- `stopBotSpeaking()`
-- `isBotSpeaking()`
-- `endConversation(stats?)`
-- `interrupt()`
-- `getTurnStats()`
-- `getFramesConsumed()`
-- `getTimeLeftMs()`
-- `isAllFramesConsumed()`
-- `reset()`
-- `getFrameAtTime(elapsedTime)`
-- `getDebugInfo()`
-
-Properties:
-
-- `length`
-
-#### `MessageHandler`
-
-Methods:
-
-- `getBlendshapeQueue()`
-- `getChatMessages()`
-- `getUserTranscription()`
-- `getIsBotResponding()`
-- `getIsSpeaking()`
-- `setRoom(room)`
-- `reset()`
-- inherited event APIs from `EventEmitter`:
-  - `on(event, callback)`
-  - `off(event, callback)`
-
-#### `EventEmitter`
-
-Methods:
-
-- `on(event, callback)`
-- `off(event, callback)`
-- `emit(event, ...args)`
-- `removeAllListeners()`
-- `listenerCount(event)`
-
-## 9. Memory Management API
-
-The Convai SDK includes a comprehensive Memory Management API that enables long-term memory storage for characters. Memories are scoped to a `(character_id, end_user_id)` pair, allowing each user to have personalized experiences that persist across conversation sessions.
-
-### Quick Start
-
-```typescript
-import { ConvaiClient } from "@convai/web-sdk/core";
-
-const client = new ConvaiClient({
-  apiKey: 'your-api-key',
-  characterId: 'your-character-id',
-  endUserId: 'user@example.com', // Required for memory operations
-});
-
-await client.connect();
-
-// Access memory manager
-const memoryManager = client.memoryManager;
-
-if (memoryManager) {
-  // Add memories
-  await memoryManager.addMemories([
-    'User prefers outdoor activities',
-    'User is allergic to peanuts'
-  ]);
-
-  // List memories
-  const memories = await memoryManager.listMemories();
-  console.log(`Total memories: ${memories.total_count}`);
-  
-  // Get a specific memory
-  const memory = await memoryManager.getMemory(memoryId);
-  
-  // Delete a memory
-  await memoryManager.deleteMemory(memoryId);
-}
-```
-
-### Requirements
-
-The `memoryManager` is available when:
-- You provide either an `apiKey` or `authToken` in the config
-- You provide an `endUserId` in the config
-
-### Memory Manager Methods
-
-| Method | Description |
-|--------|-------------|
-| `addMemories(memories: string[])` | Add one or more memory strings |
-| `listMemories(options?)` | List memories with pagination |
-| `getMemory(memoryId: string)` | Fetch a single memory by ID |
-| `deleteMemory(memoryId: string)` | Delete a single memory |
-| `deleteAllMemories()` | Delete all memories for current user/character |
-| `setEndUserId(endUserId: string)` | Switch to a different user context |
-| `setCharacterId(characterId: string)` | Switch to a different character context |
-
-### Standalone Usage
-
-You can also use the `MemoryManager` independently:
-
-```typescript
-import { MemoryManager } from "@convai/web-sdk/core";
-
-const memoryManager = new MemoryManager(
-  'your-api-key',
-  'your-character-id',
-  'user@example.com'
-);
-
-const result = await memoryManager.addMemories(['User likes coffee']);
-```
-
-### Complete Documentation
-
-For detailed documentation, examples, and API reference, see:
-- **[Memory API Documentation](./docs/MEMORY_API.md)** - Complete guide with examples
-- **[Memory API Usage Examples](./examples/memory-api-usage.ts)** - Runnable code examples
-
-### Memory Types
-
-All memory-related types are exported from the core package:
-
-```typescript
-import type {
-  Memory,
-  MemoryAddResponse,
-  MemoryListResponse,
-  MemoryGetResponse,
-  MemoryDeleteResponse,
-  MemoryDeleteAllResponse,
-  MemoryError,
-} from "@convai/web-sdk/core";
-```
-
-## 10. Message Semantics and Turn Completion
-
-### `ChatMessage` model
-
-`ChatMessage` includes:
-
-- `id`
-- `type`
-- `content`
-- `timestamp`
-- `isStreaming?` — `true` while the message is still streaming (mutable), `false` when finalized
-
-Supported message `type` values include:
-
-- `user`
-- `convai`
-- `emotion`
-- `behavior-tree`
-- `action`
-- `user-transcription`
-- `bot-llm-text`
-- `bot-emotion`
-- `user-llm-text`
-- `interrupt-bot`
-- `idle-warning` — server idle-timeout warning; `content` holds remaining seconds as a numeric string (e.g. `"45"`). Render differently from normal bot messages to prompt user activity.
-- `llm-no-response` — LLM deliberately chose not to respond; `content` is always `""`. Use the type to hide or suppress the message in your UI rather than showing an empty bubble.
-
-### Recommended way to detect response completion
-
-Use events instead of checking `isStreaming`:
-
-- `turnEnd` for the server turn-end signal (bot stopped speaking; same as `hasReceivedEndSignal()`)
-- `blendshapeStatsReceived` as additional completion marker when lipsync/animation output is enabled
-
-When driving lipsync from `BlendshapeQueue`, use `blendshapeQueue.isConversationEnded()` for definitive end-of-conversation. It returns true only when the server has signaled end and playback is complete (all expected frames consumed or queue empty). Call `blendshapeQueue.reset()` and your `onConversationEnded` when it becomes true. Use `hasReceivedEndSignal()` only when you need the raw server signal (e.g. to decide whether to keep playing remaining frames).
-
-Example:
-
-```ts
-type TurnCompletionOptions = {
-  expectBlendshapes: boolean;
-  onComplete: () => void;
-};
-
-function subscribeTurnCompletion(client: any, options: TurnCompletionOptions) {
-  let spokenDone = false;
-  let animationDone = !options.expectBlendshapes;
-
-  const invokeOnCompleteIfReady = () => {
-    if (spokenDone && animationDone) {
-      options.onComplete();
-    }
-  };
-
-  const unsubTurnEnd = client.on("turnEnd", () => {
-    spokenDone = true;
-    invokeOnCompleteIfReady();
-  });
-
-  const unsubBlendshapeStats = client.on("blendshapeStatsReceived", () => {
-    animationDone = true;
-    invokeOnCompleteIfReady();
-  });
-
-  return () => {
-    unsubTurnEnd();
-    unsubBlendshapeStats();
-  };
-}
-```
-
-When to use both signals: You only need to wait for both `turnEnd` and `blendshapeStatsReceived` when you use lipsync. Set `expectBlendshapes: false` when you do not use facial animation; then `animationDone` is effectively always true and completion runs as soon as `turnEnd` fires. Set `expectBlendshapes: true` when you drive lipsync from the queue; speech and blendshape data are separate pipelines and can finish in either order, so waiting for both ensures "turn complete" means both speech and animation are done before you run `onComplete`.
-
-## 11. React API Reference
-
-### `useConvaiClient(config?)`
-
-Import:
-
-```tsx
-import { useConvaiClient } from "@convai/web-sdk";
-```
-
-Returns full `IConvaiClient` plus React-friendly reactive fields:
-
-- `activity`
-- `chatMessages`
-- `isAudioMuted`
-- `isVideoEnabled`
-- `isScreenShareActive`
-
-### `ConvaiWidget`
-
-Import:
-
-```tsx
-import { ConvaiWidget } from "@convai/web-sdk";
-```
-
-Props:
-
-| Prop               | Type                                                                                                                  | Default  | Description                                                        |
-| ------------------ | --------------------------------------------------------------------------------------------------------------------- | -------- | ------------------------------------------------------------------ |
-| `convaiClient`     | `IConvaiClient & { activity?: string; isAudioMuted: boolean; isVideoEnabled: boolean; isScreenShareActive: boolean }` | required | Client instance returned by `useConvaiClient`.                     |
-| `showVideo`        | `boolean`                                                                                                             | `true`   | Shows video toggle in settings if connection type is video.        |
-| `showScreenShare`  | `boolean`                                                                                                             | `true`   | Shows screen-share toggle in settings if connection type is video. |
-| `defaultVoiceMode` | `boolean`                                                                                                             | `true`   | Opens in voice mode on first widget session.                       |
-
-### `useCharacterInfo(characterId?, apiKey?)`
-
-Returns:
-
-- `name`
-- `image`
-- `isLoading`
-- `error`
-
-### `useLocalCameraTrack()`
-
-Returns a LiveKit `TrackReferenceOrPlaceholder` for local camera rendering in custom React video UIs.
-
-### React audio utility exports
-
-- `AudioRenderer` from LiveKit React components
-- `AudioContext` from LiveKit React components
-
-## 12. Vanilla API Reference
-
-### `createConvaiWidget(container, options)`
-
-```ts
-import { createConvaiWidget } from "@convai/web-sdk/vanilla";
-```
-
-Creates and mounts a complete floating chat widget.
-
-#### `VanillaWidgetOptions`
-
-| Field              | Type                             | Required | Default     | Description                                        |
-| ------------------ | -------------------------------- | -------- | ----------- | -------------------------------------------------- |
-| `convaiClient`     | `IConvaiClient`                  | No\*     | -           | Existing client instance.                          |
-| `apiKey`           | `string`                         | No\*     | -           | Used only when `convaiClient` is not provided.     |
-| `characterId`      | `string`                         | No\*     | -           | Used only when `convaiClient` is not provided.     |
-| `enableVideo`      | `boolean`                        | No       | `false`     | Used for auto-created client only.                 |
-| `startWithVideoOn` | `boolean`                        | No       | `false`     | Used for auto-created client only.                 |
-| `enableLipsync`    | `boolean`                        | No       | `false`     | Used for auto-created client only.                 |
-| `blendshapeConfig` | object                           | No       | `undefined` | Used for auto-created client only.                 |
-| `showVideo`        | `boolean`                        | No       | `true`      | Show video toggle in settings.                     |
-| `showScreenShare`  | `boolean`                        | No       | `true`      | Show screen-share toggle in settings.              |
-| `defaultVoiceMode` | `boolean`                        | No       | `true`      | Start in voice mode when opened.                   |
-| `onConnect`        | `() => void`                     | No       | `undefined` | Called when widget client connects.                |
-| `onDisconnect`     | `() => void`                     | No       | `undefined` | Called when widget client disconnects.             |
-| `onMessage`        | `(message: ChatMessage) => void` | No       | `undefined` | Called on each message change with latest message. |
-
-\* You must provide either `convaiClient` OR both `apiKey` and `characterId`.
-
-#### Return type: `VanillaWidget`
-
-- `element`: root widget element
-- `client`: resolved client instance
-- `destroy()`: unmount and cleanup
-- `update?`: optional future extension field
-
-### `destroyConvaiWidget(widget)`
-
-Convenience wrapper that calls `widget.destroy()`.
-
-### `AudioRenderer` (vanilla)
-
-`AudioRenderer` listens to LiveKit room track subscriptions and auto-attaches remote audio tracks to hidden `audio` elements for playback. Use one renderer instance per active room session and destroy it during cleanup.
-
-## 13. Audio Integration Best Practices (Vanilla TypeScript)
-
-This section provides the recommended integration for stable audio playback.
-
-### Recommended reference implementation
-
-```ts
-import { ConvaiClient } from "@convai/web-sdk/core";
-import { AudioRenderer } from "@convai/web-sdk/vanilla";
-
-class ConvaiAudioSession {
-  private client: ConvaiClient;
-  private audioRenderer: AudioRenderer | null = null;
-  private audioContext: AudioContext | null = null;
-
-  constructor() {
-    this.client = new ConvaiClient({
-      apiKey: "<YOUR_CONVAI_API_KEY>",
-      characterId: "<YOUR_CHARACTER_ID>",
-      ttsEnabled: true,
-    });
-  }
-
-  async connectFromUserGesture(): Promise<void> {
-    await this.client.connect();
-
-    // Required for remote audio playback wiring.
-    this.audioRenderer = new AudioRenderer(this.client.room);
-
-    // Optional: if your app performs WebAudio analysis/effects.
-    if (!this.audioContext) {
-      this.audioContext = new AudioContext();
-    }
-    if (this.audioContext.state === "suspended") {
-      await this.audioContext.resume();
-    }
-  }
-
-  async disconnect(): Promise<void> {
-    if (this.audioRenderer) {
-      this.audioRenderer.destroy();
-      this.audioRenderer = null;
-    }
-
-    await this.client.disconnect();
-
-    if (this.audioContext && this.audioContext.state !== "closed") {
-      await this.audioContext.close();
-      this.audioContext = null;
-    }
-  }
-}
-```
-
-### AudioContext guidance
-
-- Create/resume `AudioContext` only after user interaction in browsers that enforce autoplay policy.
-- If you are not processing audio with WebAudio, you do not need a custom `AudioContext`; `AudioRenderer` is enough for playback.
-- Always close your custom `AudioContext` in teardown.
-
-### Lifecycle and cleanup order
-
-Recommended shutdown order:
-
-1. Stop UI input loops/listeners
-2. Destroy `AudioRenderer`
-3. Disconnect `ConvaiClient`
-4. Close custom `AudioContext` (if created)
-
-### Common failure modes and fixes
-
-| Symptom                 | Likely cause                              | Recommended action                                                                    |
-| ----------------------- | ----------------------------------------- | ------------------------------------------------------------------------------------- |
-| No AI audio output      | `AudioRenderer` not created               | Instantiate `new AudioRenderer(client.room)` immediately after successful connect.    |
-| No AI audio output      | Browser autoplay restriction              | Trigger connect/playback from a user click, and resume `AudioContext` if suspended.   |
-| No AI audio output      | TTS disabled                              | Ensure `ttsEnabled` is true for sessions that need speech output.                     |
-| Intermittent playback   | Multiple renderers or stale room instance | Use one renderer per session and always destroy old renderer before reconnecting.     |
-| Works once, then silent | Incomplete cleanup on previous session    | Destroy renderer and disconnect client on teardown; avoid reusing invalid room state. |
-| Random muted behavior   | App-side muting of remote tracks          | Verify no custom code is muting remote publications or media elements.                |
-
-## 14. Error Handling and Reliability Patterns
-
-### Pattern 1: Centralized SDK error handling
-
-```ts
-const unsubError = client.on("error", (error) => {
-  console.error("Convai SDK error:", error);
-  // Optional: route to telemetry/monitoring
-});
-```
-
-### Pattern 2: Retry connect with exponential backoff
-
-```ts
-async function connectWithRetry(
-  client: any,
-  attempts = 3,
-  initialDelayMs = 500,
-): Promise<void> {
-  let delay = initialDelayMs;
-
-  for (let i = 1; i <= attempts; i++) {
-    try {
-      await client.connect();
-      return;
-    } catch (error) {
-      if (i === attempts) throw error;
-      await new Promise((resolve) => setTimeout(resolve, delay));
-      delay *= 2;
-    }
-  }
-}
-```
-
-### Pattern 3: Safe send guard
-
-```ts
-function safeSendText(client: any, text: string) {
-  if (!text.trim()) return;
-  if (!client.state.isConnected) return;
-  if (!client.isBotReady) return;
-  client.sendUserTextMessage(text);
-}
-```
-
-### Pattern 4: Protect media control calls
-
-```ts
-async function safeToggleMic(client: any) {
-  try {
-    await client.audioControls.toggleAudio();
-  } catch (error) {
-    console.error("Failed to toggle microphone:", error);
-  }
-}
-```
-
-### Pattern 5: Always unsubscribe listeners
-
-```ts
-const unsubscribers = [
-  client.on("stateChange", () => {}),
-  client.on("messagesChange", () => {}),
-];
-
-function cleanupListeners() {
-  for (const unsub of unsubscribers) unsub();
-}
-```
-
-## 15. Troubleshooting
-
-### Connection issues
-
-- Verify API key and character ID are valid.
-- Ensure requests are allowed from your browser origin.
-- Set `url` explicitly if your environment does not use the SDK default endpoint.
-- Listen to `error` and inspect failed network calls in browser devtools.
-
-### `connect()` succeeds but bot never responds
-
-- Wait for `botReady` before sending messages.
-- Confirm `ttsEnabled` and message flow are configured as expected.
-- Verify `messagesChange` receives content.
-
-### Audio does not play
-
-- Ensure an `AudioRenderer` is active for the connected room (vanilla custom UI).
-- Ensure playback starts from a user gesture path to satisfy autoplay policies.
-- Confirm no custom muting code is muting remote tracks.
-
-### Microphone does not capture user voice
-
-- Ensure app is served over secure context.
-- Verify browser microphone permission.
-- Handle permission errors from `audioControls.enableAudio()/unmuteAudio()`.
-
-### Video or screen share controls fail
-
-- Use `enableVideo: true` in config when you need video capabilities.
-- Screen share can be blocked by browser policy or user denial.
-- Wrap calls in `try/catch` and provide fallback UX.
-
-### Lipsync appears out of sync or shape
-
-- Validate blendshape format (`arkit` vs `mha`) matches your rig expectations.
-- Tune `frames_buffer_duration` so you atleast have some duration of blendshapes before the audio starts playing.
-- Align lipsync start and stop with the queue: start playback when the bot starts speaking (`isBotSpeaking()` true) and treat the turn as finished when `blendshapeQueue.isConversationEnded()` is true before resetting.
-- Drive blendshape application from a single loop (e.g. `requestAnimationFrame`) and advance frame index at 60fps so mouth movement stays in sync with audio.
+---
 
-## 17. Examples
+## Example app
 
-Repository examples:
+A full React Three Fiber integration demo is in [`examples/react-three-fiber`](./examples/react-three-fiber/). It demonstrates lipsync on a 3D character, emotion display, dynamic context, and file upload.
 
-- `examples/react-three-fiber`
-- `examples/three-vanilla`
-- `examples/README.md` for example-level setup notes