@robinandeer/rtc-session-components 0.1.0 → 0.2.0

package/NOTICES

@@ -0,0 +1,41 @@
+This project includes the following third-party dependencies:
+
+================================================================================
+livekit-client
+https://github.com/livekit/client-sdk-js
+--------------------------------------------------------------------------------
+Licensed under the Apache License, Version 2.0
+
+Copyright 2023 LiveKit, Inc.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+================================================================================
+@livekit/components-react
+https://github.com/livekit/components-js
+--------------------------------------------------------------------------------
+Licensed under the Apache License, Version 2.0
+
+Copyright 2023 LiveKit, Inc.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

package/README.md

@@ -2,201 +2,317 @@
 
 React SDK for real-time AI avatar interactions with GWM-1.
 
+## Requirements
+
+- React 18+
+- A Runway API secret ([get one here](https://dev.runwayml.com/))
+- A server-side endpoint to create sessions (API secrets must not be exposed to the client)
+
 ## Installation
 
 ```bash
 npm install @runwayml/avatar-react
-# or
-bun add @runwayml/avatar-react
 ```
 
 ## Quick Start
 
+Add an avatar call to your app with just a few lines:
+
 ```tsx
-import { AvatarProvider, useAvatar, AvatarCanvas } from '@runwayml/avatar-react';
+import { AvatarCall } from '@runwayml/avatar-react';
 
 function App() {
   return (
-    <AvatarProvider config={{ apiKey: process.env.RUNWAYML_API_KEY! }}>
-      <AvatarDemo />
-    </AvatarProvider>
+    <AvatarCall
+      avatarId="game-host"
+      connectUrl="/api/avatar/connect"
+    />
   );
 }
+```
 
-function AvatarDemo() {
-  const { connect, disconnect, speak, isConnected, isConnecting } = useAvatar();
+That's it! The component handles session creation, WebRTC connection, and renders a default UI with the avatar video and controls.
 
-  const handleConnect = async () => {
-    await connect({ modelId: 'gwm-1' });
-  };
+You can use preset avatars like `game-host`, `coding-teacher`, `language-tutor`, and more. See the [Runway Developer Portal](https://dev.runwayml.com/) for the full list and creating custom avatars.
 
-  return (
-    <div>
-      <AvatarCanvas fallback={<div>Not connected</div>} showFps />
-
-      {!isConnected ? (
-        <button onClick={handleConnect} disabled={isConnecting}>
-          {isConnecting ? 'Connecting...' : 'Connect'}
-        </button>
-      ) : (
-        <>
-          <button onClick={() => speak('Hello, world!')}>Say Hello</button>
-          <button onClick={disconnect}>Disconnect</button>
-        </>
-      )}
-    </div>
-  );
+### Optional: Add Default Styles
+
+Import the optional stylesheet for a polished look out of the box:
+
+```tsx
+import '@runwayml/avatar-react/styles.css';
+```
+
+The styles use CSS custom properties for easy customization:
+
+```css
+:root {
+  --avatar-bg: #a78bfa;           /* Video background color */
+  --avatar-radius: 16px;          /* Container border radius */
+  --avatar-control-size: 48px;    /* Control button size */
+  --avatar-end-call-bg: #ef4444;  /* End call button color */
 }
 ```
 
-## API Reference
+See [`examples/`](./examples) for complete working examples:
+- [`nextjs`](./examples/nextjs) - Next.js App Router
+- [`react-router`](./examples/react-router) - React Router v7 framework mode
+- [`express`](./examples/express) - Express + Vite
 
-### Components
+## How It Works
 
-#### `<AvatarProvider>`
+1. **Client** calls your server endpoint with the `avatarId`
+2. **Server** uses your Runway API secret to create a session via `@runwayml/sdk`
+3. **Server** returns connection credentials (token, URL) to the client
+4. **Client** establishes a WebRTC connection for real-time video/audio
 
-Provides the avatar client context to child components.
+This flow keeps your API secret secure on the server while enabling low-latency communication.
 
-```tsx
-<AvatarProvider config={{ apiKey: 'your-api-key', debug: true }}>{children}</AvatarProvider>
-```
+## Server Setup
 
-**Props:**
+Your server endpoint receives the `avatarId` and returns session credentials. Use `@runwayml/sdk` to create the session:
 
-- `config.apiKey` (required) - Your RunwayML API key
-- `config.baseUrl` - Custom API base URL
-- `config.wsUrl` - Custom WebSocket URL
-- `config.timeout` - Request timeout in ms (default: 30000)
-- `config.maxRetries` - Max retry attempts (default: 2)
-- `config.debug` - Enable debug logging (default: false)
+```ts
+// /api/avatar/connect (Next.js App Router example)
+import Runway from '@runwayml/sdk';
 
-#### `<AvatarCanvas>`
+const runway = new Runway(); // Uses RUNWAYML_API_SECRET env var
 
-Canvas component that automatically renders avatar video frames.
+export async function POST(req: Request) {
+  const { avatarId } = await req.json();
 
-```tsx
-<AvatarCanvas showFps targetFps={30} fallback={<Placeholder />} style={{ width: '100%' }} />
+  const session = await runway.realtime.sessions.create({
+    model: 'calliope',
+    options: { avatar: avatarId },
+  });
+
+  return Response.json({
+    sessionId: session.id,
+    serverUrl: session.url,
+    token: session.token,
+    roomName: session.room_name,
+  });
+}
 ```
 
-### Hooks
+## Customization
 
-#### `useAvatar(options?)`
+### Custom Connect Function
 
-Main hook for managing avatar sessions.
+For more control over the connection flow:
 
 ```tsx
-const {
-  session,
-  connectionState,
-  avatarState,
-  isConnected,
-  isConnecting,
-  error,
-  connect,
-  disconnect,
-  speak,
-  sendAudio,
-  setEmotion,
-  setGaze,
-  interrupt,
-} = useAvatar({
-  autoConnect: false,
-  sessionConfig: { modelId: 'gwm-1' },
-  onConnected: () => console.log('Connected!'),
-  onError: (error) => console.error(error),
-});
-```
-
-#### `useAvatarVideo(options?)`
-
-Hook for handling avatar video frames.
+<AvatarCall
+  avatarId="game-host"
+  connect={async (avatarId) => {
+    const res = await fetch('/api/avatar/connect', {
+      method: 'POST',
+      headers: {
+        'Authorization': `Bearer ${token}`,
+        'Content-Type': 'application/json',
+      },
+      body: JSON.stringify({ avatarId }),
+    });
+    return res.json();
+  }}
+/>
+```
+
+### Custom UI with Child Components
+
+Use the built-in components for custom layouts:
 
 ```tsx
-const { canvasRef, dimensions, fps, isPlaying, frameCount } = useAvatarVideo({
-  targetFps: 30,
-  autoPlay: true,
-  onFrame: (frame) => console.log('Frame received'),
-});
+import { AvatarCall, AvatarVideo, ControlBar, UserVideo } from '@runwayml/avatar-react';
+
+<AvatarCall avatarId="game-host" connectUrl="/api/avatar/connect">
+  <div className="call-layout">
+    <AvatarVideo className="avatar" />
+    <UserVideo className="self-view" />
+    <ControlBar className="controls" />
+  </div>
+</AvatarCall>
 ```
 
-#### `useAvatarAudio(options?)`
+### Render Props
 
-Hook for handling avatar audio output.
+All components support render props for complete control:
 
 ```tsx
-const { isPlaying, isMuted, volume, setVolume, toggleMute } = useAvatarAudio({
-  autoPlay: true,
-  volume: 1,
-});
+<AvatarVideo>
+  {({ hasVideo, isConnecting, isSpeaking, trackRef }) => (
+    <div className={isSpeaking ? 'speaking' : ''}>
+      {isConnecting && <Spinner />}
+      {hasVideo && <VideoTrack trackRef={trackRef} />}
+    </div>
+  )}
+</AvatarVideo>
 ```
 
-#### `useAvatarMicrophone(options?)`
+### CSS Styling with Data Attributes
 
-Hook for capturing and sending microphone audio.
+Style connection states with CSS:
 
 ```tsx
-const { isActive, hasPermission, audioLevel, start, stop, toggleMute } = useAvatarMicrophone({
-  sampleRate: 48000,
-  echoCancellation: true,
-  noiseSuppression: true,
-});
+<AvatarCall avatarId="game-host" connectUrl="/api/avatar/connect" className="my-avatar" />
 ```
 
-### Types
+```css
+.my-avatar[data-state="connecting"] {
+  opacity: 0.5;
+}
 
-```typescript
-interface AvatarSessionConfig {
-  modelId: string;
-  resolution?: { width: number; height: number };
-  audio?: AudioConfig;
-  initialState?: Partial<AvatarState>;
+.my-avatar[data-state="error"] {
+  border: 2px solid red;
 }
 
-interface AvatarState {
-  emotion: 'neutral' | 'happy' | 'sad' | 'surprised' | 'angry' | 'confused' | 'thinking';
-  isSpeaking: boolean;
-  isListening: boolean;
-  gazeDirection: { x: number; y: number };
+.my-avatar[data-state="connected"] {
+  border: 2px solid green;
 }
+```
 
-type ConnectionState = 'disconnected' | 'connecting' | 'connected' | 'reconnecting' | 'error';
+## Callbacks
+
+```tsx
+<AvatarCall
+  avatarId="game-host"
+  connectUrl="/api/avatar/connect"
+  onEnd={() => console.log('Call ended')}
+  onError={(error) => console.error('Error:', error)}
+/>
 ```
 
-## Direct Client Usage
+## Hooks
 
-For advanced use cases, you can use the client directly:
+Use hooks for custom components within an `AvatarCall` or `AvatarSession`:
 
-```typescript
-import { createAvatarClient } from '@runwayml/avatar-react';
+### useAvatarSession
 
-const client = createAvatarClient({ apiKey: 'your-api-key' });
+Access session state and controls:
 
-await client.createSession({ modelId: 'gwm-1' });
+```tsx
+function MyComponent() {
+  const { state, sessionId, error, end } = useAvatarSession();
 
-client.on('videoFrame', (event) => {
-  // Handle video frame
-});
+  if (state === 'connecting') return <Loading />;
+  if (state === 'error') return <Error message={error.message} />;
 
-client.speak('Hello!');
-client.disconnect();
+  return <button onClick={end}>End Call</button>;
+}
 ```
 
-## Development
+### useAvatar
 
-```bash
-# Install dependencies
-bun install
+Access the remote avatar's video/audio:
 
-# Build
-bun run build
+```tsx
+function CustomAvatar() {
+  const { videoTrackRef, isSpeaking, hasVideo, hasAudio } = useAvatar();
 
-# Type check
-bun run typecheck
+  return (
+    <div data-speaking={isSpeaking}>
+      {hasVideo && <VideoTrack trackRef={videoTrackRef} />}
+    </div>
+  );
+}
+```
+
+### useLocalMedia
+
+Control local camera and microphone:
+
+```tsx
+function MediaControls() {
+  const {
+    isMicEnabled,
+    isCameraEnabled,
+    toggleMic,
+    toggleCamera,
+    toggleScreenShare,
+  } = useLocalMedia();
+
+  return (
+    <div>
+      <button onClick={toggleMic}>{isMicEnabled ? 'Mute' : 'Unmute'}</button>
+      <button onClick={toggleCamera}>{isCameraEnabled ? 'Hide' : 'Show'}</button>
+    </div>
+  );
+}
+```
+
+## Advanced: AvatarSession
+
+For full control over session management, use `AvatarSession` directly with pre-fetched credentials:
+
+```tsx
+import { AvatarSession, AvatarVideo, ControlBar } from '@runwayml/avatar-react';
+
+function AdvancedUsage({ credentials }) {
+  return (
+    <AvatarSession
+      credentials={credentials}
+      audio={true}
+      video={true}
+      onEnd={() => console.log('Ended')}
+      onError={(err) => console.error(err)}
+    >
+      <AvatarVideo />
+      <ControlBar />
+    </AvatarSession>
+  );
+}
+```
+
+## Components Reference
 
-# Run tests
-bun test
+| Component | Description |
+|-----------|-------------|
+| `AvatarCall` | High-level component that handles session creation |
+| `AvatarSession` | Low-level wrapper that requires credentials |
+| `AvatarVideo` | Renders the remote avatar video |
+| `UserVideo` | Renders the local user's camera |
+| `ControlBar` | Media control buttons (mic, camera, end call) |
+| `ScreenShareVideo` | Renders screen share content |
+| `AudioRenderer` | Handles avatar audio playback |
+
+## TypeScript
+
+All components and hooks are fully typed:
+
+```tsx
+import type {
+  AvatarCallProps,
+  SessionCredentials,
+  SessionState,
+} from '@runwayml/avatar-react';
 ```
 
+## Browser Support
+
+This SDK uses WebRTC for real-time communication. Supported browsers:
+
+- Chrome 74+
+- Firefox 78+
+- Safari 14.1+
+- Edge 79+
+
+Users must grant camera and microphone permissions when prompted.
+
+## Troubleshooting
+
+**"Failed to connect" or timeout errors**
+- Verify your server endpoint is returning the correct credential format
+- Check that `RUNWAYML_API_SECRET` is set correctly on your server
+
+**No video/audio**
+- Ensure the user has granted camera/microphone permissions
+- Check browser console for WebRTC errors
+- Verify the device has a working camera/microphone
+
+**CORS errors**
+- Your server endpoint must accept requests from your client's origin
+- For local development, ensure both client and server are on compatible origins
+
 ## License
 
 MIT