@keyframelabs/sdk 0.1.3 → 0.1.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +85 -87
  2. package/package.json +2 -2
package/README.md CHANGED
@@ -1,44 +1,51 @@
1
1
  # @keyframelabs/sdk
2
2
 
3
- Browser SDK for KeyframeLab's Persona avatar sessions. Connect any voice AI agent to a real-time avatar.
3
+ The universal, low-level SDK for Keyframe Labs.
4
+
5
+ **Which package should I use?**
6
+ - **@keyframelabs/sdk** (high control)
7
+ - You implement the UI, state management, and agent/llm binding yourself
8
+ - **@keyframelabs/elements** (custom UI)
9
+ - You implement the UI; we handle the state and agent/llm binding (framework-agnostic)
10
+ - **@keyframelabs/react**: (drop-in)
11
+ - We handle the UI, state, and agent/llm binding
4
12
 
5
13
  ## Installation
6
14
 
7
15
  ```bash
8
- npm install @keyframelabs/sdk
16
+ pnpm add @keyframelabs/sdk
9
17
  ```
10
18
 
11
- ## Quick Start
19
+ ## Quick start
20
+
21
+ ### 1. Server-side: create a session
12
22
 
13
- On your node / python backend, create a session using your KeyframeLabs API key.
23
+ From your backend, use your secret Keyframe API key to create a session.
14
24
 
15
25
  ```typescript
16
- const response = await fetch('https://api.keyframelabs.com/v1/session'
26
+ // POST https://api.keyframelabs.com/v1/session
27
+ const response = await fetch('https://api.keyframelabs.com/v1/session', {
17
28
  method: 'POST',
18
29
  headers: {
19
30
  'Content-Type': 'application/json',
20
- Authorization: `Bearer ${KFL_API_KEY}`,
31
+ 'Authorization': `Bearer ${process.env.KFL_API_KEY}`,
21
32
  },
22
33
  body: JSON.stringify({
23
- persona_id: "luna", // or cosmo or astro, etc.
24
- model_id: "persona-1-live"
34
+ persona_id: "6efx..." // or persona_slug
25
35
  }),
26
- );
27
-
28
- if (!response.ok) {
29
- throw new Error(`HTTP error! status: ${response.status}`);
30
- }
36
+ });
31
37
 
32
- // Contains your serverUrl and participantToken
33
- const result = await response.json();
38
+ // Returns { serverUrl, participantToken }
39
+ const session = await response.json();
34
40
  ```
35
41
 
36
- Then, using the client on the browser:
42
+ ### 2. Client-side: connect the client
43
+
44
+ Pass the session details to the client SDK.
37
45
 
38
46
  ```typescript
39
47
  import { createClient } from '@keyframelabs/sdk';
40
48
 
41
- // Create a Persona client
42
49
  const persona = createClient({
43
50
  serverUrl: "wss://...",
44
51
  participantToken: "A6gB...",
@@ -52,97 +59,88 @@ const persona = createClient({
52
59
  },
53
60
  });
54
61
 
55
- // Connect to the avatar
56
- await persona.connect();
62
+ // Connect to the session
63
+ await client.connect();
57
64
 
58
- // Send audio from your voice AI agent (24kHz 16-bit PCM)
59
- persona.sendAudio(pcmAudioBytes);
65
+ // Send audio (e.g. from your LLM/Agent)
66
+ client.sendAudio(pcmAudioBytes);
60
67
 
61
68
  // Signal an interruption (clears pending frames)
62
69
  persona.interrupt();
63
70
 
64
- // Close when done
65
- await persona.close();
71
+ // Cleanup
72
+ await client.close();
66
73
  ```
67
74
 
68
- ## API
69
-
70
- ### `createClient(options)`
71
-
72
- Create a new Persona client.
73
-
74
- **Options:**
75
- - `personaId` - Persona ID (e.g., 'luna', 'cosmo')
76
- - `apiUrl` - Central API URL (optional)
77
- - `apiKey` - API key for authentication (optional)
78
- - `onVideoTrack` - Callback when video track is available
79
- - `onAudioTrack` - Callback when audio track is available
80
- - `onStateChange` - Callback when session state changes
81
- - `onError` - Callback on error
82
-
83
- ### `PersonaSession`
84
-
85
- The client instance returned by `createClient()`.
75
+ ## Architecture
86
76
 
87
- **Methods:**
88
- - `connect()` - Connect to the avatar session
89
- - `sendAudio(pcmData)` - Send 24kHz 16-bit PCM audio
90
- - `interrupt()` - Signal an interruption (clears pending frames)
91
- - `close()` - Close the session
77
+ The SDK handles the real-time transport loop between your agent or real-time LLM and the Keyframe Platform, as well as synced audio/video rendering.
92
78
 
93
- **Properties:**
94
- - `state` - Current session state ('disconnected', 'connecting', 'connected', 'error')
79
+ ```
80
+ +-----------------------+ +-----------------------+
81
+ | Browser | | Keyframe Platform |
82
+ | | | |
83
+ | +-----------------+ | | +-----------------+ |
84
+ | | Microphone | | | | AvatarSession | |
85
+ | +-----------------+ | | +-----------------+ |
86
+ | | | | ^ |
87
+ | v | | | |
88
+ | +-----------------+ | DataStream | | |
89
+ | | Agent / LLM | | (PCM 16kHz) | | |
90
+ | +-----------------+ | ----------------------> | | |
91
+ | | | | | |
92
+ | v | | v |
93
+ | +-----------------+ | | +-----------------+ |
94
+ | | PersonaSession | | | | Inference | |
95
+ | +-----------------+ | | +-----------------+ |
96
+ | ^ | | | |
97
+ | | | WebRTC | | |
98
+ | | | (Audio + Video) | v |
99
+ | +-----------------+ | <---------------------- | +-----------------+ |
100
+ | | Video Element | | | | Video | |
101
+ | +-----------------+ | | +-----------------+ |
102
+ | | | |
103
+ +-----------------------+ +-----------------------+
104
+ ```
95
105
 
96
- ## Integrating Voice AI Agents
106
+ ## Integrating a specific agent or real-time LLM
97
107
 
98
- The SDK is intentionally minimal - it only handles the avatar connection. You bring your own voice AI agent (Gemini, ElevenLabs, OpenAI, etc.).
108
+ The SDK is intentionally minimal—it only handles the avatar connection. You bring your own agent or real-time LLM (e.g, Cartesia, ElevenLabs, Gemini, OpenAI).
99
109
 
100
- Example integration pattern:
110
+ ## API
101
111
 
102
- ```typescript
103
- import { createClient } from '@keyframelabs/sdk';
104
- // Your agent implementation (copy from experiments/src/agents/)
105
- import { GeminiLiveAgent } from './agents/gemini-live';
112
+ ### `createClient(options)`
106
113
 
107
- const persona = createClient({
108
- personaId: 'luna',
109
- onVideoTrack: (track) => {
110
- videoElement.srcObject = new MediaStream([track]);
111
- },
112
- });
114
+ Initializes the WebSocket connection and media managers.
113
115
 
114
- const agent = new GeminiLiveAgent();
116
+ | Option | Type | Default | Description |
117
+ | ------------------ | ----------------- | -------- | --------------------------------------------------------- |
118
+ | `serverUrl` | `string` | Required | The WSS URL returned by the `/session` API endpoint. |
119
+ | `participantToken` | `string` | Required | The access token returned by the `/session` API endpoint. |
120
+ | `onVideoTrack` | `(track) => void` | Required | Fired when the WebRTC video track is ready. |
121
+ | `onAudioTrack` | `(track) => void` | Required | Fired when the WebRTC audio track is ready. |
122
+ | `onStateChange` | `(state) => void` | — | Fired when session state changes. |
123
+ | `onError` | `(err) => void` | — | Fired on errors. |
115
124
 
116
- // Wire agent audio to persona
117
- agent.on('audio', (pcmData) => persona.sendAudio(pcmData));
125
+ ### `PersonaSession`
118
126
 
119
- // Handle interruptions
120
- agent.on('interrupted', () => persona.interrupt());
127
+ The client instance returned by `createClient()`.
121
128
 
122
- // Connect both
123
- await persona.connect();
124
- await agent.connect({ apiKey: 'your-gemini-key' });
129
+ #### Methods
125
130
 
126
- // Send microphone audio to agent
127
- // (capture and send PCM to agent.sendAudio())
128
- ```
131
+ | Method | Signature | Description |
132
+ | ----------- | ---------------------------------------------- | ------------------------------------------------ |
133
+ | `connect` | `() => Promise<void>` | Connect to the session. |
134
+ | `sendAudio` | `(pcmData: ArrayBuffer \| Int16Array) => void` | Send 24 kHz 16-bit PCM audio. |
135
+ | `interrupt` | `() => void` | Signal an interruption and clear pending frames. |
136
+ | `close` | `() => void` | Close the session and release resources. |
129
137
 
130
- ## Architecture
138
+ #### Properties
131
139
 
132
- ```
133
- Browser GPU Node
134
- ┌─────────────────┐ ┌─────────────────┐
135
- │ Microphone │──PCM 16kHz──▶│ │
136
- │ ↓ │ │ │
137
- │ Voice AI Agent │ │ AvatarSession │
138
- │ ↓ │ │ │
139
- │ PersonaSession │──DataStream─▶│ ↓ │
140
- │ ↑ │ │ Inference │
141
- │ Video Element │◀──WebRTC────│ ↓ │
142
- └─────────────────┘ │ Video │
143
- └─────────────────┘
144
- ```
140
+ | Property | Type | Description |
141
+ | -------- | ---------------------------------------------------------- | ---------------------- |
142
+ | `state` | `'disconnected' \| 'connecting' \| 'connected' \| 'error'` | Current session state. |
145
143
 
146
144
  ## License
147
145
 
148
- MIT
146
+ MIT
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@keyframelabs/sdk",
3
- "version": "0.1.3",
4
- "description": "Browser SDK for KeyframeLab's Persona avatar sessions",
3
+ "version": "0.1.5",
4
+ "description": "The universal, low-level SDK for Keyframe Labs.",
5
5
  "type": "module",
6
6
  "main": "./dist/index.js",
7
7
  "types": "./dist/index.d.ts",