@talkif/webrtc 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +178 -0
- package/package.json +1 -1
package/README.md
ADDED
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
# Talkif WebRTC JS
|
|
2
|
+
|
|
3
|
+
Browser client libraries for talking to [Talkif](https://talkif.ai) AI voice agents over WebRTC.
|
|
4
|
+
|
|
5
|
+
| Package | Description |
|
|
6
|
+
|---|---|
|
|
7
|
+
| [`@talkif/webrtc`](packages/core) | Framework-agnostic core: signaling, peer connection, audio, data-channel protocol, realtime events |
|
|
8
|
+
| [`@talkif/webrtc-react`](packages/react) | Headless React hook (`useTalkifCall`) on top of the core |
|
|
9
|
+
|
|
10
|
+
Both packages are headless — no UI ships with them. Requires a browser with WebRTC and microphone access; the React package needs React ≥ 18.
|
|
11
|
+
|
|
12
|
+
## Installation
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
npm install @talkif/webrtc
|
|
16
|
+
# React bindings (peer-depends on @talkif/webrtc)
|
|
17
|
+
npm install @talkif/webrtc-react
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Two modes
|
|
21
|
+
|
|
22
|
+
The libraries speak to the Talkif API in one of two modes, chosen by the config shape you pass:
|
|
23
|
+
|
|
24
|
+
**Authenticated mode** — for your own dashboard or backend-authenticated apps. You supply the account id and an `auth` provider returning the `Authorization` header value. Call any published flow, or test a draft flow definition.
|
|
25
|
+
|
|
26
|
+
```ts
|
|
27
|
+
const config = {
|
|
28
|
+
baseUrl: 'https://api.talkif.ai',
|
|
29
|
+
accountId: '<account-id>',
|
|
30
|
+
auth: async () => `Bearer ${await getToken()}`,
|
|
31
|
+
};
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
**Public (embed) mode** — for widgets on any web page. You ship only a publishable key (`pk_live_...`, safe in page source); the library exchanges it for a short-lived session token itself. The page's origin must be on the key's allowlist, the flow is bound to the key, and realtime call events (transcripts, word timing, speaking indicators) stream over an events WebSocket.
|
|
35
|
+
|
|
36
|
+
```ts
|
|
37
|
+
const config = {
|
|
38
|
+
baseUrl: 'https://api.talkif.ai',
|
|
39
|
+
publishableKey: 'pk_live_...',
|
|
40
|
+
};
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
If the flow's bot gate is active, the library handles the Cloudflare Turnstile challenge invisibly — no setup needed. To supply your own token instead, pass `turnstileToken: () => Promise<string>`.
|
|
44
|
+
|
|
45
|
+
## Quick start (core)
|
|
46
|
+
|
|
47
|
+
```ts
|
|
48
|
+
import { TalkifCall } from '@talkif/webrtc';
|
|
49
|
+
|
|
50
|
+
const call = new TalkifCall(config);
|
|
51
|
+
|
|
52
|
+
call.on('connected', ({ callId }) => console.log('live', callId));
|
|
53
|
+
call.on('transcript', ({ role, content }) => console.log(role, content));
|
|
54
|
+
call.on('ended', ({ reason }) => console.log('ended', reason));
|
|
55
|
+
call.on('error', ({ error }) => console.error(error.code, error.message));
|
|
56
|
+
|
|
57
|
+
await call.start({ flowId: '<published-flow-id>' }); // flowId ignored in public mode
|
|
58
|
+
|
|
59
|
+
// later
|
|
60
|
+
call.setMuted(true);
|
|
61
|
+
call.hangup();
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
### `TalkifCall` API
|
|
65
|
+
|
|
66
|
+
| Member | Description |
|
|
67
|
+
|---|---|
|
|
68
|
+
| `start(options)` | Request mic, create the call, connect. `options`: `flowId`, `draftDefinition` (authenticated test calls), `metadata`, `audio` (getUserMedia constraints), `audioElement` (your own `<audio>`, or `null` to handle the `track` event yourself) |
|
|
69
|
+
| `hangup()` | End the call locally. Always safe to call. |
|
|
70
|
+
| `dispose()` | Tear everything down (mic, peer connection, timers, listeners). Instances are single-use. |
|
|
71
|
+
| `setMuted(muted)` | Toggle the local microphone track. |
|
|
72
|
+
| `sendAppMessage(json)` | Send an arbitrary JSON message to the bot over the data channel. Returns `false` if the channel isn't open. |
|
|
73
|
+
| `on(event, fn)` / `off(event, fn)` | Subscribe / unsubscribe. `on` returns an unsubscribe function. |
|
|
74
|
+
| `state`, `callId`, `botId`, `muted`, `durationSecs` | Current snapshot getters. |
|
|
75
|
+
|
|
76
|
+
State machine: `idle → requesting → connecting → connected → ended` (or `error`).
|
|
77
|
+
|
|
78
|
+
### Events
|
|
79
|
+
|
|
80
|
+
| Event | Payload | When |
|
|
81
|
+
|---|---|---|
|
|
82
|
+
| `statechange` | `{ state, previous }` | Every state transition |
|
|
83
|
+
| `connected` | `{ callId, botId }` | Media flowing, answer applied |
|
|
84
|
+
| `track` | `{ stream }` | Remote (bot) audio track arrived |
|
|
85
|
+
| `tick` | `{ durationSecs }` | Once per second while connected |
|
|
86
|
+
| `mutechange` | `{ muted }` | Mute toggled |
|
|
87
|
+
| `appmessage` | `{ message }` | JSON from the bot over the data channel |
|
|
88
|
+
| `renegotiated` | `{ ok }` | Bot-initiated renegotiation attempted |
|
|
89
|
+
| `ended` | `{ reason }` | `local-hangup` \| `peer-left` \| `terminal-status` \| `external` |
|
|
90
|
+
| `error` | `{ error: TalkifCallError }` | Fatal failure; call is torn down |
|
|
91
|
+
|
|
92
|
+
**Realtime call events** (public mode — delivered over the events WebSocket):
|
|
93
|
+
|
|
94
|
+
| Event | Payload | Notes |
|
|
95
|
+
|---|---|---|
|
|
96
|
+
| `transcript` | `{ role, content, data }` | Final transcript line for a turn. On interruption, `content` includes text synthesized but never spoken. |
|
|
97
|
+
| `interim` | `{ data }` | Caller's in-progress transcription. Droppable under backpressure. |
|
|
98
|
+
| `ttschunk` | `{ text, timestamp, data }` | Sentence-level chunk of the agent's reply, emitted when the LLM output reaches synthesis — ahead of audio playback. Droppable. |
|
|
99
|
+
| `ttsword` | `{ word, ptsMs, timestamp, data }` | Word-level timing at playback pace (karaoke-style captions). Only words actually spoken. Droppable. |
|
|
100
|
+
| `callevent` | `{ event }` | Every delivered envelope, including types without a dedicated event (`speech`, `turn`, `node_transition`, `status`, `error`). |
|
|
101
|
+
|
|
102
|
+
"Droppable" events may be shed by the server under backpressure; the final `transcript` is authoritative — reconcile incremental displays against it.
|
|
103
|
+
|
|
104
|
+
### Errors
|
|
105
|
+
|
|
106
|
+
`error.code` on `TalkifCallError`:
|
|
107
|
+
|
|
108
|
+
| Code | Meaning |
|
|
109
|
+
|---|---|
|
|
110
|
+
| `media-unavailable` | Microphone denied or unavailable |
|
|
111
|
+
| `signaling-failed` | Call creation / SDP exchange failed |
|
|
112
|
+
| `connection-failed` | ICE/peer connection failed |
|
|
113
|
+
| `pipeline-dead` | Connected, but the bot pipeline never became healthy |
|
|
114
|
+
| `renegotiation-failed` | Bot-requested renegotiation failed |
|
|
115
|
+
| `session-rejected` | Publishable key rejected, origin not allowed, or session expired |
|
|
116
|
+
| `call-active` | This session already has an active call (another tab) |
|
|
117
|
+
| `rate-limited` | Rate/concurrency limit hit — retry later |
|
|
118
|
+
| `no-agent` | No agent available right now — retry shortly |
|
|
119
|
+
|
|
120
|
+
## Quick start (React)
|
|
121
|
+
|
|
122
|
+
```tsx
|
|
123
|
+
import { useTalkifCall } from '@talkif/webrtc-react';
|
|
124
|
+
|
|
125
|
+
function CallWidget({ config }) {
|
|
126
|
+
const { state, durationSecs, muted, start, hangup, toggleMute } = useTalkifCall({
|
|
127
|
+
config,
|
|
128
|
+
onTranscript: ({ role, content }) => console.log(role, content),
|
|
129
|
+
onTtsChunk: ({ text }) => console.log('agent (streaming):', text),
|
|
130
|
+
onEnded: (reason) => console.log('ended', reason),
|
|
131
|
+
onError: (error) => console.error(error.code),
|
|
132
|
+
});
|
|
133
|
+
|
|
134
|
+
return state === 'connected' ? (
|
|
135
|
+
<>
|
|
136
|
+
<span>{durationSecs}s</span>
|
|
137
|
+
<button onClick={toggleMute}>{muted ? 'Unmute' : 'Mute'}</button>
|
|
138
|
+
<button onClick={hangup}>Hang up</button>
|
|
139
|
+
</>
|
|
140
|
+
) : (
|
|
141
|
+
<button onClick={() => start({})} disabled={state === 'requesting' || state === 'connecting'}>
|
|
142
|
+
Start call
|
|
143
|
+
</button>
|
|
144
|
+
);
|
|
145
|
+
}
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
`useTalkifCall` returns `{ state, callId, botId, error, durationSecs, muted, call, start, hangup, toggleMute, sendAppMessage }`. Callbacks: `onAppMessage`, `onCallEvent`, `onTranscript`, `onTtsWord`, `onTtsChunk`, `onEnded`, `onError`. One active call at a time — starting a new call disposes the previous one, and unmount tears everything down (mic, peer connection, timers).
|
|
149
|
+
|
|
150
|
+
## What the core handles for you
|
|
151
|
+
|
|
152
|
+
- **Signaling** against the Talkif API: call creation (published or draft flows), TURN credential fetch, SDP offer/answer relay, public session token exchange and refresh.
|
|
153
|
+
- **Fast connect**: relay-only ICE with first-relay-candidate dispatch (no 5s+ gathering stalls), bot warmup runs in parallel with local media setup.
|
|
154
|
+
- **Data-channel protocol** spoken by Talkif bots: keepalive pings, `peerLeft` teardown, bot-initiated renegotiation, arbitrary JSON app messages both ways.
|
|
155
|
+
- **Realtime events** (public mode): WebSocket with auto-reconnect, backoff, and server-side replay after a gap — a dropped connection never loses call history.
|
|
156
|
+
- **Health watchdog**: if no health signal arrives after connect, call liveness is verified over REST before anything is torn down — a silent event stream never kills a healthy call.
|
|
157
|
+
- **Deterministic teardown**: microphone, audio element, timers, and the peer connection are always released on hangup/error/dispose.
|
|
158
|
+
|
|
159
|
+
## What it deliberately does not do
|
|
160
|
+
|
|
161
|
+
- No UI. Both packages are headless; bring your own components.
|
|
162
|
+
- No transcript/state store. Subscribe to events and feed whatever store you use.
|
|
163
|
+
- No auth storage. You supply an `auth` provider (or a publishable key); the library never persists credentials.
|
|
164
|
+
|
|
165
|
+
## Development
|
|
166
|
+
|
|
167
|
+
```bash
|
|
168
|
+
pnpm install
|
|
169
|
+
pnpm build
|
|
170
|
+
pnpm test
|
|
171
|
+
pnpm type-check
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Requires Node ≥ 20 and pnpm ≥ 10.
|
|
175
|
+
|
|
176
|
+
## License
|
|
177
|
+
|
|
178
|
+
[MIT](LICENSE)
|