@hubtalk-ai/web-sdk 0.2.0 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,17 +1,24 @@
1
1
  # @hubtalk-ai/web-sdk
2
2
 
3
- Browser SDK for running a WebRTC voice call with a HubTalk AI Conversation
4
- Flow agent from your own website. It is a thin wrapper over
5
- [`livekit-client`](https://github.com/livekit/client-sdk-js) and the
6
- platform's existing voice path (`POST /v1/calls/web`): the SDK has no
7
- transport of its own, holds no API key, and never calls the platform's HTTP
8
- API — see §9 “Web calls” of the partner integration guide
9
- (`docs/integration-guide.md` in the platform repository) for the
10
- backend → frontend → SDK sequence.
3
+ *[Русская версия — `README.ru.md`](README.ru.md).*
11
4
 
12
- ## Installation
5
+ Browser SDK for a voice call with a HubTalk AI Conversation Flow agent,
6
+ straight from your own website. A thin wrapper over
7
+ [`livekit-client`](https://github.com/livekit/client-sdk-js): it holds no
8
+ Web API key and never calls the platform's HTTP API.
9
+
10
+ The integration is three steps:
11
+
12
+ 1. Your backend calls `POST /v1/calls/web` with the workspace's Web API key —
13
+ `agent_id` in the body, plus the call's dynamic variables if you use them.
14
+ 2. The platform answers `201` with the session: `{access_token, url, …}`.
15
+ 3. Your page hands that body to `startCall()` — from there the conversation
16
+ runs between the browser and the platform, and state arrives as events.
17
+
18
+ The Web API key stays on your server and never reaches the browser. The
19
+ endpoint's contract is published in the platform's schema: `{BASE_URL}/docs`.
13
20
 
14
- With a bundler:
21
+ ## Installation
15
22
 
16
23
  ```bash
17
24
  npm install @hubtalk-ai/web-sdk
@@ -21,8 +28,8 @@ npm install @hubtalk-ai/web-sdk
21
28
  import { VoiceClient } from '@hubtalk-ai/web-sdk';
22
29
  ```
23
30
 
24
- Without a build step, load the self-contained bundle (carries its own copy
25
- of `livekit-client`):
31
+ Without a bundler, load the self-contained bundle with its own copy of
32
+ `livekit-client`:
26
33
 
27
34
  ```html
28
35
  <script src="voice-client.bundle.js"></script>
@@ -34,8 +41,6 @@ of `livekit-client`):
34
41
  ## Quickstart
35
42
 
36
43
  ```ts
37
- import { VoiceClient } from '@hubtalk-ai/web-sdk';
38
-
39
44
  const client = new VoiceClient();
40
45
 
41
46
  startCallButton.addEventListener('click', async () => {
@@ -44,16 +49,16 @@ startCallButton.addEventListener('click', async () => {
44
49
  await client.startCall(session);
45
50
  });
46
51
 
47
- client.on('connectionStateChanged', ({ state }) => console.log('connection:', state));
48
- client.on('agentSpeakingStarted', () => console.log('agent speaking'));
52
+ client.on('transcript', ({ role, text, final, id }) => renderBubble(id, role, text, final));
49
53
  client.on('callEnded', ({ reason }) => console.log('call ended:', reason));
50
54
 
51
55
  endCallButton.addEventListener('click', () => client.stopCall());
52
56
  ```
53
57
 
54
- **`startCall` must run inside a user gesture handler** (a click, a tap) — the
55
- browser's autoplay policy requires it for the agent's audio to play without
56
- being blocked, and getting microphone permission works the same way.
58
+ **Call `startCall` inside a click or tap handler** — the browser requires that
59
+ gesture both for the agent's audio and for microphone access. On iOS Safari a
60
+ network request between the click and `startCall()` can consume the gesture:
61
+ fetch the session ahead of time.
57
62
 
58
63
  ## API
59
64
 
@@ -61,449 +66,307 @@ being blocked, and getting microphone permission works the same way.
61
66
 
62
67
  | Option | Type | Default | Description |
63
68
  |---|---|---|---|
64
- | `audioContainer` | `HTMLElement` | a hidden `<div>` the SDK creates | Where agent `<audio>` elements are appended. |
65
- | `logger` | `{ error, debug }` | `console` | Custom log sink; the SDK still masks tokens before calling it. |
69
+ | `audioContainer` | `HTMLElement` | a hidden `<div>` the SDK creates | Where the agent's `<audio>` elements go. |
70
+ | `logger` | `{ error, debug }` | `console` | Your own log sink; tokens are masked either way. |
66
71
  | `logLevel` | `'silent' \| 'error' \| 'debug'` | `'error'` | Verbose logging is opt-in. |
67
- | `connectTimeoutMs` | `number` | livekit-client default (15s) | Peer connection / WebSocket connect timeout. |
72
+ | `connectTimeoutMs` | `number` | 15s | Connection timeout. |
68
73
 
69
74
  ### Methods
70
75
 
71
76
  | Method | Behavior |
72
77
  |---|---|
73
- | `startCall({ accessToken, url }, options?)` | Connects and publishes the microphone. Accepts either `{accessToken, url}` or the raw `POST /v1/calls/web` response body (`{access_token, url, …}`) — extra fields are ignored. Resolves once the microphone is published. A second call while a call is active rejects with `ALREADY_IN_CALL`. `options.audio: false` starts a chat-only call — see below. |
74
- | `stopCall()` | Ends the call. Idempotent — resolves without an `error` event if no call is active. Cancels an in-flight `startCall` if called during `connecting`. |
75
- | `getState()` | Connection axis: `idle \| connecting \| connected \| reconnecting \| disconnected \| error`. |
76
- | `getAgentState()` | Agent-activity axis: `unknown \| initializing \| listening \| thinking \| speaking`. Independent of the connection axis. |
77
- | `sendText(text)` | Sends a typed reply into the conversation (topic `lk.chat`). The agent interrupts its speech and answers it as it would answer speech. Rejects with `NOT_IN_CALL` outside a call; an empty or whitespace-only string is a silent no-op; anything longer than 1000 characters is truncated. Your own message is **not** inserted into the chat by the SDK — it comes back as a `transcript` event with `role: 'user'`. |
78
- | `getCallId()` | The call's room name (same value as `call_id` in `GET /v1/calls/{call_id}`), or `null` before connecting. |
79
- | `on(event, handler)` / `off(event, handler)` / `once(event, handler)` | Subscribe / unsubscribe / subscribe-once. |
80
- | `destroy()` | Stops the call, removes every subscription, releases all resources. The instance is not usable afterward. |
78
+ | `startCall(session, options?)` | Connects and publishes the microphone. Takes the `POST /v1/calls/web` response as it comes (extra fields ignored), or `{accessToken, url}`. A second call during an active one rejects with `ALREADY_IN_CALL`. `options.audio: false` starts a chat-only call. |
79
+ | `stopCall()` | Ends the call. With no call active it does nothing and raises no error. |
80
+ | `sendText(text)` | Sends the visitor's typed reply; the agent interrupts its speech and answers it as it would answer speech. Outside a call it rejects with `NOT_IN_CALL`; anything past 1000 characters is truncated. The SDK does not insert your message into the chat — it comes back as a `transcript` event with `role: 'user'`. |
81
+ | `getState()` | Connection state: `idle \| connecting \| connected \| reconnecting \| disconnected \| error`. |
82
+ | `getAgentState()` | What the agent is doing: `unknown \| initializing \| listening \| thinking \| speaking`. |
83
+ | `getCallId()` | The call id (the same one `GET /v1/calls/{call_id}` takes), or `null` before connecting. |
84
+ | `on` / `off` / `once` | Subscribe, unsubscribe, subscribe once. |
85
+ | `destroy()` | Stops the call and releases everything; the instance is not usable afterward. |
81
86
 
82
87
  ### Events
83
88
 
84
89
  | Event | Payload |
85
90
  |---|---|
86
91
  | `callStarted` | `{ callId }` |
87
- | `callEnded` | `{ callId, reason }` — `reason`: `local` (`stopCall`), `remote` (the agent/flow ended it), `connection_lost`, `error` |
92
+ | `callEnded` | `{ callId, reason }` — `local` (`stopCall`), `remote` (the agent ended it), `connection_lost`, `error` |
88
93
  | `connectionStateChanged` | `{ state, previous }` |
89
94
  | `agentStateChanged` | `{ state, previous }` |
90
95
  | `agentSpeakingStarted` / `agentSpeakingStopped` | `{}` |
91
- | `transcript` | `{ role, text, final, id }` — `role`: `user` \| `agent`; `text` is the reply so far; `final: false` while it is still growing; `id` is the segment id — deltas of one reply share it, so render one bubble per `id`. Agent text arrives as it is **spoken**, so an interrupted reply never shows its unspoken tail. |
92
- | `error` | `VoiceError` — see below |
96
+ | `transcript` | `{ role, text, final, id }` — a line of the conversation; `role`: `user` \| `agent`. While `final: false` the line is still growing: render one bubble per `id`. |
97
+ | `error` | `VoiceError` |
93
98
 
94
99
  ### Chat-only calls
95
100
 
96
- `startCall(session, { audio: false })` runs the same call without audio: no
97
- microphone is published, and the agent's audio is subscribed but never played,
98
- so the browser never asks for microphone permission and nothing is heard. Text
99
- keeps working in both directions — `sendText()` and the `transcript` event —
100
- which makes it the mode for a visitor who cannot or will not talk.
101
+ `startCall(session, { audio: false })` runs the same call without microphone
102
+ or sound: the browser asks for no permission and the conversation happens in
103
+ text (`sendText()` and `transcript`). It is the mode for a visitor who cannot
104
+ or will not talk. The agent still synthesizes speech, so the text arrives at
105
+ speaking pace and the call costs the same as a voice one.
101
106
 
102
107
  ```ts
103
108
  await client.startCall(session, { audio: false });
104
109
  ```
105
110
 
106
- Two consequences of that mode, both worth knowing before you ship it:
107
-
108
- - Neither `MICROPHONE_PERMISSION_DENIED` nor `MICROPHONE_UNAVAILABLE` can
109
- occur, and `WEBRTC_UNSUPPORTED` no longer requires `getUserMedia` — only a
110
- secure context and `RTCPeerConnection`, which LiveKit needs for the data
111
- channel either way.
112
- - The agent's audio is **subscribed and discarded**, not left unsubscribed.
113
- That is not waste for its own sake: in `livekit-agents`, the agent's audio
114
- output blocks on its track being subscribed
115
- (`_ParticipantAudioOutput.capture_frame` awaits a future that only
116
- `publication.wait_for_subscription()` resolves), and the agent's transcript
117
- is routed through a synchronizer bound to that same output. With no
118
- subscriber the first frame of speech blocks forever and the agent's text
119
- never arrives at all — while the echo of the visitor's own typed replies
120
- still does, because that goes out on a separate, unsynchronized path.
121
- - The agent therefore still synthesizes speech nobody hears, and LiveKit paces
122
- the transcript to that playback. So the text arrives at speaking speed
123
- rather than as fast as the model produces it, and the call costs the same in
124
- TTS as a voice call. Removing both is a platform-side change — the agent
125
- needs `RoomOutputOptions(audio_enabled=False)` — not an SDK one.
126
-
127
111
  ### Error codes
128
112
 
129
- Every failure is a structured `VoiceError`: `{ code, message, state, cause? }`
130
- — `cause` is the original browser/`livekit-client` error, for your own logs.
131
-
132
- | Code | When | Resulting state |
133
- |---|---|---|
134
- | `INVALID_PARAMS` | `accessToken` or `url` missing/empty | `error` |
135
- | `WEBRTC_UNSUPPORTED` | No WebRTC support, or not a secure context (HTTPS) | `error` |
136
- | `MICROPHONE_PERMISSION_DENIED` | Microphone access denied (voice calls only) | `error` |
137
- | `MICROPHONE_UNAVAILABLE` | No usable microphone device (voice calls only) | `error` |
138
- | `TOKEN_EXPIRED` | Token's `exp` already passed (checked locally before connecting) | `error` |
139
- | `TOKEN_INVALID` | Server rejected the token | `error` |
140
- | `CONNECTION_FAILED` | Could not establish the connection | `error` |
141
- | `CONNECTION_LOST` | Reconnect attempts exhausted after a successful connection | `disconnected` |
142
- | `ALREADY_IN_CALL` | `startCall` called while a call is already active | unchanged |
143
- | `NOT_IN_CALL` | `sendText` called outside an active call | unchanged |
144
- | `SERVER_ERROR` | Any other signalling/service failure | `error` |
145
-
146
- ## Browser matrix
147
-
148
- Bounded by [`livekit-client`](https://github.com/livekit/client-sdk-js)'s own
149
- WebRTC support matrix. **Not yet verified against a live deployment** —
150
- tracked as an open pre-check of the source requirement (`docs/patches/patch132-web-voice-sdk.md`
151
- §7.3); this table is filled in once that manual pass runs, before the first
152
- release.
153
-
154
- | Browser | Verified |
155
- |---|---|
156
- | Chrome (desktop) | — |
157
- | Firefox (desktop) | — |
158
- | Safari (desktop) | — |
159
- | Chrome (Android) | — |
160
- | Safari (iOS) | — |
113
+ Every failure is a `VoiceError`: `{ code, message, state, cause? }`. Match on
114
+ `code`; `message` is a diagnostic line, and the visitor-facing text is yours.
161
115
 
162
- iOS Safari needs special attention: both microphone publish and
163
- `startAudio()` must run strictly inside the user gesture, no exceptions.
116
+ | Code | When |
117
+ |---|---|
118
+ | `INVALID_PARAMS` | The session carries no `access_token` or `url` |
119
+ | `WEBRTC_UNSUPPORTED` | No WebRTC support, or the page is not on HTTPS |
120
+ | `MICROPHONE_PERMISSION_DENIED` | The visitor denied the microphone |
121
+ | `MICROPHONE_UNAVAILABLE` | No microphone, or it is busy |
122
+ | `TOKEN_EXPIRED` | The session expired before the call started |
123
+ | `TOKEN_INVALID` | The server rejected the session token |
124
+ | `CONNECTION_FAILED` | Could not connect |
125
+ | `CONNECTION_LOST` | The connection dropped and did not come back |
126
+ | `ALREADY_IN_CALL` | A call is already running |
127
+ | `NOT_IN_CALL` | `sendText` outside a call |
128
+ | `SERVER_ERROR` | Any other service failure |
129
+
130
+ A session token lives 2 hours and is never refreshed: mint a fresh session
131
+ per call rather than holding one on an open page.
164
132
 
165
133
  ## Share links
166
134
 
167
- You do not always need a backend of your own. The platform's cabinet can
168
- issue a temporary link to one published agent — an expiry, a cap on calls
169
- and one-click revocation — and the page behind that link mints the session
170
- itself. Whoever opens it talks to the agent with no account, no API key and
171
- no setup.
172
-
173
- That page is part of the platform, and it drives the call through this SDK
174
- without a single change to it: the response it mints has the same shape as
175
- `POST /v1/calls/web`, so `startCall(session)` takes it as it comes. Nothing
176
- in the SDK knows about links, and nothing here holds the link token — the
177
- same rule as for the API key.
135
+ You do not always need a backend. The platform's cabinet issues a temporary
136
+ link to one published agent — with an expiry, a cap on calls and one-click
137
+ revocation. The page behind the link mints the session itself and drives the
138
+ call through this same SDK, so anyone can talk to the agent with no account,
139
+ no key and no setup.
178
140
 
179
141
  Use a share link when someone just needs to hear the agent. Use your own
180
- backend, as in `example/server.py`, when the call has to start from your
181
- site, under your own domain, with variables your system knows.
182
-
183
- ## What this SDK does not do (phase 1)
184
-
185
- - No chat or call-button UI — you build the interface, the SDK reports state
186
- and takes text. `example/index.html` shows the wiring, it is not a
187
- shipped component. The reference widget of the example pages is not
188
- shipped either: copy it, do not import it.
189
- - No link handling — a share link is opened by the platform's own page, and
190
- `startCall` takes the session that page already minted. There is no
191
- `startCall({ demoToken })` and the SDK never presents a secret.
192
- - No transcript history — the SDK holds no list of replies; whoever renders
193
- them keeps them.
194
- - No dynamic variables of its own — they are set once, by your backend, in
195
- the `POST /v1/calls/web` body that mints the session. By the time the SDK
196
- has that session the variables are fixed, and there is no API to read or
197
- change them. `example/server.py` shows the backend side.
198
- - No `cf.voice`: node names and cascade transitions stay internal. Post-call
199
- data comes from `GET /v1/calls/{call_id}` or a
200
- `call_ended`/`call_analyzed` webhook.
201
- - No rate limit on `sendText` — every message is an LLM turn and TTS
202
- synthesis, so a page that sends in a loop spends the workspace's money.
203
- - No token refresh — a call cannot outlive the token's TTL
204
- (`expires_in` in the `POST /v1/calls/web` response).
205
- - No video, DTMF, custom TURN/ICE, or client-side audio processing.
206
-
207
- ## Development
208
-
209
- ```bash
210
- npm install
211
- npm run lint
212
- npm run format:check
213
- npm run build # dist/index.js (ESM) + dist/voice-client.bundle.js (script tag) + dist/*.d.ts
214
-
215
- python3 example/server.py --selftest # the demo backend's field/merge logic
216
- ```
142
+ backend when the call starts from your site, on your domain, with variables
143
+ your system knows.
217
144
 
218
- `--selftest` is the check that guards the parts of `example/server.py` with
219
- branches in them: `@today` and keyless fields in `demo_fields`, the
220
- allowlist in `merge_variables`, and every shipped scenario file parsing into
221
- a renderable form. It starts no server and needs no `.env`.
145
+ ## Example integration
222
146
 
223
- ### Running the example pages
224
-
225
- `example/server.py` is the backend half of the example: it holds the API key,
226
- mints a room through `POST /v1/calls/web` and serves the pages. Standard
227
- library only, no dependencies. Every call parameter comes from `.env` — copy
228
- `.env.example` and fill it in:
229
-
230
- ```
231
- CFLOW_API_URL=http://localhost:8100
232
- CFLOW_API_KEY=cfk_…
233
- CFLOW_AGENT_ID=my_agent_v1 # fallback when the page sends no agent_id
234
- # optional JSON objects, passed straight to POST /v1/calls/web
235
- # CFLOW_DYNAMIC_VARIABLES={"name": "Aigerim"}
236
- # CFLOW_METADATA={"source": "example"}
237
- # optional: which scenario to open with, and per-half overrides of it
238
- # CFLOW_DEMO_SCENARIO=missed-deposit
239
- # CFLOW_DEMO_FIELDS=[{"key":"rl_clientName","label":"Client name","default":"John"}]
240
- # CFLOW_DEMO_HERO={"headline":"Missed deposit follow-up"}
241
- ```
242
-
243
- **One file per demo scenario.** `example/scenarios/<id>.json` declares the
244
- dynamic variables the pages let a visitor edit — the ones the agent may
245
- reference as `{{key}}` — plus the copy of the call page's left column. The id
246
- is the file name. Two ship filled in:
247
-
248
- | id | Shows |
249
- |---|---|
250
- | `missed-deposit` | a client whose scheduled deposit was returned; 14 variables |
251
- | `identity-verification` | verifying the client, then their enrolled creditor accounts; 21 variables |
252
-
253
- **Exactly one is active, and the server picks it** — the one whose file says
254
- `"default": true`, or `CFLOW_DEMO_SCENARIO`. That is not a missing feature:
255
- these variables are the variables of the agent in `CFLOW_AGENT_ID`, and one
256
- agent is configured at a time, so a page-level switch could only ever produce
257
- variables the running agent never references. The scenario travels with the
258
- agent, in `.env`. Relying on file order for the default would let a new file
259
- quietly replace what a live demo shows, hence the explicit marker.
260
-
261
- A new demo is a new file plus one `.env` line — no HTML.
262
-
263
- Per field: `key`, `label`, and optionally `default` (`"@today"` resolves to
264
- today's date on the server), `placeholder`, `type`, `readonly`, `half` (two
265
- fields share a row).
266
-
267
- The server serves the active scenario from `GET /api/config`, `/` and `/call`
268
- render it (`/free` declares its own), and `POST /api/session` accepts values
269
- **only** for that scenario's keys — a key belonging to another scenario is
270
- dropped like an invented one, and everything else in
271
- `CFLOW_DYNAMIC_VARIABLES` stays server-side. That
272
- filter is not demo scaffolding: a production backend owes the same guarantee,
273
- since the browser names values and never the contract.
274
-
275
- **Three pages, one widget.** `/` is the widget on a host page, `/call` the
276
- call as the whole page, `/free` the widget with nothing configured — the
277
- agent name and every dynamic variable are typed into the form, `+ Variable`
278
- adds a key/value row.
279
-
280
- The allowlist follows the agent, not the page: a request for the agent in
281
- `CFLOW_AGENT_ID`, on `.env`'s own key, is filtered against the active
282
- scenario's keys, and a request for any other agent — or on a key the page
283
- brought — has its keys sent through unfiltered. That is what makes `/free`
284
- work — it brings its own agent, whose contract the active scenario does not
285
- describe — and it is why naming a different agent on `/` or `/call` drops the
286
- filter too.
287
-
288
- **Which agent is called.** The page names it: `const AGENT_ID = '…'` in the
289
- widget options of `example/index.html` and `example/call-details.html`, sent
290
- as `agent_id` in `POST /api/session`. It is the agent's **name**, the same
291
- string `CFLOW_AGENT_ID` holds — not a uuid. Left empty — the shipped
292
- default — the form grows an Agent name input and the visitor names it per
293
- call. That input rides the same form as the variables but is never one of
294
- them: it carries the reserved key `__agent_id` (`HtDemo.AGENT_KEY`), which
295
- the wiring strips out before posting. The server falls back to
296
- `CFLOW_AGENT_ID` only when the request names no agent, and answers 503 when
297
- neither does.
298
-
299
- **Which organisation the call runs as.** The key decides, and `.env`'s
300
- `CFLOW_API_KEY` is again only the fallback: `/free` carries an **Organisation
301
- API key** field in the middle of the page — not in the widget, since the key
302
- is not a call parameter — and a key typed there is sent as `api_key` and used
303
- for that call instead. One deployment can therefore demo any organisation
304
- without an `.env` edit and a restart. No organisation id goes with it:
305
- `POST /v1/calls/web` authenticates on the bearer key alone. The field is a
306
- demo affordance and not a template — a production backend holds its key
307
- server-side and never lets the browser name one — and the server's own key
308
- still never travels to a page.
147
+ `example/` holds a working backend-plus-page pair to start from: `server.py`
148
+ keeps the Web API key and mints the session, and the pages show a call
149
+ widget. `/` and `/avatar` use the optional shipped widget; the other layouts
150
+ remain examples built on the headless client.
309
151
 
310
152
  ```bash
311
- npm run build # the page loads ../dist/voice-client.bundle.js
312
- python3 example/server.py # http://localhost:8000/, /call and /free
153
+ npm install && npm run build
154
+ cp .env.example .env # CFLOW_API_URL, CFLOW_API_KEY, CFLOW_AGENT_ID
155
+ python3 example/server.py # http://localhost:8000/
313
156
  ```
314
157
 
315
- Real environment variables win over `.env`, so `CFLOW_AGENT_ID=other python3
316
- example/server.py` overrides one parameter for a single run.
317
-
318
- Every press of Call asks the server for a fresh room. That is what keeps the
319
- short-lived LiveKit `access_token` fresh: a page left open for longer than the
320
- token TTL still starts a working call. The browser never talks to `/v1`
321
- directly, so no `CFLOW_CORS_ORIGINS` entry is needed — and the API key never
322
- reaches the page. That is the shape a production integration has too, with
323
- your own backend in place of `server.py`; `startCall()` accepts the
324
- `POST /v1/calls/web` response as-is.
325
-
326
- The example server exposes only the three example pages, their stylesheet
327
- (`example/widget.css`), their shared wiring (`example/demo-call.js`), the
328
- bundle and the two `/api/*` endpoints (the scenario files are read on the
329
- server, never served); it never serves the project directory
330
- or `.env`. Every page offers both a voice call and a chat-only one:
331
-
332
158
  | Page | What it shows |
333
159
  |---|---|
334
- | `/` | a demo site with the call widget pinned in the corner — the call starts from the launcher, and “Call details” inside the panel holds the dynamic variables |
335
- | `/call` | the call as the whole page: a branded column plus the same form, switching to the conversation once the call starts |
336
- | `/free` | the corner widget with nothing configured — agent name and variables are typed in, `+ Variable` adds a row; the organisation's API key is a field in the page itself |
337
-
338
- All three are clean URLs — no `.html`, since these links get pasted to
339
- clients. The old paths (`/index.html`, `/call-details.html`,
340
- `/free-form.html`, anything under `/example/`) answer `301` to the short
341
- form, so a link already shared keeps working.
342
-
343
- On iOS Safari, note that the network round trip between the click and
344
- `startCall()` can consume the user gesture that audio playback requires. A
345
- widget that has to support iOS should fetch the session before the click and
346
- hand it straight to `startCall()`.
160
+ | `/` | a site with the call widget pinned in the corner |
161
+ | `/avatar` | draggable avatar mode: the live AI avatar once the call connects, a bundled presenter clip as the failover |
162
+ | `/call` | the call as the whole page, with a branded column |
163
+ | `/free` | the same with nothing configured: agent, variables and key typed in |
164
+ | `/file` | `/free`, plus an mp3 or wav file played into the call in place of the microphone |
347
165
 
348
- ### Embedding the widget
166
+ What to carry over into your own integration: the Web API key lives only on
167
+ the server, the browser names only variable values, and the server accepts
168
+ values only for the variables it declared itself. The shared widget is `src/widget.ts`; `example/demo-widget.js` supplies the
169
+ backend callback and uses `example/demo-call.js` for scenario fields.
349
170
 
350
- The SDK is headless: it ships no markup, no CSS and no themes. Your page owns
351
- every pixel. `example/index.html` together with `example/widget.css` is a
352
- reference you copy into your project and edit — not a component the package
353
- delivers.
171
+ ### Demo box
354
172
 
355
- **Markup.** The widget is a launcher plus a panel, both inside one root
356
- element that carries the tokens:
357
-
358
- ```html
359
- <div class="ht-widget">
360
- <button class="ht-launcher" id="launcher">…</button>
361
- <section class="ht-panel" id="panel" hidden>
362
- <header class="ht-head">
363
- <span class="ht-title">Voice agent</span>
364
- <span class="ht-pill" id="pill">Ready</span>
365
- <span class="ht-timer" id="timer" hidden>0:00</span>
366
- <button class="ht-icon" id="collapse">…</button>
367
- </header>
368
- <details class="ht-details" id="details" open>
369
- <summary>Call details</summary>
370
- <div class="ht-fields" id="fields"></div>
371
- </details>
372
- <div class="ht-log" id="log"></div>
373
- <div class="ht-debug" id="state"></div>
374
- <div class="ht-composer">
375
- <input id="message" disabled />
376
- <button class="ht-icon" id="send" disabled>…</button>
377
- </div>
378
- <div class="ht-actions">
379
- <button class="ht-btn" id="call">Call</button>
380
- <button class="ht-btn ht-btn-ghost" id="chat">Chat</button>
381
- <button class="ht-btn ht-btn-danger" id="hangup" disabled>Hang up</button>
382
- </div>
383
- </section>
384
- </div>
385
- ```
386
-
387
- The `<details>` block is where the dynamic variables are typed; the widget
388
- hides it for the duration of a call. `Chat` is the same call with
389
- `{ audio: false }` — see “Chat-only calls” above.
390
-
391
- **The wiring is shared, not duplicated.** `example/demo-call.js` holds the
392
- part that is identical on all three example pages — transcript bubbles, the
393
- timer, the composer, session creation, `startCall` — and each page passes its
394
- own nodes and its own layout hooks:
395
-
396
- ```js
397
- HtDemo.mountCall({
398
- client: new VoiceClient(),
399
- agentId: AGENT_ID, // '' → the form asks the visitor for an Agent name
400
- refs: { pill, timer, state, log, empty, message, send, call, chat, hangup, liveDot },
401
- readVariables: () => HtDemo.readValues(fieldsEl),
402
- onCallStarted: () => { /* your layout switch */ },
403
- onCallEnded: (reason) => { /* and back */ },
404
- });
405
- ```
406
-
407
- Every `refs` node is optional — a page passes what its markup has, and the
408
- wiring skips the rest. `mountCall` returns `{ startVoice, startChatOnly,
409
- stop, render }` for a page that wants to drive the call from something other
410
- than the `call` / `chat` / `hangup` buttons.
411
-
412
- The rest of `HtDemo` is the form half, which is why no page repeats it:
413
-
414
- | Helper | Does |
415
- |---|---|
416
- | `loadConfig()` | `GET /api/config` — the active scenario's `fields` and `hero` |
417
- | `renderFields(container, fields, agentId)` | the variable inputs, preceded by an Agent name input when `agentId` is empty |
418
- | `readValues(container)` | what the visitor typed, keyed by dynamic variable, read off the DOM |
419
- | `applyHero(root, hero)` | fills every `[data-hero="<key>"]` node, keeping the markup's own copy as the fallback |
420
- | `createSession(variables, agentId)` | `POST /api/session` |
421
-
422
- **Wiring.** Which SDK event drives which node:
423
-
424
- | Event | Node |
425
- |---|---|
426
- | `connectionStateChanged`, `agentStateChanged` | the state pill and the raw debug line |
427
- | `transcript` | one bubble per segment `id`; `final: false` renders as interim |
428
- | `callStarted` | clears the log, starts the timer, enables the composer and Hang up |
429
- | `callEnded` | stops the timer, disables the composer, appends why it ended |
430
- | `error` | stops the timer, hides the live dot, appends the code plus its English line to the log |
431
-
432
- Collapsing the panel is presentation only — never call `stopCall()` from it,
433
- or a visitor loses the conversation by tidying the widget away.
434
-
435
- **Restyling.** Override the tokens on `.ht-widget`; nothing else needs
436
- touching:
437
-
438
- | Token | What it paints |
439
- |---|---|
440
- | `--accent`, `--accent-contrast` | the launcher, the Call button, the visitor's bubbles |
441
- | `--bg-panel`, `--bg-elev` | the panel surface and the agent's bubbles |
442
- | `--text`, `--text-dim` | primary and secondary text |
443
- | `--border` | every hairline |
444
- | `--radius`, `--radius-sm` | corner softness |
445
- | `--hero-bg`, `--hero-text` | the branded column of the call page (deliberately the same in both schemes — a brand block does not invert) |
446
-
447
- ```css
448
- .ht-widget {
449
- --accent: #1f6feb;
450
- --accent-contrast: #ffffff;
451
- --radius: 8px;
452
- }
453
- ```
454
-
455
- The file ships both palettes and follows `prefers-color-scheme`; override a
456
- token inside your own media query to diverge per scheme.
457
-
458
- **Boundaries.** The tokens are declared on the `.ht-demo` / `.ht-call` /
459
- `.ht-widget` roots rather than on `:root`, and every class carries the `ht-`
460
- prefix — loading the file cannot repaint your site. Copy the `.ht-widget`
461
- block plus the shared form primitives (`.ht-details`, `.ht-fields`,
462
- `.ht-field*`, `.ht-btn*`) and leave the `.ht-demo` / `.ht-call` page sections
463
- behind. The package does not serve the stylesheet: copy it into your own
464
- build.
465
-
466
- ### Deploying the example
467
-
468
- The deployment script builds the SDK and runs the example server on the same
469
- machine. Put the required API configuration in `.env` at the project root.
173
+ The example server runs under the
174
+ `example/hubtalk-webrtc-example.service` unit (fix `User` and
175
+ `WorkingDirectory` before installing it elsewhere), behind an HTTPS proxy —
176
+ browsers need a secure context. A sample proxy config is
177
+ `example/caller-widget.demo.cyberdev.kz.Caddyfile`.
470
178
 
471
179
  ```bash
472
- scripts/deploy-example.sh
180
+ npm run build # required: dist/ is gitignored, so git pull never updates the bundle
181
+ sudo systemctl restart hubtalk-webrtc-example
473
182
  ```
474
183
 
475
- Set `PORT` to choose a port (default `8000`). For public access, put this
476
- server behind an HTTPS reverse proxy, since browsers require a secure context
477
- for microphone and WebRTC access.
184
+ ## Optional widget (VPT-857)
478
185
 
479
- ### Running with systemd
186
+ **What you get today.** Avatar mode works against any current HubTalk
187
+ platform: the circle plays the bundled clip failover (poster, listening loop,
188
+ speaking loop, Video.js) and the call is the ordinary voice call. The live AI
189
+ avatar described further down needs platform support that is not deployed
190
+ yet; until it is, every call shows the clips, and nothing in your integration
191
+ changes when it arrives — the widget switches to live video by itself.
480
192
 
481
- The included unit runs the example after the network is available and restarts
482
- it after a failure. It ships configured for the demo box — user
483
- `cybernet-adm`, checkout at `/home/cybernet-adm/hubtalk-web-sdk` — so edit
484
- `User` and `WorkingDirectory` before installing it anywhere else. It passes
485
- no `EnvironmentFile`: the server reads `.env` from the checkout itself.
193
+ **Try it in five minutes.**
486
194
 
487
- ```bash
488
- sudo cp example/hubtalk-webrtc-example.service /etc/systemd/system/
489
- sudo systemctl daemon-reload
490
- sudo systemctl enable --now hubtalk-webrtc-example
491
- sudo systemctl status hubtalk-webrtc-example
195
+ 1. `npm ci && npm run build` in this repository.
196
+ 2. Copy `.env.example` to `.env` in the repository root and fill in the Web API key
197
+ and the agent name.
198
+ 3. `python3 example/server.py`, then open `http://127.0.0.1:8000/avatar`.
199
+ `/` is the same widget in standard mode, `/free` the hand-written example.
200
+ 4. Press the circle to call, press it again to mute, open Chat for the
201
+ transcript and End call. Replace the three files in `example/` with your
202
+ own poster and silent clips (see `example/AVATAR-MEDIA.md`) to see your
203
+ presenter.
204
+
205
+ ```ts
206
+ import {
207
+ mountWidget,
208
+ type WidgetOptions,
209
+ type WidgetHandle,
210
+ type AvatarMedia,
211
+ type CreateSessionContext,
212
+ } from '@hubtalk-ai/web-sdk/widget';
213
+ import '@hubtalk-ai/web-sdk/widget.css';
214
+
215
+ const widget = mountWidget({
216
+ container: document.getElementById('support')!,
217
+ mode: 'avatar', // omit for the standard launcher and call/chat panel
218
+ async createSession({ avatar }: CreateSessionContext) {
219
+ // avatar is true in avatar mode: forward it to your backend, which sets
220
+ // `avatar: true` on POST /v1/calls/web (see "Live avatar" below)
221
+ const response = await fetch('/api/session', {
222
+ method: 'POST',
223
+ headers: { 'Content-Type': 'application/json' },
224
+ body: JSON.stringify({ avatar }),
225
+ });
226
+ if (!response.ok) throw new Error('Session unavailable');
227
+ return response.json(); // { access_token, url }, never a Web API key
228
+ },
229
+ // the clip FAILOVER, shown whenever there is no live avatar video
230
+ avatar: {
231
+ posterUrl: '/presenter/poster.jpg',
232
+ listeningVideoUrl: '/presenter/listening.mp4',
233
+ speakingVideoUrl: '/presenter/speaking.mp4',
234
+ size: 140, // circle diameter in px, optional; this is the default
235
+ },
236
+ });
237
+ // On page/component teardown:
238
+ widget.destroy(); // idempotent; stops the call and disposes owned DOM/media
492
239
  ```
493
240
 
494
- **`dist/` is gitignored, so `git pull` never updates the bundle the browser
495
- loads.** A pull plus a restart keeps serving the SDK that was last built on
496
- that machine — the page works, it is just the previous code, which is a very
497
- quiet way to lose an afternoon. Always build:
241
+ Without a frontend build, host these two files from `dist/`:
498
242
 
499
- ```bash
500
- npm run build
501
- sudo systemctl restart hubtalk-webrtc-example
243
+ ```html
244
+ <link rel="stylesheet" href="/sdk/widget.css">
245
+ <div id="support"></div>
246
+ <script src="/sdk/widget.bundle.js"></script>
247
+ <script>
248
+ const widget = HubTalkWidget.mountWidget({
249
+ container: document.getElementById('support'),
250
+ createSession: async () => {
251
+ const response = await fetch('/api/session', { method: 'POST' });
252
+ if (!response.ok) throw new Error('Session unavailable');
253
+ return response.json();
254
+ }
255
+ });
256
+ </script>
502
257
  ```
503
258
 
504
- Two guards make that hard to forget: the unit re-copied from
505
- `example/hubtalk-webrtc-example.service` rebuilds on every restart
506
- (`ExecStartPre`), and the server prints a loud warning at startup when
507
- `dist/voice-client.bundle.js` is older than `src/`. The pages and the bundle
508
- are served `Cache-Control: no-store`, so a plain reload is enough — no
509
- Ctrl+F5.
259
+ Both distributions expose `mountWidget(options: WidgetOptions): WidgetHandle`.
260
+ `WidgetOptions` requires `container` and `createSession`; `mode` defaults to
261
+ `standard`. Avatar mode additionally requires all three `AvatarMedia` URLs
262
+ (the failover media); optional `avatar.size` sets the circle diameter in
263
+ pixels (default 140, phones show 6/7 of it) — or set `--ht-avatar-size` on
264
+ `.ht-widget` from your CSS. `createSession` is called with
265
+ `{ avatar: boolean }` — `true` in avatar mode, `false` in standard mode;
266
+ callbacks that ignore the argument keep working. Optional
267
+ `renderCallDetails(container)` mounts your form once; `createSession` reads
268
+ its current values from a closure. The widget hides the form while a call
269
+ runs and restores it on end. See `example/demo-widget.js`.
270
+
271
+ ### Live avatar
272
+
273
+ In avatar mode the circle shows the **live video of an AI avatar** — a
274
+ LiveKit avatar worker that joins the call on the agent's behalf and publishes
275
+ video lip-synced to the agent's voice (provider Anam; the stock face "Layla"
276
+ by default, configurable per agent). Three things must all be true for it to
277
+ start, otherwise the call runs audio-only and the widget shows the clip
278
+ failover:
279
+
280
+ 1. your backend forwards the widget's ask as `"avatar": true` in the body of
281
+ `POST /v1/calls/web` (the platform treats it as a request, not a
282
+ guarantee; the response shape does not change);
283
+ 2. the agent's **Avatar** setting is enabled in the HubTalk editor
284
+ (Settings → Additional → AI avatar; off by default, cascade pipeline only);
285
+ 3. the platform contour has the avatar provider configured.
286
+
287
+ Media precedence, highest first: **live video** → **clip failover**
288
+ (`listeningVideoUrl`/`speakingVideoUrl`) → **poster**. Falling back to clips
289
+ is a normal state, not an error: it covers the time before the call, the
290
+ seconds while the avatar joins, an agent with the avatar off, a slow or
291
+ unavailable provider, and a live video that drops mid-call. While live video
292
+ plays, the clip player pauses and hides (and is not loaded at all if no clip
293
+ was needed before); when it drops, the clips resume. The live `<video>` is
294
+ muted, inline and without controls — the voice always comes from the SDK's
295
+ audio path, never from a video element, so it is heard exactly once.
296
+
297
+ Headless integrations get the same signal from `VoiceClient`:
298
+ `agentVideoChanged: { available }` fires when the worker's video track is
299
+ subscribed or goes away (reset to `false` on call end, failed startup and
300
+ `destroy()`), and `attachAgentVideo(element)` attaches the current and any
301
+ later track to your `<video>` and returns a detach function (a no-op outside
302
+ a call). The worker is the remote participant of kind `agent` whose
303
+ `lk.publish_on_behalf` attribute equals the agent's identity; agent state
304
+ stays on the agent's own `lk.agent.state`.
305
+
306
+ **Disclosure.** A visible **AI avatar** label shows on the circle while live
307
+ video plays and is also the circle's accessible description. There is no
308
+ burned-in watermark. Telling visitors that they talk to an AI, where your
309
+ jurisdiction requires it, stays the integrator's duty; the label helps but
310
+ does not replace it.
311
+
312
+ Live video arrives over the existing LiveKit connection: no new CSP origin
313
+ is needed beyond the `connect-src` and `media-src blob:` entries below. The
314
+ agent's voice and the avatar video pass through the provider's cloud —
315
+ check that with your data-protection owner before enabling the setting on a
316
+ regulated deployment.
317
+
318
+ The circle starts a voice call, then toggles **only your microphone**. Agent
319
+ audio keeps playing. The separate Chat launcher toggles the panel; End call
320
+ inside ends the call. The panel's Chat control starts a text-only call.
321
+ Keyboard Enter/Space activates buttons; focus remains visible. Dragging does
322
+ not start/mute a call. Position is clamped to the viewport and is not saved.
323
+ The panel and avatar stay separately reachable on mobile.
324
+
325
+ For the clip failover, supply a poster and silent H.264 MP4 loops of the
326
+ **same cleared person**. The clips do not synchronize phonemes: the speaking
327
+ loop follows agent state. Rings/status text distinguish speech, listening,
328
+ muted, idle and connecting. Thinking/initializing are not labelled listening.
329
+ Clip loading/failure, blocked clip autoplay and reduced motion use the poster
330
+ while call controls remain available; reduced motion keeps the live video
331
+ and disables ring animation. The example bundles a presenter cut from a
332
+ Pexels clip (Pexels License, provenance and the regenerating script in
333
+ `example/AVATAR-MEDIA.md`); it is example media, not a cleared likeness for
334
+ your site — replace it with your own.
335
+
336
+ Video.js **8.24.0** is pinned. ESM dynamically imports it only for avatar mode,
337
+ and only the first time a clip is needed. The self-contained IIFE includes the
338
+ player code and initializes it only for avatar mode; its full download cost
339
+ applies to either widget mode. Root ESM
340
+ and `voice-client.bundle.js` have no widget/Video.js imports. Widget CSS is
341
+ explicit and scoped; `sideEffects` preserves CSS in consumer builds.
342
+
343
+ Serve over HTTPS. Allow your SDK host in `script-src`/`style-src`, poster host
344
+ in `img-src`, clip host in `media-src`, and your backend and LiveKit HTTPS/WSS
345
+ origins in `connect-src`. WebRTC playback also needs `media-src blob:`; the
346
+ widget/Video.js uses inline style attributes for positioning, so permit them
347
+ with `style-src-attr 'unsafe-inline'`. Do not enable arbitrary remote origins.
348
+ A restrictive host media policy may force poster fallback. No CDN is required.
349
+
350
+ ### Microphone and playback controls
351
+
352
+ `VoiceClient.isMicrophoneEnabled()` starts false. Await
353
+ `setMicrophoneEnabled(enabled)` to change an active audio call. The
354
+ `microphoneStateChanged` event carries `{ enabled }` after confirmation and
355
+ resets to false on termination. Failed toggles preserve the last confirmed
356
+ state and the call. Connecting/reconnecting, text-only and inactive calls
357
+ reject these controls with `NOT_IN_CALL` without changing call state.
358
+
359
+ Browser playback permission is separate from microphone permission. The
360
+ initial gesture may be lost while your backend creates a session, especially
361
+ on iOS. `audioPlaybackBlocked` leaves the call connected; the widget presents
362
+ **Enable sound**, whose tap calls `resumeAudio()`. A repeated block rejects
363
+ with `AUDIO_PLAYBACK_BLOCKED`, keeps the prompt and never republishes the mic.
364
+ A denied microphone still fails voice startup. `stopCall()` ends the call;
365
+ muting does not.
366
+
367
+ ### Validation
368
+
369
+ Run `npm test`, `npm run lint`, `npm run format:check`, `npm run build`.
370
+ Browser checks: `npx playwright install chromium webkit && npm run test:browser`.
371
+ With installed Chrome: `PLAYWRIGHT_CHROME_CHANNEL=chrome npm run test:browser -- --project=chromium`.
372
+ See `VALIDATION.md` for measured build sizes and outstanding device/live-call gates.