@voctiv/agent-sdk 0.2.11 → 0.2.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +411 -50
- package/dist/index.d.ts +19 -13
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +18 -12
- package/dist/index.js.map +1 -1
- package/dist/types/events.d.ts +13 -8
- package/dist/types/events.d.ts.map +1 -1
- package/dist/types/media-channel.d.ts +1 -1
- package/dist/types/mixer.d.ts +2 -1
- package/dist/types/mixer.d.ts.map +1 -1
- package/dist/types/nlu.d.ts +9 -11
- package/dist/types/nlu.d.ts.map +1 -1
- package/dist/types/platform.d.ts +18 -9
- package/dist/types/platform.d.ts.map +1 -1
- package/dist/types/script-context.d.ts +45 -9
- package/dist/types/script-context.d.ts.map +1 -1
- package/dist/types/script-context.js.map +1 -1
- package/dist/types/sip.d.ts +81 -7
- package/dist/types/sip.d.ts.map +1 -1
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# @voctiv/agent-sdk
|
|
2
2
|
|
|
3
|
-
TypeScript SDK for
|
|
3
|
+
TypeScript SDK for voice and dialog scripts.
|
|
4
4
|
|
|
5
|
-
The package exports
|
|
5
|
+
The package exports `defineScript()` and types for media channels, SIP, ASR, TTS, LLM, dialog context, logging, and platform APIs.
|
|
6
6
|
|
|
7
|
-
The SDK
|
|
7
|
+
The SDK describes the objects injected into your script by the host runtime; it does not open calls or run services itself.
|
|
8
8
|
|
|
9
9
|
## Installation
|
|
10
10
|
|
|
@@ -121,7 +121,7 @@ For inbound calls, the script controls this explicitly:
|
|
|
121
121
|
3. Use `channel.audio.say()`, `channel.audio.play()`, `channel.createAsr()`, or `channel.sip.sendDtmf()` normally.
|
|
122
122
|
4. Call `channel.sip.answer()` when you want to send the final `200 OK`.
|
|
123
123
|
|
|
124
|
-
For outbound calls, pre-answer is controlled by the remote side. If the remote endpoint sends `183 Session Progress` with SDP,
|
|
124
|
+
For outbound calls, pre-answer is controlled by the remote side. If the remote endpoint sends `183 Session Progress` with SDP, the host runtime moves the call to `early`. If it answers directly, `waitForEarly()` resolves when the call becomes `active`.
|
|
125
125
|
|
|
126
126
|
`early` is a media-ready state, not a final answer state. `answer()` is still the explicit transition that sends final `200 OK` for inbound calls. External billing behavior depends on the carrier.
|
|
127
127
|
|
|
@@ -173,6 +173,70 @@ On SIP channels, `channel.audio.say()` and `channel.audio.play()` automatically
|
|
|
173
173
|
|
|
174
174
|
If the call terminates before media becomes available, deferred audio resolves as a no-op.
|
|
175
175
|
|
|
176
|
+
### SIP Signalling Metadata
|
|
177
|
+
|
|
178
|
+
On **SIP channels**, `channel.sip` exposes raw signalling beyond call state — useful for
|
|
179
|
+
carrier routing, diversion chains, and vendor SDP attributes.
|
|
180
|
+
|
|
181
|
+
#### INVITE headers (`inviteSipHeaders`)
|
|
182
|
+
|
|
183
|
+
Snapshot of SIP headers from an **inbound INVITE** at call setup. Includes standard and
|
|
184
|
+
extension headers (`Diversion`, `P-Asserted-Identity`, `X-Trunk-Id`, `X-Neuro-UUID`, …).
|
|
185
|
+
|
|
186
|
+
| Property | Updates during call? |
|
|
187
|
+
| --- | --- |
|
|
188
|
+
| `inviteSipHeaders` | **No** — INVITE snapshot only; outbound B-legs usually `undefined` |
|
|
189
|
+
|
|
190
|
+
```ts
|
|
191
|
+
const h = channel.sip.inviteSipHeaders;
|
|
192
|
+
const diversion = h?.Diversion; // string | string[] when multiple hops
|
|
193
|
+
const trunkId = h?.['X-Trunk-Id'];
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Header names match what the host stack exposes (case-sensitive). Duplicate headers become
|
|
197
|
+
`string[]`.
|
|
198
|
+
|
|
199
|
+
#### Remote SDP (`remoteSdp`, `getRemoteSdpDetails()`)
|
|
200
|
+
|
|
201
|
+
| Property / method | Updates during call? |
|
|
202
|
+
| --- | --- |
|
|
203
|
+
| `remoteSdp` | **Yes** — latest negotiated remote SDP (INVITE, 183, 200, re-INVITE) |
|
|
204
|
+
| `getRemoteSdpDetails()` | **Yes** — re-parses current `remoteSdp` on each call |
|
|
205
|
+
|
|
206
|
+
```ts
|
|
207
|
+
const raw = channel.sip.remoteSdp;
|
|
208
|
+
const details = channel.sip.getRemoteSdpDetails();
|
|
209
|
+
|
|
210
|
+
details?.session.connection; // e.g. "IN IP4 203.0.113.5"
|
|
211
|
+
details?.attributes.rtpmap; // e.g. "0 PCMU/8000"
|
|
212
|
+
details?.attributes['x-vendor-tag']; // custom a=x-vendor-tag:...
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
`getRemoteSdpDetails()` parses session lines (`o=`, `s=`, `c=`, `m=audio`) and all `a=`
|
|
216
|
+
attributes. It is not a full SDP validator — use `remoteSdp` when you need the raw body.
|
|
217
|
+
|
|
218
|
+
#### SIP INFO (`sipInfo$`)
|
|
219
|
+
|
|
220
|
+
Live stream of incoming SIP INFO messages (`contentType` + `body`). Prefer `dtmf$` for DTMF.
|
|
221
|
+
|
|
222
|
+
```ts
|
|
223
|
+
channel.sip.sipInfo$.subscribe(({ contentType, body }) => {
|
|
224
|
+
logger.log('SIP INFO', { contentType, body });
|
|
225
|
+
});
|
|
226
|
+
```
|
|
227
|
+
|
|
228
|
+
Events before subscribe are not replayed.
|
|
229
|
+
|
|
230
|
+
#### Relating streams to SDP
|
|
231
|
+
|
|
232
|
+
| Stream | `sdp` field |
|
|
233
|
+
| --- | --- |
|
|
234
|
+
| `progress$` | Present when a 1xx response carries SDP (e.g. 183 early media) |
|
|
235
|
+
| `sipSignal$` | Present only on callbacks that include SDP; often empty on final `active` |
|
|
236
|
+
|
|
237
|
+
For the **persisted** negotiated SDP, use `remoteSdp` / `getRemoteSdpDetails()`, not only
|
|
238
|
+
the per-event `sdp` on `sipSignal$`.
|
|
239
|
+
|
|
176
240
|
### SIP Controls
|
|
177
241
|
|
|
178
242
|
`channel.sip` also supports:
|
|
@@ -186,7 +250,43 @@ If the call terminates before media becomes available, deferred audio resolves a
|
|
|
186
250
|
- `makeCall()` to create an outbound SIP B-leg from the main SIP channel.
|
|
187
251
|
- `bridge(other)` to cross-connect two SIP channels.
|
|
188
252
|
|
|
189
|
-
`makeCall()` and `bridge()` are supported
|
|
253
|
+
`makeCall()` and `bridge()` are supported on SIP channels. The returned B-leg is a full
|
|
254
|
+
`MediaChannel` with the same API as the main channel. WS and headless channels do not create real SIP legs.
|
|
255
|
+
|
|
256
|
+
### `channel.sip` Reference
|
|
257
|
+
|
|
258
|
+
SIP-only unless noted. WS/headless: most methods are no-ops; `state` behaves as synthetic `active`.
|
|
259
|
+
|
|
260
|
+
| Member | Description |
|
|
261
|
+
| --- | --- |
|
|
262
|
+
| `state` | Sync getter: `idle` \| `ringing` \| `early` \| `active` \| `holding` \| `terminated` |
|
|
263
|
+
| `isAnswered` | `true` after 200 OK (outbound received / inbound sent via `answer()`) |
|
|
264
|
+
| `state$` | Emits on every state transition |
|
|
265
|
+
| `progress$` | SIP 1xx provisional responses (`SipProgressEvent`) |
|
|
266
|
+
| `early$` | Emits once when RTP is up before final answer |
|
|
267
|
+
| `answered$` | Emits once on 200 OK |
|
|
268
|
+
| `dtmf$` | Remote DTMF digits (`DtmfEvent`: `digit`, `duration`) |
|
|
269
|
+
| `sipInfo$` | Live incoming SIP INFO (`SipInfo`: `contentType`, `body`) |
|
|
270
|
+
| `sipSignal$` | Low-level SIP stack events (`SipSignal`; `sdp` only on that callback) |
|
|
271
|
+
| `remoteSdp` | Latest negotiated remote SDP body; updates when new SDP arrives |
|
|
272
|
+
| `inviteSipHeaders` | Inbound INVITE header snapshot (`SipInviteHeaders`); does not update |
|
|
273
|
+
| `getRemoteSdpDetails()` | Parse `remoteSdp` → `ParsedSdpDetails` (session + `a=` attributes) |
|
|
274
|
+
| `sendProgress()` | Inbound: send 183 Session Progress → `early` |
|
|
275
|
+
| `waitForEarly()` | Await `early` or `active` (Promise) |
|
|
276
|
+
| `waitForAnswer()` | Await final 200 OK (Promise) |
|
|
277
|
+
| `answer()` | Inbound: send final 200 OK → `active` |
|
|
278
|
+
| `sendDtmf(digit, duration?)` | Send DTMF tone |
|
|
279
|
+
| `sendInfo(contentType, body)` | Send SIP INFO |
|
|
280
|
+
| `hold()` / `unhold()` | SIP hold |
|
|
281
|
+
| `mute()` / `unmute()` | Suppress local outgoing audio |
|
|
282
|
+
| `hangup()` | Terminate call |
|
|
283
|
+
| `makeCall(opts)` | Outbound B-leg (`MediaChannel`); `sipUri` or `msisdn` |
|
|
284
|
+
| `bridge(other)` | Cross-connect two SIP calls; returns teardown `() => void` |
|
|
285
|
+
|
|
286
|
+
Prefer `dtmf$` over `sipInfo$` for DTMF. Prefer `state$` / `early$` / `answered$` over raw
|
|
287
|
+
`sipSignal$` for call lifecycle. Use `remoteSdp` / `getRemoteSdpDetails()` for negotiated
|
|
288
|
+
media description; use `inviteSipHeaders` for routing headers from the inbound INVITE.
|
|
289
|
+
See **SIP Signalling Metadata** above.
|
|
190
290
|
|
|
191
291
|
### SIP Bridge
|
|
192
292
|
|
|
@@ -204,7 +304,7 @@ await bLeg.sip.waitForAnswer();
|
|
|
204
304
|
const teardown = channel.sip.bridge(bLeg);
|
|
205
305
|
```
|
|
206
306
|
|
|
207
|
-
|
|
307
|
+
You can pass only `msisdn` instead of `sipUri`. The host resolves the SIP URI from agent and trunk settings and applies trunk caller-id options when configured.
|
|
208
308
|
|
|
209
309
|
```ts
|
|
210
310
|
const bLeg = await channel.sip.makeCall({
|
|
@@ -215,7 +315,7 @@ await bLeg.sip.waitForAnswer();
|
|
|
215
315
|
const teardown = channel.sip.bridge(bLeg);
|
|
216
316
|
```
|
|
217
317
|
|
|
218
|
-
|
|
318
|
+
Pass `channel` as a trunk-name override when selecting which outbound trunk to use:
|
|
219
319
|
|
|
220
320
|
```ts
|
|
221
321
|
const bLeg = await channel.sip.makeCall({
|
|
@@ -304,7 +404,7 @@ const asr = await channel.createAsr({
|
|
|
304
404
|
`AsrHandle` exposes:
|
|
305
405
|
|
|
306
406
|
- `result$`: finalized utterances.
|
|
307
|
-
- `partial$`: streaming partial hypotheses
|
|
407
|
+
- `partial$`: streaming partial hypotheses as `{ text, isFinal }`.
|
|
308
408
|
- `speechStart$` / `speechEnd$`: VAD speech boundaries.
|
|
309
409
|
- `interrupt$`: barge-in / interrupt events where the host supports them.
|
|
310
410
|
- `vadProbability$`: normalized VAD probability when available.
|
|
@@ -349,9 +449,9 @@ Each vendor connector accepts its native parameter names:
|
|
|
349
449
|
|
|
350
450
|
All vendors also accept the env-style names (`AZURE_SPEECH_KEY`, `ELEVENLABS_API_KEY`, etc.) for backwards compatibility, but vendor-native names are checked first and are preferred.
|
|
351
451
|
|
|
352
|
-
###
|
|
452
|
+
### Platform ASR Key Selection
|
|
353
453
|
|
|
354
|
-
|
|
454
|
+
When platform credential catalogs are enabled, select ASR keys by **`name`**:
|
|
355
455
|
|
|
356
456
|
```ts
|
|
357
457
|
const asr = await channel.createAsr({
|
|
@@ -360,12 +460,26 @@ const asr = await channel.createAsr({
|
|
|
360
460
|
});
|
|
361
461
|
```
|
|
362
462
|
|
|
363
|
-
The
|
|
463
|
+
The host resolves credentials from `channel.params.authentication_data` for the current dialog agent and company. If `name` is omitted, `channel.params.defaultAsrName` may be used.
|
|
364
464
|
|
|
365
465
|
When both `name` (platform key) and explicit `data` are provided, `data` values win — they are applied last and override anything resolved from the platform.
|
|
366
466
|
|
|
367
467
|
## TTS, Playback, And Mixer Queues
|
|
368
468
|
|
|
469
|
+
All audio playback goes through **`channel.audio`** (`ChannelAudio`). There are no top-level
|
|
470
|
+
`channel.say()` / `channel.play()` shortcuts on `MediaChannel`.
|
|
471
|
+
|
|
472
|
+
| Method | Purpose |
|
|
473
|
+
| --- | --- |
|
|
474
|
+
| `channel.audio.say(textOrObservable, options?)` | Synthesize text with TTS and play on a mixer queue |
|
|
475
|
+
| `channel.audio.play(source, options?)` | Play raw audio (URL, path, or platform phrase record) |
|
|
476
|
+
| `channel.audio.presay(text, options?)` | Pre-synthesize TTS into the host cache (no playback) |
|
|
477
|
+
| `channel.audio.preload(source, options?)` | Decode/warm a raw audio source (no playback) |
|
|
478
|
+
| `channel.audio.queue(index)` | Per-queue control handle (`MixerQueueControl`) |
|
|
479
|
+
| `channel.audio.remove(alias, queue?)` | Remove one queued item by alias |
|
|
480
|
+
| `channel.audio.stop(queue)` | Clear a queue **and** abort in-flight sentence TTS for it |
|
|
481
|
+
| `channel.audio.stopAll()` | Clear every queue (WS clients also get an audio interrupt) |
|
|
482
|
+
|
|
369
483
|
`channel.audio.say(textOrObservable, options?)` synthesizes text and plays it through the mixer.
|
|
370
484
|
|
|
371
485
|
```ts
|
|
@@ -384,11 +498,11 @@ await channel.audio.say('Please wait while I check that.', {
|
|
|
384
498
|
});
|
|
385
499
|
```
|
|
386
500
|
|
|
387
|
-
Use full vendor names for `ttsVendor`. Dedicated TTS vendors include `"elevenlabs"`, `"google"`, and `"voctiv"`. The default TTS path can also accept compatible aliases such as `"azure"` or `"neuro_v3"`, depending on how
|
|
501
|
+
Use full vendor names for `ttsVendor`. Dedicated TTS vendors include `"elevenlabs"`, `"google"`, and `"voctiv"`. The default TTS path can also accept compatible aliases such as `"azure"` or `"neuro_v3"`, depending on how the host runtime is configured.
|
|
388
502
|
|
|
389
503
|
Vendor-native parameter names (`api_key`, `voice_id`, `model_id`, `base_url`) are passed directly to the connector and override any platform defaults. See [TTS Credentials And Vendor Parameters](#tts-credentials-and-vendor-parameters) for the full list of accepted keys per vendor.
|
|
390
504
|
|
|
391
|
-
`channel.audio.play(source, options?)` plays raw audio from a URL/path or a `
|
|
505
|
+
`channel.audio.play(source, options?)` plays raw audio from a URL/path or a phrase record from `platform.getRecords()`.
|
|
392
506
|
|
|
393
507
|
```ts
|
|
394
508
|
await channel.audio.play('/opt/prompts/welcome.wav', {
|
|
@@ -397,9 +511,40 @@ await channel.audio.play('/opt/prompts/welcome.wav', {
|
|
|
397
511
|
});
|
|
398
512
|
```
|
|
399
513
|
|
|
400
|
-
|
|
514
|
+
### Pre-synthesis And Preload
|
|
401
515
|
|
|
402
|
-
`channel.audio.
|
|
516
|
+
`channel.audio.presay(text, options?)` runs TTS ahead of time and stores PCM in the host TTS
|
|
517
|
+
cache. Later `say()` calls with the same resolved TTS config and text can reuse the cached file.
|
|
518
|
+
Playback does **not** start. If the cache is unavailable, the runtime logs a warning and resolves
|
|
519
|
+
without throwing.
|
|
520
|
+
|
|
521
|
+
```ts
|
|
522
|
+
await channel.audio.presay('Your balance is one hundred dollars.', {
|
|
523
|
+
ttsVendor: 'elevenlabs',
|
|
524
|
+
ttsConfig: { voice_id: 'bBLRWT6MSWBFAm76ZWXY' },
|
|
525
|
+
});
|
|
526
|
+
|
|
527
|
+
// Later — cache hit, faster playback:
|
|
528
|
+
await channel.audio.say('Your balance is one hundred dollars.', {
|
|
529
|
+
alias: 'balance',
|
|
530
|
+
ttsVendor: 'elevenlabs',
|
|
531
|
+
ttsConfig: { voice_id: 'bBLRWT6MSWBFAm76ZWXY' },
|
|
532
|
+
});
|
|
533
|
+
```
|
|
534
|
+
|
|
535
|
+
`PresayOptions` accepts `ttsVendor`, `name`, `ttsConfig`, `ttsStrategy`, and optional `cache`
|
|
536
|
+
overrides (same shape as `PlayOptions.cache`).
|
|
537
|
+
|
|
538
|
+
`channel.audio.preload(source, options?)` downloads/decodes a **raw audio** source through the
|
|
539
|
+
audio player path. It does **not** synthesize TTS and does **not** populate the TTS cache used
|
|
540
|
+
by `presay()`. Use it to warm the decoder before `play()`.
|
|
541
|
+
|
|
542
|
+
```ts
|
|
543
|
+
await channel.audio.preload('/opt/prompts/welcome.wav');
|
|
544
|
+
await channel.audio.play('/opt/prompts/welcome.wav', { alias: 'welcome' });
|
|
545
|
+
```
|
|
546
|
+
|
|
547
|
+
When phrase persistence is enabled, `preload()` can store decoded audio for later playback via `platform.getRecords()`. Pass `options.cache` to override phrase name, flag, or language.
|
|
403
548
|
|
|
404
549
|
### TTS Strategies
|
|
405
550
|
|
|
@@ -413,24 +558,77 @@ When using an `Observable<string>` input, WS clients also receive text progress
|
|
|
413
558
|
|
|
414
559
|
### Mixer Queues
|
|
415
560
|
|
|
416
|
-
The mixer has queues
|
|
561
|
+
The mixer has queues **`0` through `4`**. Use separate queues for main speech, earcons, hold
|
|
562
|
+
music, or background audio so barge-in on one queue does not cut unrelated audio.
|
|
563
|
+
|
|
564
|
+
Obtain a per-queue handle with **`channel.audio.queue(index)`** (`MixerQueueControl`):
|
|
565
|
+
|
|
566
|
+
| Member | Description |
|
|
567
|
+
| --- | --- |
|
|
568
|
+
| `index` | Queue index **0–4** |
|
|
569
|
+
| `volume` | Linear gain **0.0–1.0** for the entire queue (get/set) |
|
|
570
|
+
| `itemStarted$` | Emits item **`alias`** when playback starts |
|
|
571
|
+
| `itemFinished$` | Emits **`alias`** when an item finishes, is removed, or is skipped by clear |
|
|
572
|
+
| `queueEmpty$` | Emits when the queue is empty after all PCM has been mixed out |
|
|
573
|
+
| `remove(alias)` | Drop one item on this queue |
|
|
574
|
+
| `clear()` | Drop all items on this queue (does **not** abort in-flight TTS generation) |
|
|
575
|
+
|
|
576
|
+
Top-level helpers on **`channel.audio`**:
|
|
577
|
+
|
|
578
|
+
- **`remove(alias, queue?)`** — when `queue` is omitted, searches all five queues; when set, only that queue is checked.
|
|
579
|
+
- **`stop(queue)`** — same as `clear()` **plus** aborts in-flight sentence TTS for that queue.
|
|
580
|
+
- **`stopAll()`** — `stop()` on every queue; WS clients also receive an audio interrupt signal.
|
|
417
581
|
|
|
418
582
|
```ts
|
|
583
|
+
const tts = channel.audio.queue(0);
|
|
419
584
|
const music = channel.audio.queue(2);
|
|
585
|
+
|
|
586
|
+
tts.itemStarted$.subscribe((alias) => logger.log('TTS started', { alias }));
|
|
587
|
+
tts.queueEmpty$.subscribe(() => logger.log('Agent queue idle'));
|
|
588
|
+
|
|
420
589
|
music.volume = 0.25;
|
|
421
590
|
|
|
422
591
|
await channel.audio.play('/opt/audio/hold.wav', {
|
|
423
592
|
queue: 2,
|
|
424
593
|
alias: 'hold-music',
|
|
425
594
|
loop: true,
|
|
595
|
+
loopDelayMs: 500,
|
|
426
596
|
});
|
|
427
597
|
|
|
428
|
-
|
|
598
|
+
// Remove one earcon without touching TTS:
|
|
599
|
+
channel.audio.remove('hold-music', 2);
|
|
600
|
+
|
|
601
|
+
// Barge-in: stop agent speech and abort pending sentence synthesis:
|
|
602
|
+
channel.audio.stop(0);
|
|
603
|
+
|
|
604
|
+
// Or clear music only (no TTS abort on queue 0):
|
|
605
|
+
music.clear();
|
|
429
606
|
```
|
|
430
607
|
|
|
431
|
-
`PlayOptions.volume` changes the whole queue volume, not just one item.
|
|
608
|
+
`PlayOptions.volume` changes the whole queue volume, not just one item.
|
|
609
|
+
|
|
610
|
+
For sentence-split TTS, queue item aliases are suffixed as `alias-0`, `alias-1`, and so on. Raw
|
|
611
|
+
`play()` and direct streaming TTS use the alias exactly. Pass the suffixed alias to
|
|
612
|
+
`remove()` when cancelling a single synthesized sentence.
|
|
613
|
+
|
|
614
|
+
### PlayOptions Reference
|
|
615
|
+
|
|
616
|
+
Shared by `say()`, `play()`, and (where noted) `presay()`:
|
|
617
|
+
|
|
618
|
+
| Field | Type | Applies to | Description |
|
|
619
|
+
| --- | --- | --- | --- |
|
|
620
|
+
| `queue` | `number?` | `say`, `play` | Mixer queue **0–4** (default **0**). |
|
|
621
|
+
| `alias` | `string?` | `say`, `play` | Stable item id for `remove()` and queue events. |
|
|
622
|
+
| `loop` | `boolean?` | `say`, `play` | Restart after finish until stopped/removed. |
|
|
623
|
+
| `loopDelayMs` | `number?` | `say`, `play` | Silence between loop iterations. |
|
|
624
|
+
| `volume` | `number?` | `say`, `play` | Sets **whole queue** gain **0.0–1.0** (not per-item). |
|
|
625
|
+
| `ttsStrategy` | `TtsStrategy?` | `say`, `presay` | `sentence` \| `streaming` \| `full`. |
|
|
626
|
+
| `ttsVendor` | `TtsVendor?` | `say`, `presay` | Override `channel.params.ttsVendor`. |
|
|
627
|
+
| `name` | `string?` | `say`, `presay` | Platform TTS credential **`name`** (key catalog selector). |
|
|
628
|
+
| `ttsConfig` | `Record<string, unknown>?` | `say`, `presay` | Vendor params; `name` key is stripped before send. |
|
|
629
|
+
| `cache` | `true \| CacheOptions?` | `say`, `presay` | TTS file cache; optional platform phrase persist. |
|
|
432
630
|
|
|
433
|
-
|
|
631
|
+
`play()` ignores `tts*` and `cache` for raw audio. `preload()` only accepts `cache` overrides.
|
|
434
632
|
|
|
435
633
|
## TTS Credentials And Vendor Parameters
|
|
436
634
|
|
|
@@ -460,9 +658,9 @@ Each TTS vendor connector accepts its native parameter names:
|
|
|
460
658
|
|
|
461
659
|
All vendors also accept the env-style names (`ELEVENLABS_API_KEY`, `ELEVENLABS_VOICE_ID`, etc.) for backwards compatibility, but vendor-native names are checked first and are preferred.
|
|
462
660
|
|
|
463
|
-
###
|
|
661
|
+
### Platform TTS Key Selection
|
|
464
662
|
|
|
465
|
-
|
|
663
|
+
TTS credentials can also be selected by `PlayOptions.name` or `ttsConfig.name`:
|
|
466
664
|
|
|
467
665
|
```ts
|
|
468
666
|
await channel.audio.say('Здравствуйте!', {
|
|
@@ -473,14 +671,14 @@ await channel.audio.say('Здравствуйте!', {
|
|
|
473
671
|
});
|
|
474
672
|
```
|
|
475
673
|
|
|
476
|
-
The
|
|
674
|
+
The host resolves credentials from `channel.params.authentication_data` for the current agent and company. If `name` is omitted, `channel.params.defaultTtsName` may be used.
|
|
477
675
|
|
|
478
676
|
When both `name` (platform key) and explicit `ttsConfig` values are provided, `ttsConfig` values win — they are applied last and override anything resolved from the platform.
|
|
479
677
|
|
|
480
678
|
`cache` enables TTS result caching for `say()` and `presay()`:
|
|
481
679
|
|
|
482
|
-
- **`cache: true`** — read/write TTS file cache
|
|
483
|
-
- **`cache: { phraseName, flag?, language? }`** — TTS cache **plus** persist
|
|
680
|
+
- **`cache: true`** — read/write host TTS file cache.
|
|
681
|
+
- **`cache: { phraseName, flag?, language? }`** — TTS cache **plus** persist as a platform phrase so `platform.getRecords()` can retrieve the audio later.
|
|
484
682
|
|
|
485
683
|
```ts
|
|
486
684
|
// Cache only (no platform persist):
|
|
@@ -506,7 +704,7 @@ if (records?.[0]) {
|
|
|
506
704
|
}
|
|
507
705
|
```
|
|
508
706
|
|
|
509
|
-
|
|
707
|
+
Phrase persistence requires platform phrase storage and TTS cache to be enabled on the host.
|
|
510
708
|
|
|
511
709
|
## Error Handling
|
|
512
710
|
|
|
@@ -582,14 +780,56 @@ channel.events.error$.subscribe((err) => {
|
|
|
582
780
|
| `code` | `number \| string?` | HTTP status, gRPC status, provider code, or WebSocket close code. |
|
|
583
781
|
| `vendor` | `string?` | Vendor identifier, e.g. `"yandex"`, `"elevenlabs"`, `"azure"`. |
|
|
584
782
|
| `details` | `unknown?` | Arbitrary provider-specific payload. |
|
|
585
|
-
| `cause` | `unknown?` | Original
|
|
783
|
+
| `cause` | `unknown?` | Original underlying error when available. |
|
|
586
784
|
|
|
587
785
|
Subscribing to `error$` is optional. Old scripts that do not subscribe are not affected — the observables simply go unobserved.
|
|
588
786
|
|
|
787
|
+
## Channel Events
|
|
788
|
+
|
|
789
|
+
`channel.events` exposes session-level observables that are **not** tied to a single ASR handle:
|
|
790
|
+
|
|
791
|
+
| Observable | Emits when |
|
|
792
|
+
| --- | --- |
|
|
793
|
+
| `speechStart$` | User started speaking (VAD, socket event, or synthetic text input). |
|
|
794
|
+
| `speechEnd$` | User stopped speaking (VAD end, ASR final, or synthetic text input). |
|
|
795
|
+
| `interrupt$` | Barge-in: user speech interrupted bot audio (may be inert without VAD). |
|
|
796
|
+
| `terminated$` | Session ending — hangup, WS disconnect, or `channel.destroy()`. |
|
|
797
|
+
| `message$` | Structured WS data messages (`DataMessage`: `{ event, payload }`). |
|
|
798
|
+
| `error$` | Unified media/runtime errors (see [Error Handling](#error-handling)). |
|
|
799
|
+
|
|
800
|
+
```ts
|
|
801
|
+
channel.events.speechStart$.subscribe(() => {
|
|
802
|
+
channel.audio.stop(0); // barge-in on agent TTS queue
|
|
803
|
+
});
|
|
804
|
+
|
|
805
|
+
channel.events.message$.subscribe(({ event, payload }) => {
|
|
806
|
+
logger.log('WS client event', { event, payload });
|
|
807
|
+
});
|
|
808
|
+
|
|
809
|
+
channel.events.terminated$.subscribe(() => {
|
|
810
|
+
asr.destroy();
|
|
811
|
+
channel.destroy();
|
|
812
|
+
});
|
|
813
|
+
```
|
|
814
|
+
|
|
815
|
+
**ASR vs channel events:** `AsrHandle.speechStart$` / `speechEnd$` / `interrupt$` are scoped to one
|
|
816
|
+
recognizer instance. `channel.events.*` aggregates session-level signals (useful when you do not
|
|
817
|
+
create ASR or want one subscription for the whole channel).
|
|
818
|
+
|
|
589
819
|
## LLM API
|
|
590
820
|
|
|
591
821
|
`channel.llm` talks to the Omni LLM backend.
|
|
592
822
|
|
|
823
|
+
| Method | Returns | Description |
|
|
824
|
+
| --- | --- | --- |
|
|
825
|
+
| `ask(message, options?)` | `Promise<string>` | Single-shot completion (consumes SSE stream). |
|
|
826
|
+
| `stream(message, options?)` | `Observable<LlmStreamChunk>` | Token/chunk stream; use `chunk.content` for TTS. |
|
|
827
|
+
| `extract(options?)` | `Promise<Record<string, any>>` | Structured extraction via Omni extract API. |
|
|
828
|
+
| `makePersistentStream(options?)` | `PersistentLlmStreamHandle` | Long-lived stream for multi-turn chat. |
|
|
829
|
+
|
|
830
|
+
Common `LlmOptions`: `dialogUuid`, `agentUuid`, `role`, `hidden`, `name` (LLM speaker label — **not**
|
|
831
|
+
the TTS credential `name`), `payload`, `debug`, `agentAliasFilter`, `currentAgentAlias`.
|
|
832
|
+
|
|
593
833
|
```ts
|
|
594
834
|
const answer = await channel.llm.ask('Summarize the user request', {
|
|
595
835
|
role: 'assistant',
|
|
@@ -603,6 +843,8 @@ await channel.audio.say(answer);
|
|
|
603
843
|
For streaming:
|
|
604
844
|
|
|
605
845
|
```ts
|
|
846
|
+
import { map } from 'rxjs';
|
|
847
|
+
|
|
606
848
|
const stream$ = channel.llm.stream('Answer briefly', {
|
|
607
849
|
role: 'assistant',
|
|
608
850
|
});
|
|
@@ -613,13 +855,70 @@ await channel.audio.say(
|
|
|
613
855
|
);
|
|
614
856
|
```
|
|
615
857
|
|
|
616
|
-
|
|
858
|
+
Structured extraction:
|
|
859
|
+
|
|
860
|
+
```ts
|
|
861
|
+
const fields = await channel.llm.extract({
|
|
862
|
+
prompt: 'Extract appointment date and time from the dialog.',
|
|
863
|
+
temperature: 0.2,
|
|
864
|
+
});
|
|
865
|
+
```
|
|
866
|
+
|
|
867
|
+
Persistent multi-turn stream:
|
|
868
|
+
|
|
869
|
+
```ts
|
|
870
|
+
const chat = channel.llm.makePersistentStream({
|
|
871
|
+
agentUuid: context.agentUuid,
|
|
872
|
+
dialogUuid: context.dialogUuid,
|
|
873
|
+
});
|
|
874
|
+
|
|
875
|
+
chat.stream$.pipe(map((c) => c.content)).subscribe((text) => logger.debug('LLM chunk', { text }));
|
|
876
|
+
|
|
877
|
+
chat.send('What is my balance?');
|
|
878
|
+
chat.send('And my last payment date?');
|
|
879
|
+
chat.disconnect();
|
|
880
|
+
```
|
|
881
|
+
|
|
882
|
+
## Script Return Value
|
|
883
|
+
|
|
884
|
+
Scripts may return `void` or a **`ScriptResult`**:
|
|
885
|
+
|
|
886
|
+
```ts
|
|
887
|
+
return {
|
|
888
|
+
output: { intent: 'reschedule', score: 0.92 },
|
|
889
|
+
};
|
|
890
|
+
```
|
|
891
|
+
|
|
892
|
+
- **`output`** — stored in dialog stats / host persistence.
|
|
893
|
+
- **`error`** — optional; usually auto-populated on crash, but scripts may set it explicitly.
|
|
894
|
+
|
|
895
|
+
**Do not** return `env` from the script. Persist state via `context.env$`; the runtime snapshots it
|
|
896
|
+
after completion into **`PersistedScriptResult.env`**.
|
|
897
|
+
|
|
898
|
+
Use **`getScriptPhase(context)`** to branch on lifecycle (`online`, `messaging`, `recall`, etc.):
|
|
899
|
+
|
|
900
|
+
```ts
|
|
901
|
+
import { defineScript, getScriptPhase } from '@voctiv/agent-sdk';
|
|
902
|
+
|
|
903
|
+
export default defineScript(async ({ context }) => {
|
|
904
|
+
switch (getScriptPhase(context)) {
|
|
905
|
+
case 'online':
|
|
906
|
+
// live call
|
|
907
|
+
break;
|
|
908
|
+
case 'messaging':
|
|
909
|
+
// inbound message handling
|
|
910
|
+
break;
|
|
911
|
+
}
|
|
912
|
+
});
|
|
913
|
+
```
|
|
617
914
|
|
|
618
915
|
## Platform API
|
|
619
916
|
|
|
620
|
-
`platform` exposes
|
|
917
|
+
`platform` exposes platform operations.
|
|
918
|
+
|
|
919
|
+
`platform.nlu.extract(utterance, options?)` runs intent/entity extraction. If `options.context` is omitted, current dialog params are serialized and used as NLU context.
|
|
621
920
|
|
|
622
|
-
`platform.nlu.extract(
|
|
921
|
+
`platform.nlu.extract$()` is an Observable wrapper — one `extract()` call per subscription, not a streaming NLU session.
|
|
623
922
|
|
|
624
923
|
```ts
|
|
625
924
|
const result = await platform.nlu.extract('I want to reschedule', {
|
|
@@ -629,7 +928,7 @@ const result = await platform.nlu.extract('I want to reschedule', {
|
|
|
629
928
|
});
|
|
630
929
|
```
|
|
631
930
|
|
|
632
|
-
Platform APIs
|
|
931
|
+
Platform APIs (`platform.nlu`, `platform.call`, dialog writes, messaging, phrase records) are available when the host enables platform integration.
|
|
633
932
|
|
|
634
933
|
### Dialog State
|
|
635
934
|
|
|
@@ -638,7 +937,7 @@ platform.dialog.entryPoint = 'on_recall';
|
|
|
638
937
|
platform.dialog.result = 'done';
|
|
639
938
|
```
|
|
640
939
|
|
|
641
|
-
Setters update the local value immediately and
|
|
940
|
+
Setters update the local value immediately and persist to the platform asynchronously. They are not awaitable and should not be used as transactional writes.
|
|
642
941
|
|
|
643
942
|
### Platform-Scheduled Calls
|
|
644
943
|
|
|
@@ -650,7 +949,7 @@ The destination number should be E.164 formatted.
|
|
|
650
949
|
await platform.call('+12025551234');
|
|
651
950
|
```
|
|
652
951
|
|
|
653
|
-
When options are omitted, the host fills scheduling and routing defaults from agent
|
|
952
|
+
When options are omitted, the host fills scheduling and routing defaults from agent and dialog settings. Explicit `options` always win.
|
|
654
953
|
|
|
655
954
|
By default, the platform schedules the call for immediate processing. Use `date` to schedule it for the future:
|
|
656
955
|
|
|
@@ -697,6 +996,11 @@ await platform.call('+12025551234', {
|
|
|
697
996
|
|
|
698
997
|
This means the platform may retry up to three times, waiting about five minutes between attempts.
|
|
699
998
|
|
|
999
|
+
When you omit these options, the host fills defaults from `context.recallCount` / `context.recallDelay`
|
|
1000
|
+
(effective values for the current dialog). Those in turn fall back to CMS agent contact-rules
|
|
1001
|
+
(`context.agent?.recallCount` / `context.agent?.recallDelay`, legacy `nn.get_recall_count()` /
|
|
1002
|
+
`nn.get_recall_delay()`).
|
|
1003
|
+
|
|
700
1004
|
Other scheduling options:
|
|
701
1005
|
|
|
702
1006
|
- `priority`: higher-priority calls can be processed earlier by the dialer.
|
|
@@ -715,7 +1019,7 @@ await platform.messaging.send({
|
|
|
715
1019
|
});
|
|
716
1020
|
```
|
|
717
1021
|
|
|
718
|
-
|
|
1022
|
+
`platform.messaging.message$` replays the inbound message that started a headless messaging script; it is not a live subscription to all future messages.
|
|
719
1023
|
|
|
720
1024
|
## Offline / Headless Logic
|
|
721
1025
|
|
|
@@ -740,11 +1044,7 @@ export default defineScript(async ({ channel, context, logger, platform }) => {
|
|
|
740
1044
|
});
|
|
741
1045
|
```
|
|
742
1046
|
|
|
743
|
-
Headless sessions can be started by host
|
|
744
|
-
|
|
745
|
-
- a platform dialog queue worker that loads pending dialogs;
|
|
746
|
-
- an inbound messaging worker, usually with `context.entryPoint === 'on_message_api_received'`;
|
|
747
|
-
- an HTTP/API request that asks ScriptEngine to run a script without media.
|
|
1047
|
+
Headless sessions can be started by the host for background processing, inbound messaging, or API-triggered runs without media.
|
|
748
1048
|
|
|
749
1049
|
### What Works In Headless
|
|
750
1050
|
|
|
@@ -752,7 +1052,7 @@ These APIs are available and are the intended tools for offline scripts:
|
|
|
752
1052
|
|
|
753
1053
|
- `context.dialogParams`, `context.initialData`, `context.dialogEntity`, and `context.callEntity` for platform data.
|
|
754
1054
|
- `context.env$` for persisted per-dialog state.
|
|
755
|
-
- `platform.nlu.extract()` for text NLU when
|
|
1055
|
+
- `platform.nlu.extract()` for text NLU when platform integration is enabled.
|
|
756
1056
|
- `platform.messaging.send()` for outbound messages through the configured platform messaging transport.
|
|
757
1057
|
- `platform.call()` for scheduling outbound platform-managed calls.
|
|
758
1058
|
- `platform.dialog.entryPoint` and `platform.dialog.result` for updating dialog routing and outcome.
|
|
@@ -831,7 +1131,7 @@ context.env$?.next({
|
|
|
831
1131
|
});
|
|
832
1132
|
```
|
|
833
1133
|
|
|
834
|
-
Do not return `env` from the script.
|
|
1134
|
+
Do not return `env` from the script. The runtime snapshots `context.env$` after completion and persists it for the dialog.
|
|
835
1135
|
|
|
836
1136
|
### Combining Voice And Offline In One Script
|
|
837
1137
|
|
|
@@ -860,6 +1160,72 @@ export default defineScript(async ({ channel, context, platform }) => {
|
|
|
860
1160
|
|
|
861
1161
|
`context` includes identity, telephony fields, params, routing metadata, and runtime helpers.
|
|
862
1162
|
|
|
1163
|
+
### Agent env, recall defaults, and storage
|
|
1164
|
+
|
|
1165
|
+
When the host exposes agent identity, `context.agent` provides agent-scoped helpers:
|
|
1166
|
+
|
|
1167
|
+
```ts
|
|
1168
|
+
// Read all agent env keys
|
|
1169
|
+
const all = await context.agent?.env?.();
|
|
1170
|
+
|
|
1171
|
+
// Read one key
|
|
1172
|
+
const counter = await context.agent?.env?.<number>('visitCount');
|
|
1173
|
+
|
|
1174
|
+
// Write with optional TTL (days)
|
|
1175
|
+
await context.agent?.env?.('visitCount', 42, { expire: 30 });
|
|
1176
|
+
```
|
|
1177
|
+
|
|
1178
|
+
#### Recall settings (agent defaults vs effective)
|
|
1179
|
+
|
|
1180
|
+
Recall behavior uses **two layers** on `context`:
|
|
1181
|
+
|
|
1182
|
+
| Layer | Fields | Source | Use when |
|
|
1183
|
+
|-------|--------|--------|----------|
|
|
1184
|
+
| Agent defaults | `context.agent?.recallCount`, `context.agent?.recallDelay` | CMS contact-rules (`agent.recall_count`, `agent.delay` → seconds) | Compare with CMS settings; legacy `nn.get_recall_count()` / `get_recall_delay()` parity |
|
|
1185
|
+
| Effective for this run | `context.recallCount`, `context.recallDelay` | `dialog.params` / `call.params`, then agent defaults | Schedule outbound calls, recall routing logic |
|
|
1186
|
+
|
|
1187
|
+
Precedence for effective values: **dialog/call params** (`recall_count`, `recall_delay`) **>** agent CMS defaults.
|
|
1188
|
+
|
|
1189
|
+
`context.attempt` is the current recall attempt counter from `dialog.params.attempt` (starts at 0).
|
|
1190
|
+
`context.entryPoint` is the routing branch for this run (e.g. after a failed call).
|
|
1191
|
+
|
|
1192
|
+
```ts
|
|
1193
|
+
// Use effective values when scheduling the next outbound leg
|
|
1194
|
+
await context.platform?.call?.(msisdn, {
|
|
1195
|
+
recallCount: context.recallCount,
|
|
1196
|
+
recallDelay: context.recallDelay,
|
|
1197
|
+
entryPoint: 'on_follow_up',
|
|
1198
|
+
});
|
|
1199
|
+
|
|
1200
|
+
// Log CMS defaults vs per-dialog override
|
|
1201
|
+
logger.log('recall config', {
|
|
1202
|
+
effective: { count: context.recallCount, delay: context.recallDelay },
|
|
1203
|
+
agentDefault: {
|
|
1204
|
+
count: context.agent?.recallCount,
|
|
1205
|
+
delay: context.agent?.recallDelay,
|
|
1206
|
+
},
|
|
1207
|
+
attempt: context.attempt,
|
|
1208
|
+
});
|
|
1209
|
+
```
|
|
1210
|
+
|
|
1211
|
+
Values are loaded once at script start from the legacy agent row (cached with other LE agent
|
|
1212
|
+
resolution). They are plain snapshots — not `BehaviorSubject`s and not re-fetched during the run.
|
|
1213
|
+
|
|
1214
|
+
`context.storage('key1', 'key2')` reads CMS/global variables from agent settings, then company-level
|
|
1215
|
+
fallback. Always returns an object with every requested key (value or `null`).
|
|
1216
|
+
|
|
1217
|
+
### Execution budget (`runTime`)
|
|
1218
|
+
|
|
1219
|
+
Long-running async scripts can check and extend their time budget:
|
|
1220
|
+
|
|
1221
|
+
```ts
|
|
1222
|
+
if ((context.runTime?.remainingMs() ?? Infinity) < 5000) {
|
|
1223
|
+
context.runTime?.extend(30_000);
|
|
1224
|
+
}
|
|
1225
|
+
```
|
|
1226
|
+
|
|
1227
|
+
`budgetMs` and `maxExtendMs` are fixed for the session; `extend()` grants up to the remaining quota.
|
|
1228
|
+
|
|
863
1229
|
Important fields:
|
|
864
1230
|
|
|
865
1231
|
- `context.dialogUuid`: current dialog UUID.
|
|
@@ -870,6 +1236,9 @@ Important fields:
|
|
|
870
1236
|
- `context.initialData`: shallow snapshot of params at script start.
|
|
871
1237
|
- `context.dialogParams`: live param map for the run.
|
|
872
1238
|
- `context.entryPoint`: current routing entry point.
|
|
1239
|
+
- `context.attempt`: current recall attempt number (`dialog.params.attempt`).
|
|
1240
|
+
- `context.recallCount` / `context.recallDelay`: effective recall settings for this dialog/call.
|
|
1241
|
+
- `context.agent?.recallCount` / `context.agent?.recallDelay`: CMS agent defaults (immutable snapshot).
|
|
873
1242
|
- `context.headless`: true for offline/queue/messaging sessions without a real media channel.
|
|
874
1243
|
- `context.runTime`: async execution budget helper.
|
|
875
1244
|
- `context.env$`: persisted dialog environment as an RxJS `BehaviorSubject`.
|
|
@@ -895,7 +1264,7 @@ logger.log('ASR result received', { text });
|
|
|
895
1264
|
logger.warn('Low confidence intent', { confidence });
|
|
896
1265
|
```
|
|
897
1266
|
|
|
898
|
-
`logger.enableDebug(endpoint)` streams logs from the current script instance to a remote debug endpoint. `logger.breakpoint(label, snapshot?)` pauses only when an active debug session is connected; otherwise it resolves immediately.
|
|
1267
|
+
`logger.enableDebug(endpoint)` streams logs from the current script instance to a remote debug endpoint. `logger.disableDebug()` stops streaming. `logger.breakpoint(label, snapshot?)` pauses only when an active debug session is connected; otherwise it resolves immediately.
|
|
899
1268
|
|
|
900
1269
|
## WS And Headless Behavior
|
|
901
1270
|
|
|
@@ -930,15 +1299,7 @@ This is mainly for WS debug clients and automated tests. Unknown ASR ids are ign
|
|
|
930
1299
|
|
|
931
1300
|
## Package Notes
|
|
932
1301
|
|
|
933
|
-
The package
|
|
934
|
-
|
|
935
|
-
Build locally with:
|
|
936
|
-
|
|
937
|
-
```bash
|
|
938
|
-
npm run build
|
|
939
|
-
```
|
|
940
|
-
|
|
941
|
-
The package exports only the public SDK entry point:
|
|
1302
|
+
The package ships as CommonJS with TypeScript declarations. Import from `@voctiv/agent-sdk`:
|
|
942
1303
|
|
|
943
1304
|
```ts
|
|
944
1305
|
import { defineScript, type MediaChannel, type AsrHandle, type MediaError } from '@voctiv/agent-sdk';
|