@voctiv/agent-sdk 0.3.0 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +527 -188
- package/dist/index.d.ts +2 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +3 -1
- package/dist/index.js.map +1 -1
- package/dist/types/media-providers.d.ts +87 -0
- package/dist/types/media-providers.d.ts.map +1 -0
- package/dist/types/media-providers.js +23 -0
- package/dist/types/media-providers.js.map +1 -0
- package/examples/README.md +2 -0
- package/examples/custom-media-providers-script.ts +47 -0
- package/examples/custom-media-providers.ts +112 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -70,17 +70,25 @@ export default defineScript(async ({ channel, logger, context }) => {
|
|
|
70
70
|
});
|
|
71
71
|
```
|
|
72
72
|
|
|
73
|
+
|
|
74
|
+
|
|
73
75
|
## Examples
|
|
74
76
|
|
|
75
|
-
The [
|
|
77
|
+
The `[examples/](./examples/)` folder contains copy-paste-ready scripts:
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
| Example | Description |
|
|
81
|
+
| --------------------------------------------------------------------------- | -------------------------------------------------------------------------- |
|
|
82
|
+
| [outbound-with-recall.ts](./examples/outbound-with-recall.ts) | Outbound with `recallCount` / `recallDelay` (automatic redial on failure) |
|
|
83
|
+
| [recall-routing-by-attempt.ts](./examples/recall-routing-by-attempt.ts) | Online recall: branch on `context.attempt` (`getScriptPhase` → `'online'`) |
|
|
84
|
+
| [schedule-call-with-defaults.ts](./examples/schedule-call-with-defaults.ts) | `platform.call()` without explicit recall — CMS defaults from `context` |
|
|
85
|
+
| [after-call-continuation.ts](./examples/after-call-continuation.ts) | `onSuccessCall` / `onFailedCall` vs recall (mutually exclusive) |
|
|
86
|
+
| [read-recall-from-params.ts](./examples/read-recall-from-params.ts) | `parseRecallDelaySeconds()` / `parseRecallCount()` on legacy params |
|
|
87
|
+
| [custom-media-providers.ts](./examples/custom-media-providers.ts) | Host-only `mediaProviders` module (`ScriptAsrConnector` / `ScriptTtsConnector`) |
|
|
88
|
+
| [custom-media-providers-script.ts](./examples/custom-media-providers-script.ts) | Sandboxed `defineScript` entry that uses custom vendors (no `ws` import) |
|
|
89
|
+
|
|
90
|
+
|
|
76
91
|
|
|
77
|
-
| Example | Description |
|
|
78
|
-
|---------|-------------|
|
|
79
|
-
| [outbound-with-recall.ts](./examples/outbound-with-recall.ts) | Outbound with `recallCount` / `recallDelay` (automatic redial on failure) |
|
|
80
|
-
| [recall-routing-by-attempt.ts](./examples/recall-routing-by-attempt.ts) | Online recall: branch on `context.attempt` (`getScriptPhase` → `'online'`) |
|
|
81
|
-
| [schedule-call-with-defaults.ts](./examples/schedule-call-with-defaults.ts) | `platform.call()` without explicit recall — CMS defaults from `context` |
|
|
82
|
-
| [after-call-continuation.ts](./examples/after-call-continuation.ts) | `onSuccessCall` / `onFailedCall` vs recall (mutually exclusive) |
|
|
83
|
-
| [read-recall-from-params.ts](./examples/read-recall-from-params.ts) | `parseRecallDelaySeconds()` / `parseRecallCount()` on legacy params |
|
|
84
92
|
|
|
85
93
|
### Quick recall example
|
|
86
94
|
|
|
@@ -112,17 +120,25 @@ const delaySec = parseRecallDelaySeconds(context.dialogParams?.recall_delay); //
|
|
|
112
120
|
const maxAttempts = parseRecallCount(context.dialogParams?.recall_count);
|
|
113
121
|
```
|
|
114
122
|
|
|
123
|
+
|
|
124
|
+
|
|
115
125
|
## What The SDK Contains
|
|
116
126
|
|
|
117
127
|
`defineScript(fn)` marks the default export as the script entry point. It returns the same function and exists to give TypeScript the correct `ScriptContext` shape.
|
|
118
128
|
|
|
129
|
+
`defineMediaProviders(def)` marks a named `mediaProviders` export for custom ASR/TTS factories.
|
|
130
|
+
Ship it in a **sibling** module (`media-providers/`), not in the sandboxed script entry — see
|
|
131
|
+
[Custom ASR / TTS Providers](#custom-asr--tts-providers).
|
|
132
|
+
|
|
119
133
|
**Three different “context” names:**
|
|
120
134
|
|
|
121
|
-
|
|
122
|
-
|
|
|
123
|
-
|
|
|
124
|
-
| `
|
|
125
|
-
| `
|
|
135
|
+
|
|
136
|
+
| Name | Meaning |
|
|
137
|
+
| ------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- |
|
|
138
|
+
| `ScriptContext` | Top-level injection: `{ channel, logger, context, platform }` |
|
|
139
|
+
| `context` (`ScriptDialogContext`) | Dialog identity, params, routing snapshot, `env$`, headless — see [Dialog Context](#dialog-context-and-persisted-env) |
|
|
140
|
+
| `options.context` on `platform.nlu.extract` | Opaque NLU disambiguation string/JSON — **not** dialog context |
|
|
141
|
+
|
|
126
142
|
|
|
127
143
|
`ScriptContext` is the top-level object passed to a script:
|
|
128
144
|
|
|
@@ -143,6 +159,8 @@ const maxAttempts = parseRecallCount(context.dialogParams?.recall_count);
|
|
|
143
159
|
- `channel.events` exposes speech, interrupt, termination, WS data message, and media error observables.
|
|
144
160
|
- `channel.textInput` injects synthetic ASR results for tests and debug clients.
|
|
145
161
|
|
|
162
|
+
|
|
163
|
+
|
|
146
164
|
## SIP And Pre-Answer Media
|
|
147
165
|
|
|
148
166
|
SIP sessions expose call state through `channel.sip.state`, `state$`, `progress$`, `early$`, and `answered$`.
|
|
@@ -154,6 +172,8 @@ The important states are:
|
|
|
154
172
|
- `active`: final 200 OK has been received or sent.
|
|
155
173
|
- `terminated`: the call ended and no more audio is possible.
|
|
156
174
|
|
|
175
|
+
|
|
176
|
+
|
|
157
177
|
### How Pre-Answer Works
|
|
158
178
|
|
|
159
179
|
Pre-answer means the SIP media path is open before the call is finally answered with `200 OK`. In this state the caller can already hear TTS, the script can already receive audio for ASR, and DTMF can be exchanged.
|
|
@@ -231,14 +251,16 @@ carrier routing, diversion chains, and vendor SDP attributes.
|
|
|
231
251
|
|
|
232
252
|
#### When to use what
|
|
233
253
|
|
|
234
|
-
|
|
235
|
-
|
|
|
236
|
-
|
|
|
237
|
-
|
|
|
238
|
-
|
|
|
239
|
-
|
|
|
240
|
-
|
|
|
241
|
-
|
|
|
254
|
+
|
|
255
|
+
| Need | API | When |
|
|
256
|
+
| -------------------------------------------------------------------------------------------------------------------- | ------------------------------------- | ---------------------------------- |
|
|
257
|
+
| Routing / identity / locale from the **inbound INVITE** (`Diversion`, `X-Trunk-Id`, `X-Neuro-UUID`, `X-language`, …) | `channel.sip.inviteSipHeaders` | Call **start** only (snapshot) |
|
|
258
|
+
| Peer signal **during** the call (e.g. mid-call language in INFO body) | `channel.sip.sipInfo$` | After subscribe; each INFO |
|
|
259
|
+
| DTMF digits | `channel.sip.dtmf$` | Prefer over parsing INFO |
|
|
260
|
+
| SDP / codec / connection | `remoteSdp` / `getRemoteSdpDetails()` | May change (183 / 200 / re-INVITE) |
|
|
261
|
+
| SIP response code / phrase (180, 486, …) | `sipSignal$` | Low-level; **no** response headers |
|
|
262
|
+
| Live SIP headers on 200 / re-INVITE / BYE | — | **Not exposed** |
|
|
263
|
+
|
|
242
264
|
|
|
243
265
|
Decision guide:
|
|
244
266
|
|
|
@@ -257,10 +279,12 @@ Outbound INVITE headers you **send** go through `platform.call({ protoAdditional
|
|
|
257
279
|
Snapshot of SIP headers from an **inbound INVITE** at call setup. Includes standard and
|
|
258
280
|
extension headers (`Diversion`, `P-Asserted-Identity`, `X-Trunk-Id`, `X-Neuro-UUID`, `X-language`, …).
|
|
259
281
|
|
|
260
|
-
|
|
261
|
-
|
|
|
282
|
+
|
|
283
|
+
| Property | Updates during call? |
|
|
284
|
+
| ------------------ | ---------------------------------------------------------------------------------- |
|
|
262
285
|
| `inviteSipHeaders` | **No** — INVITE snapshot only; outbound B-legs / WS / headless usually `undefined` |
|
|
263
286
|
|
|
287
|
+
|
|
264
288
|
```ts
|
|
265
289
|
const h = channel.sip.inviteSipHeaders;
|
|
266
290
|
const diversion = h?.Diversion; // string | string[] when multiple hops
|
|
@@ -273,10 +297,12 @@ Header names match what the host stack exposes (case-sensitive). Duplicate heade
|
|
|
273
297
|
|
|
274
298
|
#### Remote SDP (`remoteSdp`, `getRemoteSdpDetails()`)
|
|
275
299
|
|
|
276
|
-
|
|
277
|
-
|
|
|
278
|
-
|
|
|
279
|
-
| `
|
|
300
|
+
|
|
301
|
+
| Property / method | Updates during call? |
|
|
302
|
+
| ----------------------- | -------------------------------------------------------------------- |
|
|
303
|
+
| `remoteSdp` | **Yes** — latest negotiated remote SDP (INVITE, 183, 200, re-INVITE) |
|
|
304
|
+
| `getRemoteSdpDetails()` | **Yes** — re-parses current `remoteSdp` on each call |
|
|
305
|
+
|
|
280
306
|
|
|
281
307
|
```ts
|
|
282
308
|
const raw = channel.sip.remoteSdp;
|
|
@@ -306,11 +332,13 @@ Events before subscribe are not replayed.
|
|
|
306
332
|
|
|
307
333
|
#### Relating streams to SDP
|
|
308
334
|
|
|
309
|
-
|
|
310
|
-
|
|
|
311
|
-
|
|
|
335
|
+
|
|
336
|
+
| Stream | `sdp` field |
|
|
337
|
+
| ------------ | ------------------------------------------------------------------------- |
|
|
338
|
+
| `progress$` | Present when a 1xx response carries SDP (e.g. 183 early media) |
|
|
312
339
|
| `sipSignal$` | Present only on callbacks that include SDP; often empty on final `active` |
|
|
313
340
|
|
|
341
|
+
|
|
314
342
|
For the **persisted** negotiated SDP, use `remoteSdp` / `getRemoteSdpDetails()`, not only
|
|
315
343
|
the per-event `sdp` on `sipSignal$`.
|
|
316
344
|
|
|
@@ -337,31 +365,33 @@ unnoticed. Guard them with `context.headless` or `channel.type` when a script ru
|
|
|
337
365
|
SIP-only unless noted. WS/headless: most methods are no-ops and `state` behaves as synthetic
|
|
338
366
|
`active`, except `makeCall()` and `bridge()`, which throw.
|
|
339
367
|
|
|
340
|
-
|
|
341
|
-
|
|
|
342
|
-
|
|
|
343
|
-
| `
|
|
344
|
-
| `
|
|
345
|
-
| `
|
|
346
|
-
| `
|
|
347
|
-
| `
|
|
348
|
-
| `
|
|
349
|
-
| `
|
|
350
|
-
| `
|
|
351
|
-
| `
|
|
352
|
-
| `
|
|
353
|
-
| `
|
|
354
|
-
| `
|
|
355
|
-
| `
|
|
356
|
-
| `
|
|
357
|
-
| `
|
|
358
|
-
| `
|
|
359
|
-
| `
|
|
360
|
-
| `
|
|
361
|
-
| `
|
|
362
|
-
| `
|
|
363
|
-
| `
|
|
364
|
-
| `
|
|
368
|
+
|
|
369
|
+
| Member | Description |
|
|
370
|
+
| ----------------------------- | ----------------------------------------------------------------------------------- |
|
|
371
|
+
| `state` | Sync getter: `idle` | `ringing` | `early` | `active` | `holding` | `terminated` |
|
|
372
|
+
| `isAnswered` | `true` after 200 OK (outbound received / inbound sent via `answer()`) |
|
|
373
|
+
| `state$` | Emits on every state transition |
|
|
374
|
+
| `progress$` | SIP 1xx provisional responses (`SipProgressEvent`) |
|
|
375
|
+
| `early$` | Emits once when RTP is up before final answer |
|
|
376
|
+
| `answered$` | Emits once on 200 OK |
|
|
377
|
+
| `dtmf$` | Remote DTMF digits (`DtmfEvent`: `digit`, `duration`) |
|
|
378
|
+
| `sipInfo$` | Mid-call SIP INFO (`contentType` + `body` only; INFO headers not exposed) |
|
|
379
|
+
| `sipSignal$` | Low-level stack events (`statusCode` / `statusPhrase` / optional `sdp`; no headers) |
|
|
380
|
+
| `remoteSdp` | Latest negotiated remote SDP body; updates when new SDP arrives |
|
|
381
|
+
| `inviteSipHeaders` | Start-of-call inbound INVITE header snapshot; does not update |
|
|
382
|
+
| `getRemoteSdpDetails()` | Parse `remoteSdp` → `ParsedSdpDetails` (session + `a=` attributes) |
|
|
383
|
+
| `sendProgress()` | Inbound: send 183 Session Progress → `early` |
|
|
384
|
+
| `waitForEarly()` | Await `early` or `active` (Promise) |
|
|
385
|
+
| `waitForAnswer()` | Await final 200 OK (Promise) |
|
|
386
|
+
| `answer()` | Inbound: send final 200 OK → `active` |
|
|
387
|
+
| `sendDtmf(digit, duration?)` | Send DTMF tone |
|
|
388
|
+
| `sendInfo(contentType, body)` | Send SIP INFO |
|
|
389
|
+
| `hold()` / `unhold()` | SIP hold |
|
|
390
|
+
| `mute()` / `unmute()` | Suppress local outgoing audio |
|
|
391
|
+
| `hangup()` | Terminate call |
|
|
392
|
+
| `makeCall(opts)` | Outbound B-leg (`MediaChannel`); `sipUri` or `msisdn` |
|
|
393
|
+
| `bridge(other)` | Cross-connect two SIP calls; returns teardown `() => void` |
|
|
394
|
+
|
|
365
395
|
|
|
366
396
|
Prefer `dtmf$` over `sipInfo$` for DTMF. Prefer `state$` / `early$` / `answered$` over raw
|
|
367
397
|
`sipSignal$` for call lifecycle. Use `remoteSdp` / `getRemoteSdpDetails()` for negotiated
|
|
@@ -520,20 +550,22 @@ const asr = await channel.createAsr({
|
|
|
520
550
|
|
|
521
551
|
Each vendor connector accepts its native parameter names:
|
|
522
552
|
|
|
523
|
-
|
|
524
|
-
|
|
|
525
|
-
|
|
|
526
|
-
| **
|
|
527
|
-
| **
|
|
528
|
-
| **
|
|
529
|
-
| **
|
|
530
|
-
| **
|
|
553
|
+
|
|
554
|
+
| Vendor | Accepted `data` keys |
|
|
555
|
+
| -------------- | ----------------------------------------- |
|
|
556
|
+
| **Azure** | `subscription_key` or `api_key`, `region` |
|
|
557
|
+
| **Yandex** | `api_key` or `token`, `folder_id` |
|
|
558
|
+
| **ElevenLabs** | `api_key` (or `xi_api_key`), `model` |
|
|
559
|
+
| **Deepgram** | `api_key` |
|
|
560
|
+
| **Google** | `email`, `private_key`, `project_id` |
|
|
561
|
+
| **Whisper** | `url`, `rate`, `toFloat` |
|
|
562
|
+
|
|
531
563
|
|
|
532
564
|
All vendors also accept the env-style names (`AZURE_SPEECH_KEY`, `ELEVENLABS_API_KEY`, etc.) for backwards compatibility, but vendor-native names are checked first and are preferred.
|
|
533
565
|
|
|
534
566
|
### Platform ASR Key Selection
|
|
535
567
|
|
|
536
|
-
When platform credential catalogs are enabled, select ASR keys by
|
|
568
|
+
When platform credential catalogs are enabled, select ASR keys by `name`:
|
|
537
569
|
|
|
538
570
|
```ts
|
|
539
571
|
const asr = await channel.createAsr({
|
|
@@ -548,12 +580,15 @@ When both `name` (platform key) and explicit `data` are provided, `data` values
|
|
|
548
580
|
|
|
549
581
|
## TTS, Playback, And Mixer Queues
|
|
550
582
|
|
|
551
|
-
All audio playback goes through
|
|
583
|
+
All audio playback goes through `channel.audio` (`ChannelAudio`). There are no top-level
|
|
552
584
|
`channel.say()` / `channel.play()` shortcuts on `MediaChannel`.
|
|
553
585
|
|
|
554
586
|
Create a reusable TTS session with `channel.createTts(config?)` — same pattern as `createAsr`.
|
|
555
587
|
Call `tts.say` / `tts.say$` / `tts.presay` on the handle so synthesis reuses the warmed SSL / streaming WebSocket.
|
|
556
588
|
|
|
589
|
+
Custom engines from a logic package are documented in
|
|
590
|
+
[Custom ASR / TTS Providers](#custom-asr--tts-providers).
|
|
591
|
+
|
|
557
592
|
```ts
|
|
558
593
|
const tts = await channel.createTts({
|
|
559
594
|
vendor: 'elevenlabs',
|
|
@@ -609,20 +644,22 @@ terminate the Observable via **error** (`MediaError`), and are also mirrored on
|
|
|
609
644
|
You can still use `channel.audio.say(..., { tts })` if you prefer the channel API; a matching
|
|
610
645
|
pre-warmed session is also reused when vendor+config align.
|
|
611
646
|
|
|
612
|
-
|
|
613
|
-
|
|
|
614
|
-
|
|
|
615
|
-
| `
|
|
616
|
-
| `tts.say
|
|
617
|
-
| `tts.
|
|
618
|
-
| `
|
|
619
|
-
| `channel.audio.
|
|
620
|
-
| `channel.audio.
|
|
621
|
-
| `channel.audio.
|
|
622
|
-
| `channel.audio.
|
|
623
|
-
| `channel.audio.
|
|
624
|
-
| `channel.audio.
|
|
625
|
-
| `channel.audio.
|
|
647
|
+
|
|
648
|
+
| Method | Purpose |
|
|
649
|
+
| ----------------------------------------------- | ---------------------------------------------------------------- |
|
|
650
|
+
| `channel.createTts(config?)` | Pre-warm a TTS connector / streaming socket; returns `TtsHandle` |
|
|
651
|
+
| `tts.say(textOrObservable, options?)` | Synthesize via the handle's cached connection (`Promise`) |
|
|
652
|
+
| `tts.say$(textOrObservable, options?)` | Same path with per-utterance status events (`Observable`) |
|
|
653
|
+
| `tts.presay(text, options?)` | Pre-synthesize into the host TTS cache via the handle |
|
|
654
|
+
| `channel.audio.say(textOrObservable, options?)` | Synthesize text with TTS and play on a mixer queue |
|
|
655
|
+
| `channel.audio.play(source, options?)` | Play raw audio (URL, path, or platform phrase record) |
|
|
656
|
+
| `channel.audio.presay(text, options?)` | Pre-synthesize TTS into the host cache (no playback) |
|
|
657
|
+
| `channel.audio.preload(source, options?)` | Decode/warm a raw audio source (no playback) |
|
|
658
|
+
| `channel.audio.queue(index)` | Per-queue control handle (`MixerQueueControl`) |
|
|
659
|
+
| `channel.audio.remove(alias, queue?)` | Remove one queued item by alias |
|
|
660
|
+
| `channel.audio.stop(queue)` | Clear a queue **and** abort in-flight sentence TTS for it |
|
|
661
|
+
| `channel.audio.stopAll()` | Clear every queue (WS clients also get an audio interrupt) |
|
|
662
|
+
|
|
626
663
|
|
|
627
664
|
`channel.audio.say(textOrObservable, options?)` synthesizes text and plays it through the mixer.
|
|
628
665
|
|
|
@@ -655,6 +692,8 @@ await channel.audio.play('/opt/prompts/welcome.wav', {
|
|
|
655
692
|
});
|
|
656
693
|
```
|
|
657
694
|
|
|
695
|
+
|
|
696
|
+
|
|
658
697
|
### Pre-synthesis And Preload
|
|
659
698
|
|
|
660
699
|
`channel.audio.presay(text, options?)` runs TTS ahead of time and stores PCM in the host TTS
|
|
@@ -702,26 +741,28 @@ When using an `Observable<string>` input, WS clients also receive text progress
|
|
|
702
741
|
|
|
703
742
|
### Mixer Queues
|
|
704
743
|
|
|
705
|
-
The mixer has queues
|
|
744
|
+
The mixer has queues `0` **through** `4`. Use separate queues for main speech, earcons, hold
|
|
706
745
|
music, or background audio so barge-in on one queue does not cut unrelated audio.
|
|
707
746
|
|
|
708
|
-
Obtain a per-queue handle with
|
|
747
|
+
Obtain a per-queue handle with `channel.audio.queue(index)` (`MixerQueueControl`):
|
|
748
|
+
|
|
749
|
+
|
|
750
|
+
| Member | Description |
|
|
751
|
+
| --------------- | -------------------------------------------------------------------------- |
|
|
752
|
+
| `index` | Queue index **0–4** |
|
|
753
|
+
| `volume` | Linear gain **0.0–1.0** for the entire queue (get/set) |
|
|
754
|
+
| `itemStarted$` | Emits item `alias` when playback starts |
|
|
755
|
+
| `itemFinished$` | Emits `alias` when an item finishes, is removed, or is skipped by clear |
|
|
756
|
+
| `queueEmpty$` | Emits when the queue is empty after all PCM has been mixed out |
|
|
757
|
+
| `remove(alias)` | Drop one item on this queue |
|
|
758
|
+
| `clear()` | Drop all items on this queue (does **not** abort in-flight TTS generation) |
|
|
709
759
|
|
|
710
|
-
| Member | Description |
|
|
711
|
-
| --- | --- |
|
|
712
|
-
| `index` | Queue index **0–4** |
|
|
713
|
-
| `volume` | Linear gain **0.0–1.0** for the entire queue (get/set) |
|
|
714
|
-
| `itemStarted$` | Emits item **`alias`** when playback starts |
|
|
715
|
-
| `itemFinished$` | Emits **`alias`** when an item finishes, is removed, or is skipped by clear |
|
|
716
|
-
| `queueEmpty$` | Emits when the queue is empty after all PCM has been mixed out |
|
|
717
|
-
| `remove(alias)` | Drop one item on this queue |
|
|
718
|
-
| `clear()` | Drop all items on this queue (does **not** abort in-flight TTS generation) |
|
|
719
760
|
|
|
720
|
-
Top-level helpers on
|
|
761
|
+
Top-level helpers on `channel.audio`:
|
|
721
762
|
|
|
722
|
-
-
|
|
723
|
-
-
|
|
724
|
-
-
|
|
763
|
+
- `remove(alias, queue?)` — when `queue` is omitted, searches all five queues; when set, only that queue is checked.
|
|
764
|
+
- `stop(queue)` — same as `clear()` **plus** aborts in-flight sentence TTS for that queue.
|
|
765
|
+
- `stopAll()` — `stop()` on every queue; WS clients also receive an audio interrupt signal.
|
|
725
766
|
|
|
726
767
|
```ts
|
|
727
768
|
const tts = channel.audio.queue(0);
|
|
@@ -759,23 +800,290 @@ For sentence-split TTS, queue item aliases are suffixed as `alias-0`, `alias-1`,
|
|
|
759
800
|
|
|
760
801
|
Shared by `say()`, `play()`, and (where noted) `presay()`:
|
|
761
802
|
|
|
762
|
-
|
|
763
|
-
|
|
|
764
|
-
|
|
|
765
|
-
| `
|
|
766
|
-
| `
|
|
767
|
-
| `
|
|
768
|
-
| `
|
|
769
|
-
| `
|
|
770
|
-
| `
|
|
771
|
-
| `
|
|
772
|
-
| `
|
|
773
|
-
| `
|
|
803
|
+
|
|
804
|
+
| Field | Type | Applies to | Description |
|
|
805
|
+
| ------------- | -------------------------- | --------------- | ------------------------------------------------------ |
|
|
806
|
+
| `queue` | `number?` | `say`, `play` | Mixer queue **0–4** (default **0**). |
|
|
807
|
+
| `alias` | `string?` | `say`, `play` | Stable item id for `remove()` and queue events. |
|
|
808
|
+
| `loop` | `boolean?` | `say`, `play` | Restart after finish until stopped/removed. |
|
|
809
|
+
| `loopDelayMs` | `number?` | `say`, `play` | Silence between loop iterations. |
|
|
810
|
+
| `volume` | `number?` | `say`, `play` | Sets **whole queue** gain **0.0–1.0** (not per-item). |
|
|
811
|
+
| `ttsStrategy` | `TtsStrategy?` | `say`, `presay` | `sentence` | `streaming` | `full`. |
|
|
812
|
+
| `ttsVendor` | `TtsVendor?` | `say`, `presay` | Override `channel.params.ttsVendor`. |
|
|
813
|
+
| `name` | `string?` | `say`, `presay` | Platform TTS credential `name` (key catalog selector). |
|
|
814
|
+
| `ttsConfig` | `Record<string, unknown>?` | `say`, `presay` | Vendor params; `name` key is stripped before send. |
|
|
815
|
+
| `cache` | `true | CacheOptions?` | `say`, `presay` | TTS file cache; optional platform phrase persist. |
|
|
816
|
+
|
|
774
817
|
|
|
775
818
|
`play()` ignores `tts*` and `cache` for raw audio. `preload()` only accepts `cache` overrides.
|
|
776
819
|
|
|
820
|
+
## Custom ASR / TTS Providers
|
|
821
|
+
|
|
822
|
+
Trusted logic packages can ship their own ASR/TTS engines. The host loads
|
|
823
|
+
`export const mediaProviders` with a normal Node `require` **outside** the script VM
|
|
824
|
+
(same privileges as the API). Use this only for packages you trust.
|
|
825
|
+
|
|
826
|
+
Mid-script `registerAsr` / `registerTts` is **not** supported — declare vendors at module load.
|
|
827
|
+
|
|
828
|
+
### Package layout (required for connectors that use `ws` / Node APIs)
|
|
829
|
+
|
|
830
|
+
Put providers in a **sibling module**. Do **not** re-export them from the sandboxed entry
|
|
831
|
+
(`dist/index.js`). The script worker evaluates the entry under a restricted VM: there is
|
|
832
|
+
**no** global `process`, and loading `ws` (or similar) inside that graph fails even when
|
|
833
|
+
`net` / `tls` are allowlisted.
|
|
834
|
+
|
|
835
|
+
Recommended layout after `tsc`:
|
|
836
|
+
|
|
837
|
+
```text
|
|
838
|
+
dist/
|
|
839
|
+
index.js ← defineScript only (sandboxed)
|
|
840
|
+
media-providers/
|
|
841
|
+
index.js ← export const mediaProviders (host-only)
|
|
842
|
+
my-asr.js
|
|
843
|
+
my-tts.js
|
|
844
|
+
```
|
|
845
|
+
|
|
846
|
+
Host resolution order for `mediaProviders`:
|
|
847
|
+
|
|
848
|
+
1. `<entryDir>/media-providers/index.js`
|
|
849
|
+
2. `<entryDir>/media-providers.js`
|
|
850
|
+
3. `<scriptRoot>/media-providers/…` and `dist/media-providers/…` fallbacks
|
|
851
|
+
4. the script entry itself (backward compatible — avoid for `ws`-based connectors)
|
|
852
|
+
|
|
853
|
+
Sandboxed script code may import **lightweight** helpers from the providers package
|
|
854
|
+
(e.g. vendor id constants) if those files do **not** `require('ws')` / touch `process`.
|
|
855
|
+
Never import the connector classes or `mediaProviders` index from the entry.
|
|
856
|
+
|
|
857
|
+
Examples:
|
|
858
|
+
|
|
859
|
+
- Host module: [`examples/custom-media-providers.ts`](./examples/custom-media-providers.ts)
|
|
860
|
+
- Script entry: [`examples/custom-media-providers-script.ts`](./examples/custom-media-providers-script.ts)
|
|
861
|
+
|
|
862
|
+
### Contracts (implement the interfaces)
|
|
863
|
+
|
|
864
|
+
There are no abstract base classes — implement the SDK interfaces so TypeScript checks
|
|
865
|
+
the full surface and IDEs autocomplete correctly:
|
|
866
|
+
|
|
867
|
+
| Interface | Role |
|
|
868
|
+
| --- | --- |
|
|
869
|
+
| `ScriptAsrConnector` | Custom STT connector returned by an `asr` factory |
|
|
870
|
+
| `ScriptTtsConnector` | Unified batch + optional streaming TTS connector |
|
|
871
|
+
| `MediaConnectorContext` | `id`, flattened `config`, `dialogUuid`, `debug$` passed into factories |
|
|
872
|
+
| `MediaProviderShared` | Optional per-dialog object from `createShared` |
|
|
873
|
+
|
|
874
|
+
`defineMediaProviders` is an identity helper for typing. Factories must return objects that
|
|
875
|
+
satisfy those interfaces (classes with `implements` recommended).
|
|
876
|
+
|
|
877
|
+
### Export shape (host module)
|
|
878
|
+
|
|
879
|
+
```ts
|
|
880
|
+
// media-providers/index.ts — host-only (may use ws, https, process.env, …)
|
|
881
|
+
import {
|
|
882
|
+
defineMediaProviders,
|
|
883
|
+
type MediaConnectorContext,
|
|
884
|
+
type MediaProviderShared,
|
|
885
|
+
type ScriptAsrConnector,
|
|
886
|
+
type ScriptTtsConnector,
|
|
887
|
+
} from '@voctiv/agent-sdk';
|
|
888
|
+
import { Subject } from 'rxjs';
|
|
889
|
+
import { Readable } from 'stream';
|
|
890
|
+
|
|
891
|
+
class MyAsr implements ScriptAsrConnector { /* … */ }
|
|
892
|
+
class MyTts implements ScriptTtsConnector { /* … */ }
|
|
893
|
+
|
|
894
|
+
export const mediaProviders = defineMediaProviders({
|
|
895
|
+
// Optional: one object per dialog, shared by ASR + TTS factories
|
|
896
|
+
createShared: (ctx) => ({
|
|
897
|
+
dialogUuid: ctx.dialogUuid,
|
|
898
|
+
dispose() { /* release app state */ },
|
|
899
|
+
}),
|
|
900
|
+
asr: {
|
|
901
|
+
'my-asr': (ctx, shared) => new MyAsr(ctx, shared),
|
|
902
|
+
},
|
|
903
|
+
tts: {
|
|
904
|
+
'my-tts': (ctx, shared) => new MyTts(ctx, shared),
|
|
905
|
+
},
|
|
906
|
+
});
|
|
907
|
+
```
|
|
908
|
+
|
|
909
|
+
```ts
|
|
910
|
+
// index.ts — sandboxed entry (do NOT import ./media-providers or ws)
|
|
911
|
+
import { defineScript } from '@voctiv/agent-sdk';
|
|
912
|
+
|
|
913
|
+
export default defineScript(async ({ channel }) => {
|
|
914
|
+
// Credentials: pass via createAsr/createTts `data` (or platform/channel params).
|
|
915
|
+
// Do not rely on process.env here — `process` is not defined in the script VM.
|
|
916
|
+
const asr = await channel.createAsr({
|
|
917
|
+
vendor: 'my-asr',
|
|
918
|
+
language: 'ru-RU',
|
|
919
|
+
data: { api_key: String(channel.params.api_key ?? '') },
|
|
920
|
+
});
|
|
921
|
+
const tts = await channel.createTts({
|
|
922
|
+
vendor: 'my-tts',
|
|
923
|
+
data: {
|
|
924
|
+
api_key: String(channel.params.api_key ?? ''),
|
|
925
|
+
voice_id: '…',
|
|
926
|
+
output_format: 'pcm_16000', // preferred for telephony
|
|
927
|
+
},
|
|
928
|
+
});
|
|
929
|
+
|
|
930
|
+
await tts.say('Hello from a custom connector.', {
|
|
931
|
+
alias: 'greet',
|
|
932
|
+
ttsStrategy: 'streaming', // used when supportsStreaming() === true
|
|
933
|
+
});
|
|
934
|
+
|
|
935
|
+
tts.say$('One. Two.', { alias: 'reply' }).subscribe((_ev) => {
|
|
936
|
+
// queued | speaking | done | cancelled — same events as builtin TTS
|
|
937
|
+
});
|
|
938
|
+
|
|
939
|
+
await channel.audio.say('Again', { tts }); // warm handle reuse
|
|
940
|
+
|
|
941
|
+
asr.destroy();
|
|
942
|
+
tts.destroy();
|
|
943
|
+
});
|
|
944
|
+
```
|
|
945
|
+
|
|
946
|
+
Host-side factories **may** read `process.env` (e.g. `ELEVENLABS_API_KEY`) because they run
|
|
947
|
+
outside the VM. Prefer also accepting the same keys via `ctx.config` from `data`.
|
|
948
|
+
|
|
949
|
+
### Implement ASR (`ScriptAsrConnector`)
|
|
950
|
+
|
|
951
|
+
The host pushes **PCM S16LE mono 16 kHz** frames into `send()`. Emit partials on
|
|
952
|
+
`transcription$` and finals on both `transcription$` (`isFinal: true`) and `result$`.
|
|
953
|
+
|
|
954
|
+
| Member | Role |
|
|
955
|
+
| --- | --- |
|
|
956
|
+
| `transcription$` | `{ text, isFinal }` partials and finals |
|
|
957
|
+
| `result$` | Final utterance strings (drives `AsrHandle.result$`) |
|
|
958
|
+
| `error$` | Vendor / transport failures |
|
|
959
|
+
| `send(audio)` | Accept inbound PCM (`ArrayBuffer` / `Buffer`) |
|
|
960
|
+
| `speech(active)` | Optional VAD gate from the host (`true` = utterance open) |
|
|
961
|
+
| `finalize()` | End-of-utterance nudge (flush / commit) |
|
|
962
|
+
| `isOpen()` | Whether the vendor socket is ready |
|
|
963
|
+
| `close()` | Tear down; complete Subjects |
|
|
964
|
+
|
|
965
|
+
```ts
|
|
966
|
+
class MyAsr implements ScriptAsrConnector {
|
|
967
|
+
readonly transcription$ = new Subject<{ text: string; isFinal: boolean }>();
|
|
968
|
+
readonly result$ = new Subject<string>();
|
|
969
|
+
readonly error$ = new Subject<{ message: string; code?: number }>();
|
|
970
|
+
|
|
971
|
+
constructor(
|
|
972
|
+
private readonly ctx: MediaConnectorContext,
|
|
973
|
+
private readonly shared: MediaProviderShared | undefined,
|
|
974
|
+
) {
|
|
975
|
+
// ctx.config — flattened createAsr data + channel asrConfig
|
|
976
|
+
// ctx.dialogUuid, ctx.role, ctx.debug$
|
|
977
|
+
}
|
|
978
|
+
|
|
979
|
+
isOpen() { return true; }
|
|
980
|
+
send(audio: ArrayBufferLike) { /* forward PCM to vendor */ }
|
|
981
|
+
speech(_active: boolean) {}
|
|
982
|
+
finalize() {
|
|
983
|
+
const text = '…';
|
|
984
|
+
this.transcription$.next({ text, isFinal: true });
|
|
985
|
+
this.result$.next(text);
|
|
986
|
+
}
|
|
987
|
+
close() {
|
|
988
|
+
this.transcription$.complete();
|
|
989
|
+
this.result$.complete();
|
|
990
|
+
this.error$.complete();
|
|
991
|
+
}
|
|
992
|
+
}
|
|
993
|
+
```
|
|
994
|
+
|
|
995
|
+
Factory `throw` during create → host degraded ASR handle (same as builtin). Prefer connecting
|
|
996
|
+
lazily in `send()`; the host does not call a separate `connect()` on custom connectors.
|
|
997
|
+
|
|
998
|
+
### Implement TTS (`ScriptTtsConnector`)
|
|
999
|
+
|
|
1000
|
+
**One class per vendor** covers batch HTTP and optional streaming WebSocket. Do **not**
|
|
1001
|
+
register a separate streaming map.
|
|
1002
|
+
|
|
1003
|
+
| Member | Role |
|
|
1004
|
+
| --- | --- |
|
|
1005
|
+
| `supportsStreaming()` | `true` → host may use WS path for `ttsStrategy: 'streaming'` |
|
|
1006
|
+
| `textToSpeechStream(text, ctx?)` | Batch/HTTP synthesis → `Readable` of audio bytes |
|
|
1007
|
+
| `audio$` / `done$` | Streaming audio chunks and end-of-generation |
|
|
1008
|
+
| `open` / `startGeneration` / `sendText` / `flush` | Streaming lifecycle |
|
|
1009
|
+
| `sendSeparatorFlush?()` | Optional sentence separator flush |
|
|
1010
|
+
| `close()` | Soft-close sockets for **reuse** (do not complete Subjects if `open()` may run again) |
|
|
1011
|
+
|
|
1012
|
+
```ts
|
|
1013
|
+
class MyTts implements ScriptTtsConnector {
|
|
1014
|
+
readonly audio$ = new Subject<Buffer>();
|
|
1015
|
+
readonly done$ = new Subject<void>();
|
|
1016
|
+
|
|
1017
|
+
supportsStreaming() {
|
|
1018
|
+
return true; // or false for HTTP-only
|
|
1019
|
+
}
|
|
1020
|
+
|
|
1021
|
+
async textToSpeechStream(
|
|
1022
|
+
rawtext: string,
|
|
1023
|
+
_ctx?: { previousText?: string; nextText?: string },
|
|
1024
|
+
) {
|
|
1025
|
+
// Return PCM (preferred) or MP3/OGG. Hint format via createTts data:
|
|
1026
|
+
// output_format: 'pcm_16000' | 'mp3_…' or audioFormat: 'pcm' | 'mp3'
|
|
1027
|
+
return Readable.from([/* bytes */]);
|
|
1028
|
+
}
|
|
1029
|
+
|
|
1030
|
+
async open() { /* open long-lived WS */ }
|
|
1031
|
+
async startGeneration() { /* BOS / new utterance */ }
|
|
1032
|
+
sendText(chunk: string) { /* stream tokens */ }
|
|
1033
|
+
flush() { /* EOS; later emit done$ */ }
|
|
1034
|
+
close() { /* close WS; keep Subjects alive for fingerprint reuse */ }
|
|
1035
|
+
}
|
|
1036
|
+
```
|
|
1037
|
+
|
|
1038
|
+
- `supportsStreaming() === false` → streaming strategy falls back to sentence/full HTTP path.
|
|
1039
|
+
- Default audio assumption for custom vendors is **PCM** unless `output_format` / `audioFormat`
|
|
1040
|
+
indicates a compressed format (`mp3`, `ogg`, …).
|
|
1041
|
+
|
|
1042
|
+
### Using custom vendors in the script
|
|
1043
|
+
|
|
1044
|
+
```ts
|
|
1045
|
+
const asr = await channel.createAsr({ vendor: 'my-asr', data: { /* … */ } });
|
|
1046
|
+
const tts = await channel.createTts({ vendor: 'my-tts', data: { /* … */ } });
|
|
1047
|
+
|
|
1048
|
+
await tts.say('Hi', { alias: 'greet' });
|
|
1049
|
+
await tts.presay('Warm cache');
|
|
1050
|
+
tts.say$('Next', { alias: 'reply', ttsStrategy: 'streaming' }).subscribe(/* … */);
|
|
1051
|
+
|
|
1052
|
+
await channel.audio.say('Reuse', { tts }); // same warm session
|
|
1053
|
+
```
|
|
1054
|
+
|
|
1055
|
+
Vendor keys must be **multi-character** names. Builtin single-letter codes
|
|
1056
|
+
(`A`, `D`, `E`, `ES`, `G`, `V`, `W`, `W2`, …) **cannot** be overridden.
|
|
1057
|
+
|
|
1058
|
+
### Warm reuse vs `createShared` vs PCM cache
|
|
1059
|
+
|
|
1060
|
+
| Layer | What it caches | Scope |
|
|
1061
|
+
| --- | --- | --- |
|
|
1062
|
+
| Factory + session registries | One connector instance per `dialog + vendor + config fingerprint` (TCP/WS) | Host automatic |
|
|
1063
|
+
| `createShared` | Your app/vendor state shared by ASR↔TTS factories | One object per dialog |
|
|
1064
|
+
| `PlayOptions.cache` / `presay` | Synthesized PCM files | Host TTS file cache |
|
|
1065
|
+
|
|
1066
|
+
Call `createAsr` / `createTts` early to warm sockets. Repeated `say` with the same handle (or
|
|
1067
|
+
matching fingerprint) must **not** open a new TCP/WS per utterance.
|
|
1068
|
+
|
|
1069
|
+
The host also loads the same `mediaProviders` module into the **pipeline worker** process
|
|
1070
|
+
(separate `require`) when `PIPELINE_WORKERS` is enabled — keep the module side-effect free
|
|
1071
|
+
aside from exporting factories.
|
|
1072
|
+
|
|
1073
|
+
### Security
|
|
1074
|
+
|
|
1075
|
+
- Host `require` of `mediaProviders` runs with API privileges — **trusted packages only**.
|
|
1076
|
+
- Keep connector/`ws` code out of the sandboxed entry so the VM never evaluates it.
|
|
1077
|
+
- The script VM does **not** expose `process` (no `process.env` in `defineScript`).
|
|
1078
|
+
- Entry / provider paths are restricted to the script root (path traversal denied).
|
|
1079
|
+
- Optional host allowlists may further restrict which packages may export providers.
|
|
1080
|
+
|
|
1081
|
+
|
|
1082
|
+
|
|
777
1083
|
## TTS Credentials And Vendor Parameters
|
|
778
1084
|
|
|
1085
|
+
|
|
1086
|
+
|
|
779
1087
|
### Direct TTS Vendor Parameters
|
|
780
1088
|
|
|
781
1089
|
Pass vendor-native credentials and settings directly through `PlayOptions.ttsConfig`. These values are forwarded to the TTS connector as-is and override any defaults or platform-resolved credentials.
|
|
@@ -795,10 +1103,12 @@ await channel.audio.say('Hello!', {
|
|
|
795
1103
|
|
|
796
1104
|
Each TTS vendor connector accepts its native parameter names:
|
|
797
1105
|
|
|
798
|
-
|
|
799
|
-
|
|
|
1106
|
+
|
|
1107
|
+
| Vendor | Accepted `ttsConfig` keys |
|
|
1108
|
+
| -------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
800
1109
|
| **ElevenLabs** | `api_key` (or `xi_api_key`), `voice_id`, `model_id` (or `model`), `base_url`, `output_format`, `language_code`, `voice_settings_stability`, `voice_settings_similarity_boost`, `voice_settings_style`, `voice_settings_speed` |
|
|
801
|
-
| **Voctiv**
|
|
1110
|
+
| **Voctiv** | `url`, `voice_id`, `language`, `emotion`, `speaking_rate`, `chunk_schedule` |
|
|
1111
|
+
|
|
802
1112
|
|
|
803
1113
|
All vendors also accept the env-style names (`ELEVENLABS_API_KEY`, `ELEVENLABS_VOICE_ID`, etc.) for backwards compatibility, but vendor-native names are checked first and are preferred.
|
|
804
1114
|
|
|
@@ -821,8 +1131,8 @@ When both `name` (platform key) and explicit `ttsConfig` values are provided, `t
|
|
|
821
1131
|
|
|
822
1132
|
`cache` enables TTS result caching for `say()` and `presay()`:
|
|
823
1133
|
|
|
824
|
-
-
|
|
825
|
-
-
|
|
1134
|
+
- `cache: true` — read/write host TTS file cache.
|
|
1135
|
+
- `cache: { phraseName, flag?, language? }` — TTS cache **plus** persist as a platform phrase so `platform.getRecords()` can retrieve the audio later.
|
|
826
1136
|
|
|
827
1137
|
```ts
|
|
828
1138
|
// Cache only (no platform persist):
|
|
@@ -872,6 +1182,8 @@ try {
|
|
|
872
1182
|
}
|
|
873
1183
|
```
|
|
874
1184
|
|
|
1185
|
+
|
|
1186
|
+
|
|
875
1187
|
### ASR Errors — `error$` Observable
|
|
876
1188
|
|
|
877
1189
|
Runtime ASR errors (gRPC disconnect, auth failure, quota exceeded) are emitted on `AsrHandle.error$`:
|
|
@@ -911,20 +1223,22 @@ channel.events.error$.subscribe((err) => {
|
|
|
911
1223
|
|
|
912
1224
|
`MediaError` fields:
|
|
913
1225
|
|
|
914
|
-
|
|
915
|
-
|
|
|
916
|
-
|
|
|
917
|
-
| `
|
|
918
|
-
| `
|
|
919
|
-
| `
|
|
920
|
-
| `
|
|
921
|
-
| `
|
|
922
|
-
| `
|
|
923
|
-
| `
|
|
924
|
-
| `
|
|
925
|
-
| `
|
|
926
|
-
| `
|
|
927
|
-
| `
|
|
1226
|
+
|
|
1227
|
+
| Field | Type | Description |
|
|
1228
|
+
| ------------- | ---------------------------------------------------------------------- | ---------------------------------------------------------------------------------- |
|
|
1229
|
+
| `source` | `'asr' | 'tts' | 'sip' | 'channel' | 'llm'` | Which subsystem produced the error. |
|
|
1230
|
+
| `phase` | `'create' | 'start' | 'stream' | 'playback' | 'finalize' | 'destroy'?` | Lifecycle phase where the error happened. |
|
|
1231
|
+
| `operation` | `string?` | Public SDK operation, e.g. `createAsr`, `createTts`, `audio.say`, or `audio.play`. |
|
|
1232
|
+
| `recoverable` | `boolean?` | Whether the runtime can keep the session alive after this error. |
|
|
1233
|
+
| `handleId` | `string?` | ASR handle id when the error belongs to a recognizer instance. |
|
|
1234
|
+
| `queue` | `number?` | Mixer queue index when the error belongs to an audio operation. |
|
|
1235
|
+
| `alias` | `string?` | Queue item alias when the error belongs to playback. |
|
|
1236
|
+
| `message` | `string` | Human-readable description. |
|
|
1237
|
+
| `code` | `number | string?` | HTTP status, gRPC status, provider code, or WebSocket close code. |
|
|
1238
|
+
| `vendor` | `string?` | Vendor identifier, e.g. `"yandex"`, `"elevenlabs"`, `"azure"`. |
|
|
1239
|
+
| `details` | `unknown?` | Arbitrary provider-specific payload. |
|
|
1240
|
+
| `cause` | `unknown?` | Original underlying error when available. |
|
|
1241
|
+
|
|
928
1242
|
|
|
929
1243
|
Subscribing to `error$` is optional. Old scripts that do not subscribe are not affected — the observables simply go unobserved.
|
|
930
1244
|
|
|
@@ -932,14 +1246,16 @@ Subscribing to `error$` is optional. Old scripts that do not subscribe are not a
|
|
|
932
1246
|
|
|
933
1247
|
`channel.events` exposes session-level observables that are **not** tied to a single ASR handle:
|
|
934
1248
|
|
|
935
|
-
|
|
936
|
-
|
|
|
937
|
-
|
|
|
938
|
-
| `
|
|
939
|
-
| `
|
|
940
|
-
| `
|
|
941
|
-
| `
|
|
942
|
-
| `
|
|
1249
|
+
|
|
1250
|
+
| Observable | Emits when |
|
|
1251
|
+
| -------------- | ----------------------------------------------------------------------- |
|
|
1252
|
+
| `speechStart$` | User started speaking (VAD, socket event, or synthetic text input). |
|
|
1253
|
+
| `speechEnd$` | User stopped speaking (VAD end, ASR final, or synthetic text input). |
|
|
1254
|
+
| `interrupt$` | Barge-in: user speech interrupted bot audio (may be inert without VAD). |
|
|
1255
|
+
| `terminated$` | Session ending — hangup, WS disconnect, or `channel.destroy()`. |
|
|
1256
|
+
| `message$` | Structured WS data messages (`DataMessage`: `{ event, payload }`). |
|
|
1257
|
+
| `error$` | Unified media/runtime errors (see [Error Handling](#error-handling)). |
|
|
1258
|
+
|
|
943
1259
|
|
|
944
1260
|
```ts
|
|
945
1261
|
channel.events.speechStart$.subscribe(() => {
|
|
@@ -964,12 +1280,14 @@ create ASR or want one subscription for the whole channel).
|
|
|
964
1280
|
|
|
965
1281
|
`channel.llm` talks to the Omni LLM backend.
|
|
966
1282
|
|
|
967
|
-
|
|
968
|
-
|
|
|
969
|
-
|
|
|
970
|
-
| `
|
|
971
|
-
| `
|
|
972
|
-
| `
|
|
1283
|
+
|
|
1284
|
+
| Method | Returns | Description |
|
|
1285
|
+
| -------------------------------- | ------------------------------ | ------------------------------------------------ |
|
|
1286
|
+
| `ask(message, options?)` | `Promise<string>` | Single-shot completion (consumes SSE stream). |
|
|
1287
|
+
| `stream(message, options?)` | `Observable<LlmStreamChunk>` | Token/chunk stream; use `chunk.content` for TTS. |
|
|
1288
|
+
| `extract(options?)` | `Promise<Record<string, any>>` | Structured extraction via Omni extract API. |
|
|
1289
|
+
| `makePersistentStream(options?)` | `PersistentLlmStreamHandle` | Long-lived stream for multi-turn chat. |
|
|
1290
|
+
|
|
973
1291
|
|
|
974
1292
|
Common `LlmOptions`: `dialogUuid`, `agentUuid`, `role`, `hidden`, `name` (LLM speaker label — **not**
|
|
975
1293
|
the TTS credential `name`), `payload`, `debug`, `agentAliasFilter`, `currentAgentAlias`.
|
|
@@ -1023,9 +1341,11 @@ chat.send('And my last payment date?');
|
|
|
1023
1341
|
chat.disconnect();
|
|
1024
1342
|
```
|
|
1025
1343
|
|
|
1344
|
+
|
|
1345
|
+
|
|
1026
1346
|
## Script Return Value
|
|
1027
1347
|
|
|
1028
|
-
Scripts may return `void` or a
|
|
1348
|
+
Scripts may return `void` or a `ScriptResult`:
|
|
1029
1349
|
|
|
1030
1350
|
```ts
|
|
1031
1351
|
return {
|
|
@@ -1033,33 +1353,35 @@ return {
|
|
|
1033
1353
|
};
|
|
1034
1354
|
```
|
|
1035
1355
|
|
|
1036
|
-
-
|
|
1037
|
-
-
|
|
1356
|
+
- `output` — stored in dialog stats / host persistence. This is **not** the LE `dialog.result` lifecycle column — that is `platform.dialog.result` (see [Dialog State](#dialog-state)).
|
|
1357
|
+
- `error` — optional; usually auto-populated on crash, but scripts may set it explicitly.
|
|
1038
1358
|
|
|
1039
1359
|
**Do not** return `env` from the script. Persist state via `context.env$`; the runtime snapshots it
|
|
1040
|
-
after completion into
|
|
1360
|
+
after completion into `PersistedScriptResult.env`.
|
|
1041
1361
|
|
|
1042
1362
|
## Script Lifecycle (`getScriptPhase`)
|
|
1043
1363
|
|
|
1044
|
-
The host always runs your single
|
|
1045
|
-
name (unlike logic-executor Python `run_unit(entry_point=...)`). Use
|
|
1364
|
+
The host always runs your single `defineScript` **export** — there is no separate runtime entry per
|
|
1365
|
+
name (unlike logic-executor Python `run_unit(entry_point=...)`). Use `getScriptPhase(context)`
|
|
1046
1366
|
to tell *why* the script is running now: live call, pre-call queue, post-call continuation,
|
|
1047
1367
|
messaging, etc.
|
|
1048
1368
|
|
|
1049
|
-
|
|
1369
|
+
`context.headless === true` **only means “no live SIP/media”.** It does **not** tell you whether
|
|
1050
1370
|
the run is before or after a call. For that, use `getScriptPhase`.
|
|
1051
1371
|
|
|
1052
1372
|
The return value is the `ScriptPhase` union — one of the phases in the table below.
|
|
1053
1373
|
|
|
1054
|
-
|
|
1055
|
-
|
|
1056
|
-
|
|
|
1057
|
-
| `
|
|
1058
|
-
| `
|
|
1059
|
-
| `
|
|
1060
|
-
| `
|
|
1061
|
-
| `
|
|
1062
|
-
| `
|
|
1374
|
+
|
|
1375
|
+
| Phase | `context.headless` | When | Typical `context.entryPoint` |
|
|
1376
|
+
| -------------------- | ------------------ | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- |
|
|
1377
|
+
| `before_call` | `true` | Dialog queue / bulk outbound **before** the first platform call (often schedules `platform.call`) | empty, `main`, `default` |
|
|
1378
|
+
| `online` | `false` | Live SIP session (inbound, outbound, **and automatic recall redials**) | any (ignored for phase) |
|
|
1379
|
+
| `after_call_success` | `true` | Headless run **after** a successful call | `on_success_call`, `after_call_success`, `on_done_call` |
|
|
1380
|
+
| `after_call_failed` | `true` | Headless run **after** failed attempts (when `onFailedCall` was configured) | `on_failed_call`, `after_call_failed` |
|
|
1381
|
+
| `messaging` | `true` | Inbound message triggered the run | `on_message_api_received`, or `context.inboundMessage` set |
|
|
1382
|
+
| `recall` | `true` | Headless recall leg (legacy `entry_point`) | `on_recall`, `recall` |
|
|
1383
|
+
| `headless_other` | `true` | Any other headless run with a custom `entry_point` | your custom name |
|
|
1384
|
+
|
|
1063
1385
|
|
|
1064
1386
|
**Automatic recall** (`recallCount` + `recallDelay`) creates new **online** SIP legs with an
|
|
1065
1387
|
incremented `context.attempt`. Branch with `(context.attempt ?? 0) > 0`, **not** `phase === 'recall'`.
|
|
@@ -1113,6 +1435,8 @@ export default defineScript(async ({ channel, context, logger, platform }) => {
|
|
|
1113
1435
|
});
|
|
1114
1436
|
```
|
|
1115
1437
|
|
|
1438
|
+
|
|
1439
|
+
|
|
1116
1440
|
## Platform API
|
|
1117
1441
|
|
|
1118
1442
|
`platform` exposes platform operations.
|
|
@@ -1138,28 +1462,32 @@ Platform APIs (`platform.nlu`, `platform.call`, dialog writes, messaging, phrase
|
|
|
1138
1462
|
|
|
1139
1463
|
#### What `platform.dialog.result` is
|
|
1140
1464
|
|
|
1141
|
-
`platform.dialog.result` maps to the PostgreSQL column
|
|
1465
|
+
`platform.dialog.result` maps to the PostgreSQL column `dialog.result`: the **lifecycle status** of the dialog entity in the CMS / offline queue (in queue, in progress, closed). It is **not** the outcome of a SIP leg.
|
|
1466
|
+
|
|
1467
|
+
|
|
1468
|
+
| Value | Meaning |
|
|
1469
|
+
| --------- | -------------------------------------------------------------------------------- |
|
|
1470
|
+
| `null` | Often after-call continuation: dialog re-enters queue-api, then becomes `queued` |
|
|
1471
|
+
| `queued` | In the offline queue, not yet claimed |
|
|
1472
|
+
| `pending` | In progress (live SIP/WS or claimed queue row) |
|
|
1473
|
+
| `done` | Dialog closed successfully (terminal for the pipeline) |
|
|
1474
|
+
| `error` | Dialog closed with a logic/runtime error |
|
|
1142
1475
|
|
|
1143
|
-
| Value | Meaning |
|
|
1144
|
-
| --- | --- |
|
|
1145
|
-
| `null` | Often after-call continuation: dialog re-enters queue-api, then becomes `queued` |
|
|
1146
|
-
| `queued` | In the offline queue, not yet claimed |
|
|
1147
|
-
| `pending` | In progress (live SIP/WS or claimed queue row) |
|
|
1148
|
-
| `done` | Dialog closed successfully (terminal for the pipeline) |
|
|
1149
|
-
| `error` | Dialog closed with a logic/runtime error |
|
|
1150
1476
|
|
|
1151
1477
|
The **host** also moves these statuses (live session → `pending`; shutdown without continuation → often `done` / `error`; automatic recall → `pending`; after-call chain → `null` + `entry_point` in params). Scripts set `platform.dialog.result` when they want to **explicitly** fix the LE dialog status (commonly `'done'` in a headless after-call handler). That does **not** replace `channel.sip.hangup()`.
|
|
1152
1478
|
|
|
1153
1479
|
#### Do not confuse
|
|
1154
1480
|
|
|
1155
|
-
|
|
1156
|
-
|
|
|
1157
|
-
|
|
|
1158
|
-
| `platform.dialog.
|
|
1159
|
-
| `
|
|
1160
|
-
| `
|
|
1161
|
-
| `
|
|
1162
|
-
| `
|
|
1481
|
+
|
|
1482
|
+
| API | Layer | Does | Does not |
|
|
1483
|
+
| -------------------------------------- | --------------------- | ---------------------------------------------------------------- | ----------------------------------------------------------------- |
|
|
1484
|
+
| `platform.dialog.result` | LE `dialog` row | Lifecycle status (`done`, `pending`, …) | Hang up SIP; equal `call.result`; equal `ScriptResult.output` |
|
|
1485
|
+
| `platform.dialog.entryPoint` | LE `dialog.params` | Persist routing hint for later headless/queue runs | Change SIP state; select another script export |
|
|
1486
|
+
| `context.entryPoint` | `ScriptDialogContext` | **Snapshot** of `entry_point` at script start (`getScriptPhase`) | Persist if you assign it — write via `platform.dialog.entryPoint` |
|
|
1487
|
+
| `return { output }` / `ScriptResult` | Script return | dialog_stats / host dump | LE `dialog.result` column |
|
|
1488
|
+
| `channel.sip.hangup()` / `terminated$` | Media leg | End or observe SIP/WS media | Set `platform.dialog.result` by itself |
|
|
1489
|
+
| `call.result` (LE call row) | Per-call | SIP terminal code/phrase for CMS logs | Same as `dialog.result` |
|
|
1490
|
+
|
|
1163
1491
|
|
|
1164
1492
|
There is no `context.result` field — read/write dialog lifecycle only through `platform.dialog.result`.
|
|
1165
1493
|
|
|
@@ -1204,7 +1532,7 @@ await platform.call('+12025551234', {
|
|
|
1204
1532
|
});
|
|
1205
1533
|
```
|
|
1206
1534
|
|
|
1207
|
-
When the scheduled call connects, the host runs your
|
|
1535
|
+
When the scheduled call connects, the host runs your `defineScript` **export** again. Branch inside that handler if needed (there is no separate runtime entry per name, unlike logic-executor Python `run_unit(entry_point=...)`).
|
|
1208
1536
|
|
|
1209
1537
|
Use `dateEnd` to define the latest time when the call is still useful. If the platform cannot place the call before that deadline, it can skip the attempt.
|
|
1210
1538
|
|
|
@@ -1234,21 +1562,23 @@ After a failed **outbound** call the platform must choose **one** failure-handli
|
|
|
1234
1562
|
`recallCount` + `recallDelay` and `onFailedCall` answer the same question in different ways, so
|
|
1235
1563
|
they are **mutually exclusive** on `platform.call()` (logic-executor `nn.call` parity).
|
|
1236
1564
|
|
|
1237
|
-
|
|
1238
|
-
|
|
1239
|
-
|
|
|
1240
|
-
| **
|
|
1565
|
+
|
|
1566
|
+
| Strategy | `platform.call()` options | What happens on failure | Next script run |
|
|
1567
|
+
| --------------------------- | --------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | --------------------------------------------------------------------- |
|
|
1568
|
+
| **Automatic recall** | `recallCount` + `recallDelay` (both required on the `call` row) | Host schedules another outbound `call` after `recallDelay`; bumps `dialog.params.attempt` | **Online** SIP leg — same `defineScript`, branch on `context.attempt` |
|
|
1569
|
+
| **After-call continuation** | `onFailedCall` (and optionally `onSuccessCall`) | Host sets `dialog.params.entry_point` to the handler name and re-queues the dialog | **Headless** run — `getScriptPhase(context)` → `after_call_failed` |
|
|
1570
|
+
|
|
1241
1571
|
|
|
1242
1572
|
**Why not both?** Recall is fully platform-driven (dialer redials without running your script between
|
|
1243
1573
|
attempts). `onFailedCall` is script-driven (your handler decides logging, CRM, manual retry, etc.).
|
|
1244
|
-
If both were written to `call.params`, shutdown would be ambiguous. This host therefore **keeps
|
|
1245
|
-
`onFailedCall` and drops recall** at schedule time so the outbound still starts and the after-call
|
|
1574
|
+
If both were written to `call.params`, shutdown would be ambiguous. This host therefore **keeps**
|
|
1575
|
+
`onFailedCall` **and drops recall** at schedule time so the outbound still starts and the after-call
|
|
1246
1576
|
branch runs on failure. (If both somehow land on an existing `call.params` row, recall still wins
|
|
1247
1577
|
at shutdown — avoid writing both.)
|
|
1248
1578
|
|
|
1249
1579
|
**Do not** pass `onFailedCall` together with `recallCount` and `recallDelay` in the same
|
|
1250
1580
|
`platform.call()` invocation. If both are present (script options, dialog params, or CMS
|
|
1251
|
-
defaults), the host **keeps `onFailedCall` and drops recall** so the call still schedules —
|
|
1581
|
+
defaults), the host **keeps** `onFailedCall` **and drops recall** so the call still schedules —
|
|
1252
1582
|
after-call continuation wins over automatic redial. Prefer configuring only one strategy
|
|
1253
1583
|
explicitly.
|
|
1254
1584
|
|
|
@@ -1284,7 +1614,7 @@ export default defineScript(async ({ channel, context, logger }) => {
|
|
|
1284
1614
|
See [examples/recall-routing-by-attempt.ts](./examples/recall-routing-by-attempt.ts).
|
|
1285
1615
|
|
|
1286
1616
|
When you omit recall options, the host fills them from `context.recallCount` / `context.recallDelay`
|
|
1287
|
-
(effective values for the current dialog) **only when `onFailedCall` is not configured** for that
|
|
1617
|
+
(effective values for the current dialog) **only when** `onFailedCall` **is not configured** for that
|
|
1288
1618
|
scheduled call. Those values fall back to CMS agent contact-rules
|
|
1289
1619
|
(`context.agent?.recallCount` / `context.agent?.recallDelay`, legacy `nn.get_recall_count()` /
|
|
1290
1620
|
`nn.get_recall_delay()`). See [Failed outbound: automatic recall vs after-call continuation](#failed-outbound-automatic-recall-vs-after-call-continuation).
|
|
@@ -1298,6 +1628,8 @@ Other scheduling options:
|
|
|
1298
1628
|
- `entryPoint` (optional): stored as `entry_point` in call params for LE DB compatibility. The host still runs the same `defineScript` export; use `context.entryPoint` only if **you** branch on it inside the handler.
|
|
1299
1629
|
- `protoAdditional`: extra protocol-level parameters, such as SIP headers expected by your telephony setup.
|
|
1300
1630
|
|
|
1631
|
+
|
|
1632
|
+
|
|
1301
1633
|
### Messaging
|
|
1302
1634
|
|
|
1303
1635
|
```ts
|
|
@@ -1315,9 +1647,9 @@ await platform.messaging.send({
|
|
|
1315
1647
|
Offline, or headless, sessions run a script without a live SIP call, WebSocket audio stream, RTP pipeline, ASR, or TTS playback. They are used for platform-driven background logic, queued dialog processing, and messaging events.
|
|
1316
1648
|
|
|
1317
1649
|
The script entry point is still the same `defineScript()` handler. Detect offline mode with
|
|
1318
|
-
`context.headless`, then use
|
|
1650
|
+
`context.headless`, then use `getScriptPhase(context)` to distinguish pre-call queue runs
|
|
1319
1651
|
(`before_call`) from post-call continuations (`after_call_success` / `after_call_failed`). See
|
|
1320
|
-
[Script Lifecycle (`getScriptPhase`)](#script-lifecycle-getscriptphase).
|
|
1652
|
+
[Script Lifecycle (](#script-lifecycle-getscriptphase)`getScriptPhase`[)](#script-lifecycle-getscriptphase).
|
|
1321
1653
|
|
|
1322
1654
|
```ts
|
|
1323
1655
|
import { defineScript, getScriptPhase } from '@voctiv/agent-sdk';
|
|
@@ -1460,6 +1792,8 @@ export default defineScript(async ({ channel, context, platform }) => {
|
|
|
1460
1792
|
});
|
|
1461
1793
|
```
|
|
1462
1794
|
|
|
1795
|
+
|
|
1796
|
+
|
|
1463
1797
|
## Dialog Context And Persisted Env
|
|
1464
1798
|
|
|
1465
1799
|
`context` includes identity, telephony fields, params, routing metadata, and runtime helpers.
|
|
@@ -1479,14 +1813,18 @@ const counter = await context.agent?.env?.<number>('visitCount');
|
|
|
1479
1813
|
await context.agent?.env?.('visitCount', 42, { expire: 30 });
|
|
1480
1814
|
```
|
|
1481
1815
|
|
|
1816
|
+
|
|
1817
|
+
|
|
1482
1818
|
#### Recall settings (agent defaults vs effective)
|
|
1483
1819
|
|
|
1484
1820
|
Recall behavior uses **two layers** on `context`:
|
|
1485
1821
|
|
|
1486
|
-
|
|
1487
|
-
|
|
1488
|
-
|
|
|
1489
|
-
|
|
|
1822
|
+
|
|
1823
|
+
| Layer | Fields | Source | Use when |
|
|
1824
|
+
| ---------------------- | ---------------------------------------------------------- | ----------------------------------------------------------------- | --------------------------------------------------------------------------------------- |
|
|
1825
|
+
| Agent defaults | `context.agent?.recallCount`, `context.agent?.recallDelay` | CMS contact-rules (`agent.recall_count`, `agent.delay` → seconds) | Compare with CMS settings; legacy `nn.get_recall_count()` / `get_recall_delay()` parity |
|
|
1826
|
+
| Effective for this run | `context.recallCount`, `context.recallDelay` | `dialog.params` / `call.params`, then agent defaults | Read in script; pass explicitly to `platform.call()` when using **automatic recall** |
|
|
1827
|
+
|
|
1490
1828
|
|
|
1491
1829
|
Precedence for effective values: **dialog/call params** (`recall_count`, `recall_delay`) **>** agent CMS defaults.
|
|
1492
1830
|
|
|
@@ -1592,7 +1930,7 @@ Headless channels are for offline, queue, or messaging sessions:
|
|
|
1592
1930
|
|
|
1593
1931
|
- Audio methods are no-ops that log warnings.
|
|
1594
1932
|
- SIP methods are no-ops, except `makeCall()` and `bridge()`, which throw: there is no real leg to
|
|
1595
|
-
|
|
1933
|
+
create, and a silent no-op would hide the mistake.
|
|
1596
1934
|
- `createAsr()` returns an inert handle whose observables complete immediately.
|
|
1597
1935
|
- `createTts()` returns an inert handle (no vendor connection is opened).
|
|
1598
1936
|
- LLM, NLU, messaging, platform calls, dialog state, and `env$` still work.
|
|
@@ -1600,7 +1938,7 @@ Headless channels are for offline, queue, or messaging sessions:
|
|
|
1600
1938
|
Use `context.headless` plus `getScriptPhase(context)` when a script must behave differently without a
|
|
1601
1939
|
real media channel or across pre-call / post-call headless runs. See
|
|
1602
1940
|
[Offline / Headless Logic](#offline--headless-logic) and
|
|
1603
|
-
[Script Lifecycle (`getScriptPhase`)](#script-lifecycle-getscriptphase).
|
|
1941
|
+
[Script Lifecycle (](#script-lifecycle-getscriptphase)`getScriptPhase`[)](#script-lifecycle-getscriptphase).
|
|
1604
1942
|
|
|
1605
1943
|
## Text Input For Tests
|
|
1606
1944
|
|
|
@@ -1622,3 +1960,4 @@ The package ships as CommonJS with TypeScript declarations. Import from `@voctiv
|
|
|
1622
1960
|
```ts
|
|
1623
1961
|
import { defineScript, type MediaChannel, type AsrHandle, type MediaError } from '@voctiv/agent-sdk';
|
|
1624
1962
|
```
|
|
1963
|
+
|