@voctiv/agent-sdk 0.2.6 → 0.2.8

package/README.md

@@ -89,7 +89,7 @@ export default defineScript(async ({ channel, logger, context }) => {
 - `channel.audio` controls TTS, raw playback, pre-synthesis, and mixer queues.
 - `channel.sip` controls SIP state, pre-answer media, DTMF, hold/mute/hangup, outbound calls, and bridging.
 - `channel.llm` talks to the Omni LLM backend.
-- `channel.events` exposes speech, interrupt, termination, and WS data message observables.
+- `channel.events` exposes speech, interrupt, termination, WS data message, and media error observables.
 - `channel.textInput` injects synthetic ASR results for tests and debug clients.
 
 ## SIP And Pre-Answer Media
@@ -194,7 +194,7 @@ Create ASR with `channel.createAsr(config?)`.
 
 ```ts
 const asr = await channel.createAsr({
-  vendor: 'Y',
+  vendor: 'yandex',
   name: 'main-yandex-key',
   language: 'ru-RU',
   vad: {
@@ -217,19 +217,50 @@ const asr = await channel.createAsr({
 - `speechStart$` / `speechEnd$`: VAD speech boundaries.
 - `interrupt$`: barge-in / interrupt events where the host supports them.
 - `vadProbability$`: normalized VAD probability when available.
+- `error$`: runtime errors from the ASR provider (see [Error Handling](#error-handling)).
 - `pause()` / `resume()` to stop or resume forwarding new audio frames.
 - `finalize()` to force the current utterance to flush.
 - `destroy()` to close connector streams and subscriptions.
 
 SIP sessions use the call-level telephony VAD when it is available. WS sessions create one VAD/SmartTurn instance for the socket session on the first `createAsr()` call. Headless sessions return an inert ASR handle with empty observables.
 
-If ASR connector creation fails, SIP/WS return a degraded handle. VAD observables still mirror the channel where possible, but no real STT results are emitted.
+If ASR connector creation fails, SIP/WS return a degraded handle. VAD observables still mirror the channel where possible, but no real STT results are emitted. The creation failure is reported on `channel.events.error$`.
 
 ## ASR Credentials And Vendors
 
-`AsrConfig.vendor` is an engine hint, for example `"Y"`, `"D"`, `"yandex"`, or `"neuro_v3"`, resolved by the host vendor alias mapping.
+`AsrConfig.vendor` is an engine hint, for example `"yandex"`, `"deepgram"`, `"azure"`, `"elevenlabs"`, or `"neuro_v3"`, resolved by the host vendor alias mapping.
 
-In Voctiv legacy compatibility mode, ASR credentials can be selected by logic-executor `key_storage.name`:
+### Direct ASR Vendor Parameters
+
+Pass vendor-native credentials and settings directly through `AsrConfig.data`. These values are forwarded to the connector as-is and override any defaults or platform-resolved credentials.
+
+```ts
+const asr = await channel.createAsr({
+  vendor: 'azure',
+  language: 'ru-RU',
+  data: {
+    subscription_key: 'your-azure-key',
+    region: 'swedencentral',
+  },
+});
+```
+
+Each vendor connector accepts its native parameter names:
+
+| Vendor | Accepted `data` keys |
+| --- | --- |
+| **Azure** | `subscription_key` or `api_key`, `region` |
+| **Yandex** | `api_key` or `token`, `folder_id` |
+| **ElevenLabs** | `api_key` (or `xi_api_key`), `model` |
+| **Deepgram** | `api_key` |
+| **Google** | `email`, `private_key`, `project_id` |
+| **Whisper** | `url`, `rate`, `toFloat` |
+
+All vendors also accept the env-style names (`AZURE_SPEECH_KEY`, `ELEVENLABS_API_KEY`, etc.) for backwards compatibility, but vendor-native names are checked first and are preferred.
+
+### Voctiv Platform ASR Key Selection
+
+In Voctiv legacy compatibility mode, ASR credentials can also be selected by logic-executor `key_storage.name`:
 
 ```ts
 const asr = await channel.createAsr({
@@ -238,7 +269,9 @@ const asr = await channel.createAsr({
 });
 ```
 
-The runtime looks in `channel.params.authentication_data.legacyAsrKeysByName[name]` for the current dialog agent and company. If `name` is omitted, `channel.params.defaultAsrName` may be used. Vendor-specific overrides go into `data`; primitives are stringified and objects/arrays are JSON-serialized before connector config is built.
+The runtime looks in `channel.params.authentication_data.legacyAsrKeysByName[name]` for the current dialog agent and company. If `name` is omitted, `channel.params.defaultAsrName` may be used.
+
+When both `name` (platform key) and explicit `data` are provided, `data` values win — they are applied last and override anything resolved from the platform.
 
 ## TTS, Playback, And Mixer Queues
 
@@ -248,15 +281,22 @@ The runtime looks in `channel.params.authentication_data.legacyAsrKeysByName[nam
 await channel.audio.say('Please wait while I check that.', {
   queue: 0,
   alias: 'main-response',
-  ttsVendor: 'E',
+  ttsVendor: 'elevenlabs',
   ttsStrategy: 'sentence',
   ttsConfig: {
-    voice_id: 'voice-id',
+    api_key: 'sk_your-key',
+    voice_id: 'bBLRWT6MSWBFAm76ZWXY',
+    model_id: 'eleven_turbo_v2_5',
+    base_url: 'https://api.eu.residency.elevenlabs.io',
     output_format: 'pcm_16000',
   },
 });
 ```
 
+Use full vendor names for `ttsVendor`. Dedicated TTS vendors include `"elevenlabs"`, `"google"`, and `"voctiv"`. The default TTS path can also accept compatible aliases such as `"azure"` or `"neuro_v3"`, depending on how ScriptEngine is configured.
+
+Vendor-native parameter names (`api_key`, `voice_id`, `model_id`, `base_url`) are passed directly to the connector and override any platform defaults. See [TTS Credentials And Vendor Parameters](#tts-credentials-and-vendor-parameters) for the full list of accepted keys per vendor.
+
 `channel.audio.play(source, options?)` plays raw audio from a URL/path or a `LegacyPhraseRecord`.
 
 ```ts
@@ -301,9 +341,37 @@ channel.audio.stop(2);
 
 For sentence-split TTS, queue item aliases are suffixed as `alias-0`, `alias-1`, and so on. Raw `play()` and direct streaming TTS use the alias exactly.
 
-## TTS Credentials And Saved Phrases
+## TTS Credentials And Vendor Parameters
+
+### Direct TTS Vendor Parameters
+
+Pass vendor-native credentials and settings directly through `PlayOptions.ttsConfig`. These values are forwarded to the TTS connector as-is and override any defaults or platform-resolved credentials.
+
+```ts
+await channel.audio.say('Hello!', {
+  ttsVendor: 'elevenlabs',
+  ttsStrategy: 'streaming',
+  ttsConfig: {
+    api_key: 'sk_your-elevenlabs-key',
+    voice_id: 'bBLRWT6MSWBFAm76ZWXY',
+    model_id: 'eleven_turbo_v2_5',
+    base_url: 'https://api.eu.residency.elevenlabs.io',
+  },
+});
+```
+
+Each TTS vendor connector accepts its native parameter names:
+
+| Vendor | Accepted `ttsConfig` keys |
+| --- | --- |
+| **ElevenLabs** | `api_key` (or `xi_api_key`), `voice_id`, `model_id` (or `model`), `base_url`, `output_format`, `language_code`, `voice_settings_stability`, `voice_settings_similarity_boost`, `voice_settings_style`, `voice_settings_speed` |
+| **Voctiv** | `url`, `voice_id`, `language`, `emotion`, `speaking_rate`, `chunk_schedule` |
 
-In Voctiv legacy compatibility mode, TTS credentials can be selected by `PlayOptions.name` or `ttsConfig.name`.
+All vendors also accept the env-style names (`ELEVENLABS_API_KEY`, `ELEVENLABS_VOICE_ID`, etc.) for backwards compatibility, but vendor-native names are checked first and are preferred.
+
+### Voctiv Platform TTS Key Selection
+
+In Voctiv legacy compatibility mode, TTS credentials can also be selected by `PlayOptions.name` or `ttsConfig.name`.
 
 ```ts
 await channel.audio.say('Здравствуйте!', {
@@ -316,6 +384,8 @@ await channel.audio.say('Здравствуйте!', {
 
 The runtime looks in `channel.params.authentication_data.legacyTtsKeysByName[name]`. If `name` is omitted, `channel.params.defaultTtsName` may be used.
 
+When both `name` (platform key) and explicit `ttsConfig` values are provided, `ttsConfig` values win — they are applied last and override anything resolved from the platform.
+
 `legacySavePhrase` stores synthesized audio under the Voctiv record phrase storage root and inserts phrase metadata so it can later be loaded with `platform.getRecords()`.
 
 ```ts
@@ -340,6 +410,72 @@ if (records?.[0]) {
 
 This requires legacy compatibility mode, a trusted LE agent id/UUID, TTS cache, and `LEGACY_V3_RECORD_PHRASE_ROOT`.
 
+## Error Handling
+
+ASR and TTS errors are propagated to the script. Unhandled errors are always logged server-side, but scripts can catch them to react: fall back to a different vendor, notify the caller, or abort the dialog.
+
+### TTS Errors — Promise Rejection
+
+`say()` and `play()` reject their promises when TTS/playback fails:
+
+```ts
+try {
+  await channel.audio.say('Hello!', {
+    ttsVendor: 'elevenlabs',
+    ttsConfig: { api_key: 'invalid-key', voice_id: 'abc' },
+  });
+} catch (err) {
+  logger.error('TTS failed', { error: String(err) });
+  await channel.audio.say('Fallback message.'); // try default TTS
+}
+```
+
+### ASR Errors — `error$` Observable
+
+Runtime ASR errors (gRPC disconnect, auth failure, quota exceeded) are emitted on `AsrHandle.error$`:
+
+```ts
+const asr = await channel.createAsr({
+  vendor: 'yandex',
+  data: { api_key: 'my-key' },
+});
+
+asr.error$.subscribe((err) => {
+  logger.error('ASR provider error', {
+    message: err.message,
+    code: err.code,
+    vendor: err.vendor,
+  });
+});
+```
+
+A degraded handle (returned when connector creation itself failed) has an inert `error$` that never emits — the creation failure is reported on `channel.events.error$` instead.
+
+### Channel Error Stream
+
+`channel.events.error$` is a unified stream of all media errors — both ASR and TTS:
+
+```ts
+channel.events.error$.subscribe((err) => {
+  logger.warn(`[${err.source}] ${err.message}`, {
+    code: err.code,
+    vendor: err.vendor,
+  });
+});
+```
+
+`MediaError` fields:
+
+| Field | Type | Description |
+| --- | --- | --- |
+| `source` | `'asr' \| 'tts' \| 'sip' \| 'channel'` | Which subsystem produced the error. |
+| `message` | `string` | Human-readable description. |
+| `code` | `number?` | HTTP status, gRPC status, or WebSocket close code. |
+| `vendor` | `string?` | Vendor identifier, e.g. `"yandex"`, `"elevenlabs"`, `"azure"`. |
+| `details` | `unknown?` | Arbitrary provider-specific payload. |
+
+Subscribing to `error$` is optional. Old scripts that do not subscribe are not affected — the observables simply go unobserved.
+
 ## LLM API
 
 `channel.llm` talks to the Omni LLM backend.
@@ -469,6 +605,145 @@ await platform.messaging.send({
 
 Outbound messages are transported through legacy Redis streams. `platform.messaging.message$` currently replays the inbound message that started a headless messaging script; it is not a live subscription to all future Redis messages.
 
+## Offline / Headless Logic
+
+Offline, or headless, sessions run a script without a live SIP call, WebSocket audio stream, RTP pipeline, ASR, or TTS playback. They are used for platform-driven background logic, queued dialog processing, and messaging events.
+
+The script entry point is still the same `defineScript()` handler. Detect this mode with `context.headless`:
+
+```ts
+export default defineScript(async ({ channel, context, logger, platform }) => {
+  if (!context.headless) {
+    channel.sip.answer();
+    await channel.audio.say('Hello.');
+    return;
+  }
+
+  logger.log('Running offline logic', {
+    dialogUuid: context.dialogUuid,
+    entryPoint: context.entryPoint,
+  });
+
+  // Offline logic usually works with text, params, env, NLU, LLM, and platform APIs.
+});
+```
+
+Headless sessions can be started by host integrations such as:
+
+- a platform dialog queue worker that loads pending dialogs;
+- an inbound messaging worker, usually with `context.entryPoint === 'on_message_api_received'`;
+- an HTTP/API request that asks ScriptEngine to run a script without media.
+
+### What Works In Headless
+
+These APIs are available and are the intended tools for offline scripts:
+
+- `context.dialogParams`, `context.initialData`, `context.dialogEntity`, and `context.callEntity` for platform data.
+- `context.env$` for persisted per-dialog state.
+- `platform.nlu.extract()` for text NLU when legacy platform compatibility is enabled.
+- `platform.messaging.send()` for outbound messages through the configured platform messaging transport.
+- `platform.call()` for scheduling outbound platform-managed calls.
+- `platform.dialog.entryPoint` and `platform.dialog.result` for updating dialog routing and outcome.
+- `channel.llm.ask()`, `channel.llm.stream()`, and `channel.llm.extract()` for Omni LLM operations.
+- `logger` for structured logs.
+
+Audio and telephony APIs are intentionally inert:
+
+- `channel.audio.say()`, `play()`, `preload()`, and `presay()` do not play audio and only log warnings.
+- `channel.createAsr()` returns an inert handle with empty observables.
+- `channel.textInput` does not simulate ASR in headless mode.
+- `channel.sip.state` behaves as an already-active synthetic channel, but real SIP actions such as nested calls and bridging are not available.
+
+Use headless mode for text and platform workflows. Use SIP or WS sessions when the script needs real audio, ASR, TTS, DTMF, pre-answer media, or bridging.
+
+### Inbound Messaging
+
+When a headless script is triggered by an inbound message, the runtime exposes the message as `context.inboundMessage` and also replays it on `platform.messaging.message$`.
+
+```ts
+export default defineScript(async ({ context, platform, logger }) => {
+  const inbound = context.inboundMessage;
+  const payload = inbound?.payload ?? {};
+
+  const text =
+    typeof payload.text === 'string'
+      ? payload.text
+      : typeof payload.message === 'string'
+        ? payload.message
+        : '';
+
+  logger.log('Inbound message received', {
+    src: inbound?.src,
+    dst: inbound?.dst,
+    channelType: inbound?.channelType,
+    text,
+  });
+
+  if (!text.trim()) {
+    return { output: { reason: 'empty_message' } };
+  }
+
+  const nlu = await platform.nlu.extract(text, {
+    intents: ['support_request', 'callback_request'],
+  });
+
+  await platform.messaging.send({
+    src: inbound?.dst ?? 'bot',
+    destination: inbound?.src ?? '',
+    text: 'Thanks, I received your message.',
+  });
+
+  return {
+    output: {
+      text,
+      nlu,
+    },
+  };
+});
+```
+
+`context.inboundMessage.payload` is the raw transport payload. Different messaging providers may use different field names (`text`, `message`, `body`, `content`, etc.), so production scripts should normalize the text they need.
+
+### Persisting Offline State
+
+Use `context.env$` to keep state between offline runs for the same dialog:
+
+```ts
+const env = context.env$?.getValue() ?? {};
+const messageCount = Number(env.messageCount ?? 0) + 1;
+
+context.env$?.next({
+  ...env,
+  messageCount,
+  lastMessageAt: new Date().toISOString(),
+});
+```
+
+Do not return `env` from the script. ScriptEngine snapshots `context.env$` after completion and persists it according to the host integration.
+
+### Combining Voice And Offline In One Script
+
+One script can support both live calls and offline messages by branching on `context.headless`:
+
+```ts
+export default defineScript(async ({ channel, context, platform }) => {
+  if (context.headless) {
+    const text = String(context.inboundMessage?.payload?.text ?? '');
+
+    if (text.includes('call me')) {
+      await platform.call(context.msisdn, {
+        entryPoint: 'on_callback',
+      });
+    }
+
+    return { output: { handledOffline: true } };
+  }
+
+  channel.sip.answer();
+  await channel.audio.say('How can I help you?');
+});
+```
+
 ## Dialog Context And Persisted Env
 
 `context` includes identity, telephony fields, params, routing metadata, and runtime helpers.
@@ -524,9 +799,9 @@ Headless channels are for offline, queue, or messaging sessions:
 - Audio methods are no-ops that log warnings.
 - SIP methods are mostly no-ops.
 - `createAsr()` returns an inert handle.
-- LLM and platform APIs still work.
+- LLM, NLU, messaging, platform calls, dialog state, and `env$` still work.
 
-Use `context.headless` to branch when a script must behave differently without a real media channel.
+Use `context.headless` to branch when a script must behave differently without a real media channel. See [Offline / Headless Logic](#offline--headless-logic) for details and examples.
 
 ## Text Input For Tests
 
@@ -554,5 +829,5 @@ npm run build
 The package exports only the public SDK entry point:
 
 ```ts
-import { defineScript, type MediaChannel, type AsrHandle } from '@voctiv/agent-sdk';
+import { defineScript, type MediaChannel, type AsrHandle, type MediaError } from '@voctiv/agent-sdk';
 ```