@jambonz/schema 0.4.23 → 0.4.25

package/AGENTS.md

@@ -35,7 +35,7 @@ The verb schemas and JSON structure are identical in both modes. The difference
 - **Webhook**: Simple IVR, call routing, voicemail, basic gather-and-respond patterns.
 - **WebSocket**: LLM-powered voice agents, real-time audio streaming, complex conversational flows, anything requiring bidirectional communication, or asynchronous logic, or streaming tts.
 
-**IMPORTANT**: Any application that uses a speech-to-speech verb (`openai_s2s`, `gptlive_s2s`, `voicelive_s2s`, `google_s2s`, `deepgram_s2s`, `ultravox_s2s`, `qwen_s2s`, `elevenlabs_s2s`, `s2s`, or `agent`) MUST use WebSocket transport, not webhooks. These verbs require persistent bidirectional communication for real-time audio and events.
+**IMPORTANT**: Any application that uses a speech-to-speech verb (`openai_s2s`, `gptlive_s2s`, `voicelive_s2s`, `nova_s2s`, `google_s2s`, `deepgram_s2s`, `ultravox_s2s`, `qwen_s2s`, `elevenlabs_s2s`, `s2s`, or `agent`) MUST use WebSocket transport, not webhooks. These verbs require persistent bidirectional communication for real-time audio and events.
 
 ## Schema
 
@@ -65,6 +65,7 @@ Two tools are available:
 - **openai_s2s** / **google_s2s** / **deepgram_s2s** / **ultravox_s2s** / **qwen_s2s** — Connect the caller to a vendor-specific LLM for real-time voice conversation. These are the **preferred** verbs when the vendor is known. Each handles the full STT→LLM→TTS flow with the vendor pre-set.
 - **gptlive_s2s** — Connect the caller to OpenAI's GPT Live API (limited-access alpha). A different wire protocol from `openai_s2s` (Realtime): `llmOptions` carries only a `session_update`, there is no `response_create`, and tool calls arrive via a Responses-targeted *delegation*. See [GPT Live S2S specifics](#gpt-live-s2s-specifics) below.
 - **voicelive_s2s** — Connect the caller to the Azure Voice Live API. Requires `connectOptions.host` (your Foundry/Speech resource). Unlike `openai_s2s`, the `session_update` uses Voice Live's flat shape and is sent verbatim, so Azure-only settings (`azure_semantic_vad`, `azure_deep_noise_suppression`, Azure TTS voices) work.
+- **nova_s2s** — Connect the caller to Amazon Nova Sonic on Amazon Bedrock. `auth` takes AWS IAM credentials (`accessKeyId`, `secretAccessKey`, optional `sessionToken`) plus `region`; `model` defaults to `amazon.nova-2-5-sonic`. `llmOptions` (`systemPrompt`, `history`, `greeting`, `sessionStart`, `promptStart`) is optional. Nova waits for the caller unless a `greeting` is set. Requires the mediajam media server.
 - **elevenlabs_s2s** — Connect the caller to an ElevenLabs Conversational AI agent. **Unlike other s2s vendors**, ElevenLabs requires a pre-configured `agent_id` (created in the ElevenLabs dashboard) rather than a model and messages. See [ElevenLabs S2S specifics](#elevenlabs-s2s-specifics) below.
 - **s2s** — Generic LLM voice conversation verb. Use only when the vendor is determined at runtime (e.g. from an env var). Requires `vendor` to be specified.
 - **agent** — Higher-level voice AI agent with integrated turn detection. Mix-and-match STT, LLM, and TTS vendors.
@@ -100,7 +101,7 @@ Two tools are available:
 **IMPORTANT — Code generation rules:**
 
 1. **Always use `stream`, never `listen`** — they are synonyms; `stream` is the preferred name.
-2. **Always use the vendor-specific shortcut when the LLM vendor is known** — use `openai_s2s`, `gptlive_s2s`, `voicelive_s2s`, `google_s2s`, `elevenlabs_s2s`, `deepgram_s2s`, `qwen_s2s`, or `ultravox_s2s`. Do NOT use `llm` or `s2s` with a `vendor` property when a shortcut exists.
+2. **Always use the vendor-specific shortcut when the LLM vendor is known** — use `openai_s2s`, `gptlive_s2s`, `voicelive_s2s`, `nova_s2s`, `google_s2s`, `elevenlabs_s2s`, `deepgram_s2s`, `qwen_s2s`, or `ultravox_s2s`. Do NOT use `llm` or `s2s` with a `vendor` property when a shortcut exists.
 3. **Use `s2s` (not `llm`) when the vendor is dynamic** — e.g. the vendor comes from an env var or runtime config. Both `s2s` and `llm` are synonyms, but prefer `s2s`.
 4. **Never use `llm` in generated code** — it is a legacy name. Use either a vendor shortcut or `s2s`.
 

package/components/recognizer-mistralOptions.schema.json

@@ -0,0 +1,86 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://jambonz.org/schema/components/recognizer-mistralOptions",
+  "title": "Mistral Recognizer Options",
+  "description": "Mistral Voxtral realtime STT options. Only applies when recognizer vendor is 'mistral'. Voxtral streams text as it is recognized (sent as interim results when interim is enabled); with turn detection on (the default) a final transcript is produced when Voxtral detects the end of the speaker's turn, and in the agent verb each final ends the user's turn. Transcripts are reported with confidence 1.0. The language is sent as a two-letter code (en-US becomes en); leave it unset or 'any' for automatic detection.",
+  "type": "object",
+  "properties": {
+    "apiKey": {
+      "type": "string",
+      "description": "Mistral API key. Overrides the key configured in jambonz."
+    },
+    "model": {
+      "type": "string",
+      "description": "Realtime transcription model. Defaults to voxtral-mini-transcribe-realtime-3.",
+      "examples": ["voxtral-mini-transcribe-realtime-3"]
+    },
+    "streamingDelayMs": {
+      "type": "integer",
+      "description": "Target streaming delay in ms: how far the model looks ahead before emitting text. Lower is faster, higher is more accurate. Any value from 80 to 1200, or 2400. Voxtral's default is 480.",
+      "anyOf": [
+        { "minimum": 80, "maximum": 1200 },
+        { "const": 2400 }
+      ]
+    },
+    "verbatim": {
+      "type": "boolean",
+      "description": "Ask Voxtral for a verbatim transcript (sent as verbatim). Not in Mistral's docs yet; the realtime API accepts it."
+    },
+    "contextBias": {
+      "type": "array",
+      "items": { "type": "string" },
+      "maxItems": 100,
+      "description": "Words or phrases to favor (names, brands, jargon). When omitted, recognizer.hints are used."
+    },
+    "diarize": {
+      "type": "boolean",
+      "description": "Label each transcript with a speaker. Voxtral cannot diarize and detect turns at once, so this turns off turn detection: a final transcript closes at each sentence end or change of speaker. recognizer.diarization has the same effect. Ignored in the agent verb, which needs turn detection."
+    },
+    "speakerReferences": {
+      "type": "array",
+      "maxItems": 8,
+      "description": "Reference clips (3 to 15 seconds each) that let diarization name known speakers. Passed to Mistral as speaker_references. Requires diarize.",
+      "items": { "type": "object" }
+    },
+    "turnDetection": {
+      "type": "object",
+      "description": "Tunes when Voxtral ends a turn. Ignored when diarize is on. Thresholds must satisfy startThreshold >= eagerEndThreshold > endThreshold.",
+      "properties": {
+        "startThreshold": {
+          "type": "number",
+          "minimum": 0,
+          "maximum": 1,
+          "description": "Speech probability that starts a turn. Default 0.5."
+        },
+        "eagerEndThreshold": {
+          "type": "number",
+          "minimum": 0,
+          "maximum": 1,
+          "description": "Probability at which Voxtral signals a likely end of turn early; the agent verb uses it for early generation. Default 0.5."
+        },
+        "endThreshold": {
+          "type": "number",
+          "minimum": 0,
+          "maximum": 1,
+          "description": "Probability below which silence counts toward ending the turn. Default 0.33."
+        },
+        "minSilenceMs": {
+          "type": "integer",
+          "minimum": 0,
+          "description": "Silence in ms before a turn can end. Default 500; raised to streamingDelayMs when that is longer, since Voxtral's text lags the audio by about that much."
+        },
+        "endTimeoutMs": {
+          "type": "integer",
+          "minimum": 0,
+          "description": "Silence in ms after which the turn ends regardless of the model. Default 3000; must be at least minSilenceMs."
+        }
+      },
+      "additionalProperties": false
+    },
+    "mistralSttUri": {
+      "type": "string",
+      "description": "Custom host to connect to instead of api.mistral.ai, e.g. a proxy. A ws:// or http:// prefix disables TLS."
+    }
+  },
+  "additionalProperties": false
+}

package/components/recognizer.schema.json

@@ -8,7 +8,7 @@
     "vendor": {
       "type": "string",
       "description": "The STT vendor to use. Must match a vendor configured in the jambonz platform.",
-      "examples": ["google", "aws", "microsoft", "deepgram", "ibm", "nvidia", "soniox", "xai", "modulate", "zoom", "gradium", "assemblyai", "speechmatics", "openai", "houndify", "gladia", "elevenlabs", "custom"]
+      "examples": ["google", "aws", "microsoft", "deepgram", "ibm", "nvidia", "soniox", "xai", "modulate", "zoom", "gradium", "mistral", "assemblyai", "speechmatics", "openai", "houndify", "gladia", "elevenlabs", "custom"]
     },
     "label": {
       "type": "string",
@@ -202,6 +202,7 @@
     "modulateOptions": { "$ref": "recognizer-modulateOptions" },
     "zoomOptions": { "$ref": "recognizer-zoomOptions" },
     "gradiumOptions": { "$ref": "recognizer-gradiumOptions" },
+    "mistralOptions": { "$ref": "recognizer-mistralOptions" },
     "assemblyAiOptions": { "$ref": "recognizer-assemblyAiOptions" },
     "speechmaticsOptions": { "$ref": "recognizer-speechmaticsOptions" },
     "openaiOptions": { "$ref": "recognizer-openaiOptions" },

package/components/synthesizer.schema.json

@@ -8,7 +8,7 @@
     "vendor": {
       "type": "string",
       "description": "The TTS vendor to use. Must match a vendor configured in the jambonz platform.",
-      "examples": ["google", "aws", "microsoft", "elevenlabs", "cartesia", "fishaudio", "gradium", "kugelaudio", "nineninesix", "speechify", "deepgram", "deepgramflux", "ibm", "nvidia", "wellsaid", "whisper", "custom"]
+      "examples": ["google", "aws", "microsoft", "elevenlabs", "cartesia", "fishaudio", "gradium", "kugelaudio", "mistral", "nineninesix", "speechify", "deepgram", "deepgramflux", "ibm", "nvidia", "wellsaid", "whisper", "custom"]
     },
     "label": {
       "type": "string",

package/docs/verbs/agent.md

@@ -49,10 +49,11 @@ For other STT vendors with native turn-taking (assemblyai, speechmatics), early
 
 ## Noise isolation
 
-The `noiseIsolation` property enables server-side noise cancellation on the call audio. By default it filters the inbound (caller) audio, improving STT accuracy in noisy environments. It can also be configured to filter outbound audio via the `direction` option. Two vendors are available:
+The `noiseIsolation` property enables server-side noise cancellation on the call audio. By default it filters the inbound (caller) audio, improving STT accuracy in noisy environments. It can also be configured to filter outbound audio via the `direction` option. Three vendors are available:
 
 - **`"krisp"`** — Krisp's proprietary noise cancellation. Requires a Krisp API key on self-hosted systems.
 - **`"rnnoise"`** — Open-source RNNoise-based noise cancellation. No API key required.
+- **`"kugelaudio"`** — KugelAudio Clarity (`clarity-1`) cloud speech enhancement. Uses a KugelAudio speech credential with "use for noise isolation" enabled (pick one with `label` in the object form, or pass `auth: {apiKey, apiUri}` inline); adds a short delay (the round trip to KugelAudio) to the filtered audio. `level` is ignored.
 
 Shorthand (default settings):
 

package/jambonz-app.schema.json

@@ -30,6 +30,7 @@
         { "$ref": "verbs/qwen_s2s" },
         { "$ref": "verbs/gptlive_s2s" },
         { "$ref": "verbs/voicelive_s2s" },
+        { "$ref": "verbs/nova_s2s" },
         { "$ref": "verbs/dialogflow" },
         { "$ref": "verbs/agent" },
         { "$ref": "verbs/transfer" },

package/lib/normalize.js

@@ -17,6 +17,7 @@ const verbTransforms = new Map([
   ['qwen_s2s', {verb: 'llm', properties: {vendor: 'qwen'}}],
   ['gptlive_s2s', {verb: 'llm', properties: {vendor: 'gptlive'}}],
   ['voicelive_s2s', {verb: 'llm', properties: {vendor: 'voicelive'}}],
+  ['nova_s2s', {verb: 'llm', properties: {vendor: 'nova'}}],
 ]);
 
 function applyVerbTransform(name, data) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jambonz/schema",
-  "version": "0.4.23",
+  "version": "0.4.25",
   "description": "JSON Schema definitions and validation for jambonz verb applications",
   "main": "index.js",
   "scripts": {

package/verbs/agent.schema.json

@@ -119,7 +119,13 @@
     "noResponseTimeout": {
       "type": "number",
       "minimum": 0,
-      "description": "Seconds to wait after the assistant finishes speaking before prompting the user to respond. 0 disables. Default: 0.",
+      "description": "Seconds to wait after the assistant finishes speaking before prompting the user to respond. 0 disables. Default: 12.",
+      "default": 12
+    },
+    "noResponseMaxPrompts": {
+      "type": "integer",
+      "minimum": 0,
+      "description": "End the agent after this many no-response prompts in a row get no reply. The actionHook then fires with completionReason 'no_response', so the application decides what happens next. The count resets whenever the user speaks. 0 or unset: no limit (the agent keeps prompting). Requires feature-server with jambonz/feature-server#259.",
       "default": 0
     },
     "llm": {
@@ -229,7 +235,7 @@
       "oneOf": [
         {
           "type": "string",
-          "enum": ["krisp", "rnnoise"],
+          "enum": ["krisp", "rnnoise", "kugelaudio"],
           "description": "Shorthand — enable noise isolation with the specified vendor using default settings."
         },
         {
@@ -240,6 +246,24 @@
               "type": "string",
               "description": "Noise isolation vendor/mode (e.g. 'krisp')."
             },
+            "label": {
+              "type": "string",
+              "description": "Label of the speech credential to use, for vendors that need one (kugelaudio)."
+            },
+            "auth": {
+              "type": "object",
+              "description": "Optional inline credentials for vendors that need them (kugelaudio). When omitted, feature-server looks up the speech credential by (vendor, label).",
+              "properties": {
+                "apiKey": {
+                  "type": "string"
+                },
+                "apiUri": {
+                  "type": "string",
+                  "description": "Vendor API host, e.g. api.eu.kugelaudio.com."
+                }
+              },
+              "additionalProperties": true
+            },
             "level": {
               "type": "number",
               "minimum": 0,

package/verbs/config.schema.json

@@ -176,6 +176,24 @@
         },
         "model": {
           "type": "string"
+        },
+        "label": {
+          "type": "string",
+          "description": "Label of the speech credential to use, for vendors that need one (kugelaudio)."
+        },
+        "auth": {
+          "type": "object",
+          "description": "Optional inline credentials for vendors that need them (kugelaudio). When omitted, feature-server looks up the speech credential by (vendor, label).",
+          "properties": {
+            "apiKey": {
+              "type": "string"
+            },
+            "apiUri": {
+              "type": "string",
+              "description": "Vendor API host, e.g. api.eu.kugelaudio.com."
+            }
+          },
+          "additionalProperties": true
         }
       }
     },

package/verbs/nova_s2s.schema.json

@@ -0,0 +1,179 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://jambonz.org/schema/verbs/nova_s2s",
+  "minVersion": "11.1.7",
+  "title": "Amazon Nova Sonic S2S",
+  "description": "Shortcut for 'llm' with vendor automatically set to 'nova'. Connects the caller to Amazon Nova Sonic (Nova 2 / Nova 2.5 Sonic) on Amazon Bedrock for real-time speech-to-speech conversation. auth takes AWS IAM credentials (accessKeyId, secretAccessKey, optional sessionToken) and the Bedrock region; the credentials need bedrock:InvokeModelWithBidirectionalStream on the model. llmOptions is optional: systemPrompt, history, greeting, and sessionStart / promptStart bodies merged into Nova's own events (the audio formats are fixed by jambonz). Requires the mediajam media server.",
+  "type": "object",
+  "allOf": [
+    {
+      "$ref": "../components/llm-base"
+    }
+  ],
+  "properties": {
+    "verb": {
+      "const": "nova_s2s",
+      "description": "The verb name."
+    },
+    "vendor": {
+      "type": "string",
+      "const": "nova",
+      "description": "The LLM vendor (always 'nova' for this shortcut)."
+    },
+    "model": {
+      "type": "string",
+      "description": "The Bedrock model id. Defaults to amazon.nova-2-5-sonic.",
+      "examples": [
+        "amazon.nova-2-5-sonic",
+        "amazon.nova-2-sonic-v1:0"
+      ]
+    },
+    "auth": {
+      "type": "object",
+      "description": "AWS IAM credentials and the Bedrock region.",
+      "properties": {
+        "accessKeyId": {
+          "type": "string",
+          "minLength": 1,
+          "description": "AWS access key id."
+        },
+        "secretAccessKey": {
+          "type": "string",
+          "minLength": 1,
+          "description": "AWS secret access key."
+        },
+        "sessionToken": {
+          "type": "string",
+          "description": "Session token, for temporary (STS) credentials."
+        },
+        "region": {
+          "type": "string",
+          "description": "Bedrock region serving Nova Sonic. Defaults to us-east-1.",
+          "examples": [
+            "us-east-1",
+            "us-west-2",
+            "eu-north-1",
+            "ap-northeast-1"
+          ]
+        }
+      },
+      "required": [
+        "accessKeyId",
+        "secretAccessKey"
+      ],
+      "additionalProperties": true
+    },
+    "llmOptions": {
+      "type": "object",
+      "description": "Nova session options. A systemPrompt or other text above 50KB is split across several textInput events.",
+      "properties": {
+        "systemPrompt": {
+          "type": "string",
+          "description": "System instructions, sent as the SYSTEM text content."
+        },
+        "speechPrompt": {
+          "type": "string",
+          "description": "Sent after the system prompt with role SYSTEM_SPEECH. Nova accepts one of three fixed sentences that choose Latin, Devanagari or mixed script for Hindi transcription and responses."
+        },
+        "history": {
+          "type": "array",
+          "description": "Prior turns sent before the audio starts. Following Nova's rules, consecutive turns with the same role are merged, leading assistant turns are dropped, and the total is capped at 200KB.",
+          "items": {
+            "type": "object",
+            "properties": {
+              "role": {
+                "type": "string",
+                "enum": [
+                  "user",
+                  "assistant",
+                  "USER",
+                  "ASSISTANT"
+                ]
+              },
+              "content": {
+                "type": "string"
+              }
+            },
+            "required": [
+              "role",
+              "content"
+            ]
+          }
+        },
+        "greeting": {
+          "type": "string",
+          "description": "Sent as a user text turn once the audio starts, so Nova speaks first (Nova otherwise waits for the caller)."
+        },
+        "sessionStart": {
+          "type": "object",
+          "description": "Merged into Nova's sessionStart event: inferenceConfiguration, turnDetectionConfiguration.endpointingSensitivity (HIGH, MEDIUM, LOW)."
+        },
+        "promptStart": {
+          "type": "object",
+          "description": "Merged into Nova's promptStart event: audioOutputConfiguration.voiceId (default matthew), toolConfiguration.tools in Nova's toolSpec shape and toolConfiguration.toolChoice ({auto:{}}, {any:{}} or {tool:{name}}). The audio format fields are set by jambonz."
+        },
+        "dtmf": {
+          "type": "object",
+          "description": "Caller keypad input. Nova has no DTMF support, so each completed entry is sent as a user text turn. Takes the gather verb's digit options.",
+          "properties": {
+            "enable": {
+              "type": "boolean"
+            },
+            "finishOnKey": {
+              "type": "string"
+            },
+            "numDigits": {
+              "type": "number"
+            },
+            "minDigits": {
+              "type": "number"
+            },
+            "maxDigits": {
+              "type": "number"
+            },
+            "interDigitTimeout": {
+              "type": "number"
+            },
+            "timeout": {
+              "type": "number"
+            }
+          },
+          "required": [
+            "enable"
+          ]
+        }
+      },
+      "additionalProperties": true
+    }
+  },
+  "required": [
+    "auth"
+  ],
+  "examples": [
+    {
+      "verb": "nova_s2s",
+      "model": "amazon.nova-2-5-sonic",
+      "auth": {
+        "accessKeyId": "AKIA...",
+        "secretAccessKey": "...",
+        "region": "us-east-1"
+      },
+      "llmOptions": {
+        "systemPrompt": "You are a helpful voice assistant. Keep answers short.",
+        "greeting": "Greet the caller and ask how you can help.",
+        "sessionStart": {
+          "turnDetectionConfiguration": {
+            "endpointingSensitivity": "MEDIUM"
+          }
+        },
+        "promptStart": {
+          "audioOutputConfiguration": {
+            "voiceId": "tiffany"
+          }
+        }
+      },
+      "toolHook": "/tool-call",
+      "actionHook": "/s2s-complete"
+    }
+  ]
+}