@jambonz/schema 0.4.25 → 0.4.26

package/docs/verbs/agent.md

@@ -331,6 +331,35 @@ For Anthropic models, use `"vendor": "anthropic"` and structure messages accordi
 
 By default (`greeting: true`), the agent prompts the LLM to generate an initial greeting before the user speaks. Set `greeting: false` if you want the agent to wait silently for the user to speak first.
 
+## Controlling what is spoken
+
+Everything the LLM writes is normally sent to TTS. Two options change that. Neither changes what the LLM sees: conversation history, `llm_response` events and the `turn_end` transcript always hold the model's raw text.
+
+**`speechCleanup`** (default `true`) drops text that models write but don't mean to be heard:
+
+- a paragraph that is only a stage direction in `()`, `[]` or `**`, such as `(silence)`, `(empty response)` or `[No response needed - the call has ended]`. Models write these when they are told to stay silent. A bracketed aside inside a sentence, such as "PPO (including Anthem)", is still spoken, and so is a span with no letters in it, such as a phone number's area code.
+- `<think>`, `<thinking>` and `<reasoning>` blocks, wherever they appear.
+
+The cleanup runs on the streaming tokens and holds back only text that could still turn out to be one of these, so ordinary speech is not delayed. If nothing is left to say, the reply ends without speaking, the same way as a reply that only called tools. Set `speechCleanup: false` to send the text to TTS exactly as written.
+
+**`speak: false`** keeps all of the agent's text out of TTS. Use it for an agent that must never talk, such as one that navigates a DTMF-only phone menu by calling tools. The LLM's replies are still kept, so the agent can take notes for itself. With `speak: false`, `toolFiller` and `earlyGeneration` are turned off.
+
+```json
+{
+  "verb": "agent",
+  "speak": false,
+  "llm": {
+    "vendor": "anthropic",
+    "model": "claude-haiku-4-5",
+    "llmOptions": {
+      "messages": [
+        { "role": "system", "content": "You navigate phone menus by pressing keys with the send_dtmf tool." }
+      ]
+    }
+  }
+}
+```
+
 ## Complete example (TypeScript)
 
 A voice agent using Deepgram STT, OpenAI LLM, and Cartesia TTS with Krisp turn detection. Exposes multiple endpoints with different STT/TTS combinations:

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jambonz/schema",
-  "version": "0.4.25",
+  "version": "0.4.26",
   "description": "JSON Schema definitions and validation for jambonz verb applications",
   "main": "index.js",
   "scripts": {

package/verbs/agent.schema.json

@@ -26,12 +26,12 @@
         { "type": "boolean" },
         { "type": "string", "enum": ["always"] }
       ],
-      "description": "When using Deepgram Flux Multilingual, automatically adjust STT language hints and switch TTS voice based on detected language. Values: false (disabled), true (lock on first utterance), 'always' (continuously adapt on every turn). Default: false.",
+      "description": "Switch the TTS to the language the caller speaks, using the matching languageConfig entry. Works with any STT that reports the detected language: Deepgram (language 'multi', or Flux Multilingual), AssemblyAI (languageDetection), Google and Azure (altLanguages), Soniox (enableLanguageIdentification), Gladia and Modulate. Values: false (disabled), true (lock on the first utterance whose language is detected), 'always' (re-evaluate every turn). With Deepgram Flux, true also locks the STT language hints to the detected language. Default: false.",
       "default": false
     },
     "languageConfig": {
       "type": "object",
-      "description": "Per-language overrides for TTS. Keys are BCP-47 language codes. When autoLockLanguage detects a language switch, the agent uses the corresponding config.",
+      "description": "Per-language TTS overrides, used when autoLockLanguage detects a language. Keys are language codes such as 'es' or 'es-MX'. A detected language matches an exact key first (case-insensitive), then a key with the same primary language, so 'es' matches a detected 'es-US'. A detected language with no entry keeps the current TTS. The entry's tts is merged over the agent's tts; set its language to the code the TTS vendor expects.",
       "additionalProperties": {
         "type": "object",
         "properties": {
@@ -231,6 +231,16 @@
       "description": "Enable speculative LLM prompting before end-of-turn is confirmed. When using Krisp turn detection, set this to true to speculatively prompt the LLM before Krisp confirms the turn has ended. If the transcript matches when turn ends, buffered tokens are released immediately — reducing response latency. Note: Deepgram Flux performs early generation automatically via its native EagerEndOfTurn signal regardless of this setting. Default: false.",
       "default": false
     },
+    "speak": {
+      "type": "boolean",
+      "description": "Whether the agent's text is spoken. With false the agent never talks (for example while it drives a DTMF-only phone menu with its tools): the LLM's text is still kept in conversation history, llm_response events and the transcript, but is never sent to TTS, and toolFiller and earlyGeneration are turned off. Each reply ends when the LLM finishes, as for a reply that only called tools. Default: true.",
+      "default": true
+    },
+    "speechCleanup": {
+      "type": "boolean",
+      "description": "Keep text the model was not meant to say out of TTS: a paragraph that is only a stage direction in (), [] or ** (such as '(silence)' or '[No response needed]'), and <think>, <thinking> or <reasoning> blocks. Asides inside a sentence are still spoken. Applied as the reply streams; ordinary speech is not delayed. A reply with nothing left to say ends without speaking. History, llm_response events and the transcript keep the model's raw text. Default: true.",
+      "default": true
+    },
     "noiseIsolation": {
       "oneOf": [
         {