@jambonz/mcp-schema-server 0.1.0 → 0.1.1

package/AGENTS.md

@@ -71,7 +71,194 @@ Individual verb schemas are in `schema/verbs/`. Shared component types (synthesi
 - **answer** — Explicitly answer the call.
 - **leave** — Leave a conference or queue.
 
-## Common Patterns
+## Using the @jambonz/sdk
+
+**IMPORTANT**: Always use the `@jambonz/sdk` package (version 0.1.x+) to build jambonz applications. The older packages `@jambonz/node-client-ws` and `@jambonz/node-client` are **deprecated and replaced** by `@jambonz/sdk`. Do NOT use those old packages. Do NOT build apps with raw JSON verb arrays and plain Express/WS — use `@jambonz/sdk`.
+
+Install: `npm install @jambonz/sdk`
+
+### Webhook Application (HTTP)
+
+Import `WebhookResponse` from `@jambonz/sdk/webhook`. Create an Express server, construct a `WebhookResponse` for each request, chain verb methods, and return it via `res.json()`.
+
+**Important**: Every webhook app must include a `POST /call-status` handler. jambonz sends call status change notifications (ringing, in-progress, completed, etc.) to this endpoint. The handler should log the event and return 200.
+
+```typescript
+import express from 'express';
+import { WebhookResponse } from '@jambonz/sdk/webhook';
+
+const app = express();
+app.use(express.json());
+
+app.post('/incoming', (req, res) => {
+  const jambonz = new WebhookResponse();
+  jambonz
+    .say({ text: 'Hello! Welcome to our service.' })
+    .gather({
+      input: ['speech', 'digits'],
+      actionHook: '/handle-input',
+      numDigits: 1,
+      timeout: 10,
+      say: { text: 'Press 1 for sales or 2 for support.' },
+    })
+    .say({ text: 'We did not receive any input. Goodbye.' })
+    .hangup();
+
+  res.json(jambonz);
+});
+
+app.post('/handle-input', (req, res) => {
+  const { digits, speech } = req.body;
+  const jambonz = new WebhookResponse();
+  jambonz.say({ text: `You pressed ${digits || 'nothing'}. Goodbye.` }).hangup();
+  res.json(jambonz);
+});
+
+// Every webhook app must handle call status events
+app.post('/call-status', (req, res) => {
+  console.log(`Call ${req.body.call_sid} status: ${req.body.call_status}`);
+  res.sendStatus(200);
+});
+
+app.listen(3000, () => console.log('Listening on port 3000'));
+```
+
+### WebSocket Application
+
+Import `createEndpoint` from `@jambonz/sdk/websocket`. Create an HTTP server, call `createEndpoint` to set up WebSocket handling, then register path-based services that receive `session` objects.
+
+```typescript
+import http from 'http';
+import { createEndpoint } from '@jambonz/sdk/websocket';
+
+const server = http.createServer();
+const makeService = createEndpoint({ server, port: 3000 });
+
+const svc = makeService({ path: '/' });
+
+svc.on('session:new', (session) => {
+  console.log(`Incoming call: ${session.callSid}`);
+
+  session
+    .say({ text: 'Hello from jambonz over WebSocket!' })
+    .hangup()
+    .send();
+});
+
+console.log('jambonz ws app listening on port 3000');
+```
+
+**Key differences from webhook**: Use `session` instead of `WebhookResponse`. Chain verbs the same way, but call `.send()` at the end to transmit the initial verb array over the WebSocket.
+
+### WebSocket actionHook Events (CRITICAL)
+
+In webhook mode, an `actionHook` is just a URL that jambonz POSTs to. In WebSocket mode, the `actionHook` value becomes an **event name** emitted on the session. You MUST bind a handler for it and respond with `.reply()`.
+
+**Key rules for WebSocket actionHook handling:**
+1. Use `session.on('/hookName', (evt) => {...})` to listen for the actionHook event.
+2. The `evt` object contains the actionHook payload (same fields as the webhook POST body: `reason`, `speech`, `digits`, etc.).
+3. Respond with `.reply()` — NOT `.send()`. `.send()` is only for the initial verb array (the first response to `session:new`). `.reply()` acknowledges the actionHook and provides the next verb array.
+4. If no listener is bound for an actionHook, the SDK auto-replies with an empty verb array.
+
+### WebSocket with Gather (speech echo example)
+
+```typescript
+import http from 'http';
+import { createEndpoint } from '@jambonz/sdk/websocket';
+
+const server = http.createServer();
+const makeService = createEndpoint({ server, port: 3000 });
+
+const svc = makeService({ path: '/' });
+
+svc.on('session:new', (session) => {
+  // Bind actionHook handler BEFORE sending verbs
+  session
+    .on('close', (code: number, _reason: Buffer) => {
+      console.log(`Session ${session.callSid} closed: ${code}`);
+    })
+    .on('error', (err: Error) => {
+      console.error(`Session ${session.callSid} error:`, err);
+    })
+    .on('/echo', (evt: Record<string, any>) => {
+      // This fires when the gather verb completes (actionHook: '/echo')
+      switch (evt.reason) {
+        case 'speechDetected': {
+          const transcript = evt.speech?.alternatives?.[0]?.transcript || 'nothing';
+          session
+            .say({ text: `You said: ${transcript}.` })
+            .gather({
+              input: ['speech'],
+              actionHook: '/echo',
+              timeout: 10,
+              say: { text: 'Please say something else.' },
+            })
+            .reply();  // reply() — NOT send()
+          break;
+        }
+        case 'timeout':
+          session
+            .gather({
+              input: ['speech'],
+              actionHook: '/echo',
+              timeout: 10,
+              say: { text: 'Are you still there? I didn\'t hear anything.' },
+            })
+            .reply();
+          break;
+        default:
+          session.reply();
+          break;
+      }
+    });
+
+  // Send initial verbs to jambonz
+  session
+    .pause({ length: 1 })
+    .gather({
+      input: ['speech'],
+      actionHook: '/echo',
+      timeout: 10,
+      say: { text: 'Please say something and I will echo it back to you.' },
+    })
+    .send();  // send() — first response only
+});
+
+console.log('Speech echo WebSocket app listening on port 3000');
+```
+
+**`.send()` vs `.reply()`:**
+- `.send()` — Use ONCE for the initial verb array in response to `session:new`. This acknowledges the session.
+- `.reply()` — Use for ALL subsequent responses (actionHook events, session:redirect). This acknowledges the hook message and provides the next verb array.
+
+### SDK Verb Method Reference
+
+Both `WebhookResponse` and `Session` support the same chainable verb methods:
+
+`.say(opts)` `.play(opts)` `.gather(opts)` `.dial(opts)` `.llm(opts)` `.conference(opts)` `.enqueue(opts)` `.dequeue(opts)` `.hangup()` `.pause(opts)` `.redirect(opts)` `.config(opts)` `.tag(opts)` `.dtmf(opts)` `.listen(opts)` `.transcribe(opts)` `.message(opts)` `.stream(opts)` `.pipeline(opts)` `.dub(opts)` `.alert(opts)` `.answer(opts)` `.leave()` `.sipDecline(opts)` `.sipRefer(opts)` `.sipRequest(opts)`
+
+All methods accept the same options as the corresponding verb JSON Schema. Methods are chainable — they return `this`.
+
+### REST API Client
+
+```typescript
+import { JambonzClient } from '@jambonz/sdk/client';
+
+const client = new JambonzClient({ baseUrl: 'https://api.jambonz.us', accountSid, apiKey });
+
+// Create an outbound call
+await client.calls.create({ from: '+15085551212', to: { type: 'phone', number: '+15085551213' }, call_hook: '/incoming' });
+
+// Mid-call control
+await client.calls.whisper(callSid, { verb: 'say', text: 'Supervisor listening.' });
+await client.calls.mute(callSid, 'mute');
+await client.calls.redirect(callSid, 'https://example.com/new-flow');
+await client.calls.update(callSid, { call_status: 'completed' });
+```
+
+## Common Patterns (Raw JSON)
+
+These are the raw JSON verb arrays that the SDK generates. You should use the SDK verb methods above, but these show the underlying structure for reference.
 
 ### Simple Greeting and Gather
 ```json
@@ -151,7 +338,7 @@ When a verb completes, jambonz invokes the `actionHook` URL (webhook) or sends a
 
 **gather**: `speech` (object with `alternatives[].transcript`), `digits` (string), `reason` (`speechDetected`, `dtmfDetected`, `timeout`)
 
-**dial**: `dial_call_sid`, `dial_call_status`, `dial_sip_status`, `duration`
+**dial**: `dial_call_sid`, `dial_call_status`, `dial_sip_status`, `dial_sbc_callid`, `duration`
 
 **llm**: `completion_reason` (`normal`, `timeout`, `error`), `llm_usage` (token counts)
 
@@ -159,6 +346,62 @@ When a verb completes, jambonz invokes the `actionHook` URL (webhook) or sends a
 
 **transcribe**: `transcription` (object with transcript text)
 
+## Application Environment Variables
+
+Applications can declare configurable parameters that the jambonz portal discovers via an HTTP `OPTIONS` request. The portal renders a UI form from the schema, and configured values are delivered as `env_vars` in the initial call payload.
+
+**When to suggest env vars**: If the user describes an application that includes values likely to change between deployments or users — such as phone numbers to dial, API keys, language preferences, greeting text, queue names, or timeout values — proactively ask whether they'd like to make those configurable via environment variables rather than hardcoding them. Many users won't think of this but will benefit from the flexibility.
+
+### Declaring Environment Variables
+
+Define a schema object where each key is a parameter name and the value describes its type and UI behavior:
+
+```typescript
+const envVars = {
+  API_KEY: { type: 'string', description: 'Your API key', required: true, obscure: true },
+  LANGUAGE: { type: 'string', description: 'TTS language', default: 'en-US', enum: ['en-US', 'es-ES', 'fr-FR'] },
+  MAX_RETRIES: { type: 'number', description: 'Max retry attempts', default: 3 },
+};
+```
+
+Each parameter supports: `type` (required: `'string'` | `'number'` | `'boolean'`), `description` (required), `required`, `default`, `enum`, and `obscure` (masks value in portal UI for secrets).
+
+### WebSocket Apps
+
+Pass `envVars` to `createEndpoint`. The SDK auto-responds to OPTIONS requests:
+
+```typescript
+const makeService = createEndpoint({ server, port: 3000, envVars });
+```
+
+Access values at runtime via `session.data`:
+
+```typescript
+svc.on('session:new', (session) => {
+  const apiKey = session.data.env_vars?.API_KEY;
+});
+```
+
+### Webhook Apps
+
+Use the `envVarsMiddleware` to auto-respond to OPTIONS requests:
+
+```typescript
+import { WebhookResponse, envVarsMiddleware } from '@jambonz/sdk/webhook';
+
+app.use(envVarsMiddleware(envVars));
+```
+
+Access values at runtime via `req.body`:
+
+```typescript
+app.post('/incoming', (req, res) => {
+  const apiKey = req.body.env_vars?.API_KEY;
+});
+```
+
+**Note**: `env_vars` is only present in the initial call webhook, not in subsequent actionHook callbacks.
+
 ## Mid-Call Control
 
 Active calls can be modified asynchronously — inject verbs, mute, redirect, or start recording while the call is in progress.
@@ -223,7 +466,8 @@ session.injectCommand('redirect', { call_hook: '/new-flow' });
 |------|-------------|
 | `session:new` | New call session established. Contains call details. |
 | `session:redirect` | Call was redirected to this app. |
-| `verb:status` | A verb completed or changed status. Contains actionHook data. |
+| `verb:hook` | An actionHook fired (e.g. gather completed). Contains `hook` (the actionHook name) and `data` (the payload). The SDK emits this as `session.on('/hookName', handler)`. Respond with `.reply()`. |
+| `verb:status` | Informational verb status notification (no reply needed). |
 | `call:status` | Call state changed (e.g. `completed`). |
 | `llm:tool-call` | LLM requested a tool/function call. |
 | `llm:event` | LLM lifecycle event (connected, tokens, etc.). |
@@ -245,16 +489,185 @@ session.injectCommand('redirect', { call_hook: '/new-flow' });
 The SDK `Session` object emits events for common message types:
 
 ```typescript
-session.on('session:new', (session) => { /* new call */ });
-session.on('verb:status', (data) => { /* verb completed */ });
+// ActionHook events — the hook name IS the event name. Respond with .reply()
+session.on('/echo', (data) => { /* gather actionHook fired */ session.say({text: '...'}).reply(); });
+session.on('/dial-result', (data) => { /* dial actionHook */ session.reply(); });
+session.on('/llm-complete', (data) => { /* llm actionHook */ session.hangup().reply(); });
+
+// Fallback — fires for any verb:hook without a specific listener
+session.on('verb:hook', (hook, data) => { /* generic actionHook handler */ });
+
+// Status events (informational — no reply needed)
+session.on('verb:status', (data) => { /* verb status notification */ });
 session.on('call:status', (data) => { /* call state change */ });
+
+// LLM events
 session.on('llm:tool-call', (data) => { /* tool call from LLM */ });
 session.on('llm:event', (data) => { /* LLM event */ });
-session.on('tts:user_interrupt', () => { /* user interrupted TTS */ });
+
+// TTS streaming — specific lifecycle events
+session.on('tts:stream_open', (data) => { /* vendor connection established */ });
+session.on('tts:stream_paused', (data) => { /* backpressure: buffer full */ });
+session.on('tts:stream_resumed', (data) => { /* backpressure released */ });
+session.on('tts:stream_closed', (data) => { /* TTS stream ended */ });
+session.on('tts:user_interruption', (data) => { /* user barge-in (with event data) */ });
+session.on('tts:user_interrupt', () => { /* user barge-in (convenience, no data) */ });
+// Catch-all for any TTS streaming event
+session.on('tts:streaming-event', (data) => { /* data.event_type has the type */ });
+
+// Connection lifecycle
 session.on('close', (code, reason) => { /* connection closed */ });
 session.on('error', (err) => { /* error */ });
 ```
 
+## Audio WebSocket (Listen/Stream)
+
+The `listen` and `stream` verbs open a separate WebSocket connection from jambonz to your application, carrying raw audio. This is independent of the control WebSocket (`ws.jambonz.org`) — it uses the `audio.drachtio.org` subprotocol.
+
+### Receiving Audio in the Same Application
+
+Use `makeService.audio()` to register an audio WebSocket handler on the same server that handles the control pipe:
+
+```typescript
+import http from 'http';
+import { createEndpoint } from '@jambonz/sdk/websocket';
+
+const server = http.createServer();
+const makeService = createEndpoint({ server, port: 3000 });
+
+// Control pipe — handles call sessions
+const svc = makeService({ path: '/' });
+
+// Audio pipe — receives listen/stream audio
+const audioSvc = makeService.audio({ path: '/audio-stream' });
+
+svc.on('session:new', (session) => {
+  session
+    .answer()
+    .say({ text: 'Recording your audio.' })
+    .listen({
+      url: '/audio-stream',           // relative path — jambonz connects back to same server
+      sampleRate: 16000,
+      mixType: 'mono',
+      metadata: { purpose: 'recording' },
+    })
+    .hangup()
+    .send();
+});
+
+audioSvc.on('connection', (stream) => {
+  console.log(`Audio from call ${stream.callSid}, rate=${stream.sampleRate}`);
+  console.log('Metadata:', stream.metadata);
+
+  stream.on('audio', (pcm: Buffer) => {
+    // L16 PCM binary frames
+  });
+
+  stream.on('close', () => {
+    console.log('Audio stream closed');
+  });
+});
+```
+
+### AudioStream API
+
+The `stream` object in the `connection` event is an `AudioStream` instance:
+
+**Properties**: `metadata` (initial JSON), `callSid`, `sampleRate`
+
+**Events**:
+- `audio` — L16 PCM binary frame (`Buffer`)
+- `dtmf` — `{digit, duration}` (only if `passDtmf: true` on listen verb)
+- `playDone` — `{id}` (after non-streaming playAudio completes)
+- `mark` — `{name, event}` where event is `'playout'` or `'cleared'`
+- `close` — `(code, reason)`
+- `error` — `(err)`
+
+### Sending Audio Back (Bidirectional)
+
+The listen verb supports bidirectional audio. There are two modes, controlled by the `bidirectionalAudio.streaming` option on the listen verb.
+
+**Non-streaming mode** (`streaming: false`, the default) — send complete audio clips as base64:
+
+```typescript
+stream.playAudio(base64Content, {
+  audioContentType: 'raw',   // or 'wav'
+  sampleRate: 16000,
+  id: 'greeting',            // optional — returned in playDone event
+  queuePlay: true,           // true: queue after current; false: interrupt (default)
+});
+
+stream.on('playDone', (evt) => {
+  console.log(`Finished playing: ${evt.id}`);
+});
+```
+
+Up to 10 playAudio commands can be queued simultaneously.
+
+**Streaming mode** (`streaming: true`) — send raw binary PCM frames directly:
+
+```typescript
+// In the listen verb config:
+// bidirectionalAudio: { enabled: true, streaming: true, sampleRate: 16000 }
+
+stream.on('audio', (pcm) => {
+  // Echo audio back (or send processed/generated audio)
+  stream.sendAudio(pcm);
+});
+```
+
+### Marks (Synchronization Markers)
+
+Marks let you track when streamed audio has been played out to the caller. They work **only with bidirectional streaming mode** — you must enable `bidirectionalAudio: { enabled: true, streaming: true }` on the listen verb.
+
+The pattern is: stream audio via `sendAudio()`, then send a mark. When all the audio sent before the mark finishes playing out, jambonz sends back a mark event with `event: 'playout'`. This is how you know the caller has heard a specific chunk of audio.
+
+```typescript
+// Listen verb must enable bidirectional streaming for marks to work
+session
+  .listen({
+    url: '/audio',
+    actionHook: '/listen-done',
+    bidirectionalAudio: {
+      enabled: true,
+      streaming: true,
+      sampleRate: 8000,
+    },
+  })
+  .send();
+
+// In the audio handler:
+audioSvc.on('connection', (stream) => {
+  // Stream audio, then mark a sync point
+  stream.sendAudio(pcmBuffer);
+  stream.sendMark('chunk-1');   // fires 'playout' when audio above finishes playing
+
+  stream.sendAudio(morePcm);
+  stream.sendMark('chunk-2');   // fires 'playout' when this audio finishes
+
+  // Listen for mark events
+  stream.on('mark', (evt) => {
+    // evt.name = 'chunk-1' or 'chunk-2'
+    // evt.event = 'playout' (audio played) or 'cleared' (mark was cleared)
+  });
+
+  // Clear all pending marks (unplayed marks get event='cleared')
+  stream.clearMarks();
+});
+```
+
+**Important**: Without `bidirectionalAudio.streaming: true`, marks are accepted but never fire — there is no playout buffer to sync against. This is the most common mistake when marks appear to silently fail.
+
+### Other Commands
+
+```typescript
+stream.killAudio();           // Stop playback, flush buffer
+stream.disconnect();          // Close connection, end listen verb
+stream.sendMark('sync-pt');   // Insert synchronization marker
+stream.clearMarks();          // Clear all pending markers
+stream.close();               // Close the WebSocket
+```
+
 ## Recording
 
 jambonz supports SIPREC-based call recording. Recording is controlled mid-call via inject commands (WebSocket) or future REST API extensions.
@@ -285,14 +698,139 @@ Key resources:
 - **Calls** — Create outbound calls, query active calls, modify in-progress calls (redirect, whisper, mute, hangup)
 - **Messages** — Send SMS/MMS messages
 
+## Code Structure
+
+### Single File (default)
+
+For simple applications with 1-2 routes, put everything in a single file. This is the default for all examples in this repo and is perfectly suitable for production use.
+
+### Multi-File with Routes Directory
+
+For applications with 3+ routes or significant per-route logic, split into a `src/` directory with a routes folder:
+
+```
+src/
+  app.ts              ← entry point: server setup, route registration
+  routes/
+    incoming.ts       ← handler for one endpoint/path
+    hold-music.ts
+    queue-exit.ts
+```
+
+**Webhook pattern** — each route file exports an Express route handler:
+
+```typescript
+// src/routes/incoming.ts
+import type { Request, Response } from 'express';
+import { WebhookResponse } from '@jambonz/sdk/webhook';
+
+export default function incoming(_req: Request, res: Response) {
+  const jambonz = new WebhookResponse();
+  jambonz
+    .say({ text: 'Thank you for calling. Please hold.' })
+    .enqueue({ name: 'support', waitHook: '/hold-music', actionHook: '/queue-exit' });
+  res.json(jambonz);
+}
+```
+
+```typescript
+// src/app.ts
+import express from 'express';
+import incoming from './routes/incoming.js';
+import holdMusic from './routes/hold-music.js';
+import queueExit from './routes/queue-exit.js';
+
+const app = express();
+app.use(express.json());
+
+app.post('/incoming', incoming);
+app.post('/hold-music', holdMusic);
+app.post('/queue-exit', queueExit);
+
+app.listen(3000, () => console.log('Listening on port 3000'));
+```
+
+**WebSocket pattern** — there are two cases to consider:
+
+1. **Multiple services** (different `makeService({ path })` calls — each path gets its own `session:new`). Each route file exports a function that takes a session:
+
+```typescript
+// src/routes/caller.ts
+import type { Session } from '@jambonz/sdk/websocket';
+
+export default function caller(session: Session) {
+  session
+    .say({ text: 'Please hold.' })
+    .enqueue({ name: 'support', waitHook: '/hold-music', actionHook: '/queue-exit' })
+    .send();
+}
+```
+
+```typescript
+// src/app.ts
+import http from 'http';
+import { createEndpoint } from '@jambonz/sdk/websocket';
+import caller from './routes/caller.js';
+import agent from './routes/agent.js';
+
+const server = http.createServer();
+const makeService = createEndpoint({ server, port: 3000 });
+
+makeService({ path: '/incoming' }).on('session:new', (session) => caller(session));
+makeService({ path: '/agent' }).on('session:new', (session) => agent(session));
+```
+
+2. **Multiple actionHook handlers on one session** — extract handler functions, but register them all within `session:new`:
+
+```typescript
+// src/routes/echo-handler.ts
+import type { Session } from '@jambonz/sdk/websocket';
+
+export default function echoHandler(session: Session, evt: Record<string, any>) {
+  if (evt.reason === 'speechDetected') {
+    const text = evt.speech?.alternatives?.[0]?.transcript || 'nothing';
+    session.say({ text: `You said: ${text}` })
+      .gather({ input: ['speech'], actionHook: '/echo', timeout: 10 })
+      .reply();
+  } else {
+    session.gather({ input: ['speech'], actionHook: '/echo', timeout: 10,
+      say: { text: 'I didn\'t hear anything. Try again.' } }).reply();
+  }
+}
+```
+
+```typescript
+// src/app.ts — wire it up
+svc.on('session:new', (session) => {
+  session.on('/echo', (evt) => echoHandler(session, evt));
+  session.gather({ input: ['speech'], actionHook: '/echo', timeout: 10,
+    say: { text: 'Say something.' } }).send();
+});
+```
+
+### When to Split
+
+- **1-2 routes, simple logic** → single file
+- **3+ routes or substantial per-route logic** → `src/app.ts` + `src/routes/`
+- **Shared config, prompts, or utilities** → `src/config.ts`, `src/prompts.ts`, etc.
+
+When in doubt, start with a single file. It's easy to split later.
+
 ## Examples
 
 Complete working examples are in the `examples/` directory:
 - **hello-world** — Minimal greeting (webhook + WebSocket)
+- **echo** — Speech echo using gather with actionHook pattern (webhook + WebSocket). The canonical example for understanding actionHook event handling.
 - **ivr-menu** — Interactive menu with speech and DTMF input (webhook)
-- **voice-agent** — LLM-powered conversational AI (webhook + WebSocket)
+- **dial** — Simple outbound dial to a phone number (webhook)
+- **listen-record** — Record audio using the listen verb to stream to a WebSocket (webhook)
+- **voice-agent** — LLM-powered conversational AI with tool calls (webhook + WebSocket)
+- **openai-realtime** — OpenAI Realtime API voice agent with function calling (WebSocket)
+- **deepgram-voice-agent** — Deepgram Voice Agent API with function calling (WebSocket)
+- **llm-streaming** — Anthropic LLM with TTS token streaming and barge-in (WebSocket)
 - **queue-with-hold** — Call queue with hold music and agent dequeue (webhook + WebSocket)
 - **call-recording** — Mid-call recording control via REST API and inject commands (webhook + WebSocket)
+- **realtime-translator** — Bridges two parties with real-time speech translation using STT, Google Translate, and TTS dub tracks. Multi-file example with `src/routes/` structure (WebSocket)
 
 ## Key Concepts