qualia-framework 2.2.0 → 2.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (30) hide show
  1. package/framework/hooks/confirm-delete.sh +2 -2
  2. package/framework/hooks/migration-validate.sh +2 -2
  3. package/framework/hooks/pre-commit.sh +4 -4
  4. package/framework/hooks/pre-deploy-gate.sh +6 -21
  5. package/framework/hooks/session-context-loader.sh +1 -1
  6. package/framework/install.sh +9 -4
  7. package/framework/qualia-engine/VERSION +1 -1
  8. package/framework/qualia-engine/templates/projects/ai-agent.md +1 -1
  9. package/framework/qualia-engine/templates/projects/voice-agent.md +4 -4
  10. package/framework/qualia-engine/templates/roadmap.md +10 -0
  11. package/framework/qualia-engine/templates/state.md +3 -0
  12. package/framework/qualia-engine/workflows/new-project.md +22 -21
  13. package/framework/skills/client-handoff/SKILL.md +125 -0
  14. package/framework/skills/collab-onboard/SKILL.md +111 -0
  15. package/framework/skills/docs-lookup/SKILL.md +4 -3
  16. package/framework/skills/learn/SKILL.md +1 -1
  17. package/framework/skills/mobile-expo/SKILL.md +117 -4
  18. package/framework/skills/openrouter-agent/SKILL.md +922 -0
  19. package/framework/skills/qualia/SKILL.md +11 -5
  20. package/framework/skills/qualia-audit-milestone/SKILL.md +5 -2
  21. package/framework/skills/qualia-complete-milestone/SKILL.md +9 -5
  22. package/framework/skills/qualia-execute-phase/SKILL.md +5 -2
  23. package/framework/skills/qualia-help/SKILL.md +96 -62
  24. package/framework/skills/qualia-new-project/SKILL.md +184 -62
  25. package/framework/skills/qualia-plan-phase/SKILL.md +5 -2
  26. package/framework/skills/qualia-verify-work/SKILL.md +14 -4
  27. package/framework/skills/qualia-workflow/SKILL.md +5 -5
  28. package/framework/skills/ship/SKILL.md +32 -6
  29. package/framework/skills/voice-agent/SKILL.md +1174 -269
  30. package/package.json +1 -1
@@ -1,24 +1,26 @@
1
1
  ---
2
2
  name: voice-agent
3
- description: Build complete voice AI agents - design conversational flow, create VAPI assistant, integrate Supabase, deploy. Full voice agent development.
4
- tags: [voice, vapi, ai-agent, supabase, elevenlabs]
3
+ description: "Build complete voice AI agents with Retell AI + ElevenLabs — design conversational flow, create Retell agent, integrate Supabase backend, configure webhooks, deploy and test. Use this skill whenever the user says 'build voice agent', 'create voice assistant', 'voice AI', 'phone bot', 'retell agent', or wants to build any voice-based AI system. Also trigger on 'retell', 'elevenlabs', 'voice webhook', 'call agent'."
4
+ tags: [voice, retell, elevenlabs, ai-agent, supabase]
5
5
  ---
6
6
 
7
7
  # Voice Agent Builder
8
8
 
9
- Build complete voice AI agents from design to deployment.
9
+ Build complete voice AI agents from design to deployment using Retell AI (call orchestration) + ElevenLabs (TTS/voice cloning) + Supabase (backend).
10
+
11
+ **Announce at start:** "Activating voice agent builder. Let me set up your Retell AI + ElevenLabs agent."
10
12
 
11
13
  ## Pipeline
12
14
 
13
15
  ```
14
16
  1. Design Conversation Flow
15
-
16
- 2. Create VAPI Assistant
17
-
17
+ |
18
+ 2. Create Retell Agent
19
+ |
18
20
  3. Setup Supabase Backend
19
-
21
+ |
20
22
  4. Configure Webhooks
21
-
23
+ |
22
24
  5. Deploy & Test
23
25
  ```
24
26
 
@@ -29,375 +31,1269 @@ Say: "build voice agent for [use case]"
29
31
  Example: "build voice agent for appointment scheduling"
30
32
 
31
33
  I will:
32
- 1. Design the conversation flow
33
- 2. Create VAPI assistant configuration
34
- 3. Set up Supabase tables for state
35
- 4. Configure webhooks
36
- 5. Deploy and test
34
+ 1. Design the conversation flow (state machine + flow diagram)
35
+ 2. Create Retell agent with ElevenLabs voice
36
+ 3. Set up Supabase tables (call_logs, agent_state, domain tables)
37
+ 4. Build webhook handler as Supabase Edge Function
38
+ 5. Assign phone number and test
39
+
40
+ ## Environment Variables Needed
41
+
42
+ ```env
43
+ # Retell AI
44
+ RETELL_API_KEY=key_...
45
+
46
+ # ElevenLabs
47
+ ELEVENLABS_API_KEY=xi_...
48
+
49
+ # Supabase (auto-available in Edge Functions)
50
+ SUPABASE_URL=https://xxx.supabase.co
51
+ SUPABASE_SERVICE_ROLE_KEY=eyJ...
52
+
53
+ # LLM (if using OpenRouter for custom LLM)
54
+ OPENROUTER_API_KEY=sk-or-...
55
+ ```
56
+
57
+ ---
37
58
 
38
59
  ## Stage 1: Design Conversation Flow
39
60
 
40
61
  ### Flow Diagram
62
+
41
63
  ```
42
- ┌──────────────────┐
43
- Call Starts
44
- └────────┬─────────┘
45
-
46
- ┌──────────────────┐
47
- Greeting
48
- "Hello, how can
49
- I help you?" │
50
- └────────┬─────────┘
51
-
52
- ┌──────────────────┐
53
- │ Intent Detection │
54
- - Book appointment│
55
- - Check status │
56
- - Other inquiry │
57
- └────────┬─────────┘
58
-
59
- ┌─────┴─────┐
60
- ↓ ↓
61
- ┌──────┐ ┌──────┐
62
- │ Book │ │ Help │
63
- └──────┘ └──────┘
64
+ +--------------------+
65
+ | Call Starts |
66
+ +--------+-----------+
67
+ |
68
+ +--------v-----------+
69
+ | Greeting |
70
+ | "Hello, thanks for |
71
+ | calling. How can |
72
+ | I help you?" |
73
+ +--------+-----------+
74
+ |
75
+ +--------v-----------+
76
+ | Intent Detection |
77
+ | - Book appointment |
78
+ | - Check status |
79
+ | - General inquiry |
80
+ | - Transfer to human |
81
+ +--------+-----------+
82
+ |
83
+ +-----+------+------+
84
+ | | |
85
+ +--v---+ +----v--+ +-v--------+
86
+ | Book | | Check | | Transfer |
87
+ +--+---+ +----+--+ +----------+
88
+ | |
89
+ +--v-----------v--+
90
+ | Confirm & |
91
+ | End Call |
92
+ +-----------------+
64
93
  ```
65
94
 
66
95
  ### State Machine
96
+
67
97
  ```typescript
68
- const conversationStates = {
98
+ type ConversationState =
99
+ | 'GREETING'
100
+ | 'INTENT_DETECTION'
101
+ | 'BOOKING'
102
+ | 'COLLECT_DATE'
103
+ | 'COLLECT_TIME'
104
+ | 'COLLECT_NAME'
105
+ | 'CONFIRM_BOOKING'
106
+ | 'STATUS_CHECK'
107
+ | 'GENERAL_INQUIRY'
108
+ | 'TRANSFER'
109
+ | 'FAREWELL';
110
+
111
+ const conversationFlow: Record<ConversationState, {
112
+ next: ConversationState[];
113
+ actions: string[];
114
+ }> = {
69
115
  GREETING: {
70
116
  next: ['INTENT_DETECTION'],
71
- actions: ['playGreeting']
117
+ actions: ['playGreeting'],
72
118
  },
73
119
  INTENT_DETECTION: {
74
- next: ['BOOKING', 'STATUS_CHECK', 'TRANSFER'],
75
- actions: ['detectIntent']
120
+ next: ['BOOKING', 'STATUS_CHECK', 'GENERAL_INQUIRY', 'TRANSFER'],
121
+ actions: ['detectIntent'],
76
122
  },
77
123
  BOOKING: {
78
- next: ['COLLECT_DATE', 'COLLECT_TIME', 'CONFIRM'],
79
- actions: ['startBooking']
124
+ next: ['COLLECT_DATE'],
125
+ actions: ['startBookingFlow'],
126
+ },
127
+ COLLECT_DATE: {
128
+ next: ['COLLECT_TIME'],
129
+ actions: ['askForDate', 'validateDate'],
130
+ },
131
+ COLLECT_TIME: {
132
+ next: ['COLLECT_NAME'],
133
+ actions: ['askForTime', 'checkAvailability'],
134
+ },
135
+ COLLECT_NAME: {
136
+ next: ['CONFIRM_BOOKING'],
137
+ actions: ['askForName'],
138
+ },
139
+ CONFIRM_BOOKING: {
140
+ next: ['FAREWELL', 'COLLECT_DATE'], // retry if rejected
141
+ actions: ['readBackDetails', 'bookAppointment'],
142
+ },
143
+ STATUS_CHECK: {
144
+ next: ['FAREWELL', 'INTENT_DETECTION'],
145
+ actions: ['lookupAppointment', 'readStatus'],
146
+ },
147
+ GENERAL_INQUIRY: {
148
+ next: ['FAREWELL', 'INTENT_DETECTION'],
149
+ actions: ['answerQuestion'],
150
+ },
151
+ TRANSFER: {
152
+ next: ['FAREWELL'],
153
+ actions: ['transferToHuman'],
154
+ },
155
+ FAREWELL: {
156
+ next: [],
157
+ actions: ['endCall'],
80
158
  },
81
- // ...
82
159
  };
83
160
  ```
84
161
 
85
- ## Stage 2: Create VAPI Assistant
162
+ ### System Prompt Template
86
163
 
87
- ### Assistant Configuration
88
- ```typescript
89
- const assistant = {
90
- name: "Appointment Scheduler",
91
- model: {
92
- provider: "anthropic",
93
- model: "claude-3-5-sonnet-20241022",
94
- temperature: 0.7,
95
- systemPrompt: `You are a friendly appointment scheduling assistant.
96
- Your job is to help callers book appointments.
164
+ ```
165
+ You are a friendly and professional [ROLE] for [COMPANY].
166
+ Your job is to help callers [PRIMARY_TASK].
97
167
 
98
168
  ## Capabilities
99
- - Book new appointments
100
- - Check existing appointments
101
- - Reschedule appointments
102
- - Answer questions about services
103
-
104
- ## Conversation Flow
105
- 1. Greet the caller warmly
106
- 2. Ask how you can help
107
- 3. Collect necessary information
108
- 4. Confirm the action
109
- 5. End the call politely
110
-
111
- ## Important Rules
112
- - Always confirm before booking
113
- - Validate dates and times
114
- - Collect caller's phone for confirmation
115
- - Be concise but friendly`
169
+ - [Capability 1]
170
+ - [Capability 2]
171
+ - [Capability 3]
172
+
173
+ ## Conversation Rules
174
+ 1. Keep responses to 1-2 sentences. Callers hate long monologues.
175
+ 2. Always confirm critical info back to the caller before acting.
176
+ 3. If unsure, ask a clarifying question rather than guessing.
177
+ 4. Never make up information. If you don't know, say so.
178
+ 5. Be warm but efficient — respect the caller's time.
179
+ 6. If the caller wants a human, transfer immediately. Don't argue.
180
+
181
+ ## Available Tools
182
+ - book_appointment: Book a new appointment
183
+ - check_availability: Check open time slots
184
+ - lookup_appointment: Find existing appointment by phone
185
+
186
+ ## Escalation
187
+ If the caller is frustrated, angry, or asks for a manager:
188
+ → Apologize briefly, then transfer to a human operator.
189
+ ```
190
+
191
+ ---
192
+
193
+ ## Stage 2: Create Retell Agent
194
+
195
+ ### Agent Configuration
196
+
197
+ ```typescript
198
+ interface RetellAgentConfig {
199
+ agent_name: string;
200
+ voice_id: string; // ElevenLabs voice ID
201
+ response_engine: {
202
+ type: 'retell-llm';
203
+ llm_id: string; // Created separately via Retell LLM API
204
+ } | {
205
+ type: 'custom-llm';
206
+ llm_websocket_url: string; // Your own LLM WebSocket endpoint
207
+ };
208
+ language: string; // 'en-US', 'ar', 'el', etc.
209
+ voice_temperature: number; // 0-2, controls voice expressiveness
210
+ voice_speed: number; // 0.5-2, speech rate
211
+ responsiveness: number; // 0-1, how quickly agent responds
212
+ interruption_sensitivity: number; // 0-1, how easily caller can interrupt
213
+ enable_backchannel: boolean; // "uh huh", "I see" filler words
214
+ backchannel_frequency: number; // 0-1
215
+ reminder_trigger_ms: number; // Silence before nudge (default 10000)
216
+ reminder_max_count: number; // Max nudges before ending (default 2)
217
+ ambient_sound: string | null; // 'coffee-shop', 'convention-hall', 'summer-outdoor', 'mountain-spring', 'static-noise', 'call-center' or null
218
+ begin_message: string | null; // First message (null = wait for caller)
219
+ webhook_url: string; // Your webhook endpoint
220
+ post_call_analysis_data: Array<{
221
+ type: 'string' | 'enum' | 'boolean';
222
+ name: string;
223
+ description: string;
224
+ examples?: string[];
225
+ options?: string[];
226
+ }>;
227
+ }
228
+ ```
229
+
230
+ ### Create Agent via Retell API
231
+
232
+ ```typescript
233
+ // Create the Retell LLM first (defines the brain)
234
+ const llmResponse = await fetch('https://api.retellai.com/v2/create-retell-llm', {
235
+ method: 'POST',
236
+ headers: {
237
+ 'Authorization': `Bearer ${RETELL_API_KEY}`,
238
+ 'Content-Type': 'application/json',
116
239
  },
117
- voice: {
118
- provider: "11labs",
119
- voiceId: "21m00Tcm4TlvDq8ikWAM", // Rachel
120
- stability: 0.5,
121
- similarityBoost: 0.75
240
+ body: JSON.stringify({
241
+ model: 'claude-3-5-sonnet', // Or use 'gpt-4o', 'claude-3-5-haiku'
242
+ general_prompt: `You are a friendly appointment scheduling assistant for Qualia Solutions.
243
+ Your job is to help callers book, check, or reschedule appointments.
244
+
245
+ Rules:
246
+ - Keep responses to 1-2 sentences max
247
+ - Always confirm dates and times back to the caller
248
+ - Be warm but efficient`,
249
+ begin_message: 'Hello! Thanks for calling Qualia Solutions. How can I help you today?',
250
+ general_tools: [
251
+ {
252
+ type: 'end_call',
253
+ name: 'end_call',
254
+ description: 'End the call when the conversation is complete',
255
+ },
256
+ {
257
+ type: 'custom',
258
+ name: 'book_appointment',
259
+ description: 'Book a new appointment for the caller',
260
+ parameters: {
261
+ type: 'object',
262
+ properties: {
263
+ date: { type: 'string', description: 'Appointment date in YYYY-MM-DD format' },
264
+ time: { type: 'string', description: 'Appointment time in HH:MM format (24h)' },
265
+ service: { type: 'string', description: 'Type of service requested' },
266
+ caller_name: { type: 'string', description: 'Full name of the caller' },
267
+ },
268
+ required: ['date', 'time', 'service', 'caller_name'],
269
+ },
270
+ url: 'https://YOUR_PROJECT.supabase.co/functions/v1/retell-tool-handler',
271
+ speak_during_execution: true,
272
+ speak_after_execution: true,
273
+ execution_message_description: 'Let the caller know you are checking availability and booking their appointment.',
274
+ },
275
+ {
276
+ type: 'custom',
277
+ name: 'check_availability',
278
+ description: 'Check available appointment slots for a given date',
279
+ parameters: {
280
+ type: 'object',
281
+ properties: {
282
+ date: { type: 'string', description: 'Date to check in YYYY-MM-DD format' },
283
+ },
284
+ required: ['date'],
285
+ },
286
+ url: 'https://YOUR_PROJECT.supabase.co/functions/v1/retell-tool-handler',
287
+ speak_during_execution: true,
288
+ speak_after_execution: true,
289
+ execution_message_description: 'Let the caller know you are checking available times.',
290
+ },
291
+ ],
292
+ states: [
293
+ {
294
+ name: 'greeting',
295
+ state_prompt: 'Greet the caller warmly and ask how you can help.',
296
+ transitions: [
297
+ {
298
+ name: 'booking_intent',
299
+ description: 'Caller wants to book an appointment',
300
+ destination: 'collect_info',
301
+ },
302
+ {
303
+ name: 'check_intent',
304
+ description: 'Caller wants to check an existing appointment',
305
+ destination: 'status_check',
306
+ },
307
+ {
308
+ name: 'other_intent',
309
+ description: 'Caller has a general question',
310
+ destination: 'general_help',
311
+ },
312
+ ],
313
+ },
314
+ {
315
+ name: 'collect_info',
316
+ state_prompt: 'Collect the date, time, service type, and caller name. Use check_availability before booking. Then use book_appointment to confirm.',
317
+ transitions: [
318
+ {
319
+ name: 'booking_complete',
320
+ description: 'Appointment has been booked successfully',
321
+ destination: 'farewell',
322
+ },
323
+ ],
324
+ },
325
+ {
326
+ name: 'status_check',
327
+ state_prompt: 'Ask for the caller phone number or name to look up their appointment.',
328
+ transitions: [
329
+ {
330
+ name: 'status_done',
331
+ description: 'Status has been communicated to caller',
332
+ destination: 'farewell',
333
+ },
334
+ ],
335
+ },
336
+ {
337
+ name: 'general_help',
338
+ state_prompt: 'Answer the caller question. If you cannot help, offer to transfer to a human.',
339
+ transitions: [
340
+ {
341
+ name: 'help_done',
342
+ description: 'Question answered',
343
+ destination: 'farewell',
344
+ },
345
+ ],
346
+ },
347
+ {
348
+ name: 'farewell',
349
+ state_prompt: 'Thank the caller and end the call politely. Use end_call tool.',
350
+ transitions: [],
351
+ },
352
+ ],
353
+ starting_state: 'greeting',
354
+ }),
355
+ });
356
+
357
+ const llmData = await llmResponse.json();
358
+ const llmId = llmData.llm_id;
359
+
360
+ // Now create the agent (connects voice + LLM)
361
+ const agentResponse = await fetch('https://api.retellai.com/v2/create-agent', {
362
+ method: 'POST',
363
+ headers: {
364
+ 'Authorization': `Bearer ${RETELL_API_KEY}`,
365
+ 'Content-Type': 'application/json',
122
366
  },
123
- firstMessage: "Hello! Thanks for calling. How can I help you today?",
124
- endCallMessage: "Thank you for calling. Have a great day!",
125
- transcriber: {
126
- provider: "deepgram",
127
- model: "nova-2",
128
- language: "en"
367
+ body: JSON.stringify({
368
+ agent_name: 'Qualia Appointment Scheduler',
369
+ voice_id: '21m00Tcm4TlvDq8ikWAM', // ElevenLabs Rachel
370
+ response_engine: {
371
+ type: 'retell-llm',
372
+ llm_id: llmId,
373
+ },
374
+ language: 'en-US',
375
+ voice_temperature: 1,
376
+ voice_speed: 1,
377
+ responsiveness: 1,
378
+ interruption_sensitivity: 1,
379
+ enable_backchannel: true,
380
+ backchannel_frequency: 0.8,
381
+ reminder_trigger_ms: 10000,
382
+ reminder_max_count: 2,
383
+ ambient_sound: null,
384
+ webhook_url: 'https://YOUR_PROJECT.supabase.co/functions/v1/retell-webhook',
385
+ post_call_analysis_data: [
386
+ {
387
+ type: 'enum',
388
+ name: 'call_outcome',
389
+ description: 'The outcome of the call',
390
+ options: ['appointment_booked', 'appointment_checked', 'general_inquiry', 'transferred', 'abandoned'],
391
+ },
392
+ {
393
+ type: 'string',
394
+ name: 'call_summary',
395
+ description: 'A brief 1-2 sentence summary of what happened on the call',
396
+ },
397
+ {
398
+ type: 'boolean',
399
+ name: 'caller_satisfied',
400
+ description: 'Whether the caller seemed satisfied with the interaction',
401
+ },
402
+ ],
403
+ }),
404
+ });
405
+
406
+ const agentData = await agentResponse.json();
407
+ console.log('Agent created:', agentData.agent_id);
408
+ ```
409
+
410
+ ### Assign Phone Number
411
+
412
+ ```typescript
413
+ // Purchase and assign a phone number
414
+ const phoneResponse = await fetch('https://api.retellai.com/v2/create-phone-number', {
415
+ method: 'POST',
416
+ headers: {
417
+ 'Authorization': `Bearer ${RETELL_API_KEY}`,
418
+ 'Content-Type': 'application/json',
129
419
  },
130
- serverUrl: "https://your-app.com/api/vapi-webhook"
131
- };
420
+ body: JSON.stringify({
421
+ agent_id: agentData.agent_id,
422
+ area_code: 357, // Cyprus area code (adjust per project)
423
+ }),
424
+ });
425
+
426
+ const phoneData = await phoneResponse.json();
427
+ console.log('Phone number assigned:', phoneData.phone_number);
428
+ ```
429
+
430
+ ### Import Existing Number (Telnyx/Twilio)
431
+
432
+ ```typescript
433
+ // If bringing your own number via Telnyx
434
+ const importResponse = await fetch('https://api.retellai.com/v2/import-phone-number', {
435
+ method: 'POST',
436
+ headers: {
437
+ 'Authorization': `Bearer ${RETELL_API_KEY}`,
438
+ 'Content-Type': 'application/json',
439
+ },
440
+ body: JSON.stringify({
441
+ agent_id: agentData.agent_id,
442
+ phone_number: '+35712345678',
443
+ telephony_provider: 'telnyx', // 'telnyx' | 'twilio' | 'vonage'
444
+ telephony_credentials: {
445
+ telnyx_api_key: Deno.env.get('TELNYX_API_KEY'),
446
+ telnyx_app_connection_id: Deno.env.get('TELNYX_CONNECTION_ID'),
447
+ },
448
+ }),
449
+ });
450
+ ```
451
+
452
+ ### Update Agent (PATCH)
453
+
454
+ ```typescript
455
+ // Update an existing agent
456
+ await fetch(`https://api.retellai.com/v2/update-agent/${agentId}`, {
457
+ method: 'PATCH',
458
+ headers: {
459
+ 'Authorization': `Bearer ${RETELL_API_KEY}`,
460
+ 'Content-Type': 'application/json',
461
+ },
462
+ body: JSON.stringify({
463
+ voice_id: 'NEW_VOICE_ID',
464
+ webhook_url: 'https://YOUR_PROJECT.supabase.co/functions/v1/retell-webhook',
465
+ }),
466
+ });
132
467
  ```
133
468
 
134
- ### Create via VAPI
135
- > Use VAPI MCP tools (available via settings.json) or direct VAPI API via curl/WebFetch.
136
- > Example: `curl -X POST https://api.vapi.ai/assistant -H "Authorization: Bearer $VAPI_API_KEY" -d '<config>'`
469
+ ---
137
470
 
138
471
  ## Stage 3: Setup Supabase Backend
139
472
 
140
473
  ### Database Schema
474
+
141
475
  ```sql
142
- -- Appointments table
476
+ -- Migration: supabase/migrations/YYYYMMDD_voice_agent_tables.sql
477
+
478
+ -- Call logs — every call tracked
479
+ CREATE TABLE call_logs (
480
+ id UUID DEFAULT gen_random_uuid() PRIMARY KEY,
481
+ call_id TEXT UNIQUE NOT NULL, -- Retell call ID
482
+ agent_id TEXT NOT NULL, -- Retell agent ID
483
+ from_number TEXT, -- Caller phone
484
+ to_number TEXT, -- Agent phone
485
+ direction TEXT NOT NULL DEFAULT 'inbound'
486
+ CHECK (direction IN ('inbound', 'outbound', 'web')),
487
+ call_status TEXT NOT NULL DEFAULT 'started'
488
+ CHECK (call_status IN ('started', 'ended', 'error')),
489
+ disconnection_reason TEXT, -- 'user_hangup', 'agent_hangup', 'call_transfer', 'error', etc.
490
+ duration_ms INTEGER,
491
+ transcript TEXT,
492
+ transcript_object JSONB, -- Full structured transcript
493
+ call_analysis JSONB, -- Post-call analysis results
494
+ recording_url TEXT,
495
+ metadata JSONB DEFAULT '{}',
496
+ started_at TIMESTAMPTZ DEFAULT NOW(),
497
+ ended_at TIMESTAMPTZ,
498
+ created_at TIMESTAMPTZ DEFAULT NOW()
499
+ );
500
+
501
+ -- Agent state — persistent context between calls per phone number
502
+ CREATE TABLE agent_state (
503
+ id UUID DEFAULT gen_random_uuid() PRIMARY KEY,
504
+ phone_number TEXT UNIQUE NOT NULL,
505
+ agent_id TEXT NOT NULL,
506
+ last_call_id TEXT REFERENCES call_logs(call_id),
507
+ context JSONB DEFAULT '{}', -- Arbitrary state (name, preferences, history)
508
+ call_count INTEGER DEFAULT 0,
509
+ first_call_at TIMESTAMPTZ DEFAULT NOW(),
510
+ last_call_at TIMESTAMPTZ DEFAULT NOW(),
511
+ updated_at TIMESTAMPTZ DEFAULT NOW()
512
+ );
513
+
514
+ -- Appointments — example domain table
143
515
  CREATE TABLE appointments (
144
516
  id UUID DEFAULT gen_random_uuid() PRIMARY KEY,
517
+ call_id TEXT REFERENCES call_logs(call_id),
145
518
  customer_phone TEXT NOT NULL,
146
519
  customer_name TEXT,
147
520
  service_type TEXT NOT NULL,
148
521
  appointment_date DATE NOT NULL,
149
522
  appointment_time TIME NOT NULL,
150
- status TEXT DEFAULT 'scheduled' CHECK (status IN ('scheduled', 'confirmed', 'cancelled', 'completed')),
523
+ status TEXT DEFAULT 'scheduled'
524
+ CHECK (status IN ('scheduled', 'confirmed', 'cancelled', 'completed', 'no_show')),
151
525
  notes TEXT,
526
+ reminder_sent BOOLEAN DEFAULT FALSE,
152
527
  created_at TIMESTAMPTZ DEFAULT NOW(),
153
528
  updated_at TIMESTAMPTZ DEFAULT NOW()
154
529
  );
155
530
 
156
- -- Call logs
157
- CREATE TABLE call_logs (
158
- id UUID DEFAULT gen_random_uuid() PRIMARY KEY,
159
- call_id TEXT UNIQUE NOT NULL,
160
- phone_number TEXT NOT NULL,
161
- duration_seconds INTEGER,
162
- transcript TEXT,
163
- outcome TEXT,
164
- created_at TIMESTAMPTZ DEFAULT NOW()
165
- );
166
-
167
- -- Agent state (for context between calls)
168
- CREATE TABLE agent_state (
169
- phone_number TEXT PRIMARY KEY,
170
- last_call_id TEXT,
171
- context JSONB DEFAULT '{}',
172
- updated_at TIMESTAMPTZ DEFAULT NOW()
173
- );
531
+ -- Indexes
532
+ CREATE INDEX idx_call_logs_call_id ON call_logs(call_id);
533
+ CREATE INDEX idx_call_logs_from_number ON call_logs(from_number);
534
+ CREATE INDEX idx_call_logs_started_at ON call_logs(started_at DESC);
535
+ CREATE INDEX idx_agent_state_phone ON agent_state(phone_number);
536
+ CREATE INDEX idx_appointments_date ON appointments(appointment_date);
537
+ CREATE INDEX idx_appointments_phone ON appointments(customer_phone);
174
538
 
175
- -- Enable RLS
176
- ALTER TABLE appointments ENABLE ROW LEVEL SECURITY;
539
+ -- Enable RLS on all tables
177
540
  ALTER TABLE call_logs ENABLE ROW LEVEL SECURITY;
178
541
  ALTER TABLE agent_state ENABLE ROW LEVEL SECURITY;
542
+ ALTER TABLE appointments ENABLE ROW LEVEL SECURITY;
179
543
 
180
- -- RLS Policies: service role only (called from edge functions)
181
- CREATE POLICY "Service role full access" ON appointments
182
- FOR ALL USING (auth.role() = 'service_role');
183
-
544
+ -- RLS Policies: service_role only (Edge Functions use service_role key)
545
+ -- These tables are written to by webhooks, not by end users directly.
184
546
  CREATE POLICY "Service role full access" ON call_logs
185
547
  FOR ALL USING (auth.role() = 'service_role');
186
548
 
187
549
  CREATE POLICY "Service role full access" ON agent_state
188
550
  FOR ALL USING (auth.role() = 'service_role');
551
+
552
+ CREATE POLICY "Service role full access" ON appointments
553
+ FOR ALL USING (auth.role() = 'service_role');
554
+
555
+ -- If you also need an admin dashboard with authenticated users:
556
+ -- CREATE POLICY "Admins can read call logs" ON call_logs
557
+ -- FOR SELECT USING (
558
+ -- EXISTS (
559
+ -- SELECT 1 FROM profiles
560
+ -- WHERE profiles.id = auth.uid()
561
+ -- AND profiles.role = 'admin'
562
+ -- )
563
+ -- );
564
+
565
+ -- Updated_at trigger
566
+ CREATE OR REPLACE FUNCTION update_updated_at()
567
+ RETURNS TRIGGER AS $$
568
+ BEGIN
569
+ NEW.updated_at = NOW();
570
+ RETURN NEW;
571
+ END;
572
+ $$ LANGUAGE plpgsql;
573
+
574
+ CREATE TRIGGER tr_appointments_updated
575
+ BEFORE UPDATE ON appointments
576
+ FOR EACH ROW EXECUTE FUNCTION update_updated_at();
577
+
578
+ CREATE TRIGGER tr_agent_state_updated
579
+ BEFORE UPDATE ON agent_state
580
+ FOR EACH ROW EXECUTE FUNCTION update_updated_at();
189
581
  ```
190
582
 
191
- ### API Functions
583
+ ---
584
+
585
+ ## Stage 4: Configure Webhooks
586
+
587
+ ### Retell Webhook Events
588
+
589
+ Retell sends POST requests to your `webhook_url` with these event types:
590
+
591
+ | Event | When | Key Data |
592
+ |-------|------|----------|
593
+ | `call_started` | Call begins | call_id, from_number, to_number, agent_id |
594
+ | `call_ended` | Call finishes | call_id, duration_ms, transcript, recording_url, disconnection_reason |
595
+ | `call_analyzed` | Post-call analysis complete | call_id, call_analysis (custom fields you defined) |
596
+ | `tool_call_invoked` | Agent calls a custom tool | tool_call_id, name, arguments |
597
+ | `tool_call_result` | Tool execution completed | tool_call_id, result |
598
+
599
+ ### Webhook Signature Verification
600
+
601
+ Retell signs webhooks using `x-retell-signature` header with HMAC-SHA256.
602
+
192
603
  ```typescript
193
- // supabase/functions/book-appointment/index.ts
194
- import { z } from "https://deno.land/x/zod/mod.ts"
604
+ import { createHmac } from "node:crypto";
195
605
 
196
- const BookingSchema = z.object({
197
- phone: z.string().min(10),
198
- date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/),
199
- time: z.string().regex(/^\d{2}:\d{2}$/),
200
- service: z.string().min(1),
201
- })
606
+ function verifyRetellSignature(
607
+ body: string,
608
+ signature: string | null,
609
+ apiKey: string
610
+ ): boolean {
611
+ if (!signature) return false;
612
+
613
+ const hmac = createHmac('sha256', apiKey);
614
+ hmac.update(body);
615
+ const expectedSignature = hmac.digest('base64');
616
+
617
+ return signature === expectedSignature;
618
+ }
619
+ ```
620
+
621
+ > **Important:** Retell uses the API key itself as the HMAC secret for webhook verification. Not a separate webhook secret.
622
+
623
+ ### Webhook Handler (Supabase Edge Function)
624
+
625
+ ```typescript
626
+ // supabase/functions/retell-webhook/index.ts
627
+ import { serve } from "https://deno.land/std@0.208.0/http/server.ts";
628
+ import { createClient } from "https://esm.sh/@supabase/supabase-js@2";
629
+ import { createHmac } from "node:crypto";
630
+ import { z } from "https://deno.land/x/zod@v3.22.4/mod.ts";
631
+
632
+ // --- Signature Verification ---
633
+ function verifyRetellSignature(body: string, signature: string | null): boolean {
634
+ if (!signature) return false;
635
+ const apiKey = Deno.env.get('RETELL_API_KEY')!;
636
+ const hmac = createHmac('sha256', apiKey);
637
+ hmac.update(body);
638
+ const expected = hmac.digest('base64');
639
+ return signature === expected;
640
+ }
202
641
 
203
- serve(async (req) => {
642
+ // --- Zod Schemas ---
643
+ const CallStartedSchema = z.object({
644
+ event: z.literal('call_started'),
645
+ call: z.object({
646
+ call_id: z.string(),
647
+ agent_id: z.string(),
648
+ from_number: z.string().optional(),
649
+ to_number: z.string().optional(),
650
+ direction: z.enum(['inbound', 'outbound', 'web']),
651
+ metadata: z.record(z.unknown()).optional(),
652
+ }),
653
+ });
654
+
655
+ const CallEndedSchema = z.object({
656
+ event: z.literal('call_ended'),
657
+ call: z.object({
658
+ call_id: z.string(),
659
+ agent_id: z.string(),
660
+ from_number: z.string().optional(),
661
+ to_number: z.string().optional(),
662
+ direction: z.enum(['inbound', 'outbound', 'web']),
663
+ duration_ms: z.number().optional(),
664
+ transcript: z.string().optional(),
665
+ transcript_object: z.array(z.unknown()).optional(),
666
+ recording_url: z.string().optional(),
667
+ disconnection_reason: z.string().optional(),
668
+ metadata: z.record(z.unknown()).optional(),
669
+ }),
670
+ });
671
+
672
+ const CallAnalyzedSchema = z.object({
673
+ event: z.literal('call_analyzed'),
674
+ call: z.object({
675
+ call_id: z.string(),
676
+ call_analysis: z.record(z.unknown()).optional(),
677
+ }),
678
+ });
679
+
680
+ // --- Main Handler ---
681
+ serve(async (req: Request) => {
682
+ // Only accept POST
683
+ if (req.method !== 'POST') {
684
+ return new Response('Method not allowed', { status: 405 });
685
+ }
686
+
687
+ const body = await req.text();
688
+
689
+ // 1. Verify webhook signature
690
+ const signature = req.headers.get('x-retell-signature');
691
+ if (!verifyRetellSignature(body, signature)) {
692
+ console.error('Invalid webhook signature');
693
+ return new Response('Unauthorized', { status: 401 });
694
+ }
695
+
696
+ // 2. Parse body
697
+ let payload: unknown;
204
698
  try {
205
- const raw = await req.json();
206
- const { phone, date, time, service } = BookingSchema.parse(raw);
699
+ payload = JSON.parse(body);
700
+ } catch {
701
+ return new Response('Invalid JSON', { status: 400 });
702
+ }
207
703
 
704
+ // 3. Initialize Supabase client (service role for DB writes)
208
705
  const supabase = createClient(
209
706
  Deno.env.get('SUPABASE_URL')!,
210
707
  Deno.env.get('SUPABASE_SERVICE_ROLE_KEY')!
211
708
  );
212
709
 
213
- // Check availability
214
- const { data: existing } = await supabase
215
- .from('appointments')
216
- .select('id')
217
- .eq('appointment_date', date)
218
- .eq('appointment_time', time)
219
- .single();
220
-
221
- if (existing) {
222
- return new Response(JSON.stringify({
223
- success: false,
224
- message: "That time slot is already booked."
225
- }));
226
- }
710
+ // 4. Route by event type
711
+ const event = (payload as { event?: string }).event;
712
+
713
+ try {
714
+ switch (event) {
715
+ case 'call_started': {
716
+ const parsed = CallStartedSchema.parse(payload);
717
+ const { call } = parsed;
718
+
719
+ // Insert initial call log
720
+ await supabase.from('call_logs').insert({
721
+ call_id: call.call_id,
722
+ agent_id: call.agent_id,
723
+ from_number: call.from_number ?? null,
724
+ to_number: call.to_number ?? null,
725
+ direction: call.direction,
726
+ call_status: 'started',
727
+ metadata: call.metadata ?? {},
728
+ });
729
+
730
+ // Update agent state (track returning callers)
731
+ if (call.from_number) {
732
+ await supabase.from('agent_state').upsert(
733
+ {
734
+ phone_number: call.from_number,
735
+ agent_id: call.agent_id,
736
+ last_call_id: call.call_id,
737
+ last_call_at: new Date().toISOString(),
738
+ call_count: 1, // Will be incremented via raw SQL below
739
+ },
740
+ { onConflict: 'phone_number' }
741
+ );
742
+
743
+ // Increment call count for returning callers
744
+ await supabase.rpc('increment_call_count', {
745
+ p_phone: call.from_number,
746
+ });
747
+ }
748
+
749
+ console.log(`Call started: ${call.call_id}`);
750
+ break;
751
+ }
752
+
753
+ case 'call_ended': {
754
+ const parsed = CallEndedSchema.parse(payload);
755
+ const { call } = parsed;
756
+
757
+ // Update call log with final data
758
+ await supabase
759
+ .from('call_logs')
760
+ .update({
761
+ call_status: 'ended',
762
+ duration_ms: call.duration_ms ?? null,
763
+ transcript: call.transcript ?? null,
764
+ transcript_object: call.transcript_object ?? null,
765
+ recording_url: call.recording_url ?? null,
766
+ disconnection_reason: call.disconnection_reason ?? null,
767
+ ended_at: new Date().toISOString(),
768
+ })
769
+ .eq('call_id', call.call_id);
770
+
771
+ console.log(`Call ended: ${call.call_id} (${call.duration_ms}ms)`);
772
+ break;
773
+ }
227
774
 
228
- // Book appointment
229
- const { data, error } = await supabase
230
- .from('appointments')
231
- .insert({
232
- customer_phone: phone,
233
- appointment_date: date,
234
- appointment_time: time,
235
- service_type: service
236
- })
237
- .select()
238
- .single();
239
-
240
- return new Response(JSON.stringify({
241
- success: true,
242
- appointment: data,
243
- message: `Appointment booked for ${date} at ${time}`
244
- }));
775
+ case 'call_analyzed': {
776
+ const parsed = CallAnalyzedSchema.parse(payload);
777
+ const { call } = parsed;
778
+
779
+ // Store post-call analysis
780
+ await supabase
781
+ .from('call_logs')
782
+ .update({
783
+ call_analysis: call.call_analysis ?? null,
784
+ })
785
+ .eq('call_id', call.call_id);
786
+
787
+ console.log(`Call analyzed: ${call.call_id}`);
788
+ break;
789
+ }
790
+
791
+ default:
792
+ console.log(`Unhandled event type: ${event}`);
793
+ }
245
794
  } catch (error) {
246
- console.error("Booking error:", error);
247
- return new Response(JSON.stringify({
248
- success: false,
249
- message: "Failed to process booking request"
250
- }), { status: 500 });
795
+ console.error(`Error processing ${event}:`, error);
796
+ // Return 200 anyway to prevent Retell from retrying on validation errors.
797
+ // Log the error for debugging. Only return 5xx for transient failures.
251
798
  }
799
+
800
+ return new Response(JSON.stringify({ received: true }), {
801
+ status: 200,
802
+ headers: { 'Content-Type': 'application/json' },
803
+ });
252
804
  });
253
805
  ```
254
806
 
255
- ## Stage 4: Configure Webhooks
807
+ ### Helper RPC for Call Count Increment
808
+
809
+ ```sql
810
+ -- Migration: supabase/migrations/YYYYMMDD_increment_call_count.sql
811
+ CREATE OR REPLACE FUNCTION increment_call_count(p_phone TEXT)
812
+ RETURNS VOID AS $$
813
+ BEGIN
814
+ UPDATE agent_state
815
+ SET call_count = call_count + 1
816
+ WHERE phone_number = p_phone;
817
+ END;
818
+ $$ LANGUAGE plpgsql SECURITY DEFINER;
819
+ ```
820
+
821
+ ### Tool Handler (Supabase Edge Function)
822
+
823
+ Retell calls your custom tool URLs directly. Each tool gets its own handler or a unified one:
256
824
 
257
- ### VAPI Webhook Handler
258
825
  ```typescript
259
- // supabase/functions/vapi-webhook/index.ts
260
- serve(async (req) => {
261
- try {
262
- // Verify VAPI webhook signature
263
- const signature = req.headers.get('x-vapi-signature');
264
- const body = await req.text();
826
+ // supabase/functions/retell-tool-handler/index.ts
827
+ import { serve } from "https://deno.land/std@0.208.0/http/server.ts";
828
+ import { createClient } from "https://esm.sh/@supabase/supabase-js@2";
829
+ import { z } from "https://deno.land/x/zod@v3.22.4/mod.ts";
265
830
 
266
- if (!verifyVapiSignature(body, signature, Deno.env.get('VAPI_WEBHOOK_SECRET')!)) {
267
- return new Response('Unauthorized', { status: 401 });
831
+ // --- Zod Schemas for Tool Arguments ---
832
+ const BookAppointmentArgs = z.object({
833
+ date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/, 'Date must be YYYY-MM-DD'),
834
+ time: z.string().regex(/^\d{2}:\d{2}$/, 'Time must be HH:MM'),
835
+ service: z.string().min(1),
836
+ caller_name: z.string().min(1),
837
+ });
838
+
839
+ const CheckAvailabilityArgs = z.object({
840
+ date: z.string().regex(/^\d{4}-\d{2}-\d{2}$/),
841
+ });
842
+
843
+ // --- Main Handler ---
844
+ serve(async (req: Request) => {
845
+ if (req.method !== 'POST') {
846
+ return new Response('Method not allowed', { status: 405 });
268
847
  }
269
848
 
270
- const event = JSON.parse(body);
271
- const supabase = createClient(/* ... */);
849
+ const body = await req.json();
850
+
851
+ // Retell sends: { tool_call_id, name, arguments, call }
852
+ const { name, arguments: args, call } = body;
853
+
854
+ const supabase = createClient(
855
+ Deno.env.get('SUPABASE_URL')!,
856
+ Deno.env.get('SUPABASE_SERVICE_ROLE_KEY')!
857
+ );
858
+
859
+ try {
860
+ switch (name) {
861
+ case 'book_appointment': {
862
+ const parsed = BookAppointmentArgs.parse(args);
863
+
864
+ // Check if slot is taken
865
+ const { data: existing } = await supabase
866
+ .from('appointments')
867
+ .select('id')
868
+ .eq('appointment_date', parsed.date)
869
+ .eq('appointment_time', parsed.time)
870
+ .neq('status', 'cancelled')
871
+ .maybeSingle();
272
872
 
273
- switch (event.message.type) {
274
- case 'function-call':
275
- // Handle tool calls
276
- const { name, parameters } = event.message.functionCall;
873
+ if (existing) {
874
+ return Response.json({
875
+ result: `Sorry, that time slot on ${parsed.date} at ${parsed.time} is already booked. Please choose another time.`,
876
+ });
877
+ }
277
878
 
278
- switch (name) {
279
- case 'book_appointment':
280
- const result = await bookAppointment(parameters);
281
- return new Response(JSON.stringify({ result }));
879
+ // Book the appointment
880
+ const { data: appointment, error } = await supabase
881
+ .from('appointments')
882
+ .insert({
883
+ call_id: call?.call_id ?? null,
884
+ customer_phone: call?.from_number ?? 'unknown',
885
+ customer_name: parsed.caller_name,
886
+ service_type: parsed.service,
887
+ appointment_date: parsed.date,
888
+ appointment_time: parsed.time,
889
+ })
890
+ .select()
891
+ .single();
282
892
 
283
- case 'check_availability':
284
- const slots = await checkAvailability(parameters);
285
- return new Response(JSON.stringify({ result: slots }));
893
+ if (error) throw error;
286
894
 
287
- case 'get_customer_appointments':
288
- const appointments = await getAppointments(parameters.phone);
289
- return new Response(JSON.stringify({ result: appointments }));
895
+ return Response.json({
896
+ result: `Appointment booked successfully for ${parsed.caller_name} on ${parsed.date} at ${parsed.time} for ${parsed.service}. Appointment ID: ${appointment.id}.`,
897
+ });
290
898
  }
291
- break;
292
-
293
- case 'end-of-call-report':
294
- // Log call
295
- await supabase.from('call_logs').insert({
296
- call_id: event.call.id,
297
- phone_number: event.call.customer.number,
298
- duration_seconds: event.call.duration,
299
- transcript: event.transcript,
300
- outcome: event.call.endedReason
301
- });
302
- break;
303
899
 
304
- case 'status-update':
305
- console.log('Call status:', event.status);
306
- break;
307
- }
900
+ case 'check_availability': {
901
+ const parsed = CheckAvailabilityArgs.parse(args);
308
902
 
309
- return new Response('OK');
310
- } catch (error) {
311
- console.error("Webhook error:", error);
312
- return new Response(JSON.stringify({ error: "Internal server error" }), {
313
- status: 500,
314
- headers: { "Content-Type": "application/json" },
315
- });
316
- }
317
- });
318
- ```
903
+ // Get booked slots for that date
904
+ const { data: booked } = await supabase
905
+ .from('appointments')
906
+ .select('appointment_time')
907
+ .eq('appointment_date', parsed.date)
908
+ .neq('status', 'cancelled');
319
909
 
320
- ### Tool Definitions
321
- ```typescript
322
- const tools = [
323
- {
324
- type: "function",
325
- function: {
326
- name: "book_appointment",
327
- description: "Book a new appointment for the caller",
328
- parameters: {
329
- type: "object",
330
- properties: {
331
- date: { type: "string", description: "Date in YYYY-MM-DD format" },
332
- time: { type: "string", description: "Time in HH:MM format" },
333
- service: { type: "string", description: "Type of service" }
334
- },
335
- required: ["date", "time", "service"]
910
+ const bookedTimes = new Set((booked ?? []).map(b => b.appointment_time));
911
+
912
+ // Generate available slots (9 AM to 5 PM, 30-min intervals)
913
+ const allSlots = [];
914
+ for (let h = 9; h < 17; h++) {
915
+ for (const m of ['00', '30']) {
916
+ const slot = `${h.toString().padStart(2, '0')}:${m}`;
917
+ if (!bookedTimes.has(slot)) {
918
+ allSlots.push(slot);
919
+ }
920
+ }
921
+ }
922
+
923
+ if (allSlots.length === 0) {
924
+ return Response.json({
925
+ result: `No available slots on ${parsed.date}. Please try another date.`,
926
+ });
927
+ }
928
+
929
+ return Response.json({
930
+ result: `Available times on ${parsed.date}: ${allSlots.join(', ')}`,
931
+ });
336
932
  }
933
+
934
+ default:
935
+ return Response.json({
936
+ result: `Unknown tool: ${name}`,
937
+ });
337
938
  }
338
- },
339
- {
340
- type: "function",
341
- function: {
342
- name: "check_availability",
343
- description: "Check available appointment slots",
344
- parameters: {
345
- type: "object",
346
- properties: {
347
- date: { type: "string", description: "Date to check" }
348
- },
349
- required: ["date"]
350
- }
939
+ } catch (error) {
940
+ console.error(`Tool ${name} error:`, error);
941
+
942
+ if (error instanceof z.ZodError) {
943
+ return Response.json({
944
+ result: 'I need a bit more information. Could you provide the date and time again?',
945
+ });
351
946
  }
947
+
948
+ return Response.json({
949
+ result: 'Sorry, I encountered an error processing your request. Let me try again.',
950
+ });
352
951
  }
353
- ];
952
+ });
354
953
  ```
355
954
 
955
+ > **Key pattern:** Tool handlers return `{ result: "..." }`. The `result` string is what the agent speaks back to the caller. Write it as natural speech, not JSON data.
956
+
957
+ ---
958
+
356
959
  ## Stage 5: Deploy & Test
357
960
 
358
961
  ### Deploy Steps
962
+
359
963
  ```bash
360
- # Deploy Supabase functions
361
- supabase functions deploy book-appointment
362
- supabase functions deploy vapi-webhook
964
+ # 1. Apply database migration
965
+ supabase db push
966
+
967
+ # 2. Set secrets for Edge Functions
968
+ supabase secrets set RETELL_API_KEY=key_...
969
+ supabase secrets set ELEVENLABS_API_KEY=xi_...
970
+
971
+ # 3. Deploy Edge Functions
972
+ supabase functions deploy retell-webhook --no-verify-jwt
973
+ supabase functions deploy retell-tool-handler --no-verify-jwt
974
+
975
+ # 4. Get the function URLs
976
+ # https://YOUR_PROJECT.supabase.co/functions/v1/retell-webhook
977
+ # https://YOUR_PROJECT.supabase.co/functions/v1/retell-tool-handler
363
978
 
364
- # Update VAPI assistant with webhook URL
365
- # Use VAPI dashboard or API
979
+ # 5. Update Retell agent with webhook URL (via API or dashboard)
980
+ curl -X PATCH "https://api.retellai.com/v2/update-agent/YOUR_AGENT_ID" \
981
+ -H "Authorization: Bearer $RETELL_API_KEY" \
982
+ -H "Content-Type: application/json" \
983
+ -d '{"webhook_url": "https://YOUR_PROJECT.supabase.co/functions/v1/retell-webhook"}'
366
984
 
367
- # Test with a call
368
- vapi call --assistant-id your-assistant-id --to +1234567890
985
+ # 6. Test with a web call (no phone needed)
986
+ curl -X POST "https://api.retellai.com/v2/create-web-call" \
987
+ -H "Authorization: Bearer $RETELL_API_KEY" \
988
+ -H "Content-Type: application/json" \
989
+ -d '{"agent_id": "YOUR_AGENT_ID"}'
990
+ # Returns: { "call_id": "...", "access_token": "..." }
991
+ # Use access_token with Retell Web SDK to test in browser
992
+
993
+ # 7. Test with a phone call
994
+ curl -X POST "https://api.retellai.com/v2/create-phone-call" \
995
+ -H "Authorization: Bearer $RETELL_API_KEY" \
996
+ -H "Content-Type: application/json" \
997
+ -d '{
998
+ "from_number": "+1234567890",
999
+ "to_number": "+0987654321",
1000
+ "agent_id": "YOUR_AGENT_ID"
1001
+ }'
369
1002
  ```
370
1003
 
1004
+ > **Important:** Use `--no-verify-jwt` for webhook and tool handler functions. Retell authenticates via `x-retell-signature`, not JWT. The functions verify signatures themselves.
1005
+
371
1006
  ### Test Scenarios
372
- 1. **Happy path**: Book appointment successfully
373
- 2. **Conflict**: Time slot unavailable
374
- 3. **Edge case**: Invalid date format
375
- 4. **Error handling**: Database connection fails
376
- 5. **Context persistence**: Caller calls back
1007
+
1008
+ | # | Scenario | Expected Outcome |
1009
+ |---|----------|------------------|
1010
+ | 1 | Happy path: book appointment | Agent collects info, calls book_appointment tool, confirms booking |
1011
+ | 2 | Slot conflict | Agent reports slot unavailable, suggests alternatives |
1012
+ | 3 | Invalid date from caller | Agent asks caller to repeat/clarify the date |
1013
+ | 4 | Caller interrupts mid-sentence | Agent stops speaking and listens |
1014
+ | 5 | Caller asks for human | Agent transfers (or apologizes if no transfer configured) |
1015
+ | 6 | Returning caller | agent_state context loaded, call_count incremented |
1016
+ | 7 | Webhook failure | Returns 200 anyway, error logged for debugging |
1017
+ | 8 | Silence timeout | Agent nudges after reminder_trigger_ms, ends after max nudges |
1018
+
1019
+ ### Verify Webhook Delivery
1020
+
1021
+ ```bash
1022
+ # Check Retell webhook logs via API
1023
+ curl "https://api.retellai.com/v2/list-calls?agent_id=YOUR_AGENT_ID&limit=5" \
1024
+ -H "Authorization: Bearer $RETELL_API_KEY" | jq '.[]|{call_id,call_status,disconnection_reason,duration_ms}'
1025
+
1026
+ # Check Supabase Edge Function logs
1027
+ supabase functions logs retell-webhook --limit 20
1028
+
1029
+ # Verify data landed in DB
1030
+ supabase db execute "SELECT call_id, call_status, duration_ms, ended_at FROM call_logs ORDER BY created_at DESC LIMIT 5"
1031
+ ```
1032
+
1033
+ ---
377
1034
 
378
1035
  ## Voice Selection (ElevenLabs)
379
1036
 
380
- | Use Case | Voice | Why |
381
- |----------|-------|-----|
382
- | Professional | Rachel | Calm, clear, authoritative |
383
- | Friendly | Bella | Warm, approachable |
384
- | Energetic | Elli | Upbeat, engaging |
385
- | Corporate | Adam | Professional male |
1037
+ ### Pre-built Voices
1038
+
1039
+ | Use Case | Voice Name | Voice ID | Style | Best For |
1040
+ |----------|-----------|----------|-------|----------|
1041
+ | Professional Female | Rachel | `21m00Tcm4TlvDq8ikWAM` | Calm, clear, authoritative | Medical, legal, enterprise |
1042
+ | Friendly Female | Bella | `EXAVITQu4vr4xnSDxMaL` | Warm, approachable, casual | Retail, hospitality, support |
1043
+ | Energetic Female | Elli | `MF3mGyEYCl7XYWbV9V6O` | Upbeat, enthusiastic | Sales, marketing, events |
1044
+ | Professional Male | Adam | `pNInz6obpgDQGcFmaJgB` | Deep, confident, steady | Finance, corporate, B2B |
1045
+ | Conversational Male | Antoni | `ErXwobaYiN019PkySvjV` | Natural, relaxed | General-purpose, support |
1046
+ | Warm Male | Josh | `TxGEqnHWrfWFTfGW9XjX` | Friendly, trustworthy | Healthcare, insurance |
1047
+ | Authoritative Male | Arnold | `VR6AewLTigWG4xSOukaG` | Strong, commanding | Emergency, security |
1048
+ | Young Female | Dorothy | `ThT5KcBeYPX3keUQqHPh` | Bright, clear | Startups, tech support |
1049
+
1050
+ ### Multilingual Voices
1051
+
1052
+ | Language | Voice Name | Voice ID | Notes |
1053
+ |----------|-----------|----------|-------|
1054
+ | Arabic | Custom clone | (clone your own) | Best results with voice cloning |
1055
+ | Greek | Custom clone | (clone your own) | Limited pre-built options |
1056
+ | Multilingual | Eleven Multilingual v2 | Any v2 voice | All v2 voices support 29 languages |
1057
+
1058
+ > **Tip:** All ElevenLabs voices with "v2" or "Multilingual v2" model support automatic language detection across 29 languages. Use these for multilingual agents.
1059
+
1060
+ ### Voice Cloning (Custom Brand Voice)
1061
+
1062
+ ```typescript
1063
+ // Clone a voice from audio samples
1064
+ const formData = new FormData();
1065
+ formData.append('name', 'Brand Voice');
1066
+ formData.append('description', 'Company brand voice for phone agents');
1067
+ formData.append('files', audioFile1); // Min 1 sample, recommend 3+ minutes total
1068
+ formData.append('files', audioFile2);
1069
+ formData.append('labels', JSON.stringify({ accent: 'neutral', gender: 'female' }));
1070
+
1071
+ const response = await fetch('https://api.elevenlabs.io/v1/voices/add', {
1072
+ method: 'POST',
1073
+ headers: {
1074
+ 'xi-api-key': ELEVENLABS_API_KEY,
1075
+ },
1076
+ body: formData,
1077
+ });
1078
+
1079
+ const voice = await response.json();
1080
+ console.log('Cloned voice ID:', voice.voice_id);
1081
+ // Use this voice_id when creating the Retell agent
1082
+ ```
1083
+
1084
+ ### List Available Voices
1085
+
1086
+ ```typescript
1087
+ // Fetch all available voices from ElevenLabs
1088
+ const response = await fetch('https://api.elevenlabs.io/v1/voices', {
1089
+ headers: { 'xi-api-key': ELEVENLABS_API_KEY },
1090
+ });
1091
+ const { voices } = await response.json();
1092
+
1093
+ voices.forEach((v: { name: string; voice_id: string; labels: Record<string, string> }) => {
1094
+ console.log(`${v.name} (${v.voice_id}) — ${v.labels?.accent ?? 'unknown accent'}, ${v.labels?.gender ?? ''}`);
1095
+ });
1096
+ ```
1097
+
1098
+ ---
1099
+
1100
+ ## Advanced Patterns
1101
+
1102
+ ### Outbound Calling (Proactive Calls)
1103
+
1104
+ ```typescript
1105
+ // Trigger an outbound call (e.g., appointment reminders)
1106
+ async function makeOutboundCall(
1107
+ agentId: string,
1108
+ fromNumber: string,
1109
+ toNumber: string,
1110
+ metadata?: Record<string, string>
1111
+ ) {
1112
+ const response = await fetch('https://api.retellai.com/v2/create-phone-call', {
1113
+ method: 'POST',
1114
+ headers: {
1115
+ 'Authorization': `Bearer ${RETELL_API_KEY}`,
1116
+ 'Content-Type': 'application/json',
1117
+ },
1118
+ body: JSON.stringify({
1119
+ agent_id: agentId,
1120
+ from_number: fromNumber,
1121
+ to_number: toNumber,
1122
+ metadata: metadata ?? {},
1123
+ retell_llm_dynamic_variables: {
1124
+ // Inject variables into the LLM prompt at call time
1125
+ customer_name: 'John',
1126
+ appointment_date: '2026-04-05',
1127
+ appointment_time: '14:00',
1128
+ },
1129
+ }),
1130
+ });
1131
+
1132
+ return response.json();
1133
+ }
1134
+
1135
+ // Example: reminder cron job (Supabase Edge Function + pg_cron)
1136
+ // Run daily at 8 AM, call patients with appointments tomorrow
1137
+ ```
1138
+
1139
+ ### Dynamic Variables (Personalization)
1140
+
1141
+ Retell supports `retell_llm_dynamic_variables` to inject context into the LLM prompt at call time:
1142
+
1143
+ ```typescript
1144
+ // In your LLM prompt, use {{variable_name}} syntax:
1145
+ // "Hello {{customer_name}}, I'm calling about your appointment on {{appointment_date}}."
1146
+
1147
+ // When creating the call, pass the variables:
1148
+ const call = await fetch('https://api.retellai.com/v2/create-phone-call', {
1149
+ method: 'POST',
1150
+ headers: {
1151
+ 'Authorization': `Bearer ${RETELL_API_KEY}`,
1152
+ 'Content-Type': 'application/json',
1153
+ },
1154
+ body: JSON.stringify({
1155
+ agent_id: agentId,
1156
+ from_number: fromNumber,
1157
+ to_number: toNumber,
1158
+ retell_llm_dynamic_variables: {
1159
+ customer_name: 'Sarah',
1160
+ appointment_date: 'April 5th',
1161
+ company_name: 'Qualia Solutions',
1162
+ },
1163
+ }),
1164
+ });
1165
+ ```
1166
+
1167
+ ### Call Transfer
1168
+
1169
+ ```typescript
1170
+ // In your Retell LLM config, add a transfer tool:
1171
+ {
1172
+ type: 'transfer_call',
1173
+ name: 'transfer_to_human',
1174
+ description: 'Transfer the caller to a human agent when they request one or when you cannot help',
1175
+ number: '+35799999999', // Human agent / call center number
1176
+ }
1177
+ ```
1178
+
1179
+ ### Web Call (Browser-Based)
1180
+
1181
+ ```typescript
1182
+ // Create a web call for browser-based voice agents
1183
+ const webCall = await fetch('https://api.retellai.com/v2/create-web-call', {
1184
+ method: 'POST',
1185
+ headers: {
1186
+ 'Authorization': `Bearer ${RETELL_API_KEY}`,
1187
+ 'Content-Type': 'application/json',
1188
+ },
1189
+ body: JSON.stringify({
1190
+ agent_id: agentId,
1191
+ metadata: { source: 'website', page: '/contact' },
1192
+ retell_llm_dynamic_variables: {
1193
+ customer_name: 'Website Visitor',
1194
+ },
1195
+ }),
1196
+ });
1197
+
1198
+ const { access_token } = await webCall.json();
1199
+ // Pass access_token to Retell Web SDK on the frontend
1200
+ ```
1201
+
1202
+ ### Frontend Web SDK Integration
1203
+
1204
+ ```typescript
1205
+ // npm install retell-client-js-sdk
1206
+ import { RetellWebClient } from 'retell-client-js-sdk';
1207
+
1208
+ const retellClient = new RetellWebClient();
1209
+
1210
+ // Start call with access token from your backend
1211
+ await retellClient.startCall({
1212
+ accessToken: accessToken, // From create-web-call response
1213
+ sampleRate: 24000,
1214
+ emitRawAudioSamples: false,
1215
+ });
1216
+
1217
+ // Event handlers
1218
+ retellClient.on('call_started', () => {
1219
+ console.log('Call connected');
1220
+ });
1221
+
1222
+ retellClient.on('call_ended', () => {
1223
+ console.log('Call ended');
1224
+ });
1225
+
1226
+ retellClient.on('agent_start_talking', () => {
1227
+ // Update UI — agent is speaking
1228
+ });
1229
+
1230
+ retellClient.on('agent_stop_talking', () => {
1231
+ // Update UI — agent stopped
1232
+ });
1233
+
1234
+ retellClient.on('error', (error) => {
1235
+ console.error('Retell error:', error);
1236
+ retellClient.stopCall();
1237
+ });
1238
+
1239
+ // End call
1240
+ retellClient.stopCall();
1241
+ ```
1242
+
1243
+ ---
386
1244
 
387
1245
  ## Best Practices
388
1246
 
389
- 1. **Keep responses short** - 1-2 sentences max
390
- 2. **Confirm important info** - Repeat back dates/times
391
- 3. **Handle interruptions** - Allow user to interrupt
392
- 4. **Provide escape routes** - "Say 'operator' for help"
393
- 5. **Log everything** - For debugging and improvement
394
- 6. **Test extensively** - Real phone calls, various accents
1247
+ 1. **Keep responses short** 1-2 sentences max. Long agent monologues make callers hang up.
1248
+ 2. **Confirm critical info** Always repeat dates, times, and names back before acting.
1249
+ 3. **Handle interruptions** Set `interruption_sensitivity` to 1 for natural conversation.
1250
+ 4. **Use backchannels** Enable `enable_backchannel` for natural "uh huh" filler while listening.
1251
+ 5. **Tool results are speech** Return natural language in `result`, not JSON. The agent speaks it.
1252
+ 6. **Validate on the tool side** Use Zod in tool handlers. Return friendly error messages, not stack traces.
1253
+ 7. **Log everything** — Store full transcripts and call analysis for debugging and improvement.
1254
+ 8. **Test with real calls** — Browser testing is fast, but always test real phone calls before going live.
1255
+ 9. **Use post-call analysis** — Define `post_call_analysis_data` to auto-extract outcomes, sentiment, etc.
1256
+ 10. **Signature verification is mandatory** — Never skip `x-retell-signature` verification on webhooks.
1257
+ 11. **Ambient sound** — Use `coffee-shop` or `call-center` ambient sound for more natural feel.
1258
+ 12. **Dynamic variables** — Use `retell_llm_dynamic_variables` for personalization on outbound calls.
1259
+
1260
+ ## Retell API Quick Reference
1261
+
1262
+ | Action | Method | Endpoint |
1263
+ |--------|--------|----------|
1264
+ | Create LLM | POST | `https://api.retellai.com/v2/create-retell-llm` |
1265
+ | Create Agent | POST | `https://api.retellai.com/v2/create-agent` |
1266
+ | Update Agent | PATCH | `https://api.retellai.com/v2/update-agent/{agent_id}` |
1267
+ | Get Agent | GET | `https://api.retellai.com/v2/get-agent/{agent_id}` |
1268
+ | List Agents | GET | `https://api.retellai.com/v2/list-agents` |
1269
+ | Delete Agent | DELETE | `https://api.retellai.com/v2/delete-agent/{agent_id}` |
1270
+ | Create Phone Number | POST | `https://api.retellai.com/v2/create-phone-number` |
1271
+ | Import Phone Number | POST | `https://api.retellai.com/v2/import-phone-number` |
1272
+ | Create Phone Call | POST | `https://api.retellai.com/v2/create-phone-call` |
1273
+ | Create Web Call | POST | `https://api.retellai.com/v2/create-web-call` |
1274
+ | List Calls | GET | `https://api.retellai.com/v2/list-calls` |
1275
+ | Get Call | GET | `https://api.retellai.com/v2/get-call/{call_id}` |
1276
+ | Update Retell LLM | PATCH | `https://api.retellai.com/v2/update-retell-llm/{llm_id}` |
1277
+
1278
+ All endpoints require header: `Authorization: Bearer {RETELL_API_KEY}`
395
1279
 
396
1280
  ## Integration with Other Skills
397
1281
 
398
- - **voice-agent** - This skill (VAPI patterns, conversation flow, webhooks)
399
- - **supabase** - For database operations
400
- - **docs-lookup** - For VAPI/ElevenLabs/Deepgram API docs
1282
+ - **supabase** Database operations, Edge Functions, migrations
1283
+ - **docs-lookup** Retell AI / ElevenLabs API documentation
1284
+ - **deploy** Production deployment pipeline
1285
+ - **ship** — Full deploy with quality gates
1286
+
1287
+ ## Key Decisions to Ask User
1288
+
1289
+ - **Use case**: What should the agent do? (booking, support, sales, etc.)
1290
+ - **Voice gender/style**: Professional, friendly, energetic? (see Voice Selection table)
1291
+ - **Language**: English only or multilingual? (affects voice model selection)
1292
+ - **Custom voice**: Need voice cloning for brand consistency?
1293
+ - **Phone vs web**: Inbound phone calls, outbound calls, or browser widget?
1294
+ - **LLM**: Retell built-in (Claude/GPT) or custom LLM via OpenRouter?
1295
+ - **Transfer**: Should the agent be able to transfer to a human?
1296
+ - **Telephony**: Buy new number from Retell, or import existing (Telnyx/Twilio)?
401
1297
 
402
1298
  ## Trigger Phrases
403
1299
 
@@ -405,3 +1301,12 @@ vapi call --assistant-id your-assistant-id --to +1234567890
405
1301
  - "create voice assistant"
406
1302
  - "voice AI for"
407
1303
  - "phone bot for"
1304
+ - "retell agent"
1305
+ - "retell"
1306
+ - "elevenlabs"
1307
+ - "voice webhook"
1308
+ - "call agent"
1309
+ - "phone agent"
1310
+ - "voice automation"
1311
+ - "outbound calls"
1312
+ - "web call widget"