elevenlabs-webhook-nodejs 1.0.0

package/docs/architecture/04-WHATSAPP-AGENT.md

@@ -0,0 +1,608 @@
+# 04 - WhatsApp AI Agent
+
+> Last updated: 2026-04-02
+
+> Core WhatsApp AI agent flow -- the heart of the product.
+> Covers the full lifecycle from webhook ingestion through ElevenLabs AI response delivery.
+
+---
+
+## 1. Session Lifecycle
+
+Every patient conversation is managed through a **WhatsAppSession** record that transitions through three states:
+
+```
+                     +-----------+
+  Incoming message   |           |   Human replies within
+  ----------------->  | waiting   | -----------------------> RESOLVED (human)
+                     |           |        cancel timer
+                     +-----+-----+
+                           |
+                     grace period expires
+                     (BullMQ delayed job)
+                           |
+                           v
+                     +-----------+
+                     |           |   Message debounce
+                     |  active   | ----> AI response loop
+                     |           |
+                     +-----+-----+
+                           |
+            2h inactivity / AI timeout / manual
+                           |
+                           v
+                     +-----------+
+                     |           |
+                     | resolved  |
+                     |           |
+                     +-----------+
+```
+
+### State Definitions
+
+| State | Meaning | Transitions To |
+|-------|---------|----------------|
+| `waiting` | Contact messaged; grace timer running, waiting for human to respond first | `active` (grace expires), `resolved` (human responds) |
+| `active` | AI has taken over the conversation; ElevenLabs WS may be connected | `resolved` (inactivity, AI timeout, manual, human takeover) |
+| `resolved` | Conversation ended | Next incoming message creates a new session |
+
+### Resolution Types
+
+| `resolved_by` | Trigger |
+|----------------|---------|
+| `human` | A real human (not AI) sent a message during `waiting` state |
+| `ai_timeout` | ElevenLabs conversation ended or AI did not respond |
+| `timeout` | 2h inactivity sweep resolved the session |
+| `manual` | Operator clicked "resolve" in the UI, or grace expired with no agent configured |
+
+### Grace Period
+
+- **Default:** 180 seconds (configurable per clinic via `clinic_grace_period_seconds` -- read directly from clinic, no tenant fallback)
+- **Current implementation:** `setTimeout` in memory
+- **Target implementation:** BullMQ delayed job on `whatsapp:grace` queue
+
+```typescript
+// Target: BullMQ delayed job
+await graceQueue.add('grace-expired', { sessionId, chatId, tenantId }, {
+  delay: gracePeriodSeconds * 1000,
+  jobId: `grace:${sessionId}`,  // Deduplicate -- same session replaces the job
+});
+```
+
+On grace expiry:
+1. Verify session is still `waiting` (may have been resolved by a human in the meantime)
+2. Check if any human has ever messaged in this chat -- if so, resolve as `manual`
+3. Check AI shift schedule -- if outside AI hours, resolve as `manual`
+4. If agent is configured: transition to `active`, replay all pending contact messages to AI
+5. If no agent: resolve as `manual`
+
+### Message Debouncing
+
+When the session is `active` and a new contact message arrives, the system does not send it to AI immediately. Instead:
+
+1. Cancel any existing buffer timer for this session
+2. Set a new timer with a **random delay between 10s and 50s**
+3. When the timer fires, collect all unanswered contact messages since the last AI response
+4. Combine them into a single text payload and send to ElevenLabs
+
+**Current:** `setTimeout` in memory (`messageBufferTimers` Map)
+**Target:** BullMQ delayed job on `whatsapp:message-buffer` queue
+
+```typescript
+await messageBufferQueue.add('buffer-expired', { sessionId, chatId }, {
+  delay: randomDelay(10_000, 50_000),
+  jobId: `buffer:${sessionId}`,  // Replace existing job for same session
+});
+```
+
+### 2-Hour Inactivity Auto-Resolve
+
+A repeatable sweep runs every 5 minutes (BullMQ `whatsapp:session-cleanup`). It finds `active` sessions where `updatedAt < now - 2h` and `awaiting_operator != true`, then:
+
+1. Closes the ElevenLabs WS connection if still open
+2. Sets status to `resolved`, `resolved_by: 'timeout'`
+3. Unlinks from chat (`active_session_id = null`)
+4. Fires profile summary update and session labeling (fire-and-forget)
+
+---
+
+## 2. Message Processing Pipeline
+
+Complete step-by-step flow from webhook to AI response:
+
+### Step 1: Webhook Receives Message
+
+The webhook endpoint (`POST /webhooks/meta`) receives a payload from the Meta Business API. The payload is normalized to a common internal format.
+
+Key payload fields:
+- `account_id` -- identifies the clinic's Meta WhatsApp Business Account
+- `chat_id` -- identifies the specific conversation (Meta conversation ID)
+- `message` -- text content
+- `content_type` -- media type (e.g., `imageMessage`, `audioMessage`)
+- `is_sender` -- `true` if the message was sent by the clinic (echo)
+- `attendees[]` -- contact info (phone, name)
+- `ad_context` -- ad attribution if the patient clicked an ad to start the chat
+- `quoted_text` -- text of a quoted (replied-to) message
+
+### Step 2: System Message Filtering
+
+Over 40 regex patterns detect non-conversational messages in both Turkish and English:
+
+- Voice/video call notifications
+- Message deletions
+- End-to-end encryption notices
+- Disappearing messages toggles
+- Live location sharing
+- Payment notifications
+- Group management events
+- View-once message placeholders
+- Old WhatsApp version warnings
+
+System messages are **stored for visibility** but never trigger sessions or AI. Detection uses two layers:
+1. Structural fields: `is_event === true` or `hidden === true`
+2. Text pattern fallback: regex matching against the 40+ patterns
+
+### Step 3: Voice Transcription (STT)
+
+If `content_type === 'audioMessage'` and the tenant has `voice_transcription` enabled:
+
+1. Download audio via Meta Business API (`downloadMedia`)
+2. Send to ElevenLabs Scribe v1 STT API
+3. Replace `payload.message` with the transcript text
+4. Set `content_type = 'transcribedAudio'`
+5. Log STT usage for billing
+
+If transcription fails, the original media label is preserved as fallback.
+
+### Step 4: Operator Reply Detection
+
+Before processing as a regular message, check if this is a reply from a known operator (doctor):
+
+1. Collect all known operator phones from tenant settings (`operator_phone`, `conversation_forwarding_phone`)
+2. Search for REF code pattern (`/\b([A-Z]-[A-Z0-9]{4})\b/`) in both the message text and `quoted_text`
+3. If REF code found and sender matches an operator phone: route to `handleOperatorReply()` (see 05-OPERATOR-WORKFLOW.md)
+4. If REF code found and `is_sender === true`: verify the REF code exists as a pending request (operator using the same device)
+
+### Step 5: Patient Find-or-Create
+
+```typescript
+// Find existing patient by phone within the clinic, or create a new one
+const patient = await patientService.findOrCreate({
+  clinic_id: clinic.clinic_id,
+  phone: contactPhone,
+  display_name: contactName,
+});
+// Updates patient_last_seen_at on every message
+```
+
+Creates a `Patient` record on first contact; updates `patient_display_name` and `patient_last_seen_at` on every message. The AI summary is stored on `patient.patient_ai_summary` (encrypted).
+
+### Step 6: Chat Find/Create
+
+1. Look up `WhatsAppChat` by `chat_external_id` (Meta conversation ID)
+2. If not found, try matching by `clinic_id + contact_phone`
+3. If still not found, create a new chat record with `chat_patient_id` linking to the patient
+
+### Step 7: Session Find/Create
+
+1. Check `chat.active_session_id` for an existing non-resolved session
+2. If session is `waiting`: store the message (timer already running), return
+3. If session is `active`: forward to `handleIncomingMessage()` for debouncing
+4. If no active session: create a new session in `waiting` state with grace deadline
+
+### Step 8: Grace Period or AI Trigger
+
+- **New session (waiting):** Start grace period timer. If `fixed_first_message_enabled`, send the configured first message via Meta Business API.
+- **Existing active session:** Start/reset message buffer timer (debounce).
+
+### Step 9: Message Buffering
+
+When the buffer timer fires:
+1. Find the last AI response in this session
+2. Collect all contact messages after that AI response
+3. Combine into a single text payload (`\n`-separated)
+4. Pass to `triggerAiResponse()`
+
+### Step 10: ElevenLabs WebSocket Connection
+
+If no live connection exists for this chat:
+1. Build `user_summary` from patient record (name, preferred language, notes, AI summary, tags)
+2. If reconnecting and `chat_continuity` is enabled: build `continuation_directive` with full chat history
+3. Get a signed URL from ElevenLabs API
+4. Open WebSocket: `new WebSocket(signedUrl + '?textOnly=true')`
+5. On open: send `conversation_initiation_client_data` with dynamic variables
+6. Wait for `conversation_initiation_metadata` (15s timeout)
+7. Store conversation ID on the session record
+8. Start keepalive ping timer (30s interval)
+9. Send chat context via `contextual_update`
+
+If connection already exists and is alive: reuse it.
+
+### Step 11: AI Response -> Send via Meta API
+
+When ElevenLabs sends `agent_response`:
+1. Extract response text
+2. Send via Meta Business API to the patient's WhatsApp
+3. Store as `WhatsAppMessage` with `sender: 'human', sender_name: 'AI'`
+4. Message text (`msg_text`) is encrypted at rest using AES-256-GCM via `MESSAGE_ENCRYPTION_KEY`
+5. Increment chat and session message counts
+
+**Message encryption:** All stored messages have `msg_text` and `msg_quoted_text` encrypted with AES-256-GCM. Decryption happens on read. Messages are auto-deleted after `clinic.clinic_message_retention_days` (default 7 days) via a BullMQ scheduled job.
+
+### Step 12: Deduplication on Echo
+
+When Meta delivers our outgoing message back via webhook, it arrives as `is_sender === true`. Deduplication runs in two layers:
+
+1. **Primary:** Match by `msg_external_id` (Meta message ID) -- if already stored, skip
+2. **Fallback:** Match by `chat_id + text + sent_via_api + createdAt > 1min ago` -- covers cases where the API did not return a message ID at send time
+
+If a match is found, update the stored message with the real message ID (for future dedup) and skip processing.
+
+---
+
+## 3. Complete Sequence Diagram
+
+```
+Patient          Meta Business API    Webhook Handler       WhatsAppAgentService       ElevenLabs
+  |                   |                    |                       |                      |
+  |--- WA message --->|                    |                       |                      |
+  |                   |--- HTTP POST ----->|                       |                      |
+  |                   |                    |                       |                      |
+  |                   |                    |  1. Validate secret   |                      |
+  |                   |                    |  2. Filter system msg |                      |
+  |                   |                    |  3. STT (if audio)    |                      |
+  |                   |                    |  4. Detect operator   |                      |
+  |                   |                    |  5. Find/create patient|                     |
+  |                   |                    |  6. Find/create chat  |                      |
+  |                   |                    |  7. Find/create sess. |                      |
+  |                   |                    |                       |                      |
+  |                   |                    |--- startSession() --->|                      |
+  |                   |                    |    (if new session)   |                      |
+  |                   |                    |                       |                      |
+  |                   |                    |                       |-- BullMQ: grace ----->|
+  |                   |                    |                       |   (delayed 180s)      |
+  |                   |                    |                       |                      |
+  |              ... 180s pass, no human reply ...                 |                      |
+  |                   |                    |                       |                      |
+  |                   |                    |                       |<- grace expired       |
+  |                   |                    |                       |                      |
+  |                   |                    |                       |  Session -> active    |
+  |                   |                    |                       |  Replay pending msgs  |
+  |                   |                    |                       |                      |
+  |                   |                    |                       |-- getSignedUrl() ---->|
+  |                   |                    |                       |<-- signedUrl ---------|
+  |                   |                    |                       |                      |
+  |                   |                    |                       |== WS connect ========>|
+  |                   |                    |                       |-- init_client_data -->|
+  |                   |                    |                       |<- init_metadata ------|
+  |                   |                    |                       |                      |
+  |                   |                    |                       |-- contextual_update ->|
+  |                   |                    |                       |-- user_message ------>|
+  |                   |                    |                       |                      |
+  |                   |                    |                       |<- agent_response -----|
+  |                   |                    |                       |                      |
+  |                   |                    |                       |-- sendMessage() --+   |
+  |                   |<------- Meta API send ---------------------+                  |   |
+  |<-- WA message ----|                    |                       |                  |   |
+  |                   |                    |                       |  Store AI msg    |   |
+  |                   |                    |                       |  (encrypted)     |   |
+  |                   |                    |                       |                      |
+  |--- next message ->|--- HTTP POST ----->|                       |                      |
+  |                   |                    |--- handleIncoming() ->|                      |
+  |                   |                    |                       |                      |
+  |                   |                    |                       |-- BullMQ: buffer ---->|
+  |                   |                    |                       |   (10-50s random)     |
+  |                   |                    |                       |                      |
+  |              ... buffer timer fires ...                        |                      |
+  |                   |                    |                       |                      |
+  |                   |                    |                       |-- user_message ------>|
+  |                   |                    |                       |   (combined msgs)     |
+  |                   |                    |                       |                      |
+  |                   |                    |                       |<- client_tool_call ---|
+  |                   |                    |                       |   (e.g., reserve_slot)|
+  |                   |                    |                       |                      |
+  |                   |                    |                       |-- ToolRegistry ------>|
+  |                   |                    |                       |<- tool result --------|
+  |                   |                    |                       |                      |
+  |                   |                    |                       |-- client_tool_result >|
+  |                   |                    |                       |<- agent_response -----|
+  |                   |                    |                       |                      |
+  |                   |<------- Meta API send ---------------------+                      |
+  |<-- WA message ----|                    |                       |                      |
+```
+
+---
+
+## 4. ElevenLabs WebSocket Protocol
+
+### Connection Lifecycle
+
+```
+1. GET signed URL:   elevenlabsService.getSignedUrl({ agentId })
+2. Connect:          new WebSocket(`${signedUrl}?textOnly=true`)
+3. On WS open:       send conversation_initiation_client_data
+4. Wait for:         conversation_initiation_metadata (15s timeout)
+5. Active:           exchange user_message / agent_response / client_tool_call / client_tool_result
+6. Keepalive:        send ping + user_activity every 30s
+7. Close:            on session resolve, 2h idle cleanup, or ElevenLabs-initiated close
+```
+
+### Message Types
+
+| Type | Direction | Payload | Purpose |
+|------|-----------|---------|---------|
+| `conversation_initiation_client_data` | Client -> EL | `{ conversation_config_override, dynamic_variables }` | Initialize conversation with agent config overrides and dynamic variables |
+| `conversation_initiation_metadata` | EL -> Client | `{ conversation_id }` | Confirmation that conversation is ready; provides the conversation ID |
+| `user_message` | Client -> EL | `{ text }` | Send patient's message (or combined buffered messages) to the AI |
+| `contextual_update` | Client -> EL | `{ text }` | Send background context (chat history, operator responses) without triggering a response |
+| `agent_response` | EL -> Client | `{ agent_response }` | AI-generated response text to send to the patient |
+| `client_tool_call` | EL -> Client | `{ tool_call_id, tool_name, parameters }` | AI requests a tool execution (check_availability, reserve_slot, etc.) |
+| `client_tool_result` | Client -> EL | `{ tool_call_id, result, is_error }` | Return tool execution result to AI. **CRITICAL:** `type` must be lowercase, `is_error` is required |
+| `ping` / `pong` | Both | `{ event_id }` | ElevenLabs sends `ping`, client responds with `pong` carrying the same `event_id` |
+| `user_activity` | Client -> EL | `{}` | Keepalive signal to reset ElevenLabs' inactivity timer |
+
+### Dynamic Variables
+
+Sent in `conversation_initiation_client_data.dynamic_variables`:
+
+| Variable | Value | Source |
+|----------|-------|--------|
+| `user_summary` | Patient profile summary (name, language, notes, AI history summary, tags) or default "Ilk defa iletisime gecen hasta" | `Patient` record |
+| `current_date` | `YYYY-MM-DD` format | `new Date().toISOString().split('T')[0]` |
+| `continuation_directive` | Full chat history + "don't re-introduce yourself" instructions | Built on WS reconnection when `chat_continuity` is enabled |
+
+### Keepalive
+
+- **Interval:** 30 seconds (`WS_PING_INTERVAL_MS`)
+- **Action:** TCP-level `ws.ping()` + ElevenLabs-level `{ type: 'user_activity' }` message
+- Prevents ElevenLabs from closing the connection due to inactivity
+
+### Handshake Timeout
+
+- **15 seconds** to receive `conversation_initiation_metadata` after sending `conversation_initiation_client_data`
+- On timeout: close WebSocket, reject the connection promise, log error
+- Next incoming message will trigger a fresh connection attempt (lazy reconnect)
+
+### Chat Context Building
+
+The `buildChatContext()` method constructs a timeline string sent via `contextual_update`:
+
+1. Fetch all `WhatsAppMessage` records for the current session
+2. Fetch all `OperatorRequest` records for the current session
+3. Build timeline entries:
+   - Messages: `"Patient: {text}"` or `"Operator: {text}"`
+   - Operator request events: `"[SISTEM: Hastanin {N} medyasi doktora iletildi -- Ref: {code}]"`
+   - Operator responses: `"[SISTEM: Doktor yanit verdi -- Ref: {code}: "{response}"]"`
+   - Operator expiry: `"[SISTEM: Doktordan yanit alinamadi -- Ref: {code} suresi doldu]"`
+4. Sort by timestamp
+5. Prepend ad context if the first contact message has `ad_context` (ad attribution)
+6. Format: `"Current conversation:\n{timeline}"`
+
+---
+
+## 5. Chat Continuity (WS Reconnection)
+
+When an ElevenLabs WebSocket connection drops (ElevenLabs timeout, network issue, process restart), the system reconnects lazily on the next message. If `chat_continuity` is enabled for the tenant:
+
+### Flow
+
+1. `triggerAiResponse()` detects no live connection
+2. Check if `session.elevenlabs_conversation_id` exists (indicates a reconnection)
+3. If `chat_continuity` feature flag is enabled:
+   - Build full chat history via `buildChatContext()`
+   - Find the last AI message in the session
+   - Construct `continuation_directive`:
+     ```
+     ONEMLI -- KONUSMA DEVAMI:
+     Bu hasta ile onceden baslamis bir konusma var. Teknik bir baglanti yenilenmesi yasandi.
+     Kendini TANITMA. Selam VERME. Hosgeldin DEME. Konusmaya kaldigin yerden DEVAM ET.
+     En son gonderdigin mesaj: "{last AI message text}"
+
+     Current conversation:
+     {full timeline}
+     ```
+4. `continuation_directive` is sent as a dynamic variable at connection time
+5. `contextual_update` with chat history is sent as backup after connection is ready
+6. Previous conversation ID is pushed to `elevenlabs_previous_conversation_ids` array
+
+### Requirements
+
+- Tenant must have `chat_continuity` in `enabled_features`
+- Agent prompt on ElevenLabs dashboard must include `{{continuation_directive}}` placeholder
+
+---
+
+## 6. AI Shift Scheduling
+
+The AI agent is **active OUTSIDE clinic working hours**. The schedule defines when humans handle messages; AI fills in the gaps.
+
+### Logic
+
+```
+Schedule entries = clinic working hours (humans active)
+AI active = NOT within any scheduled window for the current day
+
+If ai_shift_enabled == false  -->  AI active 24/7
+If no entry for current day   -->  AI active all day
+If current time within entry  -->  AI inactive (clinic is open)
+If current time outside entry -->  AI active
+```
+
+### Schedule Format
+
+```typescript
+// Per-day schedule stored in clinic_ai_shift_schedule (JSONB)
+interface AiShiftEntry {
+  day: number;     // 0=Sunday, 1=Monday, ..., 6=Saturday
+  start: string;   // "09:00" (HH:mm)
+  end: string;     // "18:00" (HH:mm)
+}
+```
+
+### Timezone
+
+All schedule calculations use `Europe/Istanbul` timezone. The system converts the current UTC time to Istanbul time before comparing against schedule windows. Overnight schedules (e.g., `22:00` to `06:00`) are handled correctly.
+
+### Where AI Shift Is Checked
+
+AI shift is checked at three points:
+1. **Grace period expiry** -- if outside AI hours, session resolves as `manual` instead of activating AI
+2. **Message buffer expiry** -- if outside AI hours, buffered messages are not sent to AI
+3. **Replay pending messages** -- if outside AI hours, pending messages are not replayed
+
+---
+
+## 7. Redis State (Target Architecture)
+
+All in-memory Maps are replaced by Redis hashes and BullMQ jobs:
+
+### WebSocket Connection State
+
+```
+Key:    ws:conn:{chatId}
+Type:   Hash
+TTL:    3 hours (10800s)
+Fields:
+  agentId          string     ElevenLabs agent ID
+  sessionId        string     WhatsApp session ID
+  tenantId         string     Tenant ID
+  conversationId   string     ElevenLabs conversation ID
+  ready            '1'|'0'    Whether handshake completed
+  lastActivity     ISO date   Last message activity timestamp
+  signedUrl        string     For reconnecting WebSocket on process restart
+```
+
+**Important:** The actual WebSocket object (`ws`) cannot be stored in Redis. It lives in the NestJS process. Redis stores metadata so the system knows which connections exist. On process restart, connections are re-established from stored signed URLs.
+
+### Timer Replacement
+
+| Current (in-memory) | Target (BullMQ) | Queue | Job ID Pattern |
+|---------------------|------------------|-------|----------------|
+| `sessionTimers: Map<sessionId, Timeout>` | Delayed job | `whatsapp:grace` | `grace:{sessionId}` |
+| `messageBufferTimers: Map<sessionId, Timeout>` | Delayed job | `whatsapp:message-buffer` | `buffer:{sessionId}` |
+| `setInterval(() => sweepInactiveSessions(), 5m)` | Repeatable job | `whatsapp:session-cleanup` | Every 300,000ms |
+| `setInterval(() => sweepConversationCosts(), 10m)` | Repeatable job | `whatsapp:cost-sweep` | Every 600,000ms |
+| `setInterval(() => sweepExpiredOperatorRequests(), 15m)` | Repeatable job | `whatsapp:operator-expiry` | Every 900,000ms |
+
+---
+
+## 8. Patient Profile Management
+
+### Auto-Creation
+
+On first contact, a `Patient` record is created (find-or-create by `clinic_id + phone`) with:
+- `patient_phone`, `patient_display_name`, `patient_first_seen_at`, `patient_last_seen_at`
+- Empty: `patient_notes`, `patient_tags`, `patient_ai_summary`
+
+### AI Summary Update (Post-Session)
+
+After a session resolves (any `resolved_by`), if the session had 2+ messages:
+
+1. Build a timestamped transcript from session messages
+2. Send to the configured AI provider with a detailed system prompt:
+   - Only extract information from `[HASTA]` (patient) lines
+   - Convert relative dates to absolute dates
+   - Structure as bullet points: personal info, health, appointments, communication preferences, preferred language, important notes
+   - Preserve existing summary data not contradicted by new conversation
+3. Extract `PREFERRED_LANGUAGE` from AI response and store separately
+4. Update `patient.patient_ai_summary` (encrypted) and `patient.patient_preferred_language`
+
+This is fire-and-forget -- errors are logged but never block session resolution.
+
+### Session Labeling (Post-Session)
+
+After a session resolves, an AI call generates:
+- `sentiment`: `positive` | `neutral` | `negative`
+- `categories`: e.g., `['appointment_inquiry']`, `['callback_required']`, `['other']`
+- `llm_summary`: 2-3 sentence summary of the patient's request and outcome
+- Creates a `Lead` record linked to the session
+
+---
+
+## 9. Client Tools (AI Tool Calls)
+
+When ElevenLabs sends a `client_tool_call` message, the system dispatches to the appropriate handler:
+
+| Tool | Description | Feature Flag |
+|------|-------------|-------------|
+| `forward_to_operator` | Forward patient media to doctor (see 05-OPERATOR-WORKFLOW.md) | `operator_workflow` |
+| `forward_conversation_to_operator` | Forward conversation text to doctor | `conversation_forwarding` |
+| `send_example_photo` | Send pre-configured guide photo to patient | `operator_workflow` |
+| `check_availability` | Check free appointment slots for a date | `google_calendar` |
+| `reserve_slot` | Book an appointment | `google_calendar` |
+| `list_reservations` | List patient's appointments | `google_calendar` |
+| `cancel_reservation` | Cancel an appointment | `google_calendar` |
+| `reschedule_reservation` | Reschedule an appointment | `google_calendar` |
+| `check_patient_history` | Get patient profile and appointment history | -- |
+| `call_patient` | Initiate outbound voice call to patient | -- |
+
+**Tool result format (critical ElevenLabs requirement):**
+```json
+{
+  "type": "client_tool_result",
+  "tool_call_id": "xxx",
+  "result": "JSON string",
+  "is_error": false
+}
+```
+
+- `type` **must** be lowercase (`client_tool_result`, not `CLIENT_TOOL_RESULT`)
+- `is_error` is **required** -- omitting it causes WS 1008 disconnect
+
+### Target Architecture: Generalized Tool Registry
+
+In the NestJS rewrite, tools are extracted from the monolithic `WhatsAppAgentService` into a generalized `ToolRegistryService` with `@Tool()` decorators. See Section 16 of the Migration Reference for the full design. Channel-specific adapters (`ElevenLabsWsAdapter`, `ElevenLabsHttpAdapter`) translate between AI provider protocols and the generic `ToolContext + ToolResult` interface.
+
+---
+
+## 10. Self-Sent Message Handling
+
+When `is_sender === true` (message sent from the clinic's WhatsApp):
+
+1. **Deduplication:** Check if this message was sent via our API (Meta echo). Match by `msg_external_id` or by `text + sent_via_api + recent timestamp`.
+2. **Waiting session:** If the current session is `waiting`, resolve it as `human` -- a real person responded before the grace period expired.
+3. **Active AI session:** If the session is `active` with no human takeover, auto-switch to human mode (close ElevenLabs WS).
+4. **Store:** Save the message as `sender: 'human', sender_name: 'WhatsApp'`.
+
+---
+
+## 11. Fixed First Message
+
+If `fixed_first_message_enabled` is true on the clinic and a `fixed_first_message` string is configured, the system sends this message to the patient immediately when a new session is created (before the grace period even starts). This is typically a greeting or clinic hours notice.
+
+The message is sent fire-and-forget via Meta Business API and stored as `sender: 'system', sender_name: 'AI'`.
+
+---
+
+## 12. Background Sweeps Summary
+
+| Sweep | Interval | Queue | Purpose |
+|-------|----------|-------|---------|
+| Stuck session sweep | 5 min | `whatsapp:session-cleanup` | Find `waiting` sessions past their `grace_deadline` with no timer, trigger grace expiry |
+| Inactive session sweep | 5 min | `whatsapp:session-cleanup` | Resolve `active` sessions with no activity for 2h |
+| Conversation cost sweep | 10 min | `whatsapp:cost-sweep` | Fetch ElevenLabs conversation cost for resolved sessions (up to 20 per sweep) |
+| Operator request expiry | 15 min | `whatsapp:operator-expiry` | Expire `pending` operator requests past timeout, notify AI |
+| Idle connection cleanup | 1 min | (inline) | Close ElevenLabs WS connections with no activity for 2h |
+
+---
+
+## 13. Error Handling and Recovery
+
+### Session Recovery on Startup
+
+On process start, the system queries all `waiting` sessions from the database:
+- If `grace_deadline` has passed: trigger grace expiry immediately
+- If `grace_deadline` is in the future: set a timer for the remaining time
+
+### Lazy WS Reconnection
+
+When `triggerAiResponse()` finds no live connection:
+1. Clean up the stale connection entry
+2. Create a fresh connection with `isReconnect: true` if the session already had a conversation
+3. The new connection gets full chat context via `contextual_update`
+
+No eager reconnection is attempted on WS close. The system reconnects lazily on the next incoming message.
+
+### Blacklisted Numbers
+
+Numbers in `clinic_blacklisted_numbers` are processed (messages stored) but never trigger sessions or AI responses.