elevenlabs-webhook-nodejs 1.0.0

package/docs/SCALING.md

@@ -0,0 +1,148 @@
+# Scaling the WhatsApp Agent System
+
+## Current Architecture: Single Instance
+
+All runtime state lives in-memory in one Node.js process:
+
+| State | Storage | What It Holds |
+|-------|---------|---------------|
+| `connections` Map | In-memory | Active ElevenLabs WebSocket connections |
+| `sessionTimers` Map | In-memory | Grace period `setTimeout` callbacks |
+| `messageBufferTimers` Map | In-memory | 20s debounce timers before AI responds |
+| 3 sweep intervals | In-memory | Idle cleanup, stuck sessions, inactive sessions |
+
+**File:** `src/services/whatsapp-agent.service.ts`
+
+### Stateless Components (safe for multi-instance)
+- **Unipile API calls** (`src/services/unipile.service.ts`) — HTTP requests, no local state
+- **Webhook handler** (`src/modules/webhooks/unipile.routes.ts`) — reads from DB, delegates to agent service
+- **API routes** (`src/modules/whatsapp/whatsapp.routes.ts`) — standard CRUD, reads from DB
+
+### Session Recovery on Restart
+- `recoverSessions()` runs at startup, queries DB for `status: 'waiting'` sessions
+- Re-creates `setTimeout` timers for pending grace periods
+- Active WS connections are **lost** — lazy reconnect on next incoming message
+- Active sessions continue working but AI goes silent until contact sends another message
+
+---
+
+## What Breaks With 2+ Instances
+
+| Problem | Example | Severity |
+|---------|---------|----------|
+| **Split brain on WS connections** | Webhook arrives on Instance A, but Instance B holds the EL connection | Critical |
+| **Duplicate grace timers** | Both instances create a timer for the same session | Critical |
+| **Duplicate AI calls** | Both instances fire message buffer → 2 EL conversations for 1 chat | Critical |
+| **closeConnection fails silently** | Instance A tries to close a connection that lives on Instance B | High |
+| **Recovery misses active sessions** | `recoverSessions()` only recovers `waiting` sessions, not active WS connections | High |
+
+---
+
+## Scaling Phases
+
+### Phase 1: Graceful Shutdown (Quick Win, No Redis)
+
+**Effort:** ~20 lines in `src/index.ts`
+
+Add SIGTERM/SIGINT handlers:
+```typescript
+process.on('SIGTERM', async () => {
+  whatsappAgentService.shutdown(); // closes all WS, clears timers
+  await app.close();              // stop accepting new requests
+  process.exit(0);
+});
+```
+
+**Benefits:**
+- Clean deploys — no orphaned EL conversations
+- All active WS connections closed properly (EL gets notified)
+- All timers cleared
+
+---
+
+### Phase 2: Redis as Coordination Layer (Multi-Instance)
+
+**Effort:** Major refactor of `whatsapp-agent.service.ts`
+**Requires:** Redis instance added to `docker-compose.yml`
+
+#### 2a. Instance Ownership
+
+Track which instance owns which chat:
+```
+redis key: wa:owner:{unipileChatId} → instanceId (TTL: 30 min)
+```
+
+When a Unipile webhook arrives:
+- Check Redis for the owning instance
+- If **this instance** owns it → process locally
+- If **another instance** owns it → forward via Redis pub/sub
+- If **no owner** → claim ownership, process locally
+
+#### 2b. Timer Coordination
+
+Replace in-memory `setTimeout` with Redis-backed scheduling:
+
+**Grace period:**
+```
+redis key: wa:grace:{sessionId} → { agentId, tenantId } (TTL = grace period)
+```
+Use Redis keyspace notifications (`__keyevent@0__:expired`) or a polling sweep to detect expiry. Only the owning instance processes the expiry.
+
+**Message buffer (20s debounce):**
+```
+redis key: wa:buffer:{sessionId} → { tenantId, agentId } (TTL = 20s)
+```
+On new message: reset TTL (debounce). On expiry: owning instance triggers AI.
+
+#### 2c. Connection Registry
+
+Track EL connections in Redis so any instance can check if a connection exists:
+```
+redis key: wa:conn:{unipileChatId} → { instanceId, sessionId, conversationId }
+```
+
+Cross-instance operations via pub/sub:
+- Instance A publishes: `wa:close:{unipileChatId}`
+- Instance B (owner) receives and closes the actual WebSocket
+
+---
+
+### Phase 3: MongoDB Replica Set
+
+**Effort:** `docker-compose.yml` + connection config
+**Independent of Redis changes**
+
+- Convert standalone MongoDB to a 3-node replica set
+- Enables transactions for atomic session state changes
+- Provides automatic failover
+- Required for production-grade deployments
+
+---
+
+## Target Architecture
+
+```
+Current:
+  Unipile → Fastify (1 instance) → MongoDB (standalone)
+
+Scaled:
+  Unipile → Load Balancer → Fastify (N instances) → MongoDB (replica set)
+                                  ↕
+                                Redis
+                             (pub/sub + state)
+```
+
+---
+
+## When To Scale
+
+**Current single-instance handles the workload fine if:**
+- < 100 concurrent active chats
+- Single server with enough memory for WebSocket connections
+- Acceptable 1-2s downtime on deploys (sessions auto-recover on startup)
+
+**Consider scaling when:**
+- Need zero-downtime deploys
+- Approaching memory/CPU limits on single instance
+- Need geographic distribution
+- Regulatory requirement for high availability

package/docs/SESSION_SUMMARY_2026_03_17.md

@@ -0,0 +1,135 @@
+# Development Session Summary — 2026-03-17
+
+## Commits
+
+| Hash | Description |
+|------|-------------|
+| `8caa987` | Inbound call schedule with ElevenLabs phone number management |
+| `afb2c84` | ElevenLabs phone number dropdown for appointment validation + agent fetch limit 100 |
+| `e68c1ad` | Fix removeChild DOM errors (Google Translate + hard navigation) |
+| `467be44` | WhatsApp voice message transcription (STT) with on-demand audio download |
+| `9ef581a` | Multi-service plan quotas, billing period snapshots, dashboard redesign |
+| `2626271` | Hide plan name from tenant dashboard |
+| `5918eba` | Daily billing snapshot cron at 00:05 Istanbul |
+| `9cef1a9` | Daily billing SMS alerts + fix billable chat counting |
+| `190a298` | Billing SMS alerts documentation |
+
+## Features Implemented
+
+### 1. Inbound Call Schedule
+- `InboundCallConfig` model: tenant ↔ ElevenLabs phone number with answer schedule
+- Background service syncs agent assignment every 15 min at clock boundaries (:00, :15, :30, :45)
+- Schedule = blocked hours when agent should NOT answer calls
+- Auto-syncs agent/phone/label changes from ElevenLabs dashboard
+- Superadmin page: assign phone numbers to tenants, configure schedule, change agent, manual toggle
+- Tenant page: schedule editor integrated into Call Settings (feature flag: `inbound_call_schedule`)
+- ElevenLabs API: `phoneNumbers.update(id, { agentId })` to assign, `{ agentId: '' }` to unassign
+
+### 2. WhatsApp Voice Message Transcription (STT)
+- Feature flag: `voice_transcription`
+- On-demand audio download: Baileys REST endpoint `GET /chats/:id/download-audio/:msgId` only pulls audio when tenant has feature enabled
+- ElevenLabs Scribe v1 STT with auto language detection (no hardcoded hint)
+- Webhook handler transcribes audio before AI sees it, changes `content_type` to `transcribedAudio`
+- Falls back to label text on any failure (download, STT, empty result)
+- `SttUsage` model tracks `duration_seconds`, `character_count`, `detected_language` per transcription
+- Frontend: mic icon + "Sesli mesaj transkripsiyonu" label on transcribed message bubbles
+
+### 3. Multi-Service Plan Quotas
+- Plan model extended: `included_inbound_minutes`, `included_inbound_calls`, `included_outbound_minutes`, `included_outbound_calls`, `included_wa_chats`
+- All default 0, backward compatible with existing `included_minutes`
+- Admin can set quotas per plan — e.g., "200 dk + 500 WA sohbet"
+- Same plan assigned to different tenants at different prices via `Payment.amount`
+
+### 4. Billing Period Resolution
+- `getUsageSummary()` uses active payment period, not calendar month
+- Fallback chain: active payment → last payment's end date → tenant.createdAt
+- 6 parallel aggregations: calls, charges, top-ups, WA messages, WA sessions, STT
+- Fixed AI message count: `sender_name: 'AI'` (was incorrectly `sender: 'ai'`)
+- Fixed billable WA chats: only counts chats where AI responded (`sender_name === 'AI'`)
+
+### 5. Billing Period Snapshots
+- `BillingPeriodSnapshot` model: immutable usage record per billing period
+- Triggered on new subscription payment creation (only if previous period expired)
+- Daily cron at 00:05 Istanbul: finds expired periods and creates snapshots
+- Deduplicated by `payment_id`
+- Tenant-facing endpoint: `GET /billing/snapshots`
+
+### 6. Dashboard Redesign
+- Per-service usage bars: WhatsApp chats first, then inbound calls (minutes/count)
+- Legacy combined bar only shows as fallback when no per-service quotas set
+- WhatsApp stat cards: billable chats, AI sessions, total messages, AI messages, STT count
+- Daily trend charts: inbound calls (bar chart), WA AI messages (area chart)
+- Period dates displayed at top
+- Plan name hidden from tenants (shows "Kullanim" instead)
+- `GET /billing/daily-trend` endpoint for chart data
+
+### 7. Billing SMS Alerts
+- Daily at 09:00 Istanbul via NetGSM SMS
+- Usage threshold alerts: 75%, 90%, 100% (highest threshold only per service)
+- Period expiry alert: 3 days before end, only if no future payment exists
+- `BillingAlert` model with unique index for deduplication per billing period
+- Gated on `sms_notifications` feature + `sms_notification_enabled` + `sms_notification_phones[]`
+- Generic messages without service names or numbers
+
+### 8. Admin Panel Improvements
+- Agent direction selector (inbound/outbound/both) in create + edit forms
+- Tenant soft delete: `PUT /admin/tenants/:id/deactivate` (sets `is_active: false`)
+- Tenant hard delete: `DELETE /admin/tenants/:id` (tombstones data, mangles unique fields with `___deleted_timestamp`)
+- Must deactivate before hard delete, confirmation dialogs
+- Hidden deactivated tenants from admin list
+- Payment period defaults: new period starts day after previous ends
+- Billable items page: billing period dropdown from subscription payments
+- Tenants without payments show "Ucretsiz" period from creation date
+- Appointment validation: ElevenLabs phone number dropdown (was text input)
+- Scrollable agent create/edit dialogs (max-h-[85vh])
+- No WhatsApp notifications for outbound calls
+
+### 9. Bug Fixes
+- removeChild DOM errors: `lang="tr"` + `translate="no"` + `notranslate` meta tag
+- Auth logout: `window.dispatchEvent` instead of `window.location.href` (no DOM teardown race)
+- Badge rendering: single element with dynamic props instead of conditional swap
+- Inbound config fetch: `useRef` guard to prevent duplicate fetches
+
+## New Models Created
+
+| Model | Collection | Purpose |
+|-------|-----------|---------|
+| `InboundCallConfig` | `inboundcallconfigs` | ElevenLabs phone number ↔ tenant with schedule |
+| `SttUsage` | `sttusages` | Voice transcription tracking per message |
+| `BillingPeriodSnapshot` | `billingperiodsnapshots` | Immutable usage records per billing period |
+| `BillingAlert` | `billingalerts` | SMS alert deduplication |
+
+## New API Endpoints
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/inbound-call/phone-numbers` | List ElevenLabs phone numbers (superadmin) |
+| POST | `/inbound-call/configs` | Assign phone number to tenant |
+| DELETE | `/inbound-call/configs/:id` | Remove assignment |
+| PUT | `/inbound-call/configs/:id/admin-agent` | Change agent (superadmin) |
+| PUT | `/inbound-call/configs/:id/admin-schedule` | Update schedule (superadmin) |
+| POST | `/inbound-call/configs/:id/admin-toggle` | Manual assign/unassign (superadmin) |
+| GET | `/inbound-call/configs` | List tenant configs |
+| PUT | `/inbound-call/configs/:id/schedule` | Update schedule (tenant) |
+| POST | `/inbound-call/configs/:id/toggle-now` | Manual toggle (tenant) |
+| GET | `/billing/daily-trend` | Daily usage trend for charts |
+| GET | `/billing/snapshots` | Past billing period summaries |
+| PUT | `/admin/tenants/:id/deactivate` | Soft delete tenant |
+| PUT | `/admin/tenants/:id/reactivate` | Undo soft delete |
+| DELETE | `/admin/tenants/:id` | Hard delete (tombstone) |
+
+## Background Jobs
+
+| Schedule | Function | Timezone |
+|----------|----------|----------|
+| Every 15 min (:00, :15, :30, :45) | `inboundCallScheduleService.syncAllSchedules()` | Europe/Istanbul |
+| Daily 00:05 | `snapshotExpiredPeriods()` | Europe/Istanbul |
+| Daily 09:00 | `checkAndSendBillingAlerts()` | Europe/Istanbul |
+| Every 60s | `syncActiveBatches()` | Server time |
+
+## Feature Flags Added
+
+| Flag | Purpose |
+|------|---------|
+| `inbound_call_schedule` | Tenant access to inbound call schedule config |
+| `voice_transcription` | WhatsApp voice message STT |