elevenlabs-webhook-nodejs 1.0.0

package/docs/architecture/00-SYSTEM-OVERVIEW.md

@@ -0,0 +1,318 @@
+# System Overview
+
+> Portal Asistan 7/24 -- NestJS Architecture Reference
+> Last updated: 2026-04-02
+
+---
+
+## 1. What Is Portal Asistan 7/24
+
+Portal Asistan 7/24 is a multi-tenant clinic management SaaS platform built for Turkish healthcare clinics. It manages patients, treatment plans, appointments, and day-to-day clinic operations -- with integrated AI assistants for WhatsApp, phone calls, and Instagram DMs. The patient is the central entity; the AI is a powerful feature, not the entire product.
+
+Each tenant is a clinic or clinic chain. A tenant may operate multiple clinics (branches), each with its own phone numbers, calendars, and operator workflows. The system supports both single-clinic practices and multi-location organizations.
+
+**Core capabilities:**
+
+- Patient management (CRM) with medical history, anamnesis, documents, notes, and timeline
+- Treatment catalog and multi-visit treatment plans per clinic
+- Appointment scheduling (manual from dashboard + AI-powered via WhatsApp/phone)
+- Before/after photo documentation for procedures
+- Inbound/outbound voice calls via ElevenLabs AI agents with SIP trunking
+- WhatsApp AI chat agent with operator-in-the-loop workflows for medical photo review
+- Instagram DM support via Meta Business API
+- Google Calendar integration for appointment booking, cancellation, and rescheduling
+- Automated appointment reminders via WhatsApp and SMS
+- Batch outbound calling for appointment validation and custom campaigns
+- Patient payment collection via TekTıpPay
+- Contact profiling, lead generation, and call transcription/summarization
+- Per-tenant billing with plan-based usage limits
+- KVKK/GDPR compliance workflows (data export, deletion, retention)
+- Real-time dashboard with live chat takeover capability
+
+**Target market:** Dental clinics, aesthetic surgery centers, dermatology practices, and general healthcare providers in Turkey.
+
+---
+
+## 2. Target Tech Stack
+
+### Backend
+
+| Layer | Technology | Purpose |
+|-------|-----------|---------|
+| Framework | NestJS + Fastify adapter | Structured modular framework with the performance of Fastify |
+| Language | TypeScript (strict mode) | End-to-end type safety |
+| Database | PostgreSQL + Prisma ORM | Relational data with typed queries and migrations |
+| Cache and Queues | Redis + BullMQ | Session state, token blacklist, job queues, pub/sub |
+| Object Storage | MinIO (self-hosted, S3-compatible) | Patient media, exports, recordings -- per-tenant encrypted |
+| Auth | JWT + Passport.js + Redis blacklist | Stateless access tokens, instant revocation |
+| API Documentation | Swagger (auto-generated via NestJS decorators) | Always-current API docs |
+
+### AI
+
+| Layer | Technology | Purpose |
+|-------|-----------|---------|
+| Unified Interface | Custom AIModule (in-codebase provider pattern) | No third-party AI abstraction -- own code, full control |
+| Chat Provider A | OpenAI GPT-4o (via `openai` SDK) | Text generation, summarization |
+| Chat Provider B | Anthropic Claude Sonnet (via `@anthropic-ai/sdk`) | Alternative AI provider, per-tenant configurable |
+| Voice | ElevenLabs (WebSocket for WhatsApp, SIP trunk for calls) | AI voice agent and text-only chat sessions |
+
+### Channels and Integrations
+
+| Layer | Technology | Purpose |
+|-------|-----------|---------|
+| WhatsApp | Meta Business Cloud API | AI chat, templates, notifications, media |
+| Instagram DMs | Meta Business API (Messenger Platform) | Instagram direct message AI agent |
+| Phone / SIP | ElevenLabs SIP trunk | Inbound and outbound voice calls |
+| Calendar | Google Calendar API v3 | Appointment booking, availability checking |
+| Payments | TekTıpPay (placeholder) | Patient payment link generation and tracking -- stub module, implemented later |
+| SMS | NetGSM (placeholder) | SMS notifications and appointment reminders -- stub module, implemented later |
+| File Storage | MinIO | All media, exports, recordings -- signed URLs only |
+
+> **Note:** Baileys (self-hosted WhatsApp) is not included in the launch scope. Meta Business Cloud API is the sole WhatsApp provider. Baileys can be added later as a secondary provider for custom workflows if needed.
+
+---
+
+## 3. Architecture Diagram
+
+```
+                        +--------------------------+
+                        |   React (Vite + shadcn)  |
+                        |       :3020              |
+                        +------------+-------------+
+                                     |
+                                     | HTTP / WS (socket.io)
+                                     |
+                        +------------v-------------+
+                        |  NestJS + Fastify :3005   |
+                        |                           |
+                        |  Modules:                 |
+                        |   Auth, Tenants, Users,   |
+                        |   Clinics, Patients,      |
+                        |   Treatments,             |
+                        |   PhoneNumbers, Agents,   |
+                        |   Calls, Leads, WhatsApp, |
+                        |   Billing, Campaigns,     |
+                        |   Validation, Tools,      |
+                        |   Webhooks, Platform,     |
+                        |   Admin, Audit            |
+                        +--+-------+-------+-------+
+                           |       |       |
+              +------------+   +---+---+   +------------+
+              |                |       |                |
+   +----------v----+   +------v--+  +-v---------+
+   |  PostgreSQL    |   |  Redis  |  |  MinIO    |
+   |  (Prisma ORM)  |   | BullMQ  |  | (S3-compat)|
+   |  :5432         |   | :6379   |  | :9000     |
+   +----------------+   +----+----+  +-----------+
+                              |
+              +---------------+---------------+
+              |               |               |
+     +--------v---+  +-------v----+  +-------v------+
+     | ElevenLabs |  | Google Cal |  | Meta Business|
+     | (WS + SIP) |  | API v3     |  | API (WA+IG)  |
+     +------------+  +------------+  +--------------+
+```
+
+---
+
+## 4. Architecture Rules
+
+Five non-negotiable rules that govern every design decision in the new system:
+
+### Rule 1: Every DB Query Is Tenant-Scoped
+
+Enforced at the Prisma middleware layer. No query can return data from another tenant. Cross-tenant queries require the `superadmin` role plus an explicit `allTenants: true` flag. Clinic-level scoping is layered on top for `admin`, `doctor`, and `receptionist` roles.
+
+### Rule 2: Modules Communicate via EventEmitter
+
+Never import a service directly across module boundaries. Use `@nestjs/event-emitter` for inter-module communication. Example: the Webhooks module emits `whatsapp.message.received`, and the WhatsApp module subscribes to it. This keeps modules decoupled and independently testable.
+
+### Rule 3: No Stateful Data in Application Instances
+
+All runtime state lives in Redis or the database. WebSocket connection metadata, session timers, message buffer timers, token blacklists, Google Calendar token caches -- all in Redis. BullMQ handles all delayed jobs that were previously `setTimeout`/`setInterval`. The NestJS process can restart without losing state.
+
+### Rule 4: All Sensitive Files Served via Signed URLs
+
+Every file in MinIO is private. Access requires a pre-signed URL with a 1-hour TTL. No public URLs, no direct file serving. Patient media, operator photos, exports, recordings -- all behind signed URLs. MinIO buckets are prefixed with `{tenantId}/` to enforce tenant isolation at the storage layer.
+
+### Rule 5: No Base64 in the Database
+
+All media is stored in MinIO, referenced by an object key in the database. This keeps the database lean and queryable.
+
+---
+
+## 5. Docker Compose Topology
+
+```
++-----------------------------------------------------------+
+|  asistan724-net (internal Docker network)                  |
+|                                                            |
+|  +-------------+    +------------+    +----------+         |
+|  |   NestJS    |<-->| PostgreSQL |    |  MinIO   |         |
+|  |   :3005     |    |  :5432     |    |  :9000   |         |
+|  | (Fastify)   |    | (internal) |    | (internal)|        |
+|  +------+------+    +------------+    +----------+         |
+|         |                                                  |
+|         |  TLS        +------------+                       |
+|         +------------>|   Redis    |                       |
+|                       |   :6379    |                       |
+|                       | (internal) |                       |
+|                       +------------+                       |
+|                                                            |
+|  Exposed to host: NestJS (:3005), React (:3020)            |
+|  Everything else: internal network only                    |
++-----------------------------------------------------------+
+```
+
+**Key points:**
+
+- PostgreSQL, Redis, and MinIO have **no port binding to the host** -- only reachable within the Docker network.
+- NestJS binds to the host on `PORT` (behind a reverse proxy with TLS termination).
+- React dev server binds to the host on `:3020` (same reverse proxy).
+- WhatsApp communication uses Meta Business Cloud API directly (no self-hosted bridge).
+
+---
+
+## 6. Module List
+
+Every NestJS module in the system, organized by layer.
+
+### Infrastructure Modules
+
+| Module | Path | Description |
+|--------|------|-------------|
+| `ConfigModule` | `src/config/` | Environment variables with Zod validation |
+| `DatabaseModule` | `src/database/` | PrismaClient with tenant-scoping middleware |
+| `RedisModule` | `src/redis/` | Redis connection, BullMQ queue registration |
+| `StorageModule` | `src/storage/` | MinIO client, signed URL generation, bucket management |
+
+### Core Domain Modules
+
+| Module | Path | Description |
+|--------|------|-------------|
+| `AuthModule` | `src/auth/` | JWT authentication, Passport strategies, Redis token blacklist, 2FA |
+| `TenantsModule` | `src/tenants/` | Tenant CRUD, settings management, feature flag resolution |
+| `UsersModule` | `src/users/` | User CRUD, invitation flow, role assignment, clinic assignments |
+| `ClinicsModule` | `src/clinics/` | Multi-location clinic management, settings inheritance from tenant |
+| `PhoneNumbersModule` | `src/phone-numbers/` | Unified phone number registry across all purposes (WhatsApp, inbound, outbound), schedule sync |
+| `PlansModule` | `src/plans/` | Subscription plan definitions and limits |
+| `AgentsModule` | `src/agents/` | ElevenLabs agent configuration, per-tenant and per-clinic agents |
+
+| `PatientsModule` | `src/patients/` | Patient CRM: demographics, medical history, documents, notes, photos, timeline |
+| `TreatmentsModule` | `src/treatments/` | Treatment/service catalog per clinic, treatment plans with multi-visit items |
+
+> **Migration note:** The `WhatsAppContactProfile` model is replaced by the `Patient` model. ContactProfile data migrates into Patient.
+
+### Communication Modules
+
+| Module | Path | Description |
+|--------|------|-------------|
+| `WhatsAppModule` | `src/whatsapp/` | Core AI chat engine, session lifecycle, operator workflow, BullMQ processors (grace, buffer, cleanup, cost sweep, expiry, profile, labels) |
+| `CallsModule` | `src/calls/` | Voice call records, transcripts, detail separation for performance |
+| `LeadsModule` | `src/leads/` | Lead generation from calls and chat sessions, status tracking |
+| `OutboundCampaignsModule` | `src/outbound-campaigns/` | Customizable outbound call campaigns with agent config, extraction schema, dispatch and sync processors |
+| `AppointmentValidationModule` | `src/appointment-validation/` | Batch outbound calling to confirm existing appointments |
+| `GoogleCalendarModule` | `src/google-calendar/` | OAuth flow, availability checking, slot reservation, Redis-cached tokens |
+
+### Tools Module
+
+| Module | Path | Description |
+|--------|------|-------------|
+| `ToolsModule` | `src/tools/` | Generalized client tools system -- `@Tool()` decorator, registry, adapters (ElevenLabs WS, ElevenLabs HTTP, OpenAI), 13 tool handlers across calendar, operator, patient, validation, payment, and notification categories |
+
+### Integration Modules
+
+| Module | Path | Description |
+|--------|------|-------------|
+| `ElevenLabsModule` | `src/integrations/elevenlabs/` | ElevenLabs SDK wrapper for agents, conversations, signed URLs, batch calls, SIP, STT |
+| `MetaBusinessModule` | `src/integrations/meta-business/` | Meta Business API client for WhatsApp Cloud API and Instagram Messenger -- sole WhatsApp provider |
+| `AIModule` | `src/integrations/ai/` | Custom provider pattern -- OpenAI and Anthropic providers behind unified interface |
+| `NetGSMModule` | `src/integrations/netgsm/` | **Stub.** Interface + placeholder service. SMS sending via NetGSM -- implemented later |
+| `TekTipPayModule` | `src/integrations/tektippay/` | **Stub.** Interface + placeholder service. Patient payment links -- implemented later |
+| `MinIOModule` | `src/integrations/minio/` | S3-compatible file operations with signed URL generation |
+
+### Webhook and Event Modules
+
+| Module | Path | Description |
+|--------|------|-------------|
+| `WebhooksModule` | `src/webhooks/` | Inbound webhook controllers for ElevenLabs (post-call, tools) and Meta Business (WhatsApp + Instagram) |
+
+### Platform and Admin Modules
+
+| Module | Path | Description |
+|--------|------|-------------|
+| `PlatformModule` | `src/platform/` | Sales team endpoints -- tenant onboarding, plan assignment, tenant listing |
+| `AdminModule` | `src/admin/` | Superadmin endpoints -- system health, agent provisioning, test chat, destructive operations |
+| `BillingModule` | `src/billing/` | Usage tracking, billing events, payments, period snapshots, alerts |
+
+### Observability Modules
+
+| Module | Path | Description |
+|--------|------|-------------|
+| `LoggingModule` | `src/logging/` | Pino-based structured logging, async batched PostgreSQL writer, request context with correlation IDs |
+| `AuditModule` | `src/audit/` | Immutable audit trail -- `@Auditable()` decorator, before/after diff, 30+ tracked actions, real-time WS push |
+
+### Common (Shared)
+
+| Location | Contents |
+|----------|----------|
+| `src/common/guards/` | `JwtAuthGuard`, `PermissionsGuard`, `FeaturesGuard`, `ClinicAccessGuard` |
+| `src/common/decorators/` | `@Permissions()`, `@Features()`, `@CurrentUser()`, `@AccessibleClinics()`, `@RequireAllPermissions()` |
+| `src/common/interceptors/` | `RequestContextInterceptor` (AsyncLocalStorage for correlation IDs), `TenantScopeInterceptor` |
+| `src/common/filters/` | `HttpExceptionFilter` (standardized error responses) |
+| `src/common/interfaces/` | `TenantSettings`, `ClinicSettings` TypeScript interfaces |
+
+---
+
+## 7. Feature Flags
+
+Stored in `tenant.tenant_enabled_features` (String array in PostgreSQL). Checked at the guard level via the `@Features()` decorator and `FeaturesGuard`.
+
+| # | Feature Flag | Gated Routes / Tools | Description |
+|---|-------------|---------------------|-------------|
+| 1 | `inbound_agent` | -- | Inbound voice call handling via ElevenLabs SIP trunk |
+| 2 | `outbound_campaigns` | `/outbound-campaigns/*` | Customizable outbound call campaigns (surveys, follow-ups, marketing) |
+| 3 | `whatsapp_notifications` | -- | WhatsApp notification sending to staff and patients |
+| 4 | `whatsapp_agent` | `/whatsapp/*` | WhatsApp AI chat agent (core feature) |
+| 5 | `google_calendar` | Calendar tools (`check_availability`, `reserve_slot`, `list_reservations`, `cancel_reservation`, `reschedule_reservation`) | Google Calendar integration for appointment booking |
+| 6 | `operator_workflow` | Operator tools (`forward_to_operator`, `send_example_photo`) | Operator-in-the-loop workflow -- patient sends photo, AI forwards to doctor, doctor replies with price |
+| 7 | `sms_notifications` | `send_sms` tool | SMS sending via NetGSM for reminders and notifications |
+| 8 | `appointment_validation` | `/appointment-validation/*`, `confirm_appointment_validation` tool | Batch outbound calling to confirm upcoming appointments |
+| 9 | `inbound_call_schedule` | `/inbound-call/configs/*/schedule` | Schedule-based agent assignment for inbound calls (time-of-day routing) |
+| 10 | `voice_transcription` | -- | Speech-to-text for WhatsApp voice messages (ElevenLabs Scribe) |
+| 11 | `chat_continuity` | -- | WebSocket reconnection with conversation history (prevents AI re-introduction) |
+| 12 | `conversation_forwarding` | `forward_conversation_to_operator` tool | Forward full conversation text (not just media) to operator |
+| 13 | `payments` | `request_payment` tool | Patient payment link generation via TekTıpPay |
+
+**How flags are enforced:**
+
+```
+Request --> JwtAuthGuard --> PermissionsGuard --> FeaturesGuard --> Controller
+                                                      |
+                                           Check tenant.enabled_features
+                                           against @Features() metadata
+                                                      |
+                                          403 if required flag is missing
+```
+
+Flags are also checked at the tool execution layer: each `@Tool()` handler declares `requiredFeatures`, and the `ToolRegistryService` verifies them before execution.
+
+---
+
+## 8. Security Posture (Summary)
+
+Security is not a separate phase -- it is built into every migration phase from day one.
+
+| Layer | Implementation |
+|-------|---------------|
+| Tenant isolation | Prisma middleware auto-injects `WHERE tenantId` on every query |
+| Authentication | JWT access tokens (15min) + refresh tokens (30d) + Redis blacklist |
+| Authorization | Permission-based RBAC with clinic scoping |
+| Encryption at rest | AES-256-GCM for OAuth tokens, Baileys auth state, Meta tokens, TekTıpPay creds, 2FA secrets |
+| Encryption in transit | TLS everywhere (PostgreSQL, Redis, MinIO, inter-service) |
+| Input validation | `class-validator` DTOs on every endpoint |
+| Rate limiting | `@nestjs/throttler` with Redis storage |
+| Webhook auth | HMAC signature verification (ElevenLabs, Meta) or API key header (Baileys) |
+| File access | MinIO signed URLs with 1-hour TTL only |
+| Audit trail | Immutable `audit_logs` table, never auto-deleted |
+| 2FA | TOTP for admin/owner roles (optional for others) |