@tiledesk/tiledesk-voice-twilio-connector 0.2.0-rc3 → 0.2.0-rc6

package/README.md

@@ -1,44 +1,515 @@
-# Hello Node!
+# Tiledesk Voice Twilio Connector
 
-This project includes a Node.js server script and a web page that connects to it. The front-end page presents a form the visitor can use to submit a color name, sending the submitted value to the back-end API running on the server. The server returns info to the page that allows it to update the display with the chosen color. 🎨
+[![npm version](https://img.shields.io/npm/v/@tiledesk/tiledesk-voice-twilio-connector.svg)](https://www.npmjs.com/package/@tiledesk/tiledesk-voice-twilio-connector)
+[![Node.js Version](https://img.shields.io/node/v/@tiledesk/tiledesk-voice-twilio-connector.svg)](https://nodejs.org/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-[Node.js](https://nodejs.org/en/about/) is a popular runtime that lets you run server-side JavaScript. This project uses the [Fastify](https://www.fastify.io/) framework and explores basic templating with [Handlebars](https://handlebarsjs.com/).
+A production-ready Node.js connector that bridges **Twilio Voice API** with the **Tiledesk** conversational platform, enabling real-time voice interactions with AI-powered chatbots and human agents.
 
-_Last updated: 14 August 2023_
+## 📋 Table of Contents
 
-## Prerequisites
+- [What is it?](#what-is-it)
+- [What does it do?](#what-does-it-do)
+- [How does it work?](#how-does-it-work)
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+- [Configuration](#configuration)
+- [Architecture](#architecture)
+- [API Reference](#api-reference)
+- [Usage Examples](#usage-examples)
+- [Deployment](#deployment)
 
-You'll get best use out of this project if you're familiar with basic JavaScript. If you've written JavaScript for client-side web pages this is a little different because it uses server-side JS, but the syntax is the same!
+---
 
-## What's in this project?
+## 🎯 What is it?
 
-← `README.md`: That’s this file, where you can tell people what your cool website does and how you built it.
+The **Tiledesk Voice Twilio Connector** is a middleware service that enables voice-based conversations through Twilio's telephony infrastructure, integrated seamlessly with Tiledesk's messaging and AI platform.
 
-← `public/style.css`: The styling rules for the pages in your site.
+### Key Features
 
-← `server.js`: The **Node.js** server script for your new site. The JavaScript defines the endpoints in the site back-end, one to return the homepage and one to update with the submitted color. Each one sends data to a Handlebars template which builds these parameter values into the web page the visitor sees.
+- 🎙️ **Real-time Voice Calls**: Handle inbound and outbound voice calls via Twilio
+- 🤖 **AI Integration**: Connect voice interactions with Tiledesk's AI chatbots
+- 💬 **Unified Messaging**: Convert voice to text and text to voice for seamless conversations
+- 🔄 **Bidirectional Communication**: Sync messages between Twilio and Tiledesk in real-time
+- 📊 **Session Management**: Track and manage call sessions with Redis and MongoDB
+- 🎚️ **Speech Services**: Support for OpenAI Whisper (STT) and ElevenLabs (TTS)
+- ⚡ **Event-Driven**: Asynchronous message processing with queue-based architecture
 
-← `package.json`: The NPM packages for your project's dependencies.
+---
 
-← `src/`: This folder holds the site template along with some basic data files.
+## 🚀 What does it do?
 
-← `src/pages/index.hbs`: This is the main page template for your site. The template receives parameters from the server script, which it includes in the page HTML. The page sends the user submitted color value in the body of a request, or as a query parameter to choose a random color.
+The connector acts as a **translation layer** between two systems:
 
-← `src/colors.json`: A collection of CSS color names. We use this in the server script to pick a random color, and to match searches against color names.
+1. **Twilio Voice API** - Handles telephony infrastructure (calls, recordings, speech recognition)
+2. **Tiledesk Platform** - Manages conversations, AI chatbots, and agent routing
 
-← `src/seo.json`: When you're ready to share your new site or add a custom domain, change SEO/meta settings in here.
+### Use Cases
 
-## Try this next 🏗️
+- **Voice-Enabled Customer Support**: Allow customers to call and interact with AI chatbots via voice
+- **IVR Systems**: Build interactive voice response systems with natural language understanding
+- **Voice Assistants**: Create voice-powered assistants that can handle complex conversations
+- **Call Center Integration**: Route voice calls to human agents through Tiledesk's routing system
+- **Multi-Channel Support**: Unify voice, chat, and messaging in a single conversation thread
 
-Take a look in `TODO.md` for next steps you can try out in your new site!
+### Workflow Example
 
-___Want a minimal version of this project to build your own Node.js app? Check out [Blank Node](https://glitch.com/edit/#!/remix/glitch-blank-node)!___
+```
+1. Customer calls Twilio number
+   ↓
+2. Connector receives call webhook
+   ↓
+3. Creates Tiledesk conversation session
+   ↓
+4. Converts speech to text (OpenAI Whisper)
+   ↓
+5. Sends message to Tiledesk AI chatbot
+   ↓
+6. Receives AI response from Tiledesk
+   ↓
+7. Converts text to speech (ElevenLabs)
+   ↓
+8. Plays audio to caller via Twilio
+```
 
-![Glitch](https://cdn.glitch.com/a9975ea6-8949-4bab-addb-8a95021dc2da%2FLogo_Color.svg?v=1602781328576)
+---
 
-## You built this with Glitch!
+## ⚙️ How does it work?
 
-[Glitch](https://glitch.com) is a friendly community where millions of people come together to build web apps and websites.
+### Core Components
+
+#### 1. **Voice Channel** (`VoiceChannel`)
+- Manages call session lifecycle
+- Implements message queue polling with exponential backoff
+- Handles Redis-based state management
+- Coordinates event-driven communication
+
+#### 2. **Tiledesk Channel** (`TiledeskChannel`)
+- Communicates with Tiledesk REST API
+- Manages conversation sessions
+- Handles message queuing and retrieval
+- Syncs call status with Tiledesk
+
+#### 3. **Twilio Translator** (`TiledeskTwilioTranslator`)
+- Converts Tiledesk message format to Twilio TwiML
+- Generates TwiML verbs (Say, Play, Gather, Record)
+- Handles audio file generation and streaming
+- Manages call flow state transitions
+
+#### 4. **AI Services** (`AiService`)
+- **Speech-to-Text**: Uses OpenAI Whisper API to transcribe audio
+- **Text-to-Speech**: Uses ElevenLabs API to generate natural voice audio
+- Supports both SDK and HTTP-based implementations
+- Automatic fallback mechanisms
+
+#### 5. **Session Service** (`SessionService`)
+- Creates and manages call sessions
+- Handles conversation initialization
+- Manages integration settings per project
+- Tracks call metadata
+
+#### 6. **Message Service** (`MessageService`)
+- Polls Tiledesk for new messages
+- Implements intelligent polling with delay management
+- Handles message queuing and processing
+- Manages message lifecycle
+
+### Message Flow
+
+```
+┌─────────────┐         ┌──────────────┐         ┌─────────────┐
+│   Twilio    │◄───────►│   Connector   │◄───────►│  Tiledesk   │
+│  Voice API  │  Webhooks│  (This App)  │  REST API│  Platform   │
+└─────────────┘         └──────────────┘         └─────────────┘
+                              │
+                              │
+                    ┌─────────┴─────────┐
+                    │                   │
+              ┌─────▼─────┐      ┌──────▼──────┐
+              │   Redis   │      │  MongoDB    │
+              │  (Queue)  │      │  (Storage)  │
+              └───────────┘      └─────────────┘
+```
+
+### Request/Response Cycle
+
+1. **Incoming Call**:
+   - Twilio sends webhook to `/webhook/:projectId`
+   - Connector creates Tiledesk conversation
+   - Returns TwiML to start call flow
+
+2. **Voice Input**:
+   - User speaks → Twilio records audio
+   - Connector receives audio URL
+   - Transcribes with OpenAI Whisper
+   - Sends text to Tiledesk
+
+3. **AI Response**:
+   - Tiledesk processes message
+   - AI generates response
+   - Connector polls for response
+   - Converts text to speech with ElevenLabs
+   - Returns TwiML to play audio
+
+4. **Call Management**:
+   - Status updates via webhooks
+   - Session state in Redis
+   - Call metadata in MongoDB
+
+---
+
+## 📦 Installation
+
+### Prerequisites
+
+- **Node.js** >= 18.0.0
+- **npm** >= 8.15.0
+- **MongoDB** (for session storage)
+- **Redis** (for message queue, recommended)
+
+### Install as NPM Package
+
+```bash
+npm install @tiledesk/tiledesk-voice-twilio-connector
+```
+
+### Install from Source
+
+```bash
+git clone https://github.com/Tiledesk/tiledesk-voice-twilio-connector.git
+cd tiledesk-voice-twilio-connector
+npm install
+```
+
+---
+
+## 🏃 Quick Start
+
+### Standalone Mode
+
+1. **Set environment variables**:
+
+```bash
+export API_URL=https://api.tiledesk.com
+export MONGODB_URI=mongodb://localhost:27017/tiledesk
+export REDIS_HOST=localhost
+export REDIS_PORT=6379
+export GPT_KEY=your-openai-api-key
+```
+
+2. **Start the server**:
+
+```bash
+npm start
+```
+
+The server will start on port `3000` (or `PORT` environment variable).
+
+### As NPM Package
+
+```javascript
+const { startServer } = require('@tiledesk/tiledesk-voice-twilio-connector');
+
+// Start with environment variables
+await startServer();
+
+// Or pass configuration directly
+await startServer({
+  MONGODB_URI: 'mongodb://localhost:27017/tiledesk',
+  API_URL: 'https://api.tiledesk.com',
+  REDIS_HOST: 'localhost',
+  REDIS_PORT: 6379,
+  GPT_KEY: 'your-openai-key'
+});
+```
+
+---
+
+## ⚙️ Configuration
+
+### Environment Variables
+
+| Variable | Required | Description | Default |
+|----------|----------|-------------|---------|
+| `PORT` | No | Server port | `3000` |
+| `API_URL` | Yes | Tiledesk API base URL | - |
+| `BASE_URL` | Yes | Base URL for webhooks | - |
+| `BASE_FILE_URL` | Yes | Base URL for file uploads | - |
+| `MONGODB_URI` | Yes | MongoDB connection string | - |
+| `REDIS_HOST` | Recommended | Redis host | - |
+| `REDIS_PORT` | Recommended | Redis port | `6379` |
+| `REDIS_PASSWORD` | No | Redis password | - |
+| `GPT_KEY` | Yes | OpenAI API key for STT | - |
+| `OPENAI_ENDPOINT` | No | OpenAI API endpoint | `https://api.openai.com/v1` |
+| `ELEVENLABS_ENDPOINT` | No | ElevenLabs API endpoint | `https://api.elevenlabs.io` |
+| `USE_ELEVENLABS_SDK` | No | Use ElevenLabs SDK (vs HTTP) | `false` |
+| `BRAND_NAME` | No | Brand name for customization | - |
+| `BASE_POOLING_DELAY` | No | Base polling delay (ms) | - |
+| `MAX_POLLING_TIME` | No | Max polling time (ms) | - |
+| `VOICE_TWILIO_LOG` | No | Log level | `info` |
+
+### Configuration Priority
+
+1. **Options parameter** (highest priority)
+2. **Environment variables** (`process.env`)
+3. **Default values** (lowest priority)
+
+```javascript
+// Example: Options override environment variables
+startServer({
+  MONGODB_URI: 'mongodb://custom-host:27017/tiledesk', // Overrides process.env.MONGODB_URI
+  API_URL: 'https://custom-api.tiledesk.com'           // Overrides process.env.API_URL
+});
+```
+
+---
+
+## 🏗️ Architecture
+
+### Project Structure
+
+```
+tiledesk-voice-twilio-connector/
+├── src/
+│   ├── app.js                    # Express app setup
+│   ├── config/                    # Configuration management
+│   │   ├── index.js             # Environment config
+│   │   └── logger.js            # Winston logger config
+│   ├── controllers/              # HTTP request handlers
+│   │   └── VoiceController.js   # Main voice controller
+│   ├── middlewares/              # Express middlewares
+│   │   └── httpLogger.js         # HTTP request logging
+│   ├── models/                   # Data models
+│   │   └── KeyValueStore.js     # MongoDB abstraction
+│   ├── routes/                   # Express routes
+│   │   ├── voice.js             # Voice API routes
+│   │   └── manageApp.js         # Management routes
+│   ├── services/                 # Business logic
+│   │   ├── channels/            # Channel implementations
+│   │   │   ├── TiledeskChannel.js
+│   │   │   └── VoiceChannel.js
+│   │   ├── clients/             # API clients
+│   │   │   └── TiledeskSubscriptionClient.js
+│   │   ├── translators/         # Protocol translators
+│   │   │   └── TiledeskTwilioTranslator.js
+│   │   ├── AiService.js         # AI/LLM services
+│   │   ├── AiService.sdk.js     # SDK-based AI service
+│   │   ├── IntegrationService.js
+│   │   ├── MessageService.js
+│   │   ├── SessionService.js
+│   │   ├── SpeechService.js
+│   │   ├── TwilioService.js
+│   │   └── UploadService.js
+│   └── utils/                    # Utilities
+│       ├── constants.js
+│       ├── errors.js
+│       ├── logger.js
+│       └── utils.js
+├── index.js                      # Entry point
+└── package.json
+```
+
+### Design Patterns
+
+- **Service Layer Pattern**: Business logic separated from HTTP handling
+- **Dependency Injection**: Services receive dependencies via constructor
+- **Event-Driven Architecture**: Side effects handled via EventEmitter
+- **Repository Pattern**: Data access abstracted through models
+- **Strategy Pattern**: Multiple implementations for AI services (SDK vs HTTP)
+
+---
+
+## 📡 API Reference
+
+### Main Exports
+
+#### `startServer(options?)`
+
+Starts the Express server and initializes all services.
+
+**Parameters:**
+- `options` (Object, optional): Configuration object (see [Configuration](#configuration))
+
+**Returns:** `Promise<void>`
+
+**Example:**
+```javascript
+const { startServer } = require('@tiledesk/tiledesk-voice-twilio-connector');
+
+await startServer({
+  MONGODB_URI: 'mongodb://localhost:27017/tiledesk',
+  API_URL: 'https://api.tiledesk.com'
+});
+```
+
+#### `app`
+
+Express application instance (before server starts).
+
+**Example:**
+```javascript
+const { app } = require('@tiledesk/tiledesk-voice-twilio-connector');
+
+// Mount in existing Express app
+mainApp.use('/voice', app);
+```
+
+### API Endpoints
+
+#### `POST /webhook/:id_project`
+Receives Twilio call webhooks and initiates call flow.
+
+#### `POST /tiledesk`
+Receives messages from Tiledesk platform.
+
+#### `POST /nextblock/:callSid`
+Handles call flow progression.
+
+#### `POST /speechresult/:callSid`
+Processes speech recognition results.
+
+#### `GET /health-check`
+Health check endpoint.
+
+See [USAGE.md](./USAGE.md) for detailed usage examples.
+
+---
+
+## 💡 Usage Examples
+
+### Example 1: Basic Integration
+
+```javascript
+const { startServer } = require('@tiledesk/tiledesk-voice-twilio-connector');
+
+async function main() {
+  try {
+    await startServer({
+      MONGODB_URI: process.env.MONGODB_URI,
+      API_URL: process.env.API_URL,
+      BASE_URL: process.env.BASE_URL,
+      BASE_FILE_URL: process.env.BASE_FILE_URL,
+      REDIS_HOST: process.env.REDIS_HOST,
+      REDIS_PORT: parseInt(process.env.REDIS_PORT),
+      GPT_KEY: process.env.GPT_KEY
+    });
+    console.log('Voice connector started successfully');
+  } catch (error) {
+    console.error('Failed to start:', error);
+    process.exit(1);
+  }
+}
+
+main();
+```
+
+### Example 2: Integration with Existing Express App
+
+```javascript
+const express = require('express');
+const { app: voiceApp } = require('@tiledesk/tiledesk-voice-twilio-connector');
+
+const mainApp = express();
+
+// Configure environment
+process.env.API_URL = 'https://api.tiledesk.com';
+process.env.MONGODB_URI = 'mongodb://localhost:27017/tiledesk';
+
+// Mount voice connector
+mainApp.use('/voice', voiceApp);
+
+// Your other routes
+mainApp.get('/', (req, res) => {
+  res.json({ status: 'ok' });
+});
+
+mainApp.listen(3000);
+```
+
+### Example 3: Using with dotenv
+
+```javascript
+require('dotenv').config();
+const { startServer } = require('@tiledesk/tiledesk-voice-twilio-connector');
+
+startServer().catch(console.error);
+```
+
+---
+
+## 🐳 Deployment
+
+### Docker
+
+```bash
+# Build image
+docker build -t tiledesk-voice-twilio-connector .
+
+# Run container
+docker run -d \
+  -p 3000:3000 \
+  -e PORT=3000 \
+  -e API_URL=https://api.tiledesk.com \
+  -e MONGODB_URI=mongodb://mongo:27017/tiledesk \
+  -e REDIS_HOST=redis \
+  -e REDIS_PORT=6379 \
+  -e GPT_KEY=your-openai-key \
+  --name voice-connector \
+  tiledesk-voice-twilio-connector
+```
+
+### Docker Compose
+
+```yaml
+version: '3.8'
+services:
+  voice-connector:
+    build: .
+    ports:
+      - "3000:3000"
+    environment:
+      - PORT=3000
+      - API_URL=https://api.tiledesk.com
+      - MONGODB_URI=mongodb://mongo:27017/tiledesk
+      - REDIS_HOST=redis
+      - REDIS_PORT=6379
+    depends_on:
+      - mongo
+      - redis
+```
+
+---
+
+## 📚 Additional Resources
+
+- [USAGE.md](./USAGE.md) - Detailed usage guide
+- [Tiledesk Documentation](https://developer.tiledesk.com)
+- [Twilio Voice API Docs](https://www.twilio.com/docs/voice)
+
+---
+
+## 🤝 Contributing
+
+Contributions are welcome! Please read our contributing guidelines before submitting PRs.
+
+---
+
+## 📄 License
+
+MIT License - see [LICENSE](./LICENSE) file for details.
+
+---
+
+## 👥 Authors
+
+- **Gabriele Panico** - Initial work
+
+---
+
+## 🙏 Acknowledgments
+
+- [Tiledesk](https://tiledesk.com) - Conversational platform
+- [Twilio](https://www.twilio.com) - Voice API infrastructure
+- [OpenAI](https://openai.com) - Speech-to-text services
+- [ElevenLabs](https://elevenlabs.io) - Text-to-speech services
 
-- Need more help? [Check out our Help Center](https://help.glitch.com/) for answers to any common questions.
-- Ready to make it official? [Become a paid Glitch member](https://glitch.com/pricing) to boost your app with private sharing, more storage and memory, domains and more.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tiledesk/tiledesk-voice-twilio-connector",
-  "version": "0.2.0-rc3",
+  "version": "0.2.0-rc6",
   "description": "Tiledesk VOICE Twilio connector",
   "license": "MIT",
   "author": "Gabriele Panico",
@@ -33,7 +33,8 @@
     "redis": "^4.6.13",
     "remove-markdown": "^0.6.2",
     "twilio": "^5.2.1",
-    "winston": "^3.3.3"
+    "winston": "^3.3.3",
+    "uuid": "^9.0.0"
   },
   "engines": {
     "node": ">=18.0.0",

package/src/app.js

@@ -81,10 +81,18 @@ async function startServer(options = {}) {
       logger.error("Missing redis parameters");
     }
 
-    // Mongo
+    // Mongo - Reuse existing connection if provided, otherwise create new one
     const db = new KVBaseMongo('kvstore');
-    await db.connect(mergedConfig.database.mongodbUri);
-    logger.info("KVBaseMongo successfully connected.");
+    if (options.dbconnection) {
+      await db.reuseConnection(options.dbconnection);
+      logger.debug("KVBaseMongo reused existing db connection");
+    } else {
+      if (!mergedConfig.database.mongodbUri) {
+        throw new Error("MONGODB_URI is required when dbconnection is not provided");
+      }
+      await db.connect(mergedConfig.database.mongodbUri);
+      logger.info("KVBaseMongo successfully connected.");
+    }
 
     // Services
     const voiceChannel = new VoiceChannel({