@makemore/agent-frontend 1.3.0 → 1.4.0

package/README.md

@@ -23,14 +23,17 @@ Most chat widgets are tightly coupled to specific frameworks or require complex
 | Feature | Description |
 |---------|-------------|
 | 💬 **Real-time Streaming** | SSE-based message streaming for instant, token-by-token responses |
+| 🔊 **Text-to-Speech** | ElevenLabs integration with secure Django proxy support |
 | 🎨 **Theming** | Customize colors, titles, messages, and position |
 | 🌙 **Dark Mode** | Automatic dark mode based on system preferences |
 | 📱 **Responsive** | Works seamlessly on desktop and mobile |
 | 🔧 **Debug Mode** | Toggle visibility of tool calls and results |
-| 🤖 **Demo Flows** | Built-in auto-run mode for showcasing agent journeys |
+| 🤖 **Demo Flows** | Built-in auto-run mode with automatic, confirm, and manual modes |
 | 🔒 **Sessions** | Automatic anonymous session creation and management |
 | 💾 **Persistence** | Conversations persist across page reloads via localStorage |
 | 🛡️ **Isolated CSS** | Scoped styles that won't leak into or from your page |
+| 🎯 **Configurable APIs** | Customize backend endpoints to match your server structure |
+| 📝 **Enhanced Markdown** | Optional rich markdown with tables, code blocks, and syntax highlighting |
 
 ## Installation
 
@@ -85,7 +88,7 @@ The widget automatically detects and uses the enhanced markdown parser if availa
 
 ## Quick Start
 
-### Initialize the widget
+### Basic Setup
 
 ```html
 <script>
@@ -98,6 +101,23 @@ The widget automatically detects and uses the enhanced markdown parser if availa
 </script>
 ```
 
+### With Text-to-Speech (Recommended: Django Proxy)
+
+```html
+<script>
+  ChatWidget.init({
+    backendUrl: 'https://your-api.com',
+    agentKey: 'your-agent',
+    title: 'Voice-Enabled Chat',
+    primaryColor: '#0066cc',
+    enableTTS: true,
+    ttsProxyUrl: 'https://your-api.com/api/tts/speak/',
+  });
+</script>
+```
+
+See `django-tts-example.py` for the complete Django backend implementation.
+
 ### With custom API paths
 
 ```html
@@ -139,6 +159,85 @@ The widget automatically detects and uses the enhanced markdown parser if availa
 | `apiPaths` | object | See below | API endpoint paths (customizable for different backends) |
 | `autoRunMode` | string | `'automatic'` | Demo flow mode: `'automatic'`, `'confirm'`, or `'manual'` |
 | `autoRunDelay` | number | `1000` | Delay in milliseconds before auto-generating next message (automatic mode) |
+| `enableTTS` | boolean | `false` | Enable text-to-speech for messages |
+| `ttsProxyUrl` | string | `null` | Django proxy URL for TTS (recommended for security) |
+| `elevenLabsApiKey` | string | `null` | ElevenLabs API key (only if not using proxy) |
+| `ttsVoices` | object | `{ assistant: null, user: null }` | Voice IDs (only if not using proxy) |
+| `ttsModel` | string | `'eleven_turbo_v2_5'` | ElevenLabs model (only if not using proxy) |
+| `ttsSettings` | object | See below | ElevenLabs voice settings (only if not using proxy) |
+
+### Text-to-Speech (ElevenLabs)
+
+Add realistic voice narration to your chat widget using ElevenLabs. Two integration options:
+
+#### Option 1: Secure Django Proxy (Recommended)
+
+Keep your API key secure on the server:
+
+```javascript
+ChatWidget.init({
+  enableTTS: true,
+  ttsProxyUrl: 'https://your-backend.com/api/tts/speak/',
+  // No API key or voice IDs needed - configured on server
+});
+```
+
+**Django Setup:**
+
+See `django-tts-example.py` for a complete Django REST Framework implementation. Quick setup:
+
+1. Install: `pip install requests`
+2. Add to `settings.py`:
+```python
+ELEVENLABS_API_KEY = 'your_api_key_here'
+ELEVENLABS_VOICES = {
+    'assistant': 'EXAVITQu4vr4xnSDxMaL',  # Bella
+    'user': 'pNInz6obpgDQGcFmaJgB',       # Adam
+}
+```
+3. Add view from `django-tts-example.py` to your Django app
+4. Add URL route: `path('api/tts/speak/', views.text_to_speech)`
+
+#### Option 2: Direct API (Client-Side)
+
+For testing or simple deployments:
+
+```javascript
+ChatWidget.init({
+  enableTTS: true,
+  elevenLabsApiKey: 'your_elevenlabs_api_key',  // ⚠️ Exposed to client
+  ttsVoices: {
+    assistant: 'EXAVITQu4vr4xnSDxMaL',  // Bella
+    user: 'pNInz6obpgDQGcFmaJgB',       // Adam
+  },
+  ttsModel: 'eleven_turbo_v2_5',
+  ttsSettings: {
+    stability: 0.5,
+    similarity_boost: 0.75,
+    style: 0.0,
+    use_speaker_boost: true,
+  },
+});
+```
+
+**Features:**
+- Speaks assistant responses automatically
+- Speaks simulated user messages in demo mode
+- Queues messages to prevent overlap
+- Waits for speech to finish before continuing demo (automatic mode)
+- Toggle TTS on/off with button in header
+- Visual indicator when speaking (pulsing icon)
+
+**Get Voice IDs:**
+1. Go to https://elevenlabs.io/app/voice-library
+2. Choose voices and copy their IDs
+3. Or use the API: https://api.elevenlabs.io/v1/voices
+
+**Control TTS:**
+```javascript
+ChatWidget.toggleTTS();  // Toggle on/off
+ChatWidget.stopSpeech(); // Stop current speech and clear queue
+```
 
 ### Demo Flow Control
 
@@ -233,6 +332,10 @@ ChatWidget.send('Hello, I need help!');
 // Clear the conversation
 ChatWidget.clearMessages();
 
+// Text-to-speech controls
+ChatWidget.toggleTTS();  // Toggle TTS on/off
+ChatWidget.stopSpeech(); // Stop current speech and clear queue
+
 // Start a demo flow
 ChatWidget.startDemoFlow('quote');
 
@@ -377,6 +480,36 @@ agent-frontend/
 
 Requires: `EventSource` (SSE), `fetch`, `localStorage`
 
+## Version History
+
+### v1.4.0 (Latest)
+- ✨ **Text-to-Speech**: ElevenLabs integration with secure Django proxy support
+- 🔊 Automatic speech for assistant and simulated user messages
+- 🎛️ Smart speech queuing to prevent overlap
+- 🔐 Secure proxy approach keeps API keys on server
+
+### v1.3.0
+- 🎮 **Demo Flow Control**: Three modes (automatic, confirm-next, manual)
+- ⏱️ Configurable delay for automatic mode (0-5000ms)
+- 🎯 Real-time mode switching via dropdown menu
+- ▶️ Continue button for confirm mode
+
+### v1.2.0
+- 📝 **Enhanced Markdown**: Optional rich markdown with tables and code blocks
+- 🎨 Syntax highlighting support via highlight.js
+- 🔧 Automatic detection of markdown addon
+
+### v1.1.0
+- 🔌 **Configurable API Paths**: Customize backend endpoints
+- 🛠️ Support for different backend URL structures
+
+### v1.0.0
+- 🎉 Initial release
+- 💬 Real-time SSE streaming
+- 🎨 Theming and customization
+- 🤖 Demo flows
+- 🔒 Session management
+
 ## License
 
 MIT © 2024

package/dist/chat-widget.css

@@ -159,6 +159,19 @@
   color: #ffd700;
 }
 
+.cw-btn-speaking {
+  animation: pulse-speaking 1.5s ease-in-out infinite;
+}
+
+@keyframes pulse-speaking {
+  0%, 100% {
+    background: rgba(255, 255, 255, 0.3);
+  }
+  50% {
+    background: rgba(255, 255, 255, 0.5);
+  }
+}
+
 /* Status bar */
 .cw-status-bar {
   display: flex;

package/dist/chat-widget.js

@@ -46,6 +46,21 @@
     // Demo flow control
     autoRunDelay: 1000, // Delay in ms before auto-generating next message
     autoRunMode: 'automatic', // 'automatic', 'confirm', or 'manual'
+    // Text-to-speech (ElevenLabs)
+    enableTTS: false,
+    ttsProxyUrl: null, // If set, uses Django proxy instead of direct API calls
+    elevenLabsApiKey: null, // Only needed if not using proxy
+    ttsVoices: {
+      assistant: null, // ElevenLabs voice ID for assistant (not needed if using proxy)
+      user: null, // ElevenLabs voice ID for simulated user (not needed if using proxy)
+    },
+    ttsModel: 'eleven_turbo_v2_5', // ElevenLabs model (not needed if using proxy)
+    ttsSettings: {
+      stability: 0.5,
+      similarity_boost: 0.75,
+      style: 0.0,
+      use_speaker_boost: true,
+    },
   };
 
   // State
@@ -64,6 +79,9 @@
     sessionToken: null,
     error: null,
     eventSource: null,
+    currentAudio: null,
+    isSpeaking: false,
+    speechQueue: [],
   };
 
   // DOM elements
@@ -138,6 +156,135 @@
     }
   }
 
+  // ============================================================================
+  // Text-to-Speech (ElevenLabs)
+  // ============================================================================
+
+  async function speakText(text, role) {
+    if (!config.enableTTS) return;
+
+    // Check if we have either proxy or direct API access
+    if (!config.ttsProxyUrl && !config.elevenLabsApiKey) return;
+
+    // If using direct API, check for voice ID
+    if (!config.ttsProxyUrl) {
+      const voiceId = role === 'assistant' ? config.ttsVoices.assistant : config.ttsVoices.user;
+      if (!voiceId) return;
+    }
+
+    // Add to queue
+    state.speechQueue.push({ text, role });
+
+    // Process queue if not already speaking
+    if (!state.isSpeaking) {
+      processSpeechQueue();
+    }
+  }
+
+  async function processSpeechQueue() {
+    if (state.speechQueue.length === 0) {
+      state.isSpeaking = false;
+      render();
+
+      // If auto-run is waiting for speech to finish, continue
+      if (state.autoRunActive && state.autoRunPaused && config.autoRunMode === 'automatic') {
+        setTimeout(() => {
+          if (state.autoRunActive && !state.isSpeaking) {
+            continueAutoRun();
+          }
+        }, config.autoRunDelay);
+      }
+      return;
+    }
+
+    state.isSpeaking = true;
+    render();
+
+    const { text, role } = state.speechQueue.shift();
+
+    try {
+      let response;
+
+      if (config.ttsProxyUrl) {
+        // Use Django proxy
+        response = await fetch(config.ttsProxyUrl, {
+          method: 'POST',
+          headers: {
+            'Content-Type': 'application/json',
+            ...(state.sessionToken ? { [config.anonymousTokenHeader]: state.sessionToken } : {}),
+          },
+          body: JSON.stringify({
+            text: text,
+            role: role,
+          }),
+        });
+      } else {
+        // Direct ElevenLabs API call
+        const voiceId = role === 'assistant' ? config.ttsVoices.assistant : config.ttsVoices.user;
+        response = await fetch(`https://api.elevenlabs.io/v1/text-to-speech/${voiceId}`, {
+          method: 'POST',
+          headers: {
+            'Accept': 'audio/mpeg',
+            'Content-Type': 'application/json',
+            'xi-api-key': config.elevenLabsApiKey,
+          },
+          body: JSON.stringify({
+            text: text,
+            model_id: config.ttsModel,
+            voice_settings: config.ttsSettings,
+          }),
+        });
+      }
+
+      if (!response.ok) {
+        throw new Error(`TTS API error: ${response.status}`);
+      }
+
+      const audioBlob = await response.blob();
+      const audioUrl = URL.createObjectURL(audioBlob);
+      const audio = new Audio(audioUrl);
+
+      state.currentAudio = audio;
+
+      audio.onended = () => {
+        URL.revokeObjectURL(audioUrl);
+        state.currentAudio = null;
+        processSpeechQueue();
+      };
+
+      audio.onerror = () => {
+        console.error('[ChatWidget] Audio playback error');
+        URL.revokeObjectURL(audioUrl);
+        state.currentAudio = null;
+        processSpeechQueue();
+      };
+
+      await audio.play();
+    } catch (err) {
+      console.error('[ChatWidget] TTS error:', err);
+      state.currentAudio = null;
+      processSpeechQueue();
+    }
+  }
+
+  function stopSpeech() {
+    if (state.currentAudio) {
+      state.currentAudio.pause();
+      state.currentAudio = null;
+    }
+    state.speechQueue = [];
+    state.isSpeaking = false;
+    render();
+  }
+
+  function toggleTTS() {
+    config.enableTTS = !config.enableTTS;
+    if (!config.enableTTS) {
+      stopSpeech();
+    }
+    render();
+  }
+
   // ============================================================================
   // Session Management
   // ============================================================================
@@ -347,10 +494,21 @@
       state.eventSource = null;
       render();
 
+      // Speak assistant message if TTS enabled
+      if (assistantContent && !state.error) {
+        speakText(assistantContent, 'assistant');
+      }
+
       // Trigger auto-run if enabled
       if (state.autoRunActive && !state.error) {
         if (config.autoRunMode === 'automatic') {
-          setTimeout(() => triggerAutoRun(), config.autoRunDelay);
+          // Wait for speech to finish before continuing
+          if (config.enableTTS && assistantContent) {
+            state.autoRunPaused = true;
+            // processSpeechQueue will continue when done
+          } else {
+            setTimeout(() => triggerAutoRun(), config.autoRunDelay);
+          }
         } else if (config.autoRunMode === 'confirm') {
           state.autoRunPaused = true;
           render();
@@ -402,6 +560,12 @@
         const data = await response.json();
         if (data.response) {
           state.isSimulating = false;
+
+          // Speak simulated user message if TTS enabled
+          if (config.enableTTS && config.ttsVoices.user) {
+            await speakText(data.response, 'user');
+          }
+
           await sendMessage(data.response);
           return;
         }
@@ -674,6 +838,13 @@
                 </svg>
               </button>
             ` : ''}
+            ${config.elevenLabsApiKey ? `
+              <button class="cw-header-btn ${config.enableTTS ? 'cw-btn-active' : ''} ${state.isSpeaking ? 'cw-btn-speaking' : ''}"
+                      data-action="toggle-tts"
+                      title="${config.enableTTS ? (state.isSpeaking ? 'Speaking...' : 'TTS Enabled') : 'TTS Disabled'}">
+                ${state.isSpeaking ? '🔊' : (config.enableTTS ? '🔉' : '🔇')}
+              </button>
+            ` : ''}
             ${renderJourneyDropdown()}
             <button class="cw-header-btn" data-action="toggle-expand" title="${state.isExpanded ? 'Minimize' : 'Expand'}">
               ${state.isExpanded ? '⊖' : '⊕'}
@@ -721,6 +892,7 @@
           case 'close': closeWidget(); break;
           case 'toggle-expand': toggleExpand(); break;
           case 'toggle-debug': toggleDebugMode(); break;
+          case 'toggle-tts': toggleTTS(); break;
           case 'clear': clearMessages(); break;
           case 'stop-autorun': stopAutoRun(); break;
           case 'continue-autorun': continueAutoRun(); break;
@@ -838,6 +1010,8 @@
     continueAutoRun,
     setAutoRunMode,
     setAutoRunDelay,
+    toggleTTS,
+    stopSpeech,
     getState: () => ({ ...state }),
     getConfig: () => ({ ...config }),
   };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@makemore/agent-frontend",
-  "version": "1.3.0",
+  "version": "1.4.0",
   "description": "A standalone, zero-dependency chat widget for AI agents. Embed conversational AI into any website with a single script tag.",
   "main": "dist/chat-widget.js",
   "files": [