aillom-vox-client 1.0.0

package/docs/TOOLS.md

@@ -0,0 +1,314 @@
+# 🛠️ Client Tools Guide
+
+Client Tools allow the AI to interact with your application's UI. When the AI determines that a specific action is needed (like ending a call or showing an alert), it calls a tool, and your application executes it.
+
+---
+
+## 📋 Architecture
+
+### CLIENT-side Tools
+Executed **locally** in your application (browser, mobile app, CLI):
+- Control UI elements
+- Handle navigation
+- Manage connections
+- Display notifications
+
+### SERVER-side Tools
+Executed on **AillomVox servers** (you cannot register these):
+- Access databases
+- Call external APIs
+- Query RAG systems
+- Server-side logic
+
+> **Rule**: If it controls **your app's UI**, it's a CLIENT tool. If it accesses **server resources**, it's a SERVER tool.
+
+---
+
+## 🚀 Quick Start
+
+### 1. Register Tools
+
+```javascript
+await client.connect({
+    provider: 'gemini',
+    voice: 'Puck',
+    tools: [
+        {
+            name: 'hangup',
+            description: 'End the current call',
+            parameters: {
+                type: 'object',
+                properties: {},
+                required: []
+            }
+        }
+    ]
+});
+```
+
+### 2. Handle Tool Calls
+
+```javascript
+client.on('tool_call', async (tool) => {
+    console.log(`AI requested: ${tool.name}`, tool.args);
+    
+    if (tool.name === 'hangup') {
+        await client.disconnect();
+        return 'Call ended successfully';
+    }
+});
+```
+
+### 3. Send Result Back
+
+The return value from your handler is automatically sent back to the AI, allowing it to continue the conversation.
+
+---
+
+## 📦 Tool Definition Format
+
+```javascript
+{
+    name: 'tool_name',           // Unique identifier (match in handler)
+    description: 'What it does', // AI uses this to decide when to call
+    parameters: {                // JSON Schema for arguments
+        type: 'object',
+        properties: {
+            arg1: { 
+                type: 'string', 
+                description: 'First argument' 
+            },
+            arg2: { 
+                type: 'number', 
+                description: 'Second argument' 
+            }
+        },
+        required: ['arg1']      // Mandatory arguments
+    }
+}
+```
+
+---
+
+## 🎯 Common Tools
+
+### 1. Hangup (End Call)
+
+```javascript
+{
+    name: 'hangup',
+    description: 'End the call when user says goodbye',
+    parameters: { type: 'object', properties: {} }
+}
+
+// Handler
+client.on('tool_call', async (tool) => {
+    if (tool.name === 'hangup') {
+        await client.disconnect();
+        return 'Call ended';
+    }
+});
+```
+
+### 2. Show Alert
+
+```javascript
+{
+    name: 'show_alert',
+    description: 'Display important message to user',
+    parameters: {
+        type: 'object',
+        properties: {
+            message: { 
+                type: 'string', 
+                description: 'Alert message' 
+            },
+            severity: { 
+                type: 'string', 
+                enum: ['info', 'warning', 'error'],
+                description: 'Alert type'
+            }
+        },
+        required: ['message']
+    }
+}
+
+// Handler
+client.on('tool_call', async (tool) => {
+    if (tool.name === 'show_alert') {
+        alert(`[${tool.args.severity}] ${tool.args.message}`);
+        return 'Alert displayed';
+    }
+});
+```
+
+### 3. Navigate to Page
+
+```javascript
+{
+    name: 'navigate_to_page',
+    description: 'Navigate user to a specific page',
+    parameters: {
+        type: 'object',
+        properties: {
+            url: { 
+                type: 'string', 
+                description: 'Target URL' 
+            }
+        },
+        required: ['url']
+    }
+}
+
+// Handler
+client.on('tool_call', async (tool) => {
+    if (tool.name === 'navigate_to_page') {
+        window.location.href = tool.args.url;
+        return 'Navigating';
+    }
+});
+```
+
+### 4. Update UI Element
+
+```javascript
+{
+    name: 'highlight_element',
+    description: 'Highlight UI element to draw user attention',
+    parameters: {
+        type: 'object',
+        properties: {
+            element_id: { type: 'string', description: 'DOM element ID' },
+            color: { type: 'string', description: 'Highlight color' }
+        },
+        required: ['element_id']
+    }
+}
+
+// Handler
+client.on('tool_call', async (tool) => {
+    if (tool.name === 'highlight_element') {
+        const el = document.getElementById(tool.args.element_id);
+        el.style.border = `3px solid ${tool.args.color || 'yellow'}`;
+        return 'Element highlighted';
+    }
+});
+```
+
+---
+
+## 🧪 Testing Tools
+
+### Manual Testing
+
+```javascript
+// Simulate AI calling a tool
+client.emit('tool_call', {
+    name: 'show_alert',
+    args: { message: 'Test alert', severity: 'info' }
+});
+```
+
+### Debugging
+
+```javascript
+client.on('tool_call', async (tool) => {
+    console.log('=== TOOL CALL ===');
+    console.log('Name:', tool.name);
+    console.log('Args:', tool.args);
+    console.log('ID:', tool.call_id);
+    
+    // Your handler logic here
+    const result = await handleTool(tool);
+    
+    console.log('Result:', result);
+    return result;
+});
+```
+
+---
+
+## ⚠️ Best Practices
+
+### 1. Clear Descriptions
+```javascript
+// ❌ Bad
+description: 'Does something'
+
+// ✅ Good
+description: 'End the call when user says goodbye or wants to finish'
+```
+
+### 2. Validate Arguments
+```javascript
+client.on('tool_call', async (tool) => {
+    if (tool.name === 'navigate_to_page') {
+        // Validate
+        if (!tool.args.url || !tool.args.url.startsWith('http')) {
+            return 'Error: Invalid URL';
+        }
+        
+        window.location.href = tool.args.url;
+        return 'Navigating to ' + tool.args.url;
+    }
+});
+```
+
+### 3. Return Meaningful Results
+```javascript
+// ❌ Bad
+return 'ok';
+
+// ✅ Good
+return 'Alert shown successfully with message: "' + tool.args.message + '"';
+```
+
+This helps the AI understand what happened and continue the conversation naturally.
+
+---
+
+## 🔒 Security
+
+### Client Tools are Safe
+- Executed in **your app**, not on our servers
+- You have **full control** over what they do
+- No sensitive data is sent to the server
+
+### Never Trust User Input
+```javascript
+// ❌ Dangerous
+client.on('tool_call', async (tool) => {
+    eval(tool.args.code); // NEVER DO THIS
+});
+
+// ✅ Safe
+client.on('tool_call', async (tool) => {
+    const allowedUrls = ['https://myapp.com/dashboard', 'https://myapp.com/profile'];
+    if (allowedUrls.includes(tool.args.url)) {
+        window.location.href = tool.args.url;
+    }
+});
+```
+
+---
+
+## 📚 Examples
+
+See complete working examples in:
+- [Browser Example](../examples/browser/) - Full UI with tools
+- [Node.js CLI](../examples/nodejs/) - CLI bot with hangup
+- [Python Example](../examples/python/) - Python integration
+
+---
+
+## 💡 Tips
+
+1. **Start Simple**: Begin with just `hangup`, then add more
+2. **Test Locally**: Use manual emit to test without calling the AI
+3. **Return Clear Messages**: AI uses return value to continue conversation
+4. **Validate Everything**: Never trust arguments blindly
+5. **Keep it Fast**: Tool execution should be instant (< 100ms)
+
+---
+
+Happy building! 🎉

package/docs/TROUBLESHOOTING.md

@@ -0,0 +1,86 @@
+# Troubleshooting Guide
+
+Common issues and how to resolve them when using AillomVox.
+
+---
+
+## Authentication Errors
+
+### `401 Unauthorized` / `error: unauthorized`
+- **Cause**: API Key is missing, invalid, or expired.
+- **Fix**: Check that you're sending `apikey` in your config handshake. Ensure you are copying the key exactly as provided in your dashboard.
+
+### `error: insufficient_balance`
+- **Cause**: Your account balance is $0.00 or below.
+- **Fix**: Add credits via the [Billing page](https://vox.aillom.com/billing).
+
+---
+
+## Connection Issues
+
+### `Connection Refused` / `Socket Hang Up`
+- **Cause**: Server unreachable or firewall blocking outbound port 443 (WSS).
+- **Fix**: Ensure you can reach `https://vox.aillom.com/health`. Check your network settings.
+
+### `1008 Policy Violation`
+- **Cause**: Sent binary data before the config handshake.
+- **Fix**: The **first message** must be a JSON config. Wait for it to be acknowledged before sending audio.
+
+### `1006 Abnormal Closure`
+- **Cause**: Generic WebSocket error. Often caused by:
+  - Sending malformed JSON
+  - Sending audio in wrong format (e.g., MP3 instead of PCM)
+  - Network timeout (keep-alive)
+
+---
+
+## Audio Problems
+
+### "Static" or "Noise" instead of Voice
+- **Cause**: Wrong encoding or endianness.
+- **Fix**: AillomVox expects **PCM 16-bit Signed Little Endian**. 
+  - If you send Float32 (Web Audio API default), you get static. Convert to Int16 first.
+  - If you send Big Endian, you get static. Use `true` for little-endian in `DataView.setInt16()`.
+
+### High Latency
+- **Cause**: Buffering too much audio before sending.
+- **Fix**: Send small chunks (20ms–50ms) as soon as recorded. Do not batch 1 second of audio.
+
+### "Choppy" Audio Output
+- **Cause**: Network jitter or empty buffer underruns.
+- **Fix**: Implement a jitter buffer — wait for 2–3 chunks before starting playback.
+
+### No Audio Received
+- **Cause**: `sample_rate` mismatch or missing `system_prompt`.
+- **Fix**: Ensure `sample_rate` is `8000`, `16000`, or `24000`. The `system_prompt` field is mandatory.
+
+---
+
+## Tool Call Issues
+
+### AI Hangs After Tool Call
+- **Cause**: Missing `tool_result` response.
+- **Fix**: Always respond to `tool_call` events with a `tool_result` within 15 seconds:
+  ```javascript
+  ws.send(JSON.stringify({
+    type: "tool_result",
+    call_id: msg.call_id,
+    result: "Success"
+  }));
+  ```
+
+### Tools Not Triggering
+- **Cause**: Vague tool descriptions or wrong provider.
+- **Fix**: Use clear, specific descriptions. Note: Qwen does not support tools — use AillomVox, OpenAI, Gemini, or AWS.
+
+---
+
+## Session Issues
+
+### `max_duration_reached`
+- **Cause**: Session exceeded the `max_duration` limit.
+- **Fix**: Increase `max_duration` (up to 3600) or handle the farewell message gracefully.
+
+### Multiple Connections Rejected
+- **Cause**: Exceeded concurrent connection limit (3 per user, 2 per API key).
+- **Fix**: Close unused connections before opening new ones.

package/docs/VOICES.md

@@ -0,0 +1,219 @@
+# Voice Catalog
+
+Complete reference of all available voices across providers.
+
+---
+
+## AillomVox (Inworld TTS 1.5)
+
+**65 voices** verified via the Inworld TTS API. Each voice has a native language but supports multilingual synthesis across all 15 languages.
+
+### English (25 voices)
+
+**Male (14)**
+
+| Voice | Style |
+| :--- | :--- |
+| **Edward** | Fast-talking, emphatic (default EN) |
+| **Dennis** | Smooth, calm, friendly |
+| **Alex** | Energetic, expressive |
+| **Craig** | Older British, refined |
+| **Mark** | Energetic, rapid delivery |
+| **Ronald** | Confident British, deep |
+| **Shaun** | Friendly, dynamic |
+| **Theodore** | Gravelly, time-worn |
+| **Timothy** | Lively, upbeat American |
+| **Carter** | Mature radio announcer |
+| **Blake** | Rich, intimate |
+| **Clive** | British, calm, cordial |
+| **Dominus** | Robotic, deep, menacing |
+| **Hades** | Commanding, gruff |
+
+**Female (11)**
+
+| Voice | Style |
+| :--- | :--- |
+| **Ashley** | Warm, natural |
+| **Deborah** | Gentle, elegant |
+| **Elizabeth** | Professional, narrations |
+| **Julia** | Quirky, high-pitched |
+| **Olivia** | Young British, upbeat |
+| **Priya** | Even-toned, Indian accent |
+| **Sarah** | Fast-talking, curious |
+| **Wendy** | Posh British |
+| **Luna** | Calm, relaxing |
+| **Hana** | Bright, expressive |
+| **Pixie** | High-pitched, childlike |
+
+### Portuguese (2)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Heitor** | Male | Composed, neutral (default PT) |
+| **Maitê** | Female | Middle-aged, professional |
+
+### Spanish (4)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Diego** | Male | Soothing, gentle (default ES) |
+| **Miguel** | Male | Calm, storytelling |
+| **Rafael** | Male | Deep, composed |
+| **Lupita** | Female | Vibrant, energetic |
+
+### French (4)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Alain** | Male | Deep, smooth |
+| **Mathieu** | Male | Nasal quality |
+| **Étienne** | Male | Calm, young adult |
+| **Hélène** | Female | Musical, graceful |
+
+### German (2)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Josef** | Male | Articulate, announcer |
+| **Johanna** | Female | Calm, low, smoky |
+
+### Italian (2)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Gianni** | Male | Deep, smooth, rapid |
+| **Orietta** | Female | Calm, soothing |
+
+### Chinese (4)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Yichen** | Male | Calm, flat |
+| **Xiaoyin** | Female | Youthful, sweet |
+| **Xinyi** | Female | Neutral, narrations |
+| **Jing** | Female | Energetic, fast |
+
+### Dutch (4)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Erik** | Male | Older, weathered |
+| **Lennart** | Male | Confident, calm |
+| **Katrien** | Female | Expressive |
+| **Lore** | Female | Clear, calm |
+
+### Japanese (2)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Satoshi** | Male | Dramatic, expressive |
+| **Asuka** | Female | Friendly, young |
+
+### Korean (4)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Hyunwoo** | Male | Young adult |
+| **Seojun** | Male | Deep, mature |
+| **Minji** | Female | Energetic, friendly |
+| **Yoona** | Female | Gentle, soothing |
+
+### Polish (2)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Szymon** | Male | Warm, friendly |
+| **Wojciech** | Male | Middle-aged |
+
+### Russian (4)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Dmitry** | Male | Deep, commanding |
+| **Nikolai** | Male | Deep, theatrical |
+| **Svetlana** | Female | Soft, high-pitched |
+| **Elena** | Female | Clear, smooth |
+
+### Hindi (2)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Manoj** | Male | Professional |
+| **Riya** | Female | Polished, approachable |
+
+### Hebrew (2)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Oren** | Male | Steady |
+| **Yael** | Female | Mid-range, narrations |
+
+### Arabic (2)
+
+| Voice | Gender | Style |
+| :--- | :--- | :--- |
+| **Omar** | Male | Bright, confident |
+| **Nour** | Female | Polished, friendly |
+
+---
+
+## OpenAI Realtime
+
+| Voice | Style |
+| :--- | :--- |
+| **alloy** | Neutral, balanced |
+| **ash** | Warm, conversational |
+| **coral** | Clear, professional |
+| **echo** | Smooth, calm |
+| **sage** | Wise, measured |
+| **shimmer** | Bright, energetic |
+
+---
+
+## Google Gemini
+
+| Voice | Style |
+| :--- | :--- |
+| **Puck** | Soft, higher pitch |
+| **Kore** | Soft, higher pitch |
+| **Charon** | Deep, confident |
+| **Fenrir** | Deep, confident |
+| **Aoede** | Confident, higher pitch |
+
+---
+
+## AWS Bedrock (Nova Sonic)
+
+| Voice | Style |
+| :--- | :--- |
+| **matthew** | Male, neutral |
+| **ruth** | Female, professional |
+| **tiffany** | Female, warm |
+
+---
+
+## Ultravox
+
+| Voice | Style |
+| :--- | :--- |
+| **Mark** | Male |
+| **Jessica** | Female |
+
+---
+
+## Language Support
+
+### AillomVox Languages (15)
+
+```
+en, pt, es, fr, de, it, ja, zh, ko, hi, ar, ru, pl, nl, he
+```
+
+### Language Defaults (AillomVox)
+
+| Code | Language | Default Voice |
+| :--- | :--- | :--- |
+| `en-US` | English | Edward |
+| `pt-BR` | Portuguese (Brazil) | Heitor |
+| `es-ES` | Spanish | Diego |
+| All others | — | Edward |