aillom-vox-client 1.0.2 → 2.1.0

package/docs/PROTOCOL.md

@@ -1,156 +0,0 @@
-# WebSocket Protocol
-
-The AillomVox Gateway uses a WebSocket protocol for full-duplex audio streaming and control messaging.
-
-## Connection
-
-**Endpoint**: `wss://vox.aillom.com/ws`
-
-> **Note**: Authentication is performed in-band (inside the first message), not via HTTP headers or query params.
-
-## Message Flow
-
-```
-Client                              Server
-  |                                    |
-  |--- WebSocket Connect (GET /ws) --->|
-  |<--- 101 Switching Protocols -------|
-  |                                    |
-  |--- JSON Config (type: "config") -->|  ← MUST be first message
-  |                                    |  (Auth + Billing + Provider init)
-  |        ~500ms stabilization~       |
-  |                                    |
-  |--- Binary PCM Audio Chunks ------->|  ← 16-bit LE Mono
-  |<--- Binary PCM Audio Chunks -------|  ← AI response audio
-  |<--- JSON Events (transcript, etc) -|
-  |                                    |
-  |--- JSON { type: "hangup" } ------->|  ← or server-initiated
-  |<--- WebSocket Close ----------------|
-```
-
-## 1. Handshake (Client → Server)
-
-The **first message** must be a flat JSON config object. Sending binary data before this message results in connection termination (code `1008`).
-
-```json
-{
-  "type": "config",
-  "apikey": "av_your_api_key_here",
-  "provider": "aillomvox",
-  "voice": "Edward",
-  "language": "en-US",
-  "sample_rate": 16000,
-  "system_prompt": "You are a helpful assistant.",
-  "first_message": "Hello! How can I help you?",
-  "farewell_message": "Thank you for calling. Goodbye!",
-  "max_duration": 300,
-  "tools": [
-    {
-      "name": "hangup",
-      "description": "End the call when user says goodbye.",
-      "parameters": { "type": "object", "properties": {} }
-    }
-  ]
-}
-```
-
-| Field | Required | Type | Description |
-| :--- | :--- | :--- | :--- |
-| `type` | ✅ | string | Must be `"config"` |
-| `apikey` | ✅ | string | Your AillomVox API key |
-| `provider` | ✅ | string | `aillomvox`, `openai`, `gemini`, `aws`, `ultravox`, `grok`, `qwen` |
-| `sample_rate` | ✅ | number | `8000`, `16000`, or `24000` Hz |
-| `system_prompt` | ✅ | string | AI persona instructions |
-| `voice` | | string | Provider-specific voice ID |
-| `language` | | string | Locale code (e.g., `en-US`, `pt-BR`) |
-| `first_message` | | string | Greeting spoken on connect |
-| `farewell_message` | | string | Message before session close |
-| `max_duration` | | number | Session limit in seconds (60–3600) |
-| `tools` | | array | Client-side tool definitions |
-
-## 2. Audio (Binary Messages)
-
-After the handshake, audio flows as raw binary WebSocket messages in both directions.
-
-- **Format**: PCM 16-bit Signed Integer, Little Endian
-- **Channels**: Mono (1 channel)
-- **Rate**: Must match `sample_rate` from the handshake
-- **Direction**: Full duplex (bidirectional)
-
-> There is no JSON wrapper for audio. If the WebSocket message is binary, it is a raw audio chunk.
-
-## 3. Server → Client Events
-
-### Transcript
-```json
-{
-  "type": "transcript",
-  "role": "user",
-  "text": "Hello world",
-  "final": true
-}
-```
-- `role`: `"user"` or `"assistant"`
-- `final`: `true` when the sentence is complete. Only render `final: true` transcripts to avoid UI flickering.
-
-### Tool Call
-```json
-{
-  "type": "tool_call",
-  "call_id": "abc123",
-  "name": "hangup",
-  "args": {}
-}
-```
-Client must respond with a `tool_result` within 15 seconds (see below).
-
-### Playback Clear Buffer
-```json
-{
-  "type": "playback_clear_buffer"
-}
-```
-Sent when the user interrupts the AI. Client **must** immediately flush its audio playback queue.
-
-### Hangup
-```json
-{
-  "type": "hangup"
-}
-```
-Server-initiated session end. Client should stop audio, close socket, and reset UI.
-
-### Error
-```json
-{
-  "type": "error",
-  "message": "Invalid API Key"
-}
-```
-Error codes: `unauthorized`, `insufficient_balance`, `max_duration_reached`.
-
-## 4. Client → Server Events
-
-### Tool Result
-```json
-{
-  "type": "tool_result",
-  "call_id": "abc123",
-  "result": "Order status: shipped"
-}
-```
-**Mandatory** response after receiving a `tool_call`. The AI pauses execution until it receives this. Timeout: 15 seconds.
-
-### Hangup
-```json
-{
-  "type": "hangup"
-}
-```
-Client-initiated session end.
-
-## 5. Session Governance
-
-- **Max Duration**: 60–3600 seconds (default: 300)
-- **Farewell Warning**: At 15 seconds remaining, the AI speaks the `farewell_message`
-- **Force Close**: At 0 seconds, connection closes with `max_duration_reached`

package/docs/PROVIDERS.md

@@ -1,40 +0,0 @@
-# Supported AI Providers
-
-AillomVox aggregates top-tier Voice AI providers. Click on a provider to see detailed configuration and examples.
-
-## Provider Index
-
-| Provider | Model | Voices | Best For | Documentation |
-| :--- | :--- | :--- | :--- | :--- |
-| **AillomVox** | *Groq + Inworld TTS* | 65 voices | **Speed & Telephony**. Lowest latency, 8kHz native. | [Docs](providers/AILLOMVOX.md) |
-| **OpenAI** | `gpt-realtime-mini` | 6 voices | **Complex Logic**. Math, coding, strict reasoning. | [Docs](providers/OPENAI.md) |
-| **Gemini** | `gemini-2.5-flash` | 5 voices | **Long Context**. Massive memory, complex prompts. | [Docs](providers/GEMINI.md) |
-| **AWS** | `nova-2-sonic` | 3 voices | **Enterprise**. High reliability, AWS compliance. | [Docs](providers/AWS.md) |
-| **UltraVox** | `ultravox-v0.7` | 2 voices | **Emotion**. High emotional intelligence. | [Docs](providers/ULTRAVOX.md) |
-| **Grok** | `grok-beta` | Model-dependent | **Casual/Fun**. Witty, less robotic interactions. | [Docs](providers/GROK.md) |
-| **Qwen** | `qwen3-omni` | Model-dependent | **Cost & Asia**. High performance at lower cost. | [Docs](providers/QWEN.md) |
-
-See the complete [Voice Catalog](VOICES.md) for all voices across providers.
-
-## Quick Config Example
-
-```javascript
-// Connect to AillomVox
-ws.send(JSON.stringify({
-  type: "config",
-  apikey: "YOUR_API_KEY",
-  provider: "aillomvox",
-  voice: "Edward",
-  language: "en-US",
-  sample_rate: 16000,
-  system_prompt: "You are a helpful assistant."
-}));
-```
-
----
-
-## ⚠️ Known Limitations
-
-| Provider | Limitation | Workaround |
-| :--- | :--- | :--- |
-| **Qwen** (`qwen3-omni-flash-realtime`) | **Function calling / Client Tools not supported** in WebSocket Realtime mode. The model will respond with text instead of emitting tool calls. | Use **AWS**, **OpenAI**, or **Gemini** for scenarios requiring tools (e.g., `hangup`, `transfer`). |

package/docs/TOOLS.md

@@ -1,314 +0,0 @@
-# 🛠️ Client Tools Guide
-
-Client Tools allow the AI to interact with your application's UI. When the AI determines that a specific action is needed (like ending a call or showing an alert), it calls a tool, and your application executes it.
-
----
-
-## 📋 Architecture
-
-### CLIENT-side Tools
-Executed **locally** in your application (browser, mobile app, CLI):
-- Control UI elements
-- Handle navigation
-- Manage connections
-- Display notifications
-
-### SERVER-side Tools
-Executed on **AillomVox servers** (you cannot register these):
-- Access databases
-- Call external APIs
-- Query RAG systems
-- Server-side logic
-
-> **Rule**: If it controls **your app's UI**, it's a CLIENT tool. If it accesses **server resources**, it's a SERVER tool.
-
----
-
-## 🚀 Quick Start
-
-### 1. Register Tools
-
-```javascript
-await client.connect({
-    provider: 'gemini',
-    voice: 'Puck',
-    tools: [
-        {
-            name: 'hangup',
-            description: 'End the current call',
-            parameters: {
-                type: 'object',
-                properties: {},
-                required: []
-            }
-        }
-    ]
-});
-```
-
-### 2. Handle Tool Calls
-
-```javascript
-client.on('tool_call', async (tool) => {
-    console.log(`AI requested: ${tool.name}`, tool.args);
-    
-    if (tool.name === 'hangup') {
-        await client.disconnect();
-        return 'Call ended successfully';
-    }
-});
-```
-
-### 3. Send Result Back
-
-The return value from your handler is automatically sent back to the AI, allowing it to continue the conversation.
-
----
-
-## 📦 Tool Definition Format
-
-```javascript
-{
-    name: 'tool_name',           // Unique identifier (match in handler)
-    description: 'What it does', // AI uses this to decide when to call
-    parameters: {                // JSON Schema for arguments
-        type: 'object',
-        properties: {
-            arg1: { 
-                type: 'string', 
-                description: 'First argument' 
-            },
-            arg2: { 
-                type: 'number', 
-                description: 'Second argument' 
-            }
-        },
-        required: ['arg1']      // Mandatory arguments
-    }
-}
-```
-
----
-
-## 🎯 Common Tools
-
-### 1. Hangup (End Call)
-
-```javascript
-{
-    name: 'hangup',
-    description: 'End the call when user says goodbye',
-    parameters: { type: 'object', properties: {} }
-}
-
-// Handler
-client.on('tool_call', async (tool) => {
-    if (tool.name === 'hangup') {
-        await client.disconnect();
-        return 'Call ended';
-    }
-});
-```
-
-### 2. Show Alert
-
-```javascript
-{
-    name: 'show_alert',
-    description: 'Display important message to user',
-    parameters: {
-        type: 'object',
-        properties: {
-            message: { 
-                type: 'string', 
-                description: 'Alert message' 
-            },
-            severity: { 
-                type: 'string', 
-                enum: ['info', 'warning', 'error'],
-                description: 'Alert type'
-            }
-        },
-        required: ['message']
-    }
-}
-
-// Handler
-client.on('tool_call', async (tool) => {
-    if (tool.name === 'show_alert') {
-        alert(`[${tool.args.severity}] ${tool.args.message}`);
-        return 'Alert displayed';
-    }
-});
-```
-
-### 3. Navigate to Page
-
-```javascript
-{
-    name: 'navigate_to_page',
-    description: 'Navigate user to a specific page',
-    parameters: {
-        type: 'object',
-        properties: {
-            url: { 
-                type: 'string', 
-                description: 'Target URL' 
-            }
-        },
-        required: ['url']
-    }
-}
-
-// Handler
-client.on('tool_call', async (tool) => {
-    if (tool.name === 'navigate_to_page') {
-        window.location.href = tool.args.url;
-        return 'Navigating';
-    }
-});
-```
-
-### 4. Update UI Element
-
-```javascript
-{
-    name: 'highlight_element',
-    description: 'Highlight UI element to draw user attention',
-    parameters: {
-        type: 'object',
-        properties: {
-            element_id: { type: 'string', description: 'DOM element ID' },
-            color: { type: 'string', description: 'Highlight color' }
-        },
-        required: ['element_id']
-    }
-}
-
-// Handler
-client.on('tool_call', async (tool) => {
-    if (tool.name === 'highlight_element') {
-        const el = document.getElementById(tool.args.element_id);
-        el.style.border = `3px solid ${tool.args.color || 'yellow'}`;
-        return 'Element highlighted';
-    }
-});
-```
-
----
-
-## 🧪 Testing Tools
-
-### Manual Testing
-
-```javascript
-// Simulate AI calling a tool
-client.emit('tool_call', {
-    name: 'show_alert',
-    args: { message: 'Test alert', severity: 'info' }
-});
-```
-
-### Debugging
-
-```javascript
-client.on('tool_call', async (tool) => {
-    console.log('=== TOOL CALL ===');
-    console.log('Name:', tool.name);
-    console.log('Args:', tool.args);
-    console.log('ID:', tool.call_id);
-    
-    // Your handler logic here
-    const result = await handleTool(tool);
-    
-    console.log('Result:', result);
-    return result;
-});
-```
-
----
-
-## ⚠️ Best Practices
-
-### 1. Clear Descriptions
-```javascript
-// ❌ Bad
-description: 'Does something'
-
-// ✅ Good
-description: 'End the call when user says goodbye or wants to finish'
-```
-
-### 2. Validate Arguments
-```javascript
-client.on('tool_call', async (tool) => {
-    if (tool.name === 'navigate_to_page') {
-        // Validate
-        if (!tool.args.url || !tool.args.url.startsWith('http')) {
-            return 'Error: Invalid URL';
-        }
-        
-        window.location.href = tool.args.url;
-        return 'Navigating to ' + tool.args.url;
-    }
-});
-```
-
-### 3. Return Meaningful Results
-```javascript
-// ❌ Bad
-return 'ok';
-
-// ✅ Good
-return 'Alert shown successfully with message: "' + tool.args.message + '"';
-```
-
-This helps the AI understand what happened and continue the conversation naturally.
-
----
-
-## 🔒 Security
-
-### Client Tools are Safe
-- Executed in **your app**, not on our servers
-- You have **full control** over what they do
-- No sensitive data is sent to the server
-
-### Never Trust User Input
-```javascript
-// ❌ Dangerous
-client.on('tool_call', async (tool) => {
-    eval(tool.args.code); // NEVER DO THIS
-});
-
-// ✅ Safe
-client.on('tool_call', async (tool) => {
-    const allowedUrls = ['https://myapp.com/dashboard', 'https://myapp.com/profile'];
-    if (allowedUrls.includes(tool.args.url)) {
-        window.location.href = tool.args.url;
-    }
-});
-```
-
----
-
-## 📚 Examples
-
-See complete working examples in:
-- [Browser Example](../examples/browser/) - Full UI with tools
-- [Node.js CLI](../examples/nodejs/) - CLI bot with hangup
-- [Python Example](../examples/python/) - Python integration
-
----
-
-## 💡 Tips
-
-1. **Start Simple**: Begin with just `hangup`, then add more
-2. **Test Locally**: Use manual emit to test without calling the AI
-3. **Return Clear Messages**: AI uses return value to continue conversation
-4. **Validate Everything**: Never trust arguments blindly
-5. **Keep it Fast**: Tool execution should be instant (< 100ms)
-
----
-
-Happy building! 🎉

package/docs/TROUBLESHOOTING.md

@@ -1,86 +0,0 @@
-# Troubleshooting Guide
-
-Common issues and how to resolve them when using AillomVox.
-
----
-
-## Authentication Errors
-
-### `401 Unauthorized` / `error: unauthorized`
-- **Cause**: API Key is missing, invalid, or expired.
-- **Fix**: Check that you're sending `apikey` in your config handshake. Ensure you are copying the key exactly as provided in your dashboard.
-
-### `error: insufficient_balance`
-- **Cause**: Your account balance is $0.00 or below.
-- **Fix**: Add credits via the [Billing page](https://vox.aillom.com/billing).
-
----
-
-## Connection Issues
-
-### `Connection Refused` / `Socket Hang Up`
-- **Cause**: Server unreachable or firewall blocking outbound port 443 (WSS).
-- **Fix**: Ensure you can reach `https://vox.aillom.com/health`. Check your network settings.
-
-### `1008 Policy Violation`
-- **Cause**: Sent binary data before the config handshake.
-- **Fix**: The **first message** must be a JSON config. Wait for it to be acknowledged before sending audio.
-
-### `1006 Abnormal Closure`
-- **Cause**: Generic WebSocket error. Often caused by:
-  - Sending malformed JSON
-  - Sending audio in wrong format (e.g., MP3 instead of PCM)
-  - Network timeout (keep-alive)
-
----
-
-## Audio Problems
-
-### "Static" or "Noise" instead of Voice
-- **Cause**: Wrong encoding or endianness.
-- **Fix**: AillomVox expects **PCM 16-bit Signed Little Endian**. 
-  - If you send Float32 (Web Audio API default), you get static. Convert to Int16 first.
-  - If you send Big Endian, you get static. Use `true` for little-endian in `DataView.setInt16()`.
-
-### High Latency
-- **Cause**: Buffering too much audio before sending.
-- **Fix**: Send small chunks (20ms–50ms) as soon as recorded. Do not batch 1 second of audio.
-
-### "Choppy" Audio Output
-- **Cause**: Network jitter or empty buffer underruns.
-- **Fix**: Implement a jitter buffer — wait for 2–3 chunks before starting playback.
-
-### No Audio Received
-- **Cause**: `sample_rate` mismatch or missing `system_prompt`.
-- **Fix**: Ensure `sample_rate` is `8000`, `16000`, or `24000`. The `system_prompt` field is mandatory.
-
----
-
-## Tool Call Issues
-
-### AI Hangs After Tool Call
-- **Cause**: Missing `tool_result` response.
-- **Fix**: Always respond to `tool_call` events with a `tool_result` within 15 seconds:
-  ```javascript
-  ws.send(JSON.stringify({
-    type: "tool_result",
-    call_id: msg.call_id,
-    result: "Success"
-  }));
-  ```
-
-### Tools Not Triggering
-- **Cause**: Vague tool descriptions or wrong provider.
-- **Fix**: Use clear, specific descriptions. Note: Qwen does not support tools — use AillomVox, OpenAI, Gemini, or AWS.
-
----
-
-## Session Issues
-
-### `max_duration_reached`
-- **Cause**: Session exceeded the `max_duration` limit.
-- **Fix**: Increase `max_duration` (up to 3600) or handle the farewell message gracefully.
-
-### Multiple Connections Rejected
-- **Cause**: Exceeded concurrent connection limit (3 per user, 2 per API key).
-- **Fix**: Close unused connections before opening new ones.