openclaw-liveavatar 1.0.0

package/README.md

@@ -0,0 +1,241 @@
+# OpenClaw LiveAvatar
+
+Give your OpenClaw agent a face and voice! Talk face-to-face with a real-time AI avatar powered by [LiveAvatar](https://liveavatar.com).
+
+![OpenClaw LiveAvatar Demo](public/demo.png)
+
+## Features
+
+- **Voice-to-Voice Conversation**: Speak naturally and hear your agent respond
+- **Real-time Avatar**: Lip-synced video avatar with natural expressions
+- **OpenClaw Integration**: Connects to your local OpenClaw Gateway
+- **Smart TTS Summarization**: Long responses are summarized for natural speech
+- **Echo Cancellation**: Won't respond to itself
+- **Multiple Avatar Choices**: Select from custom or public avatars
+- **Chat Transcript**: View the full conversation history
+
+## Installation
+
+### Option 1: ClawHub Skill (Recommended)
+
+If you have [OpenClaw](https://openclaw.ai) installed:
+
+```bash
+clawhub install liveavatar
+```
+
+Then run the `/liveavatar` command in any OpenClaw chat.
+
+### Option 2: NPX (Quick Start)
+
+```bash
+# Set your API key
+export LIVEAVATAR_API_KEY=your_key_here
+
+# Start OpenClaw Gateway (in another terminal)
+openclaw gateway
+
+# Run LiveAvatar
+npx openclaw-liveavatar
+```
+
+### Option 3: Global Install
+
+```bash
+npm install -g openclaw-liveavatar
+
+# Then run anytime with:
+openclaw-liveavatar
+```
+
+### Option 4: Development Setup
+
+```bash
+git clone https://github.com/eNNNo/openclaw-liveavatar.git
+cd openclaw-liveavatar
+npm install
+cp .env.example .env.local
+# Edit .env.local with your API key
+npm run dev
+```
+
+## Prerequisites
+
+- Node.js 18+
+- [OpenClaw](https://openclaw.ai) installed with Gateway running
+- [LiveAvatar API Key](https://app.liveavatar.com) (free tier available)
+
+## Setup
+
+### 1. Get Your API Key (Free)
+
+1. Go to [app.liveavatar.com](https://app.liveavatar.com)
+2. Create a free account
+3. Copy your API key from the dashboard
+
+### 2. Set Your API Key
+
+**Option A: Environment variable**
+```bash
+export LIVEAVATAR_API_KEY=your_api_key_here
+```
+
+**Option B: OpenClaw config** (`~/.openclaw/openclaw.json`)
+```json
+{
+  "skills": {
+    "entries": {
+      "liveavatar": {
+        "env": {
+          "LIVEAVATAR_API_KEY": "your_api_key_here"
+        }
+      }
+    }
+  }
+}
+```
+
+### 3. Start OpenClaw Gateway
+
+```bash
+openclaw gateway
+```
+
+### 4. Launch LiveAvatar
+
+```bash
+npx openclaw-liveavatar
+# Or: /liveavatar (if installed as skill)
+```
+
+The interface will open at http://localhost:3001
+
+> **Demo Mode**: If OpenClaw Gateway isn't running, the app will start in Demo Mode where you can interact with the avatar and learn about the integration.
+
+## How It Works
+
+```
+┌─────────────────┐     ┌──────────────────┐     ┌─────────────────┐
+│   You Speak     │────▶│   LiveAvatar     │────▶│  OpenClaw       │
+│   (Microphone)  │     │   (Transcribe)   │     │  Gateway        │
+└─────────────────┘     └──────────────────┘     └────────┬────────┘
+                                                          │
+┌─────────────────┐     ┌──────────────────┐              │
+│   Avatar Speaks │◀────│   LiveAvatar     │◀─────────────┘
+│   (Lip-sync)    │     │   (TTS + Video)  │     Agent Response
+└─────────────────┘     └──────────────────┘
+```
+
+1. **You speak** into your microphone
+2. **LiveAvatar transcribes** your speech to text
+3. **OpenClaw Gateway** receives the text and sends it to your agent
+4. **Your agent responds** with text
+5. **LiveAvatar synthesizes** the response as speech
+6. **The avatar speaks** with synchronized lip movements
+
+## Configuration
+
+### Environment Variables
+
+| Variable | Description | Default |
+|----------|-------------|---------|
+| `LIVEAVATAR_API_KEY` | Your LiveAvatar API key (required) | - |
+| `OPENCLAW_GATEWAY_URL` | WebSocket URL for OpenClaw Gateway | `ws://127.0.0.1:18789` |
+| `OPENCLAW_GATEWAY_TOKEN` | Token for remote Gateway access | - |
+
+### OpenClaw Skill Installation
+
+This can also be installed as an OpenClaw skill:
+
+```bash
+clawhub install liveavatar
+```
+
+Or invoke directly:
+
+```
+/liveavatar
+```
+
+## Development
+
+```bash
+# Install dependencies
+npm install
+
+# Start development server
+npm run dev
+
+# Build for production
+npm run build
+
+# Start production server
+npm start
+```
+
+## Architecture
+
+```
+openclaw-liveavatar/
+├── app/                    # Next.js app directory
+│   ├── api/               # API routes
+│   │   ├── config.ts      # Configuration
+│   │   ├── start-session/ # Session token generation
+│   │   └── get-avatars/   # Avatar listing
+│   ├── layout.tsx         # Root layout
+│   └── page.tsx           # Main page
+├── src/
+│   ├── components/        # React components
+│   │   ├── OpenClawDemo.tsx      # Setup/landing UI
+│   │   └── LiveAvatarSession.tsx # Session UI
+│   ├── gateway/           # OpenClaw Gateway client
+│   │   ├── client.ts      # WebSocket client
+│   │   └── types.ts       # Protocol types
+│   └── liveavatar/        # LiveAvatar SDK hooks
+│       ├── context.tsx    # React context with Gateway bridge
+│       ├── useSession.ts  # Session management
+│       ├── useVoiceChat.ts # Voice chat controls
+│       └── ...
+├── skills/                # OpenClaw skill definition
+│   └── liveavatar/
+│       └── SKILL.md
+└── openclaw.plugin.json   # Channel plugin manifest
+```
+
+## Troubleshooting
+
+### "OpenClaw Disconnected"
+
+Make sure your OpenClaw Gateway is running:
+
+```bash
+openclaw gateway
+```
+
+### "No avatars available"
+
+Verify your `LIVEAVATAR_API_KEY` is set correctly in `.env.local`
+
+### Avatar not responding to speech
+
+1. Check microphone permissions in your browser
+2. Ensure the mic is not muted (green button = active)
+3. Verify Gateway shows "Connected" status
+4. Try a different microphone from the dropdown
+
+### Connection quality issues
+
+LiveAvatar uses WebRTC. For best results:
+- Use a stable internet connection
+- Close other video/audio applications
+- Use Chrome or Edge for best WebRTC support
+
+## Credits
+
+- [HeyGen LiveAvatar](https://liveavatar.com) - Real-time AI avatar technology
+- [OpenClaw](https://openclaw.ai) - AI agent gateway
+- Based on [liveavatar-ai-sdr](https://github.com/eNNNo/liveavatar-ai-sdr)
+
+## License
+
+MIT

package/app/api/config.ts

@@ -0,0 +1,18 @@
+// LiveAvatar Configuration
+// Get your free API key from https://app.liveavatar.com/developers
+export const LIVEAVATAR_API_KEY = process.env.LIVEAVATAR_API_KEY || "";
+export const LIVEAVATAR_API_URL = "https://api.liveavatar.com";
+
+// Default avatar and voice (can be overridden by user selection)
+export const DEFAULT_AVATAR_ID = "1c690fe7-23e0-49f9-bfba-14344450285b"; // Stock avatar
+export const DEFAULT_VOICE_ID = ""; // Will use avatar's default voice
+
+// Sandbox mode for development (uses minimal credits)
+// Note: Not all avatars support sandbox mode - set to false for full avatar access
+export const IS_SANDBOX = false;
+
+export const LANGUAGE = "en";
+
+// OpenClaw Gateway Configuration
+export const OPENCLAW_GATEWAY_URL = process.env.OPENCLAW_GATEWAY_URL || "ws://127.0.0.1:18789";
+export const OPENCLAW_GATEWAY_TOKEN = process.env.OPENCLAW_GATEWAY_TOKEN || "";

package/app/api/get-avatars/route.ts

@@ -0,0 +1,117 @@
+import { LIVEAVATAR_API_KEY, LIVEAVATAR_API_URL } from "../config";
+
+interface Avatar {
+  id: string;
+  name: string;
+  preview_url?: string;
+  status?: string;
+  is_custom?: boolean;
+  is_expired?: boolean;
+  default_voice?: {
+    id: string;
+    name: string;
+  };
+}
+
+interface AvatarResponse {
+  data?: {
+    results?: Avatar[];
+    total?: number;
+    page?: number;
+    limit?: number;
+  };
+}
+
+// Fetch all pages from a paginated endpoint
+async function fetchAllAvatars(endpoint: string): Promise<Avatar[]> {
+  const allAvatars: Avatar[] = [];
+  let page = 1;
+  const limit = 100;
+
+  while (true) {
+    const res = await fetch(`${LIVEAVATAR_API_URL}${endpoint}?page=${page}&limit=${limit}`, {
+      headers: {
+        "X-API-KEY": LIVEAVATAR_API_KEY!,
+      },
+    });
+
+    if (!res.ok) {
+      console.error(`Failed to fetch ${endpoint} page ${page}:`, res.status);
+      break;
+    }
+
+    const data: AvatarResponse = await res.json();
+    const pageAvatars = data.data?.results || [];
+    allAvatars.push(...pageAvatars);
+
+    const total = data.data?.total || pageAvatars.length;
+    console.log(`[Avatars] ${endpoint} page ${page}: ${pageAvatars.length} avatars (total: ${total})`);
+
+    // Check if we've fetched all
+    if (allAvatars.length >= total || pageAvatars.length === 0) {
+      break;
+    }
+    page++;
+  }
+
+  return allAvatars;
+}
+
+export async function GET() {
+  if (!LIVEAVATAR_API_KEY) {
+    return new Response(
+      JSON.stringify({
+        error: "LiveAvatar API key not configured",
+        customAvatars: [],
+        publicAvatars: [],
+      }),
+      { status: 200, headers: { "Content-Type": "application/json" } }
+    );
+  }
+
+  try {
+    // Fetch both custom (user) avatars and public avatars in parallel
+    const [customAvatars, publicAvatars] = await Promise.all([
+      fetchAllAvatars("/v1/avatars"),        // User's custom avatars
+      fetchAllAvatars("/v1/avatars/public"), // Public avatars
+    ]);
+
+    // Mark custom avatars
+    const markedCustomAvatars = customAvatars.map(avatar => ({
+      ...avatar,
+      is_custom: true,
+    }));
+
+    // Mark public avatars (they don't expire)
+    const markedPublicAvatars = publicAvatars.map(avatar => ({
+      ...avatar,
+      is_custom: false,
+      is_expired: false, // Public avatars don't expire
+    }));
+
+    console.log(`[Avatars] Total: ${markedCustomAvatars.length} custom, ${markedPublicAvatars.length} public`);
+
+    return new Response(
+      JSON.stringify({
+        customAvatars: markedCustomAvatars,
+        publicAvatars: markedPublicAvatars,
+      }),
+      {
+        status: 200,
+        headers: { "Content-Type": "application/json" },
+      }
+    );
+  } catch (error) {
+    console.error("Error fetching avatars:", error);
+    return new Response(
+      JSON.stringify({
+        customAvatars: [],
+        publicAvatars: [],
+      }),
+      {
+        status: 200,
+        headers: { "Content-Type": "application/json" },
+      }
+    );
+  }
+}

package/app/api/start-session/route.ts

@@ -0,0 +1,95 @@
+import {
+  LIVEAVATAR_API_KEY,
+  LIVEAVATAR_API_URL,
+  DEFAULT_AVATAR_ID,
+  DEFAULT_VOICE_ID,
+  IS_SANDBOX,
+  LANGUAGE,
+} from "../config";
+
+export async function POST(request: Request) {
+  let session_token = "";
+  let session_id = "";
+
+  // Allow dynamic parameters from request body, fall back to defaults
+  let avatarId = DEFAULT_AVATAR_ID;
+  let voiceId = DEFAULT_VOICE_ID;
+
+  try {
+    const body = await request.json();
+    if (body.avatarId) {
+      avatarId = body.avatarId;
+    }
+    if (body.voiceId) {
+      voiceId = body.voiceId;
+    }
+  } catch {
+    // No body or invalid JSON, use defaults
+  }
+
+  // Check for API key
+  if (!LIVEAVATAR_API_KEY) {
+    return new Response(
+      JSON.stringify({
+        error:
+          "LiveAvatar API key not configured. Please set LIVEAVATAR_API_KEY in your .env.local file.",
+      }),
+      { status: 500, headers: { "Content-Type": "application/json" } }
+    );
+  }
+
+  try {
+    // For OpenClaw integration, we use FULL mode but without a predefined context
+    // The avatar will act as a passthrough - we'll control what it says via the bridge
+    const res = await fetch(`${LIVEAVATAR_API_URL}/v1/sessions/token`, {
+      method: "POST",
+      headers: {
+        "X-API-KEY": LIVEAVATAR_API_KEY,
+        "Content-Type": "application/json",
+      },
+      body: JSON.stringify({
+        mode: "FULL",
+        avatar_id: avatarId,
+        avatar_persona: {
+          // Only include voice_id if provided
+          ...(voiceId ? { voice_id: voiceId } : {}),
+          // No context_id - we'll use OpenClaw for the intelligence
+          language: LANGUAGE,
+        },
+        is_sandbox: IS_SANDBOX,
+      }),
+    });
+
+    if (!res.ok) {
+      const resp = await res.json();
+      const errorMessage =
+        resp.data?.[0]?.message ?? "Failed to retrieve session token";
+      return new Response(JSON.stringify({ error: errorMessage }), {
+        status: res.status,
+        headers: { "Content-Type": "application/json" },
+      });
+    }
+
+    const data = await res.json();
+    session_token = data.data.session_token;
+    session_id = data.data.session_id;
+  } catch (error) {
+    console.error("Error retrieving session token:", error);
+    return new Response(JSON.stringify({ error: (error as Error).message }), {
+      status: 500,
+      headers: { "Content-Type": "application/json" },
+    });
+  }
+
+  if (!session_token) {
+    return new Response(
+      JSON.stringify({ error: "Failed to retrieve session token" }),
+      { status: 500, headers: { "Content-Type": "application/json" } }
+    );
+  }
+
+  return new Response(JSON.stringify({ session_token, session_id }), {
+    status: 200,
+    headers: { "Content-Type": "application/json" },
+  });
+}

package/app/globals.css

@@ -0,0 +1,3 @@
+@tailwind base;
+@tailwind components;
+@tailwind utilities;

package/app/layout.tsx

@@ -0,0 +1,37 @@
+import "./globals.css";
+import type { Metadata } from "next";
+
+export const metadata: Metadata = {
+  title: "OpenClaw LiveAvatar",
+  description: "Talk to your OpenClaw agent face-to-face with a real-time AI avatar",
+};
+
+// Gateway configuration from environment variables
+const gatewayConfig = {
+  url: process.env.OPENCLAW_GATEWAY_URL || "ws://127.0.0.1:18789",
+  token: process.env.OPENCLAW_GATEWAY_TOKEN || "",
+};
+
+export default function RootLayout({
+  children,
+}: {
+  children: React.ReactNode;
+}) {
+  return (
+    <html lang="en">
+      <head>
+        <script
+          dangerouslySetInnerHTML={{
+            __html: `
+              window.__OPENCLAW_GATEWAY_URL = ${JSON.stringify(gatewayConfig.url)};
+              window.__OPENCLAW_GATEWAY_TOKEN = ${JSON.stringify(gatewayConfig.token)};
+            `,
+          }}
+        />
+      </head>
+      <body className="bg-gray-950 text-white min-h-screen">
+        {children}
+      </body>
+    </html>
+  );
+}

package/app/page.tsx

@@ -0,0 +1,9 @@
+import { OpenClawDemo } from "../src/components/OpenClawDemo";
+
+export default function Home() {
+  return (
+    <main className="min-h-screen flex items-center justify-center">
+      <OpenClawDemo />
+    </main>
+  );
+}

package/bin/cli.js

@@ -0,0 +1,100 @@
+#!/usr/bin/env node
+
+import { spawn } from 'child_process';
+import { fileURLToPath } from 'url';
+import { dirname, join } from 'path';
+import open from 'open';
+
+const __filename = fileURLToPath(import.meta.url);
+const __dirname = dirname(__filename);
+const projectRoot = join(__dirname, '..');
+
+const PORT = process.env.PORT || 3001;
+
+console.log('');
+console.log('  \x1b[38;5;208m\x1b[1m🦞 OpenClaw LiveAvatar\x1b[0m');
+console.log('  ─────────────────────────────────────');
+console.log('');
+
+// Check for LIVEAVATAR_API_KEY
+if (!process.env.LIVEAVATAR_API_KEY) {
+  console.log('  \x1b[33m⚠️  Warning: LIVEAVATAR_API_KEY not set\x1b[0m');
+  console.log('');
+  console.log('  To use LiveAvatar, you need an API key:');
+  console.log('  1. Get your free key at \x1b[36mhttps://app.liveavatar.com\x1b[0m');
+  console.log('  2. Set it: \x1b[90mexport LIVEAVATAR_API_KEY=your_key_here\x1b[0m');
+  console.log('');
+}
+
+console.log(`  Starting server on port ${PORT}...`);
+console.log('');
+
+// Start the Next.js server
+const server = spawn('npx', ['next', 'start', '-p', String(PORT)], {
+  cwd: projectRoot,
+  stdio: ['inherit', 'pipe', 'pipe'],
+  env: { ...process.env, PORT: String(PORT) }
+});
+
+let serverReady = false;
+
+server.stdout.on('data', (data) => {
+  const output = data.toString();
+
+  // Detect when server is ready
+  if (!serverReady && (output.includes('Ready') || output.includes('started') || output.includes('localhost'))) {
+    serverReady = true;
+    const url = `http://localhost:${PORT}`;
+
+    console.log(`  \x1b[32m✓ Server ready!\x1b[0m`);
+    console.log('');
+    console.log(`  \x1b[1mOpen in browser:\x1b[0m \x1b[36m${url}\x1b[0m`);
+    console.log('');
+    console.log('  \x1b[90mThe LiveAvatar interface will connect to your\x1b[0m');
+    console.log('  \x1b[90mOpenClaw Gateway on port 18789 automatically.\x1b[0m');
+    console.log('');
+    console.log('  \x1b[90mPress Ctrl+C to stop\x1b[0m');
+    console.log('');
+
+    // Open browser
+    open(url).catch(() => {
+      console.log('  \x1b[90m(Could not open browser automatically)\x1b[0m');
+    });
+  }
+});
+
+server.stderr.on('data', (data) => {
+  const output = data.toString();
+  // Filter out noisy Next.js output
+  if (!output.includes('Compiling') && !output.includes('compiled')) {
+    process.stderr.write(data);
+  }
+});
+
+server.on('error', (err) => {
+  console.error('  \x1b[31m✗ Failed to start server:\x1b[0m', err.message);
+  console.log('');
+  console.log('  Make sure you have run:');
+  console.log('  \x1b[90mnpm install\x1b[0m');
+  console.log('');
+  process.exit(1);
+});
+
+server.on('close', (code) => {
+  if (code !== 0 && code !== null) {
+    console.log('');
+    console.log(`  \x1b[31mServer exited with code ${code}\x1b[0m`);
+  }
+  process.exit(code || 0);
+});
+
+// Handle graceful shutdown
+process.on('SIGINT', () => {
+  console.log('');
+  console.log('  \x1b[90mShutting down...\x1b[0m');
+  server.kill('SIGINT');
+});
+
+process.on('SIGTERM', () => {
+  server.kill('SIGTERM');
+});

package/package.json

@@ -0,0 +1,66 @@
+{
+  "name": "openclaw-liveavatar",
+  "version": "1.0.0",
+  "description": "Real-time video avatar chat interface for OpenClaw - talk face-to-face with your AI agent",
+  "type": "module",
+  "bin": {
+    "openclaw-liveavatar": "./bin/cli.js"
+  },
+  "scripts": {
+    "dev": "next dev -p 3001 --turbopack",
+    "build": "next build",
+    "start": "next start -p 3001",
+    "prepublishOnly": "npm run build"
+  },
+  "keywords": [
+    "openclaw",
+    "liveavatar",
+    "heygen",
+    "ai",
+    "avatar",
+    "voice",
+    "chat",
+    "video",
+    "assistant"
+  ],
+  "author": "eNNNo",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/eNNNo/openclaw-liveavatar.git"
+  },
+  "bugs": {
+    "url": "https://github.com/eNNNo/openclaw-liveavatar/issues"
+  },
+  "homepage": "https://github.com/eNNNo/openclaw-liveavatar#readme",
+  "files": [
+    "bin",
+    "app",
+    "src",
+    "public",
+    ".next",
+    "next.config.ts",
+    "tailwind.config.ts",
+    "postcss.config.mjs",
+    "tsconfig.json"
+  ],
+  "dependencies": {
+    "@heygen/liveavatar-web-sdk": "^0.0.10",
+    "next": "^15.4.2",
+    "open": "^10.1.0",
+    "react": "^19.1.0",
+    "react-dom": "^19.1.0"
+  },
+  "devDependencies": {
+    "@types/node": "22.14.0",
+    "@types/react": "19.1.6",
+    "@types/react-dom": "19.1.6",
+    "autoprefixer": "^10.4.21",
+    "postcss": "^8.5.3",
+    "tailwindcss": "^3.4.17",
+    "typescript": "5.8.2"
+  },
+  "engines": {
+    "node": ">=18.0.0"
+  }
+}