talkiebot 0.1.0

package/README.md

@@ -0,0 +1,263 @@
+# Talkie
+
+A voice-first interface for Claude Code, styled after the classic Talkie cassette recorder from Home Alone 2.
+
+Talk to Claude with push-to-talk, wake words, or continuous listening. Manage conversations as cassette tapes. Works with web UI, Telegram bot, and MCP integration.
+
+## Quick Start
+
+```bash
+npx talkie
+```
+
+This starts the HTTPS server and opens https://localhost:5173 in your browser.
+
+> Requires Chrome or Edge (Web Speech API).
+
+### Setup
+
+1. Complete the onboarding flow to configure voice settings
+2. Choose your preferences:
+   - **Wake word**: Enable "hey talkie" hands-free activation
+   - **Continuous listening**: Always-on with trigger word
+   - **Text-to-speech**: Have responses read aloud
+3. Start talking!
+
+## Features
+
+### Voice Input
+
+- **Push-to-talk**: Hold spacebar or click the record button
+- **Wake word**: Say "hey talkie" (customizable) for hands-free activation
+- **Continuous listening**: Always-on mode that waits for your trigger word
+- **Trigger word**: Say "over" (customizable) to end your turn
+- **Silence detection**: Configurable delay (0.5-3.0s) after trigger word
+- **Streaming TTS**: Responses spoken back in real-time
+
+### Cassette Tape UI
+
+Conversations are "tapes" in a tape deck:
+- **Tape Deck**: Input bar with mini cassette display showing current conversation
+- **Tape Collection**: Drawer with all conversations, eject button to browse
+- **Switch tapes**: Click any tape to load that conversation
+- **New tape**: Create a fresh conversation
+
+### Claude Integration
+
+Two modes:
+
+- **Claude Code mode** (default): Spawns `claude -p` with full tool access. Shows real-time activity feed.
+- **Direct API mode**: Uses your Anthropic API key. Supports streaming TTS.
+
+### Image Handling
+
+- Drag and drop images onto the chat for Claude vision analysis
+- Media library to browse images across all conversations
+- Image lightbox viewer
+
+### Activity Feed
+
+When Claude Code runs tools, the UI shows tool name, icon, input details, and live status (spinner, checkmark, error). Collapsible per-message activity history persisted to SQLite.
+
+### Mobile Support
+
+- Floating Action Button (FAB) for recording — draggable and resizable
+- Mobile dropdown menu for settings, media library, tape collection
+
+### Keyboard Shortcuts
+
+| Key | Action |
+|-----|--------|
+| `Spacebar` (hold) | Push-to-talk (when not in continuous mode) |
+| `Escape` | Cancel recording |
+| `Cmd/Ctrl+K` | Search |
+| `Cmd/Ctrl+E` | Export conversation |
+
+### Export
+
+Export conversations as Markdown, JSON, or plain text.
+
+## Server Management
+
+```bash
+talkie-server start [-f]    # Start (background, or -f for foreground)
+talkie-server stop          # Stop the server
+talkie-server restart       # Restart
+talkie-server status        # Show status (running, port, launchd, DB)
+talkie-server logs [-f]     # View logs (-f to follow)
+talkie-server install       # Install as macOS launchd daemon (auto-start on login)
+talkie-server uninstall     # Remove launchd daemon
+```
+
+Set `TALKIE_PORT` to change the default port (5173).
+
+## Telegram Bot
+
+Chat with Claude from your phone via Telegram.
+
+### Setup
+
+1. Create a bot via [@BotFather](https://t.me/BotFather)
+2. Provide the token:
+   - Environment variable: `TELEGRAM_BOT_TOKEN=your-token`
+   - Or token file: `~/.talkie/telegram.token`
+3. Start the server — bot starts automatically
+
+### Commands
+
+| Command | Description |
+|---------|-------------|
+| `/start` | Welcome message |
+| `/help` | Show all commands |
+| `/conversations` | List recent conversations with selection buttons |
+| `/new <name>` | Create a new conversation |
+| `/current` | Show current conversation info |
+| `/status` | Check server and Claude status |
+
+Send text messages to chat. Send photos (with optional captions) for image analysis.
+
+## MCP Integration
+
+Claude Code can launch and interact with Talkie via MCP tools.
+
+### Setup
+
+Add to `~/.claude/settings.json`:
+
+```json
+{
+  "mcpServers": {
+    "talkie": {
+      "command": "npx",
+      "args": ["talkie-mcp"]
+    }
+  }
+}
+```
+
+### Tools
+
+| Tool | Description |
+|------|-------------|
+| `launch_talkie` | Start server and open browser |
+| `get_talkie_status` | Check running status and avatar state |
+| `get_transcript` | Get latest voice transcript |
+| `get_conversation_history` | Get current conversation messages |
+| `get_claude_session` | Get connected session ID |
+| `set_claude_session` | Connect to a Claude Code session |
+| `disconnect_claude_session` | Disconnect session |
+| `get_pending_message` | Poll for user messages (IPC mode) |
+| `respond_to_talkie` | Send response to user (IPC mode) |
+| `update_talkie_state` | Set avatar state, transcript |
+| `analyze_image` | Analyze image via Claude vision |
+| `open_url` | Open URL in default browser |
+
+## Persistence
+
+- **Browser**: localStorage for settings and conversation cache
+- **Server**: SQLite at `~/.talkie/talkie.db` (WAL mode, FTS5 full-text search)
+- **Migration**: On first server connection, localStorage data auto-migrates to SQLite
+
+## Architecture
+
+```
+Browser (React 18 + TypeScript + Zustand)
+    │
+    ├── Voice: Web Speech API (STT/TTS)
+    ├── State: Zustand + localStorage cache
+    └── API ──► HTTPS Server (Hono)
+                    │
+                    ├── SQLite (conversations, messages, activities, images)
+                    ├── Claude Code CLI (spawn per message)
+                    ├── Telegram Bot (grammy)
+                    └── Static files (dist/)
+
+MCP Server (stdio) ──► HTTPS Server API
+```
+
+```
+src/
+├── App.tsx                    Main orchestration
+├── components/
+│   ├── activity/              Real-time tool usage feed
+│   ├── avatar/                Animated avatar with states
+│   ├── cassette/              Tape deck, collection, visuals
+│   ├── chat/                  Timeline, sidebar, input bar
+│   ├── dropzone/              Image drag-and-drop
+│   ├── media/                 Lightbox and library
+│   ├── onboarding/            First-run setup
+│   ├── settings/              Settings drawer
+│   └── voice/                 Recognition, synthesis, wake word
+├── lib/
+│   ├── claude.ts              Claude API (direct + CLI + vision)
+│   ├── store.ts               Zustand state
+│   └── api.ts                 Server API client
+├── hooks/                     Keyboard shortcuts, sound effects
+└── contexts/                  Theme context
+
+server/
+├── index.ts                   Server startup (HTTPS, DB, Telegram)
+├── api.ts                     All API routes
+├── state.ts                   In-memory IPC state
+├── ssl.ts                     Self-signed cert generation
+├── db/
+│   ├── schema.ts              Versioned migrations
+│   └── repositories/          CRUD modules
+└── telegram/                  Bot commands and handlers
+
+mcp-server/index.js            MCP server (12 tools, stdio)
+bin/
+├── talkie.js                 Start server + open browser
+├── talkie-server.js          Server lifecycle CLI
+└── talkie-mcp.js             MCP server entry
+```
+
+### API Endpoints
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/api/status` | GET | Health check, avatar state, DB status |
+| `/api/transcript` | GET | Latest voice transcript and messages |
+| `/api/history` | GET | Conversation messages for MCP |
+| `/api/state` | GET/POST | Get or sync browser state |
+| `/api/session` | GET/POST/DELETE | Claude Code session management |
+| `/api/pending` | GET | Check for pending IPC messages |
+| `/api/respond` | POST | Send IPC response |
+| `/api/send` | POST | Send message with SSE streaming |
+| `/api/claude-code` | POST | Execute Claude CLI, stream response |
+| `/api/analyze-image` | POST | Claude vision analysis |
+| `/api/open-url` | POST | Open URL in native browser |
+| `/api/conversations` | GET/POST | List or create conversations |
+| `/api/conversations/:id` | GET/PATCH/DELETE | Get, update, or delete conversation |
+| `/api/conversations/:id/messages` | POST | Add message with images/activities |
+| `/api/search` | GET | Full-text search across messages |
+| `/api/migrate` | POST | Migrate localStorage to server |
+
+See [docs/API.md](docs/API.md) for detailed request/response formats.
+
+## Settings
+
+Configurable via the settings drawer:
+
+- **Claude Code mode**: Toggle between CLI and Direct API
+- **Session connection**: View/disconnect Claude Code session
+- **TTS**: Enable/disable text-to-speech
+- **Continuous listening**: Auto-restart recording after responses
+- **Trigger word**: Custom end-of-turn word (default: "over")
+- **Silence delay**: Wait time after trigger word (0.5-3.0s)
+- **API key**: Anthropic API key for Direct API mode
+- **Reset onboarding**: Re-run the setup wizard
+
+## Development
+
+```bash
+npm install       # Install dependencies
+npm run dev       # Vite dev server (frontend only)
+npm run build     # Production build
+npm run test      # Run tests
+npm run lint      # ESLint
+```
+
+## License
+
+MIT

package/bin/talkie-server.js

@@ -0,0 +1,336 @@
+#!/usr/bin/env node
+
+import { spawn, execSync } from 'child_process'
+import { existsSync, mkdirSync, writeFileSync, readFileSync, unlinkSync, createReadStream } from 'fs'
+import { join, dirname } from 'path'
+import { homedir } from 'os'
+import { fileURLToPath } from 'url'
+import { createInterface } from 'readline'
+
+const __dirname = dirname(fileURLToPath(import.meta.url))
+const PLIST_NAME = 'com.talkie.server'
+const PLIST_PATH = join(homedir(), 'Library', 'LaunchAgents', `${PLIST_NAME}.plist`)
+const LOG_DIR = join(homedir(), '.talkie', 'logs')
+const PID_FILE = join(homedir(), '.talkie', 'server.pid')
+const PORT = parseInt(process.env.TALKIE_PORT || '5173', 10)
+
+function ensureDirs() {
+  const talkieDir = join(homedir(), '.talkie')
+  if (!existsSync(talkieDir)) {
+    mkdirSync(talkieDir, { recursive: true })
+  }
+  if (!existsSync(LOG_DIR)) {
+    mkdirSync(LOG_DIR, { recursive: true })
+  }
+  const launchAgentsDir = dirname(PLIST_PATH)
+  if (!existsSync(launchAgentsDir)) {
+    mkdirSync(launchAgentsDir, { recursive: true })
+  }
+}
+
+function generatePlist() {
+  const serverScript = join(__dirname, '..', 'server', 'index.js')
+
+  return `<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key>
+    <string>${PLIST_NAME}</string>
+    <key>ProgramArguments</key>
+    <array>
+        <string>/usr/bin/env</string>
+        <string>node</string>
+        <string>${serverScript}</string>
+    </array>
+    <key>EnvironmentVariables</key>
+    <dict>
+        <key>PORT</key>
+        <string>${PORT}</string>
+        <key>PATH</key>
+        <string>/usr/local/bin:/usr/bin:/bin:/opt/homebrew/bin</string>
+    </dict>
+    <key>WorkingDirectory</key>
+    <string>${join(__dirname, '..')}</string>
+    <key>RunAtLoad</key>
+    <true/>
+    <key>KeepAlive</key>
+    <true/>
+    <key>StandardOutPath</key>
+    <string>${join(LOG_DIR, 'stdout.log')}</string>
+    <key>StandardErrorPath</key>
+    <string>${join(LOG_DIR, 'stderr.log')}</string>
+</dict>
+</plist>`
+}
+
+function isLaunchctlLoaded() {
+  try {
+    const result = execSync(`launchctl list ${PLIST_NAME} 2>/dev/null`, { encoding: 'utf-8' })
+    return result.includes(PLIST_NAME)
+  } catch {
+    return false
+  }
+}
+
+function isServerRunning() {
+  try {
+    execSync(`curl -s -k https://localhost:${PORT}/api/status`, { encoding: 'utf-8' })
+    return true
+  } catch {
+    return false
+  }
+}
+
+function getPid() {
+  if (existsSync(PID_FILE)) {
+    return parseInt(readFileSync(PID_FILE, 'utf-8').trim(), 10)
+  }
+  return null
+}
+
+async function commands() {
+  const cmd = process.argv[2]
+  const args = process.argv.slice(3)
+
+  switch (cmd) {
+    case 'start':
+      await startServer(args.includes('--foreground') || args.includes('-f'))
+      break
+
+    case 'stop':
+      await stopServer()
+      break
+
+    case 'restart':
+      await stopServer()
+      await startServer(false)
+      break
+
+    case 'status':
+      await showStatus()
+      break
+
+    case 'logs':
+      await showLogs(args.includes('--follow') || args.includes('-f'))
+      break
+
+    case 'install':
+      await installDaemon()
+      break
+
+    case 'uninstall':
+      await uninstallDaemon()
+      break
+
+    default:
+      showHelp()
+  }
+}
+
+async function startServer(foreground = false) {
+  ensureDirs()
+
+  if (isServerRunning()) {
+    console.log('Server is already running.')
+    return
+  }
+
+  if (foreground) {
+    console.log(`Starting Talkie server in foreground on port ${PORT}...`)
+    const { startServer } = await import('../server/index.js')
+    await startServer(PORT)
+  } else {
+    // Check if launchd plist is installed
+    if (existsSync(PLIST_PATH)) {
+      console.log('Starting via launchd...')
+      execSync(`launchctl load ${PLIST_PATH}`)
+    } else {
+      // Start as background process
+      console.log('Starting as background process...')
+      const serverScript = join(__dirname, '..', 'server', 'index.js')
+      const child = spawn('node', [serverScript], {
+        detached: true,
+        stdio: ['ignore', 'pipe', 'pipe'],
+        env: { ...process.env, PORT: String(PORT) },
+        cwd: join(__dirname, '..'),
+      })
+
+      // Save PID
+      writeFileSync(PID_FILE, String(child.pid))
+
+      // Pipe logs
+      const stdout = createWriteStream(join(LOG_DIR, 'stdout.log'), { flags: 'a' })
+      const stderr = createWriteStream(join(LOG_DIR, 'stderr.log'), { flags: 'a' })
+      child.stdout?.pipe(stdout)
+      child.stderr?.pipe(stderr)
+
+      child.unref()
+    }
+
+    // Wait for server to be ready
+    console.log('Waiting for server to start...')
+    for (let i = 0; i < 30; i++) {
+      await new Promise(r => setTimeout(r, 500))
+      if (isServerRunning()) {
+        console.log(`Talkie server running at https://localhost:${PORT}`)
+        return
+      }
+    }
+
+    console.error('Server failed to start. Check logs with: talkie-server logs')
+  }
+}
+
+async function stopServer() {
+  if (isLaunchctlLoaded()) {
+    console.log('Stopping via launchd...')
+    try {
+      execSync(`launchctl unload ${PLIST_PATH}`)
+    } catch {
+      // Ignore errors
+    }
+  }
+
+  const pid = getPid()
+  if (pid) {
+    console.log(`Stopping process ${pid}...`)
+    try {
+      process.kill(pid, 'SIGTERM')
+      unlinkSync(PID_FILE)
+    } catch {
+      // Process may already be dead
+    }
+  }
+
+  // Wait for server to stop
+  for (let i = 0; i < 10; i++) {
+    await new Promise(r => setTimeout(r, 500))
+    if (!isServerRunning()) {
+      console.log('Server stopped.')
+      return
+    }
+  }
+
+  if (isServerRunning()) {
+    console.log('Warning: Server may still be running.')
+  } else {
+    console.log('Server stopped.')
+  }
+}
+
+async function showStatus() {
+  const running = isServerRunning()
+  const launchd = isLaunchctlLoaded()
+
+  console.log('Talkie Server Status')
+  console.log('=====================')
+  console.log(`Server:   ${running ? '✅ Running' : '❌ Stopped'}`)
+  console.log(`Port:     ${PORT}`)
+  console.log(`launchd:  ${launchd ? '✅ Loaded' : '⚪ Not loaded'}`)
+  console.log(`Plist:    ${existsSync(PLIST_PATH) ? '✅ Installed' : '⚪ Not installed'}`)
+
+  if (running) {
+    try {
+      const response = await fetch(`https://localhost:${PORT}/api/status`, {
+        // @ts-expect-error
+        agent: new (await import('https')).Agent({ rejectUnauthorized: false })
+      })
+      const data = await response.json()
+      console.log(`Database: ${data.dbStatus === 'connected' ? '✅ Connected' : '⚠️ Unavailable'}`)
+      console.log(`Avatar:   ${data.avatarState}`)
+    } catch {
+      // Ignore
+    }
+  }
+}
+
+async function showLogs(follow = false) {
+  const logFile = join(LOG_DIR, 'stdout.log')
+  const errFile = join(LOG_DIR, 'stderr.log')
+
+  if (!existsSync(logFile) && !existsSync(errFile)) {
+    console.log('No logs found.')
+    return
+  }
+
+  if (follow) {
+    console.log('Following logs (Ctrl+C to stop)...\n')
+    const tail = spawn('tail', ['-f', logFile, errFile])
+    tail.stdout.pipe(process.stdout)
+    tail.stderr.pipe(process.stderr)
+
+    process.on('SIGINT', () => {
+      tail.kill()
+      process.exit(0)
+    })
+
+    await new Promise(() => {}) // Wait forever
+  } else {
+    // Show last 50 lines
+    if (existsSync(logFile)) {
+      console.log('=== stdout.log ===')
+      const stdout = readFileSync(logFile, 'utf-8').split('\n').slice(-50).join('\n')
+      console.log(stdout)
+    }
+
+    if (existsSync(errFile)) {
+      console.log('\n=== stderr.log ===')
+      const stderr = readFileSync(errFile, 'utf-8').split('\n').slice(-50).join('\n')
+      console.log(stderr)
+    }
+  }
+}
+
+async function installDaemon() {
+  ensureDirs()
+
+  const plist = generatePlist()
+  writeFileSync(PLIST_PATH, plist)
+  console.log(`Installed plist to ${PLIST_PATH}`)
+
+  // Load the daemon
+  execSync(`launchctl load ${PLIST_PATH}`)
+  console.log('Daemon loaded and started.')
+  console.log(`Server will start automatically on login.`)
+  console.log(`\nTo uninstall: talkie-server uninstall`)
+}
+
+async function uninstallDaemon() {
+  if (isLaunchctlLoaded()) {
+    execSync(`launchctl unload ${PLIST_PATH}`)
+    console.log('Daemon unloaded.')
+  }
+
+  if (existsSync(PLIST_PATH)) {
+    unlinkSync(PLIST_PATH)
+    console.log(`Removed ${PLIST_PATH}`)
+  }
+
+  console.log('Daemon uninstalled.')
+}
+
+function showHelp() {
+  console.log(`
+Talkie Server CLI
+
+Usage: talkie-server <command> [options]
+
+Commands:
+  start [-f]      Start the server (use -f for foreground)
+  stop            Stop the server
+  restart         Restart the server
+  status          Show server status
+  logs [-f]       Show logs (use -f to follow)
+  install         Install as launchd daemon (auto-start on login)
+  uninstall       Remove launchd daemon
+
+Environment:
+  TALKIE_PORT    Server port (default: 5173)
+`)
+}
+
+// Import createWriteStream for logging
+import { createWriteStream } from 'fs'
+
+commands().catch(console.error)

package/bin/talkie.js

@@ -0,0 +1,43 @@
+#!/usr/bin/env node
+
+import { startServer } from '../server/index.js'
+import open from 'open'
+
+const PORT = parseInt(process.env.TALKIE_PORT || '5173', 10)
+const URL = `https://localhost:${PORT}`
+
+async function main() {
+  console.log('Starting Talkie...')
+
+  try {
+    await startServer(PORT)
+
+    // Open browser after server starts (prefer Chrome for Web Speech API support)
+    console.log(`Opening ${URL}...`)
+    console.log('(Requires Chrome/Edge - Firefox does not support Web Speech API)')
+
+    // Try to open in Chrome, fall back to default browser
+    try {
+      await open(URL, { app: { name: 'google chrome' } })
+    } catch {
+      await open(URL)
+    }
+
+    console.log('\nTalkie is running. Press Ctrl+C to stop.')
+  } catch (err) {
+    if (err instanceof Error && err.message.includes('already in use')) {
+      console.log(`Talkie appears to already be running on port ${PORT}`)
+      console.log(`Opening ${URL}...`)
+      try {
+        await open(URL, { app: { name: 'google chrome' } })
+      } catch {
+        await open(URL)
+      }
+    } else {
+      console.error('Failed to start Talkie:', err)
+      process.exit(1)
+    }
+  }
+}
+
+main()