agentvibes 2.4.1 → 2.4.2

package/docs/whatsapp-tts-plugin-feasibility.md

@@ -0,0 +1,418 @@
+# WhatsApp TTS Plugin - Feasibility Analysis
+
+## Executive Summary
+
+**Status**: ✅ **HIGHLY FEASIBLE** - All components exist and can be integrated with minimal code changes.
+
+The WhatsApp TTS plugin for AgentVibes is completely feasible and can be implemented as an **MCP server extension** that integrates the existing WhatsApp MCP server with AgentVibes TTS capabilities.
+
+## Project Overview
+
+Create an AgentVibes plugin that enables voice notifications for WhatsApp messages through MCP (Model Context Protocol). When activated, the plugin will:
+
+1. Monitor incoming WhatsApp messages via the WhatsApp MCP server
+2. Use voice differentiation (round-robin) for different contacts in group chats
+3. Provide MCP commands to enable/disable notifications per contact/channel
+4. Automatically read messages aloud using AgentVibes TTS
+
+## Architecture Analysis
+
+### Component 1: WhatsApp MCP Server (Existing)
+
+**Location**: `/home/fire/claude/whatsapp-mcp/`
+
+**Key Components**:
+- **Go Bridge** (`whatsapp-bridge/main.go`): Connects to WhatsApp API, stores messages in SQLite
+  - Message handler: `handleMessage()` at line 412
+  - Captures: sender, content, timestamp, chat JID, media type
+  - Outputs to console: `fmt.Printf("[%s] %s %s: %s\n", timestamp, direction, sender, content)` at line 468
+
+- **Python MCP Server** (`whatsapp-mcp-server/main.py`): Exposes tools via MCP protocol
+  - Existing tools: `search_contacts`, `list_messages`, `list_chats`, `send_message`, etc.
+  - Built on FastMCP framework
+
+**Database Structure**:
+```sql
+chats (jid, name, last_message_time)
+messages (id, chat_jid, sender, content, timestamp, is_from_me, media_type, ...)
+```
+
+### Component 2: AgentVibes TTS System (Existing)
+
+**Location**: `/home/fire/claude/AgentVibes/`
+
+**Key Components**:
+- **MCP Server** (`mcp-server/server.py`): Exposes TTS capabilities via MCP
+  - Tool: `text_to_speech(text, voice, personality, language)`
+  - Voice management: `list_voices()`, `set_voice(voice_name)`
+  - Supports: ElevenLabs (150+ voices), Piper TTS (50+ voices)
+
+- **Voice Manager** (`.claude/hooks/voice-manager.sh`): Controls voice selection
+- **Play TTS** (`.claude/hooks/play-tts.sh`): Core TTS execution script
+
+**Available Voices**: 27+ unique voices with multilingual support
+
+## Proposed Architecture
+
+### ✅ RECOMMENDED: Standalone Plugin (Independent Repository)
+
+**UPDATE**: After analysis, a **completely standalone plugin** is the superior approach.
+
+Create an independent repository that depends on AgentVibes as an external service:
+
+```
+┌─────────────────────────────────────┐
+│   whatsapp-agentvibes-plugin        │ ← New Standalone Repo
+│   (Independent MCP Server)          │
+└──────────────┬──────────────────────┘
+               │ depends on (subprocess calls)
+               │
+      ┌────────┴─────────┬────────────────┐
+      ▼                  ▼                ▼
+┌────────────┐   ┌──────────────┐   ┌─────────────┐
+│ WhatsApp   │   │ AgentVibes   │   │ FastMCP     │
+│ MCP Server │   │ (installed)  │   │ (Python)    │
+└────────────┘   └──────────────┘   └─────────────┘
+```
+
+**Key Architecture Points**:
+- **Standalone repo**: `whatsapp-agentvibes` (separate from AgentVibes codebase)
+- **AgentVibes integration**: Calls `play-tts.sh` via subprocess (no code imports)
+- **WhatsApp data**: Reads SQLite database directly (read-only)
+- **Distribution**: pip package on PyPI
+- **Configuration**: Separate `~/.config/whatsapp-agentvibes/config.json`
+
+### Why Standalone?
+
+✅ **Separation of Concerns**: Plugin development doesn't affect AgentVibes core
+✅ **Independent Versioning**: Plugin can release updates independently
+✅ **Easier Maintenance**: Different maintainers can own different repos
+✅ **Cleaner Dependencies**: Users only install what they need
+✅ **Community Contributions**: Easier for external contributors
+✅ **Plugin Ecosystem**: Opens path for Signal-TTS, Telegram-TTS, Slack-TTS, etc.
+
+### ❌ Rejected: Integrated Approach
+
+Building inside AgentVibes repo would:
+- Couple messaging features to TTS engine
+- Make AgentVibes responsible for messaging integrations
+- Complicate versioning and releases
+- Limit community contributions
+- Mix concerns (TTS vs messaging)
+
+## Feature Design
+
+### Round-Robin Voice Assignment
+
+**Strategy**: Hash-based voice assignment with persistent storage
+
+```python
+def assign_voice_to_contact(contact_jid: str, available_voices: list) -> str:
+    """Assigns a consistent voice to a contact using hash-based selection"""
+    # Generate stable hash from contact JID
+    contact_hash = hash(contact_jid) % len(available_voices)
+    return available_voices[contact_hash]
+```
+
+**Benefits**:
+- Same contact always gets same voice (consistency)
+- Automatic distribution across available voices
+- No manual voice assignment needed
+
+**Persistence**:
+```json
+{
+  "contact_voices": {
+    "1234567890@s.whatsapp.net": "Aria",
+    "0987654321@s.whatsapp.net": "Northern Terry",
+    "group123@g.us": {
+      "sender1@s.whatsapp.net": "Cowboy Bob",
+      "sender2@s.whatsapp.net": "Kristin"
+    }
+  }
+}
+```
+
+### MCP Commands (New Tools)
+
+1. **`whatsapp_enable_notifications(contact_or_chat: str)`**
+   - Enable TTS notifications for a specific contact or group
+   - Assigns voice automatically via round-robin
+   - Returns: "✅ Enabled notifications for {name} with voice {voice}"
+
+2. **`whatsapp_disable_notifications(contact_or_chat: str)`**
+   - Disable TTS notifications for a specific contact or group
+   - Returns: "❌ Disabled notifications for {name}"
+
+3. **`whatsapp_list_notification_channels()`**
+   - Shows all contacts/groups with notifications enabled
+   - Displays assigned voices
+   - Returns formatted list
+
+4. **`whatsapp_set_contact_voice(contact: str, voice: str)`**
+   - Override automatic voice assignment
+   - Manually set specific voice for a contact
+   - Returns: "✅ Set {contact} voice to {voice}"
+
+5. **`whatsapp_get_notification_status()`**
+   - Show global notification status
+   - List enabled contacts with their voices
+   - Show round-robin statistics
+
+### Message Monitoring Implementation
+
+**Approach**: Polling SQLite database with timestamp tracking
+
+```python
+class WhatsAppTTSMonitor:
+    def __init__(self):
+        self.last_checked = datetime.now()
+        self.enabled_contacts = self.load_config()
+
+    async def check_new_messages(self):
+        """Poll database for messages since last check"""
+        query = """
+            SELECT id, chat_jid, sender, content, timestamp, is_from_me
+            FROM messages
+            WHERE timestamp > ? AND is_from_me = 0
+            ORDER BY timestamp ASC
+        """
+        messages = await db.execute(query, (self.last_checked,))
+
+        for msg in messages:
+            await self.process_message(msg)
+
+        self.last_checked = datetime.now()
+
+    async def process_message(self, msg):
+        """Determine if message should trigger TTS"""
+        # Check if notifications enabled for this contact/chat
+        if msg.chat_jid in self.enabled_contacts:
+            voice = self.get_voice_for_sender(msg.chat_jid, msg.sender)
+            await self.speak_message(msg.content, voice, msg.sender)
+```
+
+**Alternative**: Event-based (requires modifying Go bridge)
+- More efficient but requires code changes to `handleMessage()` in Go
+- Can add webhook/callback after line 470 in main.go
+- Polling is simpler and doesn't modify existing systems
+
+## Implementation Plan
+
+### Phase 1: Core Integration (Week 1)
+
+1. Create new MCP server: `agentvibes-whatsapp-mcp/`
+   - Merge WhatsApp MCP tools
+   - Add AgentVibes TTS integration
+   - Implement configuration storage (JSON file)
+
+2. Implement voice assignment logic
+   - Hash-based round-robin
+   - Persistent voice-to-contact mapping
+   - Configuration save/load
+
+3. Add basic MCP tools
+   - `whatsapp_enable_notifications`
+   - `whatsapp_disable_notifications`
+   - `whatsapp_list_notification_channels`
+
+### Phase 2: Message Monitoring (Week 2)
+
+1. Implement SQLite polling
+   - Connect to WhatsApp messages.db
+   - Track last checked timestamp
+   - Query new messages periodically
+
+2. Message filtering
+   - Filter by enabled contacts
+   - Skip messages from self (is_from_me = true)
+   - Handle group vs direct messages
+
+3. TTS integration
+   - Call AgentVibes play-tts.sh script
+   - Pass contact name + message content
+   - Handle voice selection
+
+### Phase 3: Advanced Features (Week 3)
+
+1. Voice management tools
+   - `whatsapp_set_contact_voice` (manual override)
+   - `whatsapp_get_notification_status` (statistics)
+
+2. Group chat handling
+   - Per-sender voice assignment within groups
+   - Optional: Prefix speaker name ("John says: Hello")
+
+3. Configuration UI enhancements
+   - Natural language commands
+   - Bulk enable/disable operations
+
+### Phase 4: Testing & Documentation (Week 4)
+
+1. End-to-end testing
+2. Documentation
+3. Example usage scenarios
+4. Error handling and edge cases
+
+## Technical Considerations
+
+### Pros
+✅ **Zero modification to existing systems** - WhatsApp bridge and AgentVibes remain unchanged
+✅ **Leverages MCP protocol** - Native Claude Desktop/Code integration
+✅ **Reuses existing infrastructure** - Both message storage and TTS already work
+✅ **Natural language control** - "Enable notifications for Mom's chat"
+✅ **Voice variety** - 27+ voices available for round-robin
+
+### Challenges
+⚠️ **Database polling latency** - 1-5 second delay before TTS (acceptable for notifications)
+⚠️ **Voice assignment persistence** - Need to store contact-voice mappings
+⚠️ **Group chat complexity** - Multiple senders require per-sender voice tracking
+⚠️ **Media messages** - How to announce images/videos/voice notes?
+
+### Solutions
+- **Polling**: Use 2-second intervals for responsive notifications
+- **Persistence**: JSON config file in `.claude/whatsapp-notifications.json`
+- **Groups**: Extend mapping to `{group_jid: {sender_jid: voice}}`
+- **Media**: Speak "[Name] sent a photo" or "[Name] sent a voice message"
+
+## Resource Requirements
+
+### Development Time
+- **Phase 1**: 8-12 hours (core integration)
+- **Phase 2**: 6-8 hours (monitoring)
+- **Phase 3**: 4-6 hours (advanced features)
+- **Phase 4**: 4-6 hours (testing/docs)
+- **Total**: ~25-35 hours
+
+### Runtime Resources
+- **Memory**: +50-100 MB (Python process with polling)
+- **CPU**: Negligible (2-second polls, TTS on demand)
+- **Storage**: ~10 KB (configuration file)
+
+### Dependencies
+- Existing: WhatsApp MCP server, AgentVibes
+- New: None (uses existing FastMCP framework)
+
+## Example Usage Scenarios
+
+### Scenario 1: Enable notifications for a contact
+```
+User: "Enable WhatsApp notifications for my mom"
+Claude: [Calls whatsapp_search_contacts("mom")]
+        [Finds: Jane Smith, +1234567890]
+        [Calls whatsapp_enable_notifications("+1234567890@s.whatsapp.net")]
+        ✅ Enabled notifications for Jane Smith with voice "Aria"
+
+[Later, message arrives from Jane]
+TTS: "Message from Jane Smith: Don't forget to call me tonight"
+```
+
+### Scenario 2: Group chat with multiple voices
+```
+User: "Enable notifications for the Family group"
+Claude: [Calls whatsapp_list_chats("Family")]
+        [Calls whatsapp_enable_notifications("group123@g.us")]
+        ✅ Enabled notifications for Family group
+        Voices will be assigned per sender automatically
+
+[Later, messages arrive]
+TTS (Aria voice): "Dad: Can someone pick up milk?"
+TTS (Terry voice): "Mom: I'll get it on the way home"
+TTS (Bob voice): "Sister: Thanks Mom!"
+```
+
+### Scenario 3: Managing notifications
+```
+User: "Show me all WhatsApp notification channels"
+Claude: [Calls whatsapp_list_notification_channels()]
+
+🔔 WhatsApp Notifications Active:
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+  1. Jane Smith (+1234567890)
+     Voice: Aria
+     Last message: 2 minutes ago
+
+  2. Family Group
+     Voice assignments:
+       • Dad → Northern Terry
+       • Mom → Kristin
+       • Sister → Cowboy Bob
+     Last message: 5 minutes ago
+
+  3. Work Team
+     Voice: Jenny
+     Last message: 1 hour ago
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+```
+
+## Security & Privacy
+
+### Data Access
+- **Read-only access** to WhatsApp SQLite database
+- **No message forwarding** - all processing local
+- **User-controlled** - notifications only for explicitly enabled contacts
+
+### Configuration Storage
+- Stored locally in `.claude/whatsapp-notifications.json`
+- Contains: JIDs (identifiers), voice assignments
+- No message content stored by plugin
+
+### Permissions
+- User must explicitly enable per-contact
+- Can disable anytime via MCP command
+- Respects WhatsApp MCP server's existing security
+
+## Alternatives Considered
+
+### Alternative 1: Direct Go Bridge Modification
+**Pros**: Real-time, no polling lag
+**Cons**: Modifies existing codebase, harder to maintain
+**Verdict**: ❌ Rejected - maintain separation of concerns
+
+### Alternative 2: Webhook-based System
+**Pros**: Event-driven, more efficient
+**Cons**: Requires network layer, authentication complexity
+**Verdict**: ❌ Rejected - overkill for local system
+
+### Alternative 3: Browser Extension
+**Pros**: Works with WhatsApp Web directly
+**Cons**: No access to MCP, can't use AgentVibes voices
+**Verdict**: ❌ Rejected - loses MCP integration benefits
+
+## Conclusion
+
+This plugin is **highly feasible** and should be implemented as a **standalone Python package** in an independent repository.
+
+**Key Success Factors**:
+1. ✅ No modifications to AgentVibes or WhatsApp MCP required
+2. ✅ Clean separation of concerns (TTS engine vs messaging integration)
+3. ✅ Independent versioning and release cycles
+4. ✅ Opens path for plugin ecosystem (Signal, Telegram, Slack, etc.)
+5. ✅ Subprocess-based AgentVibes integration (simple, reliable)
+6. ✅ Simple SQLite polling for message detection
+7. ✅ Hash-based voice assignment for consistency
+8. ✅ Natural language command interface via MCP
+
+**Repository Structure**:
+- **Name**: `whatsapp-agentvibes` or `agentvibes-whatsapp-plugin`
+- **Location**: New independent GitHub repository
+- **Distribution**: PyPI package (`pip install whatsapp-agentvibes`)
+- **Dependencies**: AgentVibes (subprocess), WhatsApp MCP (SQLite), FastMCP (Python)
+
+**Recommendation**: Proceed with **Standalone Plugin Architecture** using the phased implementation plan above.
+
+**See also**: `/docs/whatsapp-tts-standalone-plugin.md` for detailed standalone architecture design.
+
+## Next Steps
+
+1. Create GitHub issue with detailed implementation tasks
+2. Set up project structure: `agentvibes-whatsapp-plugin/`
+3. Implement Phase 1 (core integration)
+4. Create demo video showing the feature in action
+
+---
+
+**Document Version**: 1.0
+**Date**: 2025-11-12
+**Author**: AgentVibes + Claude AI
+**Status**: Ready for Implementation