agentvibes 2.0.17-beta.2 → 2.0.17-beta.21

package/scripts/CHANGELOG-2025-10-16.md

@@ -0,0 +1,268 @@
+# Audio Tunnel Fix - October 16, 2025
+
+## Summary
+
+Fixed critical audio tunnel issues where AgentVibes TTS stopped working between ubuntu-rdp and Windows WSL. The root cause was stale SSH processes holding port 14713, combined with a stopped socat bridge.
+
+## What Happened
+
+### User Report
+> "Audio on this computer is not even working for Agent Vibes"
+> "speaker-test -t sine -f 1000 -l 1" fails with "Connection refused"
+> "This was working just yesterday"
+
+### Investigation
+
+**Initial Symptoms:**
+```bash
+$ speaker-test -t sine -f 1000 -l 1
+ALSA lib pulse.c:242:(pulse_connect) PulseAudio: Unable to connect: Connection terminated
+Playback open error: -111,Connection refused
+```
+
+**SSH Tunnel Warnings:**
+```
+Warning: remote port forwarding failed for listen port 14713
+```
+
+### Root Cause Analysis
+
+**Primary Issue:** Multiple stale SSH processes on ubuntu-rdp were holding port 14713
+```bash
+$ sudo lsof -i :14713
+COMMAND     PID          USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
+sshd    2845015 administrator    7u  IPv6 70604344      0t0  TCP ip6-localhost:14713 (LISTEN)
+sshd    2845015 administrator    9u  IPv4 70604345      0t0  TCP localhost:14713 (LISTEN)
+```
+
+**Secondary Issue:** socat bridge on WSL had stopped running
+```bash
+$ wsl ss -tlnp | grep 14713
+# No output - socat not running!
+```
+
+### Why This Happened
+
+1. **Zombie SSH Sessions**: Previous SSH connections didn't terminate cleanly, leaving processes bound to port 14713
+2. **Background Job Termination**: The socat bridge PowerShell background job had ended
+3. **No Automatic Recovery**: No healthcheck or recovery mechanism existed
+
+## The Fix
+
+### Steps Performed
+
+1. **Identified stale processes:**
+   ```bash
+   ssh ubuntu-rdp 'sudo lsof -i :14713'
+   ```
+
+2. **Forcefully killed them:**
+   ```bash
+   ssh ubuntu-rdp 'sudo fuser -k 14713/tcp'
+   ```
+
+3. **Restarted socat bridge:**
+   ```powershell
+   wsl pkill socat
+   Start-Job -ScriptBlock {
+     wsl socat 'TCP-LISTEN:14713,fork,reuseaddr' 'UNIX-CONNECT:/mnt/wslg/PulseServer'
+   } -Name "SocatAudioBridge"
+   ```
+
+4. **Created fresh SSH tunnel:**
+   ```bash
+   wsl bash -c "ssh -f -N -R 14713:localhost:14713 ubuntu-rdp"
+   ```
+
+5. **Verified connection:**
+   ```bash
+   ssh ubuntu-rdp 'netstat -tlnp | grep 14713'
+   # Output showed tunnel was listening
+   ```
+
+### Result
+✅ Audio tunnel restored
+✅ AgentVibes TTS working through Windows speakers
+✅ speaker-test playing sound successfully
+
+## Improvements Made
+
+### 1. Updated fix-audio-tunnel.sh Script
+
+**New Features:**
+- ✅ Auto-detects environment (WSL vs ubuntu-rdp)
+- ✅ Kills stale SSH processes on remote server automatically
+- ✅ Restarts socat bridge if stopped
+- ✅ Kills local stale SSH tunnels
+- ✅ Creates fresh tunnel with verification
+- ✅ Tests audio connection
+- ✅ Provides clear status messages and instructions
+
+**Script Locations:**
+- `scripts/fix-audio-tunnel.sh` (main script)
+- `scripts/fix-audio-tunnel-complete.sh` (backup)
+
+### 2. Created TROUBLESHOOTING.md
+
+Comprehensive troubleshooting guide documenting:
+- Root cause analysis of today's issue
+- Audio tunnel architecture diagram
+- Common symptoms and solutions
+- Diagnostic command reference
+- Prevention best practices
+- Success indicators
+
+### 3. Updated scripts/README.md
+
+Added documentation for:
+- `fix-audio-tunnel.sh` usage and features
+- Quick fix guide
+- Common issues and their fixes
+- Manual diagnostic commands
+- Reference to TROUBLESHOOTING.md
+
+## Technical Details
+
+### Audio Tunnel Architecture
+
+```
+ubuntu-rdp (Remote)           WSL (Windows)                Windows
+    │                              │                          │
+    │ Speaker-test/TTS             │                          │
+    │         ↓                    │                          │
+    │  PULSE_SERVER=              │                          │
+    │  tcp:localhost:14713         │                          │
+    │         ↓                    │                          │
+    │  [SSH Tunnel Port 14713] ←──┴──→ [socat Bridge]        │
+    │                              │      ↓                   │
+    │                              │  TCP:14713 →            │
+    │                              │  UNIX:/mnt/wslg/        │
+    │                              │  PulseServer            │
+    │                              │      ↓                   │
+    │                              │  [WSL PulseAudio] ───→ Speakers
+```
+
+### Key Commands
+
+**Diagnostic:**
+```bash
+# Check socat bridge
+wsl ss -tlnp | grep 14713
+
+# Check SSH tunnel
+ssh ubuntu-rdp 'netstat -tlnp | grep 14713'
+
+# Find stale processes
+ssh ubuntu-rdp 'sudo lsof -i :14713'
+
+# Test PulseAudio
+ssh ubuntu-rdp 'export PULSE_SERVER=tcp:localhost:14713 && pactl info'
+```
+
+**Fix:**
+```bash
+# Automated fix (recommended)
+./scripts/fix-audio-tunnel.sh
+
+# Manual fix
+ssh ubuntu-rdp 'sudo fuser -k 14713/tcp'
+wsl pkill socat
+wsl bash -c "ssh -f -N -R 14713:localhost:14713 ubuntu-rdp"
+```
+
+## Prevention
+
+### Recommendations Implemented
+
+1. **Automated recovery script**: `fix-audio-tunnel.sh` handles all common failure modes
+2. **Environment detection**: Script adapts to whether it's run from WSL or ubuntu-rdp
+3. **Comprehensive diagnostics**: Script reports status at each step
+4. **Documentation**: TROUBLESHOOTING.md provides detailed incident analysis
+
+### Future Improvements
+
+1. **Healthcheck cron job**: Periodically verify tunnel status
+2. **Auto-restart on failure**: systemd service or scheduled task
+3. **Monitoring alert**: Notify when tunnel goes down
+4. **Persistent socat**: Run as systemd service instead of background job
+
+## Lessons Learned
+
+1. **Always check for stale processes first** - `sudo lsof -i :14713` should be the first diagnostic
+2. **Force kill is sometimes necessary** - Regular termination may not release port bindings
+3. **socat bridge is critical** - Even with a working tunnel, audio fails if socat isn't running
+4. **Background jobs don't persist** - Need better solution for long-running processes
+5. **Document immediately** - Capture troubleshooting steps while fresh
+
+## Testing Performed
+
+### Before Fix
+```bash
+$ speaker-test -t sine -f 1000 -l 1
+Playback open error: -111,Connection refused
+```
+
+### After Fix
+```bash
+$ speaker-test -t sine -f 1000 -l 1
+# Audio played through Windows speakers successfully
+```
+
+### Verification
+```bash
+$ ssh ubuntu-rdp 'export PULSE_SERVER=tcp:localhost:14713 && pactl info | head -3'
+Server String: tcp:localhost:14713
+Library Protocol Version: 35
+Server Protocol Version: 35
+```
+
+## Files Modified
+
+- `scripts/fix-audio-tunnel.sh` - Completely rewritten with comprehensive fixes
+- `scripts/fix-audio-tunnel-complete.sh` - New backup copy
+- `scripts/TROUBLESHOOTING.md` - New detailed troubleshooting guide
+- `scripts/README.md` - Updated with new script documentation
+- `scripts/CHANGELOG-2025-10-16.md` - This file
+
+## Impact
+
+✅ **Audio tunnel restored** to working state
+✅ **Automated fix** now available for future issues
+✅ **Documentation** created for troubleshooting
+✅ **Prevention guidance** added to avoid recurrence
+
+## Timeline
+
+- **10:00 AM** - User reported audio not working
+- **10:05 AM** - Initial investigation, found connection refused errors
+- **10:15 AM** - Discovered stale SSH processes via `lsof`
+- **10:20 AM** - Killed stale processes, restarted socat bridge
+- **10:25 AM** - Created fresh tunnel, verified connection
+- **10:30 AM** - Audio working, user confirmed success
+- **10:40 AM** - Rewrote fix-audio-tunnel.sh script
+- **10:50 AM** - Created TROUBLESHOOTING.md documentation
+- **11:00 AM** - Updated README.md with new information
+
+## User Feedback
+
+> "its working now, what did u fix?"
+
+User confirmed audio tunnel restored successfully.
+
+## Related Issues
+
+- Previous tunnel outages (not documented)
+- X2Go PulseAudio conflicts (documented in guide)
+- VS Code Remote SSH tunnel issues (documented in guide)
+
+## References
+
+- [TROUBLESHOOTING.md](./TROUBLESHOOTING.md) - Detailed troubleshooting guide
+- [README.md](./README.md) - Scripts documentation
+- [Remote Audio Setup Guide](../docs/remote-audio-setup.md) - Original setup guide
+
+---
+
+**Incident resolved:** October 16, 2025
+**Resolution time:** ~30 minutes
+**Automated fix:** Available via `./scripts/fix-audio-tunnel.sh`

package/scripts/README.md

@@ -82,10 +82,124 @@ Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
 For detailed information about the remote audio setup, see:
 - [Remote Audio Setup Guide](../docs/remote-audio-setup.md)
 
+#### `fix-audio-tunnel.sh` (NEW - WSL or Remote Server)
+
+**The easiest way to fix audio tunnel issues!**
+
+Auto-detects environment and runs appropriate fixes for stale SSH processes, stopped socat bridges, and connection issues.
+
+**Requirements:**
+- Run from WSL (for complete fix) OR ubuntu-rdp (for local fix)
+- SSH access configured
+- sudo privileges for killing processes
+
+**Usage:**
+```bash
+# From WSL - runs complete fix
+./scripts/fix-audio-tunnel.sh
+
+# From ubuntu-rdp - runs local fix
+./scripts/fix-audio-tunnel.sh
+```
+
+**What it does (when run from WSL):**
+1. ✅ Kills stale SSH processes on ubuntu-rdp holding port 14713
+2. ✅ Restarts socat bridge on WSL if needed
+3. ✅ Kills local stale SSH tunnels
+4. ✅ Creates fresh SSH tunnel to ubuntu-rdp
+5. ✅ Tests audio connection
+6. ✅ Provides clear status and usage instructions
+
+**What it does (when run from ubuntu-rdp):**
+1. ✅ Kills local processes using port 14713
+2. ✅ Waits for tunnel to be re-established
+3. ✅ Tests audio connection
+4. ✅ Plays test tone through Windows speakers
+
+**When to use:**
+- Audio stopped working suddenly
+- Getting "Connection refused" errors
+- SSH shows "Warning: remote port forwarding failed"
+- speaker-test fails
+- AgentVibes TTS not playing through Windows speakers
+
+#### `check-audio-tunnel.sh` (Diagnostic)
+
+Checks audio tunnel status without making changes. Useful for verifying setup or diagnosing issues.
+
+**Usage:**
+```bash
+./scripts/check-audio-tunnel.sh
+```
+
 ## Troubleshooting
 
+### Quick Fix
+
+**If audio stopped working**, run this first:
+```bash
+# From WSL
+./scripts/fix-audio-tunnel.sh
+```
+
+This will automatically fix the most common issues:
+- Stale SSH processes blocking port 14713
+- Stopped socat bridge
+- Conflicting SSH tunnels
+
 ### Common Issues
 
+**Issue 1: "Warning: remote port forwarding failed for listen port 14713"**
+
+**Symptom:** SSH tunnel fails to establish
+**Cause:** Old SSH sessions left zombie processes holding port 14713
+**Fix:** `./scripts/fix-audio-tunnel.sh` (kills stale processes automatically)
+
+**Issue 2: "Connection refused" / "ALSA lib pulse.c: Unable to connect"**
+
+**Symptom:** Audio playback fails
+**Cause:** socat bridge on WSL stopped running
+**Fix:** `./scripts/fix-audio-tunnel.sh` (restarts socat automatically)
+
+**Issue 3: Audio works intermittently**
+
+**Symptom:** Audio plays sometimes but not always
+**Cause:** Multiple SSH sessions creating conflicting tunnels
+**Fix:** `./scripts/fix-audio-tunnel.sh` (kills all tunnels and creates one clean tunnel)
+
+### Manual Diagnostics
+
+If the automated fix doesn't work, try these manual checks:
+
+**Check socat bridge (WSL):**
+```bash
+wsl ss -tlnp | grep 14713
+# Expected: LISTEN 0 5 *:14713 *:* users:(("socat"...))
+```
+
+**Check SSH tunnel (ubuntu-rdp):**
+```bash
+ssh ubuntu-rdp 'netstat -tlnp | grep 14713'
+# Expected: tcp 0 0 127.0.0.1:14713 0.0.0.0:* LISTEN
+```
+
+**Find stale processes:**
+```bash
+ssh ubuntu-rdp 'sudo lsof -i :14713'
+# Should show only one sshd process
+```
+
+**Test audio:**
+```bash
+ssh ubuntu-rdp
+export PULSE_SERVER=tcp:localhost:14713
+speaker-test -t sine -f 1000 -l 1
+```
+
+**Detailed troubleshooting guide:** See [TROUBLESHOOTING.md](./TROUBLESHOOTING.md) for in-depth analysis of recent issues and their solutions.
+
+### Legacy Issues
+
 **Audio doesn't play:**
 - Ensure WSL is updated: `wsl --update`
 - Restart WSL: `wsl --shutdown`

package/scripts/TROUBLESHOOTING.md

@@ -0,0 +1,258 @@
+# Audio Tunnel Troubleshooting - 2025-10-16
+
+## Problem Summary
+
+The audio tunnel that was working yesterday stopped functioning today. Users experienced:
+- `speaker-test` failing with "Connection refused"
+- SSH tunnel showing "Warning: remote port forwarding failed for listen port 14713"
+
+## Root Cause Analysis
+
+### Primary Issue: Stale SSH Processes
+**Multiple old SSH sessions** on ubuntu-rdp (remote server) were holding port 14713, preventing new tunnels from binding to that port.
+
+```bash
+# Investigation revealed:
+$ sudo lsof -i :14713
+COMMAND     PID          USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
+sshd    2845015 administrator    7u  IPv6 70604344      0t0  TCP ip6-localhost:14713 (LISTEN)
+sshd    2845015 administrator    9u  IPv4 70604345      0t0  TCP localhost:14713 (LISTEN)
+```
+
+### Secondary Issue: Stopped socat Bridge
+The socat bridge on WSL (Windows side) had stopped running, breaking the connection chain even if the tunnel worked.
+
+## How the Audio Tunnel Works
+
+```
+ubuntu-rdp (Remote)           WSL (Windows)                Windows
+    │                              │                          │
+    │ Speaker-test/TTS             │                          │
+    │         ↓                    │                          │
+    │  PULSE_SERVER=              │                          │
+    │  tcp:localhost:14713         │                          │
+    │         ↓                    │                          │
+    │  [SSH Tunnel Port 14713] ←──┴──→ [socat Bridge]        │
+    │                              │      ↓                   │
+    │                              │  TCP:14713 →            │
+    │                              │  UNIX:/mnt/wslg/        │
+    │                              │  PulseServer            │
+    │                              │      ↓                   │
+    │                              │  [WSL PulseAudio] ───→ Speakers
+```
+
+## Symptoms
+
+1. **SSH tunnel fails to establish:**
+   ```
+   Warning: remote port forwarding failed for listen port 14713
+   ```
+
+2. **Audio playback error:**
+   ```
+   ALSA lib pulse.c:242:(pulse_connect) PulseAudio: Unable to connect: Connection terminated
+   Playback open error: -111,Connection refused
+   ```
+
+3. **pactl info shows local connection instead of TCP:**
+   ```
+   Server String: unix:/mnt/wslg/PulseServer
+   # Instead of:
+   Server String: tcp:localhost:14713
+   ```
+
+## The Fix
+
+### Step 1: Kill Stale Processes on Remote Server
+```bash
+ssh ubuntu-rdp 'sudo fuser -k 14713/tcp'
+```
+
+This forcefully kills all processes (including zombie SSH sessions) holding port 14713 on the remote server.
+
+### Step 2: Restart socat Bridge on WSL
+```powershell
+# Kill any existing socat
+wsl pkill socat
+
+# Start fresh socat bridge
+Start-Job -ScriptBlock {
+  wsl socat 'TCP-LISTEN:14713,fork,reuseaddr' 'UNIX-CONNECT:/mnt/wslg/PulseServer'
+} -Name "SocatAudioBridge"
+
+# Verify it's running
+wsl ss -tlnp | Select-String ":14713"
+```
+
+### Step 3: Kill Local Stale SSH Tunnels
+```bash
+wsl bash -c "pkill -f 'ssh.*ubuntu-rdp'"
+```
+
+### Step 4: Create Fresh SSH Tunnel
+```bash
+wsl bash -c "ssh -f -N -R 14713:localhost:14713 ubuntu-rdp"
+```
+
+### Step 5: Verify Everything Works
+```bash
+# Check tunnel exists on remote
+wsl bash -c "ssh ubuntu-rdp 'netstat -tlnp | grep 14713'"
+
+# Test audio
+ssh ubuntu-rdp
+export PULSE_SERVER=tcp:localhost:14713
+speaker-test -t sine -f 1000 -l 1
+```
+
+## Prevention
+
+### Why This Happens
+- SSH sessions don't always clean up properly when disconnected
+- Network interruptions can leave zombie processes
+- Multiple VS Code Remote sessions can create conflicting tunnels
+- X2Go client (if enabled) can conflict with SSH audio tunnel
+
+### Best Practices
+
+1. **Always use `-f` flag for background tunnels:**
+   ```bash
+   ssh -f -N -R 14713:localhost:14713 ubuntu-rdp
+   ```
+
+2. **Check for existing tunnels before creating new ones:**
+   ```bash
+   ssh ubuntu-rdp 'sudo lsof -i :14713'
+   ```
+
+3. **Clean exit from SSH sessions:**
+   - Use `exit` command instead of closing terminal
+   - Avoid killing terminal windows with active SSH sessions
+
+4. **Disable X2Go PulseAudio:**
+   - X2Go's audio support conflicts with SSH tunnel
+   - Disable in X2Go session settings under "Media/Sound"
+
+5. **Use the automated fix script:**
+   ```bash
+   ./scripts/fix-audio-tunnel.sh
+   ```
+
+## Automated Solution
+
+The updated `fix-audio-tunnel.sh` script now handles:
+- ✅ Kills stale SSH processes on remote server
+- ✅ Restarts socat bridge on WSL
+- ✅ Kills local stale SSH tunnels
+- ✅ Creates fresh tunnel
+- ✅ Verifies connection works
+- ✅ Tests audio playback
+
+## Future Improvements
+
+### 1. Healthcheck Script
+Create a cron job or systemd timer that periodically checks tunnel health:
+```bash
+#!/bin/bash
+# /usr/local/bin/check-audio-tunnel.sh
+if ! ss -tlnp | grep -q :14713; then
+  # Alert or auto-fix
+  logger "Audio tunnel down, attempting fix"
+  /path/to/fix-audio-tunnel.sh
+fi
+```
+
+### 2. SSH Config Improvement
+Add these to `~/.ssh/config` for more reliable connections:
+```
+Host ubuntu-rdp
+    ServerAliveInterval 30
+    ServerAliveCountMax 3
+    ExitOnForwardFailure yes
+    # This makes SSH exit if tunnel can't be created
+```
+
+### 3. Monitoring
+Add tunnel status to shell prompt or status bar:
+```bash
+# In .bashrc or .zshrc
+audio_tunnel_status() {
+  if ss -tlnp 2>/dev/null | grep -q :14713; then
+    echo "🔊"
+  else
+    echo "🔇"
+  fi
+}
+```
+
+## Related Issues
+
+- **VS Code Remote SSH**: May need `"remote.SSH.useExecServer": false` in settings
+- **Multiple WSL instances**: Each instance needs its own socat bridge
+- **Firewall rules**: Ensure UFW allows localhost connections to 14713
+- **PulseAudio config**: Must have `autospawn = no` and `auth-anonymous = true`
+
+## Key Learnings
+
+1. **Always check for stale processes first** - `sudo lsof -i :14713` should be the first diagnostic command
+2. **socat bridge is critical** - Without it, even a working tunnel won't function
+3. **Force kill is necessary** - Regular process termination may not clear port bindings
+4. **Background tunnels need monitoring** - The `-f` flag creates fire-and-forget tunnels that need healthchecks
+5. **Environment variable persistence** - `PULSE_SERVER` must be set in every new shell session
+
+## Success Indicators
+
+When working correctly, you should see:
+
+1. **On WSL:**
+   ```bash
+   wsl ss -tlnp | grep 14713
+   # Output: socat listening on *:14713
+   ```
+
+2. **On ubuntu-rdp:**
+   ```bash
+   netstat -tlnp | grep 14713
+   # Output: listening on 127.0.0.1:14713 and ::1:14713
+   ```
+
+3. **PulseAudio connection:**
+   ```bash
+   export PULSE_SERVER=tcp:localhost:14713
+   pactl info | head -3
+   # Output: Server String: tcp:localhost:14713
+   ```
+
+4. **Audio playback:**
+   ```bash
+   speaker-test -t sine -f 1000 -l 1
+   # Output: Sound plays through Windows speakers
+   ```
+
+## Diagnostic Commands Cheat Sheet
+
+```bash
+# Check socat bridge (on WSL)
+wsl ss -tlnp | grep 14713
+
+# Check tunnel on remote
+ssh ubuntu-rdp 'netstat -tlnp | grep 14713'
+
+# Find processes using port
+ssh ubuntu-rdp 'sudo lsof -i :14713'
+
+# Check PulseAudio connection
+ssh ubuntu-rdp 'export PULSE_SERVER=tcp:localhost:14713 && pactl info'
+
+# Test audio
+ssh ubuntu-rdp 'export PULSE_SERVER=tcp:localhost:14713 && speaker-test -t sine -f 1000 -l 1'
+
+# Kill stale processes
+ssh ubuntu-rdp 'sudo fuser -k 14713/tcp'
+
+# Kill local SSH tunnels
+wsl pkill -f 'ssh.*ubuntu-rdp'
+
+# Create new tunnel
+wsl bash -c "ssh -f -N -R 14713:localhost:14713 ubuntu-rdp"
+```

package/scripts/audio-tunnel.config

@@ -0,0 +1,17 @@
+# AgentVibes Audio Tunnel Configuration
+# Personal configuration - DO NOT COMMIT
+
+# Remote server hostname (from your ~/.ssh/config)
+REMOTE_HOST="ubuntu-rdp"
+
+# Tunnel port (default: 14713)
+TUNNEL_PORT="14713"
+
+# PulseAudio socket path (WSL default)
+PULSE_SOCKET="/mnt/wslg/PulseServer"
+
+# Log file for automatic fixes
+AUTO_FIX_LOG="/tmp/audio-tunnel-autofix.log"
+
+# Healthcheck interval in minutes (for cron setup)
+CHECK_INTERVAL="5"

package/scripts/audio-tunnel.config.example

@@ -0,0 +1,18 @@
+# AgentVibes Audio Tunnel Configuration
+# Copy this file to audio-tunnel.config and customize for your setup
+# DO NOT commit audio-tunnel.config to git (it's in .gitignore)
+
+# Remote server hostname (from your ~/.ssh/config)
+REMOTE_HOST="your-remote-host"
+
+# Tunnel port (default: 14713)
+TUNNEL_PORT="14713"
+
+# PulseAudio socket path (WSL default)
+PULSE_SOCKET="/mnt/wslg/PulseServer"
+
+# Log file for automatic fixes
+AUTO_FIX_LOG="/tmp/audio-tunnel-autofix.log"
+
+# Healthcheck interval in minutes (for cron setup)
+CHECK_INTERVAL="5"