agentgui 1.0.690 → 1.0.692

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agentgui",
-  "version": "1.0.690",
+  "version": "1.0.692",
   "description": "Multi-agent ACP client with real-time communication",
   "type": "module",
   "main": "server.js",

package/CACHE_DESYNC_PREVENTION.md

@@ -1,219 +0,0 @@
-# Cache Desync Prevention Implementation
-
-## Overview
-
-The conversation thread cache in AgentGUI (stored in `ConversationManager.conversations`) is now protected against desynchronization through atomic mutation points and version tracking.
-
-## Problem Solved
-
-**Before:** Multiple code paths could mutate `this.conversations` independently:
-- `loadConversations()` polling every 30s
-- WebSocket handlers creating/updating/deleting conversations in real-time
-- Manual delete operations
-
-Result: Array could enter intermediate states during concurrent operations, causing:
-- Stale UI displays
-- Lost updates when poll overwrites WebSocket changes
-- Race conditions between server and client state
-
-**After:** All mutations route through a single atomic operation with:
-- Version tracking for cache coherency
-- Source attribution for debugging
-- Timestamp recording for audit trails
-- No intermediate states visible to UI
-
-## Implementation Details
-
-### Single Mutation Point: _updateConversations()
-
-Location: `static/js/conversations.js` lines 110-128
-
-```javascript
-_updateConversations(newArray, source, context = {}) {
-  const oldLen = this.conversations.length;
-  const newLen = Array.isArray(newArray) ? newArray.length : 0;
-  const mutationId = ++this._conversationVersion;
-  const timestamp = Date.now();
-
-  this.conversations = Array.isArray(newArray) ? newArray : [];
-  this._lastMutationSource = source;
-  this._lastMutationTime = timestamp;
-
-  window._conversationCacheVersion = mutationId;
-
-  if (context.verbose) {
-    console.log(`[ConvMgr] mutation #${mutationId} (${source}): ${oldLen} → ${newLen} items, ts=${timestamp}`);
-  }
-
-  return { version: mutationId, timestamp, oldLen, newLen };
-}
-```
-
-**Key Features:**
-- Atomic: Replaces entire array reference, never partial mutations
-- Versioned: Increments counter on every mutation
-- Sourced: Records where mutation originated (poll, add, update, delete, clear_all, ws_clear_all)
-- Timestamped: Records when mutation occurred
-- Observable: Exposes version via `window._conversationCacheVersion` for debugging
-
-### All Mutation Paths Routed
-
-| Method | Source | Line | What It Does |
-|--------|--------|------|-----------|
-| `loadConversations()` | 'poll' | 445 | Server poll every 30s |
-| `addConversation(conv)` | 'add' | 558 | New conversation created |
-| `updateConversation(id, updates)` | 'update' | 567 | Conversation metadata changed |
-| `deleteConversation(id)` | 'delete' | 577 | Conversation deleted |
-| `confirmDeleteAll()` | 'clear_all' | 316 | All conversations cleared |
-| WebSocket handler | 'ws_clear_all' | 596 | Server broadcast clear |
-
-### Version Tracking
-
-State variables added to constructor:
-- `this._conversationVersion = 0` - Current mutation counter
-- `this._lastMutationSource = null` - Source of last mutation
-- `this._lastMutationTime = 0` - Timestamp of last mutation
-
-Global exposure:
-- `window._conversationCacheVersion` - Updated on each mutation
-- `getConversationCacheVersion()" - Getter for version
-
-## Testing
-
-Comprehensive test suite covers 8 scenarios:
-
-1. Single add operation
-2. Version increments on each mutation
-3. Poll overwrites cache atomically
-4. Concurrent add + poll (race condition)
-5. Update preserves order
-6. Delete maintains array integrity
-7. Mutation source tracking
-8. No intermediate states
-
-Run tests:
-```bash
-node tests/cache-desync-test.js
-```
-
-Result: All 8/8 tests pass.
-
-## Preventing Cache Desync: How It Works
-
-### Scenario 1: Concurrent WebSocket Add + Poll
-
-```
-t0: WebSocket 'conversation_created' arrives
-    → addConversation() called
-    → _updateConversations([new_conv, ...old], 'add')
-    → version = 1, array contains new + old
-
-t1: 30s poll timer fires
-    → loadConversations() called with old cached server data
-    → _updateConversations([old1, old2, ...], 'poll')
-    → version = 2, array overwrites with server snapshot
-
-Result: Consistent state - either new+old (version 1) or server data (version 2)
-        Never partial/intermediate state visible to UI
-```
-
-### Scenario 2: Update During Transition
-
-```
-t0: WebSocket 'conversation_updated' arrives for conv #1
-    → updateConversation('conv-1', {title: 'New'})
-    → Creates new array with updated object at index 1
-    → _updateConversations(newArray, 'update')
-    → Entire array replaced atomically
-
-Result: All items stay in original order + positions
-        Update is transactional - either fully applied or not at all
-```
-
-## Observability
-
-### Debugging Cache State
-
-In browser console:
-```javascript
-// Get current version
-window._conversationCacheVersion  // → 15
-
-// Get conversation manager instance
-window.conversationManager.getConversationCacheVersion()  // → 15
-
-// Last mutation source
-window.conversationManager._lastMutationSource  // → 'update'
-
-// Last mutation timestamp
-window.conversationManager._lastMutationTime  // → 1705412890123
-
-// Full conversations array
-window.conversationManager.conversations  // → [...]
-
-// Enable verbose logging
-window.conversationManager._updateConversations(
-  window.conversationManager.conversations,
-  'debug',
-  { verbose: true }
-)
-// Output: [ConvMgr] mutation #16 (debug): 3 → 3 items, ts=...
-```
-
-### Mutation Log
-
-Each mutation source ('poll', 'add', 'update', 'delete', 'clear_all', 'ws_clear_all') can be filtered to understand timing:
-
-```javascript
-// Capture mutations for 1 minute
-const mutations = [];
-const originalUpdate = window.conversationManager._updateConversations;
-window.conversationManager._updateConversations = function(arr, src, ctx) {
-  mutations.push({ src, time: Date.now() });
-  return originalUpdate.call(this, arr, src, ctx);
-};
-
-// Later: analyze
-mutations.filter(m => m.src === 'poll')  // All polls
-mutations.filter(m => m.src.includes('ws'))  // All WebSocket events
-```
-
-## Edge Cases Handled
-
-1. **Nonexistent conversation update** - No version increment if not found
-2. **Duplicate add** - Already-exists check prevents duplicate
-3. **Empty load** - Handles `data.conversations || []` safely
-4. **Rapid mutations** - Each increments version, no race condition
-5. **Concurrent add + delete** - Both atomic, no orphaned references
-
-## Future Enhancements
-
-Potential follow-ups (not implemented):
-
-- **Conflict detection:** Track last-write-wins vs. merge strategies
-- **Optimistic updates:** Append version to pending updates
-- **Cache invalidation:** TTL-based refresh of stale entries
-- **Replay capability:** Use version counter to detect gaps in WebSocket stream
-- **CRDT integration:** Replace array with conflict-free replicated data type
-
-## Files Modified
-
-- `static/js/conversations.js` (+45 lines, -8 lines)
-  - Added atomic mutation point
-  - Routed all 6 mutation paths through it
-  - Added version tracking and observability
-
-## Testing
-
-Created `tests/cache-desync-test.js` with 8 comprehensive test cases covering:
-- Basic mutations
-- Concurrent scenarios
-- Race conditions
-- State preservation
-- Source tracking
-
-All tests pass (8/8).
-
----
-
-**Summary:** Cache desync is prevented by enforcing all mutations through a single atomic operation with version tracking. No intermediate states exist. Concurrent WebSocket and polling scenarios are safe.

package/SYNC_ARCHITECTURE.md

@@ -1,279 +0,0 @@
-# AgentGUI Sync-to-Display Architecture Report
-
-## Executive Summary
-
-The sync-to-display system handles real-time synchronization of execution state from server to client display. Current implementation has been improved through multiple fixes, particularly for queue display consistency and steering support. This document consolidates the architecture and identifies remaining optimization opportunities.
-
-## Architecture Overview
-
-### 1. Broadcast System (server.js:4269-4317)
-
-**BROADCAST_TYPES** - Global broadcast events sent to ALL connected clients:
-- `message_created`, `conversation_created/updated/deleted`
-- `queue_status`, `queue_updated`
-- `streaming_start`, `streaming_progress`, `streaming_complete`, `streaming_error`
-- `rate_limit_hit`, `rate_limit_clear`
-- Tool-related events (`tool_*`, `tools_*`)
-- Voice/speech events
-- PM2 monitoring events
-
-**broadcastSync()** function (line 4288):
-```
-1. Check if event.type is in BROADCAST_TYPES
-2. If broadcast: send to ALL syncClients
-3. If targeted: send to sessionId or conversationId subscribers only
-4. Also dispatch to SSE stream handlers if active
-```
-
-### 2. Client-Side Message Reception (static/js/client.js)
-
-**handleWsMessage()** routes events by type:
-```javascript
-case 'message_created': handleMessageCreated(data)
-case 'queue_status': handleQueueStatus(data)
-case 'streaming_start': handleStreamingStart(data)
-case 'streaming_progress': handleStreamingProgress(data)
-```
-
-### 3. Display Rendering Layers
-
-**Layer 1: Message Rendering** (handleMessageCreated)
-- Checks if message already exists (prevents duplicates)
-- For user messages: finds optimistic message, updates it
-- For assistant messages: skips if streaming (handled by chunks)
-
-**Layer 2: Streaming Progress** (handleStreamingProgress)
-- Adds chunks to queue
-- Batches rendering via StreamingRenderer
-- Handles thinking blocks directly
-
-**Layer 3: Queue Indicator** (fetchAndRenderQueue)
-- Polls via `q.ls` RPC every interval
-- Displays queued items in yellow control blocks
-- Renders with Edit/Delete/Steer buttons
-
-**Layer 4: Conversation State** (handleConversationUpdated)
-- Updates isStreaming flag
-- Refreshes UI based on new state
-
-## Recent Fixes & Status
-
-### Fixed Issues
-
-✅ **Queue Message Duplication** (commit fbfd1ad)
-- Problem: Queued messages appeared as both user prompt + queue item
-- Solution: Skip optimistic message when conversation is streaming
-- Impact: Clean queue display, no duplication
-
-✅ **Steering Support** (commit 81d83af)
-- Problem: Steering showed "Process not available" error
-- Solution: Keep stdin open (supportsStdin=true, closeStdin=false)
-- Impact: Users can send follow-up prompts during execution
-
-✅ **Chunk Rendering Race** (Session 3a)
-- Problem: Early chunks missed before polling started
-- Solution: Immediate chunk fetch on streaming_start
-- Impact: No missing initial output
-
-✅ **Dark Mode Selection** (fsbrowse)
-- Problem: Selection states hard to see in dark mode
-- Solution: Theme-aware colors for hover/focus
-- Impact: Better visual feedback in both themes
-
-### Remaining Consistency Issues
-
-⚠️ **Queue State Sync Timing**
-- Queue created on server via enqueue()
-- message_created broadcast may arrive before queue is populated
-- Client fetchAndRenderQueue() polls but timing is unpredictable
-- No atomic operation ensuring client sees both message and queue
-
-⚠️ **Multiple Sources of Truth**
-- Server: database, active execution map, queue map
-- Client: message list, streaming set, queue indicator
-- Conversation state: isStreaming flag can desync if connection drops
-- No single authoritative state object
-
-⚠️ **Optimistic Message Updates**
-- Client creates optimistic "sending" message on submit
-- Server creates actual message and broadcasts
-- handleMessageCreated() finds and updates optimistic message
-- If network is slow, both may briefly exist
-
-⚠️ **Queue Execution Transition**
-- Queue items don't explicitly transition to executed
-- Client must poll queue indicator to see it disappear
-- No event indicating "queue item now executing"
-- Confusing UX when queue suddenly empties
-
-## Data Flow Examples
-
-### Normal Execution (Ctrl+Enter with no active stream)
-
-```
-User input: "Write a function"
-    ↓
-startExecution()
-    → _showOptimisticMessage(pendingId, content)  // yellow "sending..." message
-    ↓
-streamToConversation() → msg.stream RPC
-    ↓
-Server: msg.stream handler
-    → createMessage(p.id, 'user', prompt)
-    → broadcastSync({ type: 'message_created', ... })
-    → startExecution() -> streaming_start broadcast
-    ↓
-Client: message_created event
-    → handleMessageCreated()
-    → Finds optimistic message by ID
-    → Updates it (removes "sending" style, adds timestamp)
-    ↓
-Client: streaming_start event
-    → Disables input, shows "thinking..."
-    → Subscribes to session
-    → Starts chunk polling
-```
-
-### Queue Case (Ctrl+Enter while streaming) - NOW FIXED
-
-```
-User input: "Now add subtraction" during execution
-    ↓
-startExecution()
-    → isStreaming = true
-    → SKIP _showOptimisticMessage ✓ (FIXED)
-    ↓
-streamToConversation() → msg.stream RPC
-    ↓
-Server: msg.stream handler
-    → createMessage(p.id, 'user', prompt)
-    → broadcastSync({ type: 'message_created', ... })
-    → activeExecutions.has(p.id) = true
-    → enqueue(p.id, prompt, ...)
-    → broadcastSync({ type: 'queue_status', ... })
-    ↓
-Client: message_created event
-    → handleMessageCreated()
-    → Message is not in current conversation display
-    → Emit event but don't render
-    ↓
-Client: queue_status event
-    → handleQueueStatus()
-    → fetchAndRenderQueue()
-    → Displays in yellow queue indicator ✓
-    ↓
-Result: Message only in queue, no duplication ✓
-```
-
-### Steering (Ctrl+Enter + Steer button during execution)
-
-```
-User clicks Steer button on queued message
-    ↓
-Client: conv.steer RPC with JSON-RPC format
-    ↓
-Server: conv.steer handler
-    → Finds active process stdin
-    → Writes JSON-RPC prompt request
-    → Removes from queue (optional)
-    ↓
-Claude Code: Receives JSON-RPC on stdin
-    → Processes prompt immediately
-    → Continues execution with new direction
-    ↓
-Client: receives streaming_progress chunks
-    → Renders new content below existing output
-```
-
-## Consistency Guarantees
-
-### What IS Consistent
-- Messages don't duplicate (handled in handleMessageCreated)
-- Streaming chunks are ordered (via session ID subscription)
-- Queue items maintain order (FIFO in server queue map)
-- Tool installations tracked atomically (per-tool in database)
-
-### What IS NOT Guaranteed
-- Client queue display and server queue state may briefly desync
-- If connection drops mid-queue, client state becomes stale
-- No explicit confirmation that user message was queued
-- Queue item execution not signaled with explicit event
-
-## Recommendations for Further Improvement
-
-### Priority 1: Explicit Queue Lifecycle
-Add dedicated broadcast events:
-```javascript
-'message_queued'        // Sent when msg added to queue
-'queue_item_executing'  // Sent when queue item becomes active
-'queue_item_completed'  // Sent when queue item finished
-```
-
-### Priority 2: Atomic State Snapshots
-Periodically broadcast conversation state:
-```javascript
-{
-  type: 'conversation_state',
-  conversationId,
-  state: {
-    isStreaming: boolean,
-    messageCount: number,
-    queueLength: number,
-    lastMessageId: string,
-    lastUpdate: timestamp
-  }
-}
-```
-
-### Priority 3: Deterministic Client State Machine
-Each conversation has explicit state mode:
-```javascript
-state = {
-  mode: 'IDLE' | 'STREAMING' | 'QUEUED',
-  messages: [],
-  queue: [],
-  isTransitioning: boolean  // true when queue→execute
-}
-```
-
-### Priority 4: Connection Recovery
-After reconnect, fetch complete conversation state:
-```javascript
-await wsClient.rpc('conv.sync', { id: conversationId })
-// Returns: { messages, queue, isStreaming, chunks }
-```
-
-## Testing Matrix
-
-| Scenario | Before Fix | After Fix | Status |
-|----------|-----------|-----------|--------|
-| Ctrl+Enter (not streaming) | ✓ Works | ✓ Works | ✓ OK |
-| Ctrl+Enter (streaming) | ✗ Duplicate | ✓ Single queue item | ✓ FIXED |
-| Queue indicator updates | ⚠️ Polling | ⚠️ Polling | ⚠️ Could improve |
-| Steering works | ✗ "Not available" | ✓ Works | ✓ FIXED |
-| Multiple queued items | ⚠️ Order unclear | ⚠️ Order unclear | ⚠️ OK but could confirm |
-| Page reload with queue | ✗ Queue lost | ✓ Persisted | ✓ OK |
-| Network disconnect | ⚠️ State stale | ⚠️ State stale | ⚠️ Could improve |
-
-## Files Modified in Latest Session
-
-1. **lib/claude-runner.js** - Steering support
-   - Changed `supportsStdin: false` → `true`
-   - Changed `closeStdin: true` → `false`
-   - Removed positional prompt argument
-
-2. **static/js/client.js** - Queue display fix
-   - Skip optimistic message when streaming
-   - Skip confirm/fail handlers for queued messages
-
-3. **static/index.html** - Hamburger animation
-   - Added `transition: none` to .sidebar
-
-4. **fsbrowse/public/style.css** - Dark mode colors
-   - Improved hover states
-   - Theme-aware modal focus colors
-
-## Conclusion
-
-The sync-to-display system is now more consistent with the queue display fix and steering support. The architecture handles the primary use cases well, but could benefit from explicit lifecycle events and atomic state snapshots for guaranteed consistency. The current implementation is pragmatic and performant, trading off some guarantees for simplicity and responsiveness.
-

package/TOOLS-FIX-SUMMARY.md

@@ -1,122 +0,0 @@
-# CLI Tools Detection and Agent Dropdown Fixes
-
-## Summary
-Three critical issues were identified and fixed in the tools and agent systems:
-1. CLI tools showing as "Not installed" despite being in PATH
-2. Agent dropdown not showing Claude Code
-3. /api/tools endpoint hanging during tool detection
-
-## Issues Fixed
-
-### Issue 1: CLI Tools Not Detected (lib/tool-manager.js)
-
-**Root Cause**: `execSync` was being used but not imported at module level. The code had:
-```javascript
-// BEFORE (broken)
-const { execSync } = require('child_process');  // Inside function
-```
-
-This caused "require is not defined" errors in ES modules.
-
-**Fix**: Import `execSync` at the top of the file:
-```javascript
-// AFTER (fixed)
-import { spawn, execSync } from 'child_process';
-```
-
-**Changes Made**:
-- Line 1: Added `execSync` to the child_process import
-- checkCliInstalled() now uses the imported execSync directly
-- getCliVersion() now uses the imported execSync directly
-- Reduced version check timeout from 5000ms to 2000ms to prevent hangs
-- Added better error handling for CLI version detection
-
-**Impact**: CLI tools (claude, opencode, gemini, kilo, codex) are now properly detected as installed when present in PATH.
-
-### Issue 2: /api/tools Endpoint Hanging (server.js)
-
-**Root Cause**: The `/api/tools` endpoint was calling `getAllToolsAsync()` which attempted to fetch published versions from npm registry. Some tools like opencode have long-running or hanging --version commands, causing the entire endpoint to hang.
-
-**Fix**: Use `Promise.race` with a timeout to return immediately with cached data:
-```javascript
-// Lines 1844-1846
-const tools = await Promise.race([
-  toolManager.getAllToolsAsync(true), // skipPublishedVersion=true for fast response
-  new Promise((_, reject) => setTimeout(() => reject(new Error('timeout')), 1500))
-]);
-```
-
-**Changes Made**:
-- Added timeout of 1500ms to the API call
-- Pass `skipPublishedVersion=true` to skip network calls during initial detection
-- Fall back to cached tool statuses if timeout occurs
-- Ensures UI remains responsive when fetching tool data
-
-**Impact**: `/api/tools` endpoint now responds within 2 seconds instead of hanging indefinitely.
-
-### Issue 3: Claude Code Agent Dropdown
-
-**Current State**: Claude Code agent SHOULD appear in the dropdown because:
-
-1. **Server-side** (server.js lines 461-492):
-   - `discoverAgents()` function checks for `claude` command in PATH
-   - If found → agent is added with detected path
-   - If NOT found → fallback ensures Claude Code is ALWAYS added as npx-launchable (lines 486-488)
-
-2. **Agent RPC Handler** (lib/ws-handlers-session.js lines 104-111):
-   - `agent.ls` RPC handler returns all discoveredAgents
-   - Includes Claude Code from the fallback mechanism
-
-3. **Frontend** (static/js/client.js lines 2145-2149):
-   - Calls `window.wsClient.rpc('agent.ls')` to fetch agents
-   - Populates UI with returned agents
-
-**Verification Needed**:
-- Check if browser console shows agent list includes 'claude-code'
-- Verify /api/agents HTTP endpoint returns Claude Code
-- Check if agent.ls RPC message logs show Claude Code being returned
-
-## Files Modified
-
-1. **lib/tool-manager.js**
-   - Line 1: Import execSync from child_process
-   - Lines 180-215: Enhanced CLI detection functions
-   - Line 208: Reduced timeout for version checks
-   - Lines 290, 475: Pass skipPublishedVersion parameter
-
-2. **lib/speech.js**
-   - Line 47: Removed unused `splitSentences` export (webtalk handles splitting)
-
-3. **server.js**
-   - Lines 1844-1847: Added Promise.race with timeout for /api/tools
-   - Lines 1862-1864: Added timeout error handling with fallback
-
-4. **.prd** (Work tracking)
-   - Updated with completion status for CLI detection fixes
-
-## Testing Checklist
-
-- [ ] `claude --version` returns a version number in PATH
-- [ ] GET /api/tools returns installed=true for cli-claude
-- [ ] GET /api/tools completes within 2 seconds
-- [ ] /api/agents includes Claude Code agent
-- [ ] agent.ls RPC message returns Claude Code
-- [ ] Agent dropdown in UI shows Claude Code
-- [ ] Can create conversation with Claude Code agent
-- [ ] No "Not installed" messages for detected tools
-- [ ] Server startup auto-detection logs show CLI tools found
-
-## Known Limitations
-
-1. **CLI Version Detection**: Some tools (opencode, gemini, kilo) may return 'unknown' version if --version command doesn't return standard semantic version format
-
-2. **Timeout Edge Case**: If /api/tools times out, it returns cached data from previous request. New tools won't show until cache expires (30 minutes) or server restarts
-
-3. **Fallback Behavior**: Claude Code appears as npx-launchable even if `claude` binary is not in PATH. This allows users to still launch Claude Code via npm.
-
-## Next Steps
-
-1. Verify agent dropdown shows Claude Code after fixes
-2. Test auto-installation of CLI tools on startup
-3. Verify version detection works for all tools
-4. Test update detection and installation for CLI tools