claude-self-reflect 2.8.2 → 2.8.3

package/README.md

@@ -126,7 +126,7 @@ Files are categorized by age and processed with priority queuing to ensure newes
 
 - **Docker Desktop** (macOS/Windows) or **Docker Engine** (Linux)
 - **Node.js** 16+ (for the setup wizard)
-- **Claude Desktop** app
+- **Claude Code** CLI
 
 ## 📖 Documentation
 
@@ -231,6 +231,98 @@ docker compose run --rm importer python /app/scripts/delta-metadata-update-safe.
 
 </details>
 
+## 🔧 Troubleshooting
+
+### Common Issues and Solutions
+
+#### 1. "No collections created" after import
+**Symptom**: Import runs but Qdrant shows no collections  
+**Cause**: Docker can't access Claude projects directory  
+**Solution**:
+```bash
+# Run diagnostics to identify the issue
+claude-self-reflect doctor
+
+# Fix: Re-run setup to set correct paths
+claude-self-reflect setup
+
+# Verify .env has full paths (no ~):
+cat .env | grep CLAUDE_LOGS_PATH
+# Should show: CLAUDE_LOGS_PATH=/Users/YOUR_NAME/.claude/projects
+```
+
+#### 2. MCP server shows "ERROR" but it's actually INFO
+**Symptom**: `[ERROR] MCP server "claude-self-reflect" Server stderr: INFO Starting MCP server`  
+**Cause**: Claude Code displays all stderr output as errors  
+**Solution**: This is not an actual error - the MCP is working correctly. The INFO message confirms successful startup.
+
+#### 3. "No JSONL files found"
+**Symptom**: Setup can't find any conversation files  
+**Cause**: Claude Code hasn't been used yet or stores files elsewhere  
+**Solution**:
+```bash
+# Check if files exist
+ls ~/.claude/projects/
+
+# If empty, use Claude Code to create some conversations first
+# The watcher will import them automatically
+```
+
+#### 4. Docker volume mount issues
+**Symptom**: Import fails with permission errors  
+**Cause**: Docker can't access home directory  
+**Solution**:
+```bash
+# Ensure Docker has file sharing permissions
+# macOS: Docker Desktop → Settings → Resources → File Sharing
+# Add: /Users/YOUR_USERNAME/.claude
+
+# Restart Docker and re-run setup
+docker compose down
+claude-self-reflect setup
+```
+
+#### 5. Qdrant not accessible
+**Symptom**: Can't connect to localhost:6333  
+**Solution**:
+```bash
+# Start services
+docker compose --profile mcp up -d
+
+# Check if running
+docker compose ps
+
+# View logs for errors
+docker compose logs qdrant
+```
+
+### Diagnostic Tools
+
+Run comprehensive diagnostics:
+```bash
+claude-self-reflect doctor
+```
+
+This checks:
+- Docker installation and configuration
+- Environment variables and paths
+- Claude projects and JSONL files
+- Import status and collections
+- Service health
+
+### Still Having Issues?
+
+1. Check the logs:
+   ```bash
+   docker compose logs -f
+   ```
+
+2. Report issues with diagnostic output:
+   ```bash
+   claude-self-reflect doctor > diagnostic.txt
+   ```
+   Then include diagnostic.txt when [reporting issues](https://github.com/ramakay/claude-self-reflect/issues)
+
 ## 👥 Contributors
 
 Special thanks to our contributors:

package/installer/cli.js

@@ -1,6 +1,6 @@
 #!/usr/bin/env node
 
-import { spawn } from 'child_process';
+import { spawn, execSync } from 'child_process';
 import { fileURLToPath } from 'url';
 import { dirname, join } from 'path';
 import fs from 'fs/promises';
@@ -76,71 +76,76 @@ async function status() {
 }
 
 async function doctor() {
-  console.log('🔍 Checking Claude Self-Reflect installation...\n');
+  console.log('🔍 Running comprehensive diagnostics...\n');
   
-  const checks = [
-    {
-      name: 'Python 3.10+',
-      check: async () => {
-        try {
-          const { execSync } = await import('child_process');
-          const version = execSync('python3 --version').toString().trim();
-          return { passed: true, message: version };
-        } catch {
-          return { passed: false, message: 'Python 3.10+ not found' };
-        }
+  // Use the new Python doctor script for comprehensive checks
+  const doctorScript = join(__dirname, '..', 'scripts', 'doctor.py');
+  
+  try {
+    // Check if Python is available
+    try {
+      execSync('python3 --version', { stdio: 'ignore' });
+    } catch {
+      console.log('❌ Python 3 is required but not found');
+      console.log('   Please install Python 3.10 or later');
+      process.exit(1);
+    }
+    
+    // Run the doctor script
+    const child = spawn('python3', [doctorScript], { 
+      stdio: 'inherit',
+      cwd: join(__dirname, '..')
+    });
+    
+    child.on('exit', (code) => {
+      if (code === 0) {
+        console.log('\n✅ All checks passed!');
+      } else if (code === 2) {
+        console.log('\n⚠️  Some issues found - see recommendations above');
+      } else {
+        console.log('\n❌ Critical issues found - please address them first');
       }
-    },
-    {
-      name: 'Qdrant',
-      check: async () => {
-        try {
-          const response = await fetch('http://localhost:6333');
-          const data = await response.json();
-          if (data.title && data.title.includes('qdrant')) {
-            return { passed: true, message: `Qdrant ${data.version} is running on port 6333` };
+      process.exit(code || 0);
+    });
+  } catch (error) {
+    // Fallback to basic checks if doctor script fails
+    console.log('⚠️  Could not run comprehensive diagnostics, using basic checks...\n');
+    
+    const checks = [
+      {
+        name: 'Docker',
+        check: async () => {
+          try {
+            execSync('docker info', { stdio: 'ignore' });
+            return { passed: true, message: 'Docker is running' };
+          } catch {
+            return { passed: false, message: 'Docker not running or not installed' };
           }
-        } catch {}
-        return { passed: false, message: 'Qdrant not accessible on localhost:6333' };
-      }
-    },
-    {
-      name: 'MCP Server',
-      check: async () => {
-        const mcpPath = join(__dirname, '..', 'mcp-server', 'pyproject.toml');
-        try {
-          await fs.access(mcpPath);
-          return { passed: true, message: 'MCP server files found' };
-        } catch {
-          return { passed: false, message: 'MCP server files not found' };
         }
-      }
-    },
-    {
-      name: 'Environment Variables',
-      check: async () => {
-        const envPath = join(__dirname, '..', '.env');
-        try {
-          const content = await fs.readFile(envPath, 'utf-8');
-          const hasVoyageKey = content.includes('VOYAGE_KEY=') && !content.includes('VOYAGE_KEY=your-');
-          if (hasVoyageKey) {
-            return { passed: true, message: '.env file configured with VOYAGE_KEY' };
-          }
-          return { passed: false, message: '.env file missing VOYAGE_KEY' };
-        } catch {
-          return { passed: false, message: '.env file not found' };
+      },
+      {
+        name: 'Qdrant',
+        check: async () => {
+          try {
+            const response = await fetch('http://localhost:6333');
+            const data = await response.json();
+            if (data.title && data.title.includes('qdrant')) {
+              return { passed: true, message: `Qdrant ${data.version} is running` };
+            }
+          } catch {}
+          return { passed: false, message: 'Qdrant not accessible on localhost:6333' };
         }
       }
+    ];
+    
+    for (const check of checks) {
+      const result = await check.check();
+      const icon = result.passed ? '✅' : '❌';
+      console.log(`${icon} ${check.name}: ${result.message}`);
     }
-  ];
-  
-  for (const check of checks) {
-    const result = await check.check();
-    const icon = result.passed ? '✅' : '❌';
-    console.log(`${icon} ${check.name}: ${result.message}`);
+    
+    console.log('\n💡 Run "claude-self-reflect setup" to fix any issues');
   }
-  
-  console.log('\n💡 Run "claude-self-reflect setup" to fix any issues');
 }
 
 const ASCII_ART = `

package/installer/setup-wizard-docker.js

@@ -191,6 +191,12 @@ async function configureEnvironment() {
   if (!envContent.includes('CONFIG_PATH=')) {
     envContent += `CONFIG_PATH=${userConfigDir}\n`;
   }
+  // CRITICAL: Set CLAUDE_LOGS_PATH with expanded home directory
+  // Docker doesn't expand ~ in volume mounts
+  if (!envContent.includes('CLAUDE_LOGS_PATH=')) {
+    const claudeLogsPath = path.join(os.homedir(), '.claude', 'projects');
+    envContent += `CLAUDE_LOGS_PATH=${claudeLogsPath}\n`;
+  }
   
   await fs.writeFile(envPath, envContent.trim() + '\n');
   console.log('✅ Environment configured');
@@ -294,30 +300,36 @@ async function startDockerServices() {
 }
 
 async function configureClaude() {
-  console.log('\n🤖 Configuring Claude Desktop...');
+  console.log('\n🤖 Configuring Claude Code...');
   
-  const mcpScript = join(projectRoot, 'mcp-server', 'run-mcp-docker.sh');
+  // Use the clean wrapper if running locally, Docker script for Docker mode
+  const isDockerMode = process.env.USE_DOCKER_MCP === 'true';
+  const mcpScript = isDockerMode 
+    ? join(projectRoot, 'mcp-server', 'run-mcp-docker.sh')
+    : join(projectRoot, 'mcp-server', 'run-mcp-clean.sh');
   
-  // Create a script that runs the MCP server in Docker
-  const scriptContent = `#!/bin/bash
+  if (isDockerMode) {
+    // Create a script that runs the MCP server in Docker
+    const scriptContent = `#!/bin/bash
 # Run the MCP server in the Docker container with stdin attached
 # Using python -u for unbuffered output
 # Using the main module which properly supports local embeddings
 docker exec -i claude-reflection-mcp python -u -m src
 `;
-  
-  await fs.writeFile(mcpScript, scriptContent, { mode: 0o755 });
+    
+    await fs.writeFile(mcpScript, scriptContent, { mode: 0o755 });
+  }
   
   // Check if Claude CLI is available
   try {
     safeExec('which', ['claude'], { stdio: 'ignore' });
     
-    console.log('🔧 Adding MCP to Claude Desktop...');
+    console.log('🔧 Adding MCP to Claude Code...');
     try {
       const mcpArgs = ['mcp', 'add', 'claude-self-reflect', mcpScript];
       safeExec('claude', mcpArgs, { stdio: 'inherit' });
       console.log('✅ MCP added successfully!');
-      console.log('\n⚠️  Please restart Claude Desktop for changes to take effect.');
+      console.log('\n⚠️  Please restart Claude Code for changes to take effect.');
     } catch {
       console.log('⚠️  Could not add MCP automatically');
       showManualConfig(mcpScript);
@@ -329,7 +341,7 @@ docker exec -i claude-reflection-mcp python -u -m src
 }
 
 function showManualConfig(mcpScript) {
-  console.log('\nAdd this to your Claude Desktop config manually:');
+  console.log('\nAdd this to your Claude Code config manually:');
   console.log('```json');
   console.log(JSON.stringify({
     "claude-self-reflect": {
@@ -342,6 +354,42 @@ function showManualConfig(mcpScript) {
 async function importConversations() {
   console.log('\n📚 Checking conversation baseline...');
   
+  // First check if Claude projects directory exists and has JSONL files
+  const claudeProjectsDir = path.join(os.homedir(), '.claude', 'projects');
+  let totalJsonlFiles = 0;
+  let projectCount = 0;
+  
+  try {
+    if (fsSync.existsSync(claudeProjectsDir)) {
+      const projects = fsSync.readdirSync(claudeProjectsDir);
+      for (const project of projects) {
+        const projectPath = path.join(claudeProjectsDir, project);
+        if (fsSync.statSync(projectPath).isDirectory()) {
+          const jsonlFiles = fsSync.readdirSync(projectPath).filter(f => f.endsWith('.jsonl'));
+          if (jsonlFiles.length > 0) {
+            projectCount++;
+            totalJsonlFiles += jsonlFiles.length;
+          }
+        }
+      }
+    }
+    
+    if (totalJsonlFiles === 0) {
+      console.log('\n⚠️  No Claude conversation files found!');
+      console.log(`   Checked: ${claudeProjectsDir}`);
+      console.log('\n   This could mean:');
+      console.log('   • You haven\'t used Claude Code yet');
+      console.log('   • Claude stores conversations in a different location');
+      console.log('   • Permissions issue accessing the directory');
+      console.log('\n   The watcher will monitor for new conversations.');
+      return;
+    }
+    
+    console.log(`✅ Found ${totalJsonlFiles} conversation files across ${projectCount} projects`);
+  } catch (e) {
+    console.log(`⚠️  Could not access Claude projects directory: ${e.message}`);
+  }
+  
   // Check if baseline exists by looking for imported files state
   const configDir = path.join(os.homedir(), '.claude-self-reflect', 'config');
   const stateFile = path.join(configDir, 'imported-files.json');
@@ -492,7 +540,7 @@ async function showFinalInstructions() {
   console.log('   • Stop all: docker compose down');
   
   console.log('\n🎯 Next Steps:');
-  console.log('1. Restart Claude Desktop');
+  console.log('1. Restart Claude Code');
   console.log('2. Look for "claude-self-reflect" in the MCP tools');
   console.log('3. Try: "Search my past conversations about Python"');
   

package/mcp-server/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "claude-self-reflect-mcp"
-version = "2.7.3"
+version = "2.8.3"
 description = "MCP server for Claude self-reflection with memory decay"
 # readme = "README.md"
 requires-python = ">=3.10"

package/mcp-server/src/status.py

@@ -6,57 +6,66 @@ Designed for <20ms execution time to support status bars and shell scripts.
 
 import json
 import time
+import sys
 from pathlib import Path
 from collections import defaultdict
 
-
-def extract_project_name_from_path(file_path: str) -> str:
-    """Extract project name from JSONL file path.
-    
-    Handles paths like:
-    - ~/.claude/projects/-Users-username-projects-claude-self-reflect/file.jsonl
-    - /logs/-Users-username-projects-n8n-builder/file.jsonl
-    """
-    # Get the directory name containing the JSONL file
-    path_obj = Path(file_path)
-    dir_name = path_obj.parent.name
-    
-    # Extract project name from dash-encoded path
-    # Format: -Users-username-projects-PROJECT_NAME (PROJECT_NAME can have dashes)
-    if dir_name.startswith('-') and 'projects' in dir_name:
-        parts = dir_name.split('-')
-        # Find 'projects' and take everything after it as the project name
-        try:
-            projects_idx = parts.index('projects')
-            if projects_idx + 1 < len(parts):
-                # Join all parts after 'projects' to handle multi-part project names
-                # like "claude-self-reflect", "n8n-builder", etc.
-                project_parts = parts[projects_idx + 1:]
-                return '-'.join(project_parts)
-        except ValueError:
-            pass
-    
-    # Fallback: use the directory name as-is
-    return dir_name.lstrip('-')
-
-
-def normalize_file_path(file_path: str) -> str:
-    """Normalize file paths to handle Docker vs local path differences.
-    
-    Converts:
-    - /logs/PROJECT_DIR/file.jsonl -> ~/.claude/projects/PROJECT_DIR/file.jsonl
-    - Already normalized paths remain unchanged
-    """
-    if file_path.startswith("/logs/"):
-        # Convert Docker path to local path
-        projects_dir = str(Path.home() / ".claude" / "projects")
-        return file_path.replace("/logs/", projects_dir + "/", 1)
-    return file_path
+# Try to import shared utilities
+try:
+    # Add scripts directory to path
+    scripts_dir = Path(__file__).parent.parent.parent / "scripts"
+    if scripts_dir.exists():
+        sys.path.insert(0, str(scripts_dir))
+    from shared_utils import (
+        normalize_file_path, 
+        extract_project_name_from_path,
+        get_claude_projects_dir,
+        get_csr_config_dir
+    )
+except ImportError:
+    # Fallback implementations if shared_utils is not available
+    def extract_project_name_from_path(file_path: str) -> str:
+        """Extract project name from JSONL file path."""
+        path_obj = Path(file_path)
+        dir_name = path_obj.parent.name
+        
+        if dir_name.startswith('-') and 'projects' in dir_name:
+            parts = dir_name.split('-')
+            try:
+                projects_idx = parts.index('projects')
+                if projects_idx + 1 < len(parts):
+                    project_parts = parts[projects_idx + 1:]
+                    return '-'.join(project_parts)
+            except ValueError:
+                pass
+        
+        return dir_name.lstrip('-')
+    
+    def normalize_file_path(file_path: str) -> str:
+        """Normalize file paths to handle Docker vs local path differences."""
+        if file_path.startswith("/logs/"):
+            projects_dir = str(Path.home() / ".claude" / "projects")
+            return file_path.replace("/logs/", projects_dir + "/", 1)
+        return file_path
+    
+    def get_claude_projects_dir() -> Path:
+        """Get Claude projects directory."""
+        import os
+        if 'CLAUDE_PROJECTS_DIR' in os.environ:
+            return Path(os.environ['CLAUDE_PROJECTS_DIR'])
+        return Path.home() / ".claude" / "projects"
+    
+    def get_csr_config_dir() -> Path:
+        """Get CSR config directory."""
+        import os
+        if 'CSR_CONFIG_DIR' in os.environ:
+            return Path(os.environ['CSR_CONFIG_DIR'])
+        return Path.home() / '.claude-self-reflect' / 'config'
 
 
 def get_watcher_status() -> dict:
     """Get streaming watcher status if available."""
-    watcher_state_file = Path.home() / "config" / "csr-watcher.json"
+    watcher_state_file = get_csr_config_dir() / "csr-watcher.json"
     
     if not watcher_state_file.exists():
         return {"running": False, "status": "not configured"}
@@ -85,7 +94,7 @@ def get_status() -> dict:
     Returns:
         dict: JSON structure with overall and per-project indexing status, plus watcher status
     """
-    projects_dir = Path.home() / ".claude" / "projects"
+    projects_dir = get_claude_projects_dir()
     project_stats = defaultdict(lambda: {"indexed": 0, "total": 0})
     
     # Build a mapping of normalized file paths to project names
@@ -99,18 +108,24 @@ def get_status() -> dict:
             project_stats[project_name]["total"] += 1
             file_to_project[file_str] = project_name
     
+    # Track which files have been counted to avoid duplicates
+    counted_files = set()
+    
     # Read imported-files.json to count indexed files per project
-    config_paths = [
-        Path.home() / ".claude-self-reflect" / "config" / "imported-files.json",
-        Path(__file__).parent.parent.parent / "config" / "imported-files.json",
-        Path("/config/imported-files.json")  # Docker path
-    ]
-    
-    imported_files_path = None
-    for path in config_paths:
-        if path.exists():
-            imported_files_path = path
-            break
+    config_dir = get_csr_config_dir()
+    imported_files_path = config_dir / "imported-files.json"
+    
+    # Fallback to other locations if not found
+    if not imported_files_path.exists():
+        config_paths = [
+            Path(__file__).parent.parent.parent / "config" / "imported-files.json",
+            Path("/config/imported-files.json")  # Docker path
+        ]
+        
+        for path in config_paths:
+            if path.exists():
+                imported_files_path = path
+                break
     
     if imported_files_path:
         try:
@@ -123,9 +138,10 @@ def get_status() -> dict:
                 # Count all files in imported_files object (they are all fully imported)
                 for file_path in imported_files.keys():
                     normalized_path = normalize_file_path(file_path)
-                    if normalized_path in file_to_project:
+                    if normalized_path in file_to_project and normalized_path not in counted_files:
                         project_name = file_to_project[normalized_path]
                         project_stats[project_name]["indexed"] += 1
+                        counted_files.add(normalized_path)
                 
                 # Also check file_metadata for partially imported files
                 file_metadata = data.get("file_metadata", {})
@@ -134,9 +150,10 @@ def get_status() -> dict:
                         # Only count if not already in imported_files
                         if file_path not in imported_files:
                             normalized_path = normalize_file_path(file_path)
-                            if normalized_path in file_to_project:
+                            if normalized_path in file_to_project and normalized_path not in counted_files:
                                 project_name = file_to_project[normalized_path]
                                 project_stats[project_name]["indexed"] += 1
+                                counted_files.add(normalized_path)
                 
                 # Also check stream_position if it contains file paths
                 stream_position = data.get("stream_position", {})
@@ -148,15 +165,33 @@ def get_status() -> dict:
                         # Only count if not already counted
                         if file_path not in imported_files:
                             normalized_path = normalize_file_path(file_path)
-                            if normalized_path in file_to_project:
+                            if normalized_path in file_to_project and normalized_path not in counted_files:
                                 project_name = file_to_project[normalized_path]
-                                # Only increment if not already counted
-                                if project_stats[project_name]["indexed"] < project_stats[project_name]["total"]:
-                                    project_stats[project_name]["indexed"] += 1
+                                project_stats[project_name]["indexed"] += 1
+                                counted_files.add(normalized_path)
         except (json.JSONDecodeError, KeyError, OSError):
             # If config file is corrupted or unreadable, continue with zero indexed counts
             pass
     
+    # Also read csr-watcher.json to count files imported by the watcher
+    watcher_state_file = config_dir / "csr-watcher.json"
+    if watcher_state_file.exists():
+        try:
+            with open(watcher_state_file, 'r') as f:
+                watcher_data = json.load(f)
+                
+                # Count files imported by the watcher
+                watcher_imports = watcher_data.get("imported_files", {})
+                for file_path in watcher_imports.keys():
+                    normalized_path = normalize_file_path(file_path)
+                    if normalized_path in file_to_project and normalized_path not in counted_files:
+                        project_name = file_to_project[normalized_path]
+                        project_stats[project_name]["indexed"] += 1
+                        counted_files.add(normalized_path)
+        except (json.JSONDecodeError, KeyError, OSError):
+            # If watcher file is corrupted or unreadable, continue
+            pass
+    
     # Calculate overall stats
     total_all = sum(p["total"] for p in project_stats.values())
     indexed_all = sum(p["indexed"] for p in project_stats.values())

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-self-reflect",
-  "version": "2.8.2",
+  "version": "2.8.3",
   "description": "Give Claude perfect memory of all your conversations - Installation wizard for Python MCP server",
   "keywords": [
     "claude",

package/scripts/import-conversations-unified.py

@@ -338,7 +338,19 @@ def main():
     logger.info(f"Loaded state with {len(state.get('imported_files', {}))} previously imported files")
     
     # Find all projects
-    logs_dir = Path(os.getenv("LOGS_DIR", "/logs"))
+    # Use LOGS_DIR env var, or fall back to Claude projects directory, then /logs for Docker
+    logs_dir_env = os.getenv("LOGS_DIR")
+    if logs_dir_env:
+        logs_dir = Path(logs_dir_env)
+    elif (Path.home() / ".claude" / "projects").exists():
+        logs_dir = Path.home() / ".claude" / "projects"
+    else:
+        logs_dir = Path("/logs")  # Docker fallback
+    
+    if not logs_dir.exists():
+        logger.error(f"Projects directory not found: {logs_dir}")
+        sys.exit(1)
+    
     project_dirs = [d for d in logs_dir.iterdir() if d.is_dir()]
     logger.info(f"Found {len(project_dirs)} projects to import")