claude-self-reflect 3.2.3 → 3.3.0

package/.claude/agents/documentation-writer.md

@@ -48,7 +48,7 @@ You are a technical documentation specialist for the Claude Self Reflect project
  * @param query - Natural language search query
  * @param options - Search configuration options
  * @param options.limit - Maximum results to return (default: 10)
- * @param options.threshold - Minimum similarity score 0-1 (default: 0.7)
+ * @param options.threshold - Minimum similarity score 0-1 (removed in v3.2.4 - uses natural scoring)
  * @param options.project - Filter by specific project name
  * @returns Promise resolving to array of search results
  * 

package/.claude/agents/qdrant-specialist.md

@@ -12,7 +12,7 @@ You are a Qdrant vector database specialist for the claude-self-reflect project.
 - Collections use per-project isolation: `conv_<md5_hash>_local` or `conv_<md5_hash>_voyage` naming
 - Project paths: ~/.claude/projects/-Users-{username}-projects-{project-name}/*.jsonl
 - Project name is extracted from path and MD5 hashed for collection naming
-- Cross-collection search enabled with 0.7 similarity threshold
+- Cross-collection search uses Qdrant's natural scoring (no artificial thresholds since v3.2.4)
 - Streaming importer detects file growth and processes new lines incrementally
 - MCP server expects collections to match project name MD5 hash
 
@@ -195,7 +195,7 @@ docker stats qdrant
 
 ## Project-Specific Rules
 - Always use Voyage AI embeddings for consistency
-- Maintain 0.7 similarity threshold as baseline
+- Use Qdrant's natural scoring (no artificial thresholds since v3.2.4)
 - Preserve per-project collection isolation
 - Do not grep JSONL files unless explicitly asked
 - Always verify the MCP integration works end-to-end

package/.claude/agents/reflection-specialist.md

@@ -1,7 +1,7 @@
 ---
 name: reflection-specialist
 description: Conversation memory expert for searching past conversations, storing insights, and self-reflection. Use PROACTIVELY when searching for previous discussions, storing important findings, or maintaining knowledge continuity.
-tools: mcp__claude-self-reflect__reflect_on_past, mcp__claude-self-reflect__store_reflection
+tools: mcp__claude-self-reflect__reflect_on_past, mcp__claude-self-reflect__store_reflection, mcp__claude-self-reflect__get_recent_work, mcp__claude-self-reflect__search_by_recency, mcp__claude-self-reflect__get_timeline, mcp__claude-self-reflect__quick_search, mcp__claude-self-reflect__search_summary, mcp__claude-self-reflect__get_more_results, mcp__claude-self-reflect__search_by_file, mcp__claude-self-reflect__search_by_concept, mcp__claude-self-reflect__get_full_conversation, mcp__claude-self-reflect__get_next_results
 ---
 
 You are a conversation memory specialist for the Claude Self Reflect project. Your expertise covers semantic search across all Claude conversations, insight storage, and maintaining knowledge continuity across sessions.
@@ -117,9 +117,65 @@ Save important insights and decisions for future retrieval.
 }
 ```
 
-### Specialized Search Tools (NEW in v2.4.5)
+### Temporal Query Tools (v3.x)
 
-**Note**: These specialized tools are available through this reflection-specialist agent. Due to FastMCP limitations, they cannot be called directly via MCP (e.g., `mcp__claude-self-reflect__quick_search`), but work perfectly when used through this agent.
+These tools answer time-based questions about your work and conversations.
+
+#### get_recent_work
+Returns recent conversations to answer "What did we work on last?" queries.
+
+```javascript
+// Get recent work (default: current project)
+{
+  limit: 10,
+  group_by: "conversation",  // Or "day" or "session"
+  include_reflections: true
+}
+
+// Get recent work across all projects
+{
+  project: "all",
+  limit: 20,
+  group_by: "day"
+}
+```
+
+#### search_by_recency
+Time-constrained semantic search for queries like "docker issues last week".
+
+```javascript
+// Search with natural language time
+{
+  query: "authentication bugs",
+  time_range: "last week",
+  limit: 10
+}
+
+// Search with specific dates
+{
+  query: "performance optimization",
+  since: "2025-01-01",
+  until: "2025-01-10",
+  project: "all"
+}
+```
+
+#### get_timeline
+Show activity timeline for a project or across all projects.
+
+```javascript
+// Get activity timeline
+{
+  time_range: "last week",
+  granularity: "day",  // Or "hour", "week", "month"
+  include_stats: true,
+  project: "all"
+}
+```
+
+### Specialized Search Tools
+
+**Note**: These specialized tools complement the temporal tools for non-time-based queries.
 
 #### quick_search
 Fast search that returns only the count and top result. Perfect for quick checks and overview.
@@ -128,7 +184,7 @@ Fast search that returns only the count and top result. Perfect for quick checks
 // Quick overview of matches
 {
   query: "authentication patterns",
-  min_score: 0.5,  // Optional, defaults to 0.7
+  min_score: 0.5,  // Optional (v3.2.4+ ignores this - uses natural scoring)
   project: "all"    // Optional, defaults to current project
 }
 ```
@@ -165,7 +221,7 @@ Pagination support for getting additional results after an initial search.
   query: "original search query",  // Must match original query
   offset: 3,                      // Skip first 3 results
   limit: 3,                       // Get next 3 results
-  min_score: 0.7,                 // Optional
+  min_score: 0.7,                 // Optional (v3.2.4+ ignores this)
   project: "all"                  // Optional
 }
 ```

package/.claude/agents/search-optimizer.md

@@ -9,7 +9,7 @@ You are a search optimization specialist for the claude-self-reflect project. Yo
 ## Project Context
 - Current baseline: 66.1% search accuracy with Voyage AI
 - Gemini comparison showed 70-77% accuracy but 50% slower
-- Default similarity threshold: 0.7
+- Search scoring: Uses Qdrant's natural scoring (no artificial thresholds as of v3.2.4)
 - Cross-collection search adds ~100ms overhead
 - 24+ projects with 10,165+ conversation chunks
 
@@ -71,9 +71,11 @@ python scripts/analyze-search-quality.py
 ### Threshold Tuning
 ```bash
 # Test different thresholds
-for threshold in 0.5 0.6 0.7 0.8 0.9; do
-  echo "Testing threshold: $threshold"
-  SIMILARITY_THRESHOLD=$threshold npm test
+# Note: As of v3.2.4, artificial thresholds removed
+# Focus on embedding model comparison instead
+for model in voyage openai gemini; do
+  echo "Testing model: $model"
+  EMBEDDING_MODEL=$model npm test
 done
 
 # Find optimal threshold
@@ -237,7 +239,7 @@ def calculate_mrr(queries, results):
 interface ABTestConfig {
   control: {
     model: 'voyage',
-    threshold: 0.7,
+    scoring: 'natural',
     limit: 10
   },
   variant: {
@@ -285,7 +287,7 @@ async function abTestSearch(query: string, userId: string) {
 ### Recommended Settings
 ```env
 # Search Configuration
-SIMILARITY_THRESHOLD=0.7
+# SIMILARITY_THRESHOLD removed in v3.2.4 - uses natural scoring
 SEARCH_LIMIT=10
 CROSS_COLLECTION_LIMIT=5
 
@@ -300,7 +302,7 @@ SAMPLE_RATE=0.1
 ```
 
 ## Project-Specific Rules
-- Maintain 0.7 similarity threshold as baseline
+- Use Qdrant's natural scoring (no artificial thresholds since v3.2.4)
 - Always compare against Voyage AI baseline (66.1%)
 - Consider search latency alongside accuracy
 - Test with real conversation data

package/README.md

@@ -116,15 +116,18 @@ Works with [Claude Code Statusline](https://github.com/sirmalloc/ccstatusline) -
 <summary><b>MCP Tools Available to Claude</b></summary>
 
 **Search & Memory Tools:**
-- `reflect_on_past` - Search past conversations using semantic similarity with time decay
+- `reflect_on_past` - Search past conversations using semantic similarity with time decay (supports quick/summary modes)
 - `store_reflection` - Store important insights or learnings for future reference
-- `quick_search` - Fast search returning only count and top result
-- `search_summary` - Get aggregated insights without individual details
-- `get_more_results` - Paginate through additional search results
+- `get_next_results` - Paginate through additional search results
 - `search_by_file` - Find conversations that analyzed specific files
 - `search_by_concept` - Search for conversations about development concepts
 - `get_full_conversation` - Retrieve complete JSONL conversation files (v2.8.8)
 
+**NEW: Temporal Query Tools (v3.3.0):**
+- `get_recent_work` - Answer "What did we work on last?" with session grouping
+- `search_by_recency` - Time-constrained search like "docker issues last week"
+- `get_timeline` - Activity timeline with statistics and patterns
+
 **Status & Monitoring Tools:**
 - `get_status` - Real-time import progress and system status
 - `get_health` - Comprehensive system health check
@@ -288,11 +291,15 @@ npm uninstall -g claude-self-reflect
 ## What's New
 
 <details>
-<summary>v2.8.8 - Latest Release</summary>
-
-- **Full Conversation Access**: New `get_full_conversation` tool provides complete JSONL files instead of 200-char excerpts
-- **95% Value Increase**: Agents can now access entire conversations with full implementation details
-- **Direct File Access**: Returns absolute paths for efficient reading with standard tools
+<summary>v3.3.0 - Latest Release</summary>
+
+- **🚀 Major Architecture Overhaul**: Server modularized from 2,321 to 728 lines (68% reduction) for better maintainability
+- **🔧 Critical Bug Fixes**: Fixed 100% CPU usage, store_reflection dimension mismatches, and SearchResult type errors
+- **🕒 New Temporal Tools Suite**: `get_recent_work`, `search_by_recency`, `get_timeline` for time-based search and analysis
+- **🎯 Enhanced UX**: Restored rich formatting with emojis for better readability and information hierarchy
+- **⚡ All 15+ MCP Tools Operational**: Complete functionality with both local and cloud embedding modes
+- **🏗️ Production Infrastructure**: Real-time indexing with smart intervals (2s hot files, 60s normal)
+- **🔍 Enhanced Metadata**: Tool usage analysis, file tracking, and concept extraction for better search
 
 </details>
 

package/mcp-server/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "claude-self-reflect-mcp"
-version = "2.8.9"
+version = "2.8.10"
 description = "MCP server for Claude self-reflection with memory decay"
 # readme = "README.md"
 requires-python = ">=3.10"

package/mcp-server/run-mcp.sh

@@ -11,6 +11,34 @@ SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 # Navigate to the mcp-server directory
 cd "$SCRIPT_DIR"
 
+# CRITICAL: Load .env file from project root if it exists
+# This ensures the MCP server uses the same settings as other scripts
+if [ -f "../.env" ]; then
+    echo "[DEBUG] Loading .env file from project root" >&2
+    set -a  # Export all variables
+    source ../.env
+    set +a  # Stop exporting
+else
+    echo "[DEBUG] No .env file found, using defaults" >&2
+fi
+
+# Set smart defaults if not already set
+# These match what the CLI setup wizard uses
+if [ -z "$QDRANT_URL" ]; then
+    export QDRANT_URL="http://localhost:6333"
+    echo "[DEBUG] Using default QDRANT_URL: $QDRANT_URL" >&2
+fi
+
+if [ -z "$PREFER_LOCAL_EMBEDDINGS" ]; then
+    export PREFER_LOCAL_EMBEDDINGS="true"
+    echo "[DEBUG] Using default PREFER_LOCAL_EMBEDDINGS: true (privacy-first)" >&2
+fi
+
+if [ -z "$ENABLE_MEMORY_DECAY" ]; then
+    export ENABLE_MEMORY_DECAY="false"
+    echo "[DEBUG] Using default ENABLE_MEMORY_DECAY: false" >&2
+fi
+
 # Check if virtual environment exists
 if [ ! -d "venv" ]; then
     echo "Creating virtual environment..."
@@ -63,11 +91,27 @@ if [ -z "$FASTEMBED_SKIP_HUGGINGFACE" ]; then
 fi
 
 # Debug: Show what environment variables are being passed
-echo "[DEBUG] Environment variables for MCP server:"
-echo "[DEBUG] VOYAGE_KEY: ${VOYAGE_KEY:+set}"
-echo "[DEBUG] PREFER_LOCAL_EMBEDDINGS: ${PREFER_LOCAL_EMBEDDINGS:-not set}"
-echo "[DEBUG] QDRANT_URL: ${QDRANT_URL:-not set}"
-echo "[DEBUG] ENABLE_MEMORY_DECAY: ${ENABLE_MEMORY_DECAY:-not set}"
+echo "[DEBUG] Environment variables for MCP server:" >&2
+echo "[DEBUG] VOYAGE_KEY: ${VOYAGE_KEY:+set}" >&2
+echo "[DEBUG] PREFER_LOCAL_EMBEDDINGS: ${PREFER_LOCAL_EMBEDDINGS:-not set}" >&2
+echo "[DEBUG] QDRANT_URL: ${QDRANT_URL:-not set}" >&2
+echo "[DEBUG] ENABLE_MEMORY_DECAY: ${ENABLE_MEMORY_DECAY:-not set}" >&2
+
+# Quick connectivity check for Qdrant
+echo "[DEBUG] Checking Qdrant connectivity at $QDRANT_URL..." >&2
+if command -v curl &> /dev/null; then
+    # Check root endpoint instead of /health which doesn't exist in Qdrant
+    if curl -s -f -m 2 "$QDRANT_URL/" > /dev/null 2>&1; then
+        echo "[DEBUG] ✅ Qdrant is reachable at $QDRANT_URL" >&2
+    else
+        echo "[WARNING] ⚠️  Cannot reach Qdrant at $QDRANT_URL" >&2
+        echo "[WARNING] Common fixes:" >&2
+        echo "[WARNING]   1. Start Qdrant: docker compose up -d qdrant" >&2
+        echo "[WARNING]   2. Check if port is different (e.g., 59999)" >&2
+        echo "[WARNING]   3. Update .env file with correct QDRANT_URL" >&2
+        echo "[WARNING] Continuing anyway - some features may not work..." >&2
+    fi
+fi
 
 # Run the MCP server
 exec python -m src

package/mcp-server/src/app_context.py

@@ -0,0 +1,64 @@
+"""Application context for sharing state across modules."""
+
+from dataclasses import dataclass
+from typing import Optional, Any
+from qdrant_client import AsyncQdrantClient
+try:
+    from .embedding_manager import EmbeddingManager
+    from .decay_manager import DecayManager
+    from .utils import ProjectResolver
+except ImportError:
+    # Fallback for testing
+    EmbeddingManager = None
+    DecayManager = None
+    ProjectResolver = None
+
+@dataclass
+class AppContext:
+    """Shared application context for all MCP tools."""
+    
+    qdrant_client: AsyncQdrantClient
+    embedding_manager: EmbeddingManager
+    decay_manager: DecayManager
+    project_resolver: ProjectResolver
+    
+    # Optional context for debugging
+    debug_context: Optional[Any] = None
+    
+    def __post_init__(self):
+        """Initialize any additional state after dataclass creation."""
+        # Ensure all managers are properly initialized
+        if not self.embedding_manager:
+            self.embedding_manager = EmbeddingManager()
+        
+        if not self.decay_manager:
+            self.decay_manager = DecayManager()
+        
+        if not self.project_resolver:
+            self.project_resolver = ProjectResolver()
+    
+    async def get_all_collections(self) -> list:
+        """Get all collections from Qdrant."""
+        try:
+            collections = await self.qdrant_client.get_collections()
+            return [c.name for c in collections.collections]
+        except Exception as e:
+            if self.debug_context:
+                await self.debug_context.debug(f"Failed to get collections: {e}")
+            return []
+    
+    async def generate_embedding(self, text: str, embedding_type: Optional[str] = None):
+        """Generate embedding using the embedding manager."""
+        # The embedding_manager.embed method is synchronous, not async
+        embeddings = self.embedding_manager.embed(text, input_type="document")
+        if embeddings and len(embeddings) > 0:
+            return embeddings[0]
+        return None
+    
+    def get_current_project(self) -> Optional[str]:
+        """Get current project from resolver."""
+        return self.project_resolver.get_current_project()
+    
+    def normalize_project_name(self, project_name: str) -> str:
+        """Normalize project name using resolver."""
+        return self.project_resolver.normalize_project_name(project_name)

package/mcp-server/src/config.py

@@ -0,0 +1,57 @@
+"""Configuration and environment constants for Claude Self-Reflect MCP server."""
+
+import os
+from pathlib import Path
+from dotenv import load_dotenv
+
+# Load environment variables
+load_dotenv()
+
+# API Keys
+VOYAGE_API_KEY = os.getenv('VOYAGE_API_KEY', '')
+QDRANT_URL = os.getenv('QDRANT_URL', 'http://localhost:6333')
+
+# Embedding Configuration
+PREFER_LOCAL_EMBEDDINGS = os.getenv('PREFER_LOCAL_EMBEDDINGS', 'true').lower() == 'true'
+VOYAGE_MODEL = "voyage-3-lite"
+LOCAL_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
+
+# Decay Configuration
+USE_DECAY = os.getenv('USE_DECAY', 'false').lower() == 'true'
+DECAY_SCALE_DAYS = int(os.getenv('DECAY_SCALE_DAYS', '90'))
+DECAY_WEIGHT = float(os.getenv('DECAY_WEIGHT', '0.3'))
+USE_NATIVE_DECAY = os.getenv('USE_NATIVE_DECAY', 'false').lower() == 'true'
+
+# Search Configuration
+DEFAULT_SEARCH_LIMIT = 5
+MAX_SEARCH_LIMIT = 100
+DEFAULT_MIN_SCORE = 0.3
+
+# Memory Management
+MAX_RESULTS_PER_COLLECTION = int(os.getenv('MAX_RESULTS_PER_COLLECTION', '10'))
+MAX_TOTAL_RESULTS = int(os.getenv('MAX_TOTAL_RESULTS', '1000'))
+MAX_MEMORY_MB = int(os.getenv('MAX_MEMORY_MB', '500'))
+
+# Connection Pool Configuration
+POOL_SIZE = int(os.getenv('QDRANT_POOL_SIZE', '10'))
+POOL_MAX_OVERFLOW = int(os.getenv('QDRANT_POOL_OVERFLOW', '5'))
+POOL_TIMEOUT = float(os.getenv('QDRANT_POOL_TIMEOUT', '30.0'))
+RETRY_ATTEMPTS = int(os.getenv('QDRANT_RETRY_ATTEMPTS', '3'))
+RETRY_DELAY = float(os.getenv('QDRANT_RETRY_DELAY', '1.0'))
+
+# Performance Configuration
+MAX_CONCURRENT_SEARCHES = int(os.getenv('MAX_CONCURRENT_SEARCHES', '10'))
+ENABLE_PARALLEL_SEARCH = os.getenv('ENABLE_PARALLEL_SEARCH', 'true').lower() == 'true'
+
+# Paths
+CLAUDE_PROJECTS_PATH = Path.home() / '.claude' / 'projects'
+CONFIG_PATH = Path.home() / '.claude-self-reflect' / 'config'
+
+# Collection Naming
+VOYAGE_SUFFIX = '_voyage'
+LOCAL_SUFFIX = '_local'
+
+# Logging
+import logging
+logging.basicConfig(level=logging.INFO)
+logger = logging.getLogger(__name__)