claude-self-reflect 2.5.2 → 2.5.3

package/README.md

@@ -26,18 +26,9 @@ Your conversations become searchable. Your decisions stay remembered. Your conte
 - **Node.js** 16+ (for the setup wizard)
 - **Claude Desktop** app
 
-## System Requirements
+## Quick Install
 
-### Memory
-- **Docker Memory**: 2GB minimum (4GB recommended for initial setup)
-- **First Import**: May take 2-7 minutes to process all conversations
-- **Subsequent Imports**: <60 seconds (only processes new/changed files)
-
-💡 **First-Time User Note**: The initial import processes your entire conversation history. This is a one-time operation. After that, the system only imports new conversations, making it much faster and using less memory.
-
-## Install
-
-### Quick Start (Local Mode - Default)
+### Local Mode (Default - Your Data Stays Private)
 ```bash
 # Install and run automatic setup
 npm install -g claude-self-reflect
@@ -50,8 +41,6 @@ claude-self-reflect setup
 # ✅ Start monitoring for new conversations
 # ✅ Verify the reflection tools work
 # 🔒 Keep all data local - no API keys needed
-# 🚀 Import watcher runs every 60 seconds
-# ⚡ Memory decay enabled by default (90-day half-life)
 ```
 
 ### Cloud Mode (Better Search Accuracy)
@@ -67,15 +56,6 @@ claude-self-reflect setup --voyage-key=YOUR_ACTUAL_KEY_HERE
 
 5 minutes. Everything automatic. Just works.
 
-### 🔒 Privacy & Data Exchange
-
-| Mode | Data Storage | External API Calls | Data Sent | Search Quality |
-|------|--------------|-------------------|-----------|----------------|
-| **Local (Default)** | Your machine only | None | Nothing leaves your computer | Good - uses efficient local embeddings |
-| **Cloud (Opt-in)** | Your machine | Voyage AI | Conversation text for embedding generation | Better - uses state-of-the-art models |
-
-**Note**: Cloud mode sends conversation content to Voyage AI for processing. Review their [privacy policy](https://www.voyageai.com/privacy) before enabling.
- 
 ## The Magic
 
 ![Self Reflection vs The Grind](docs/images/red-reflection.webp)
@@ -103,25 +83,65 @@ Claude: "3 conversations found:
         - Nov 20: Added rate limiting per authenticated connection"
 ```
 
-## The Secret Sauce: Sub-Agents
-
-Here's what makes this magical: **The Reflection Specialist sub-agent**.
-
-When you ask about past conversations, Claude doesn't search in your main chat. Instead, it spawns a specialized sub-agent that:
-- Searches your conversation history in its own context
-- Brings back only the relevant results
-- Keeps your main conversation clean and focused
-
-**Your main context stays pristine**. No clutter. No token waste.
-
-![Reflection Agent in Action](docs/images/Reflection-specialist.png)
-
-## How It Works (10 Second Version)
+## How It Works
 
 Your conversations → Vector embeddings → Semantic search → Claude remembers
 
 Technical details exist. You don't need them to start.
 
+## Import Architecture
+
+Here's how your conversations get imported and prioritized:
+
+```mermaid
+flowchart TB
+    subgraph "Real-Time Import System"
+        NewFile([New/Modified<br/>Conversation]) --> Watcher{Streaming<br/>Watcher}
+        
+        Watcher -->|Every 60s| Check[Check Files]
+        Check --> Priority{File Age?}
+        
+        Priority -->|< 5 min| Hot[🔥 HOT<br/>2-sec mode]
+        Priority -->|< 24 hr| Warm[🌡️ WARM<br/>Normal]
+        Priority -->|> 24 hr| Cold[❄️ COLD<br/>Batch-5]
+        
+        Hot --> FastMode[Fast Mode<br/>2-sec intervals<br/>Can interrupt]
+        Warm --> NormalMode[Normal Mode<br/>60-sec intervals]
+        Cold --> BatchMode[Batch Mode<br/>Max 5 files]
+        
+        FastMode --> Import{Import<br/>Strategy}
+        NormalMode --> Import
+        BatchMode --> Import
+        
+        Import -->|Never seen| Full[Baseline Import<br/>From start]
+        Import -->|Gap > 100 lines| Catchup[Catch-up Import<br/>From position]
+        Import -->|Small changes| Stream[Stream Import<br/>Continue]
+        
+        Full --> Process[Process<br/>& Embed]
+        Catchup --> Process
+        Stream --> Process
+        
+        Process --> Memory{Memory<br/>< 50MB?}
+        Memory -->|Yes| Store[(Qdrant<br/>Vector DB)]
+        Memory -->|No| GC[Garbage<br/>Collect]
+        
+        GC --> Store
+        Store --> State[Update State]
+        State --> Wait[Wait for<br/>next cycle]
+        Wait --> Watcher
+        
+        style Hot fill:#ff6b6b
+        style Warm fill:#ffd93d
+        style Cold fill:#6bcf7f
+        style FastMode stroke:#ff6b6b,stroke-width:3px
+    end
+```
+
+**The system intelligently prioritizes your conversations:**
+- **🔥 HOT** (< 5 minutes): Switches to 2-second intervals for near real-time import
+- **🌡️ WARM** (< 24 hours): Normal priority, processed every 60 seconds
+- **❄️ COLD** (> 24 hours): Batch processed, max 5 per cycle to prevent blocking
+
 ## Using It
 
 Once installed, just talk naturally:
@@ -132,54 +152,10 @@ Once installed, just talk naturally:
 
 The reflection specialist automatically activates. No special commands needed.
 
-## Performance & Usage Guide
-
-### 🚀 Lightning Fast Search
-Optimized to deliver results in **200-350ms** (10-40x faster than v2.4.4)
-
-### 🎯 Recommended Usage: Through Reflection-Specialist Agent
-
-**Why use the agent instead of direct MCP tools?**
-- **Preserves your main conversation context** - Search results don't clutter your working memory
-- **Rich formatted responses** - Clean markdown instead of raw XML in your conversation
-- **Better user experience** - Real-time streaming feedback and progress indicators
-- **Proper tool counting** - Shows actual tool usage instead of "0 tool uses"
-- **Automatic cross-project search** - Agent suggests searching across projects when relevant
-- **Specialized search tools** - Access to quick_search, search_summary, and pagination
-
-**Context Preservation Benefit:**
-When you use the reflection-specialist agent, all the search results and processing happen in an isolated context. This means:
-- Your main conversation stays clean and focused
-- No XML dumps or raw data in your chat history
-- Multiple searches won't exhaust your context window
-- You get just the insights, not the implementation details
-
-**Example:**
-```
-You: "What Docker issues did we solve?"
-[Claude automatically spawns reflection-specialist agent]
-⏺ reflection-specialist(Search Docker issues)
-  ⎿ Searching 57 collections...
-  ⎿ Found 5 relevant conversations
-  ⎿ Done (1 tool use · 12k tokens · 2.3s)
-[Returns clean, formatted insights without cluttering your context]
-```
-
-### ⚡ Performance Baselines
-
-| Method | Search Time | Total Time | Context Impact | Best For |
-|--------|------------|------------|----------------|----------|
-| Direct MCP | 200-350ms | 200-350ms | Uses main context | Programmatic use, when context space matters |
-| Via Agent | 200-350ms | 24-30s* | Isolated context | Interactive use, exploration, multiple searches |
-
-*Note: The 24-30s includes context preservation overhead, which keeps your main conversation clean
-
-**Note**: The specialized tools (`quick_search`, `search_summary`, `get_more_results`) only work through the reflection-specialist agent due to MCP protocol limitations.
-
 ## Key Features
 
 ### 🎯 Project-Scoped Search
-Searches are **project-aware by default** (v2.4.3+). Claude automatically searches within your current project:
+Searches are **project-aware by default**. Claude automatically searches within your current project:
 
 ```
 # In ~/projects/MyApp
@@ -191,133 +167,23 @@ You: "Search all projects for WebSocket implementations"
 Claude: [Searches across ALL your projects]
 ```
 
-| Search Scope | How to Trigger | Example |
-|------------|----------------|---------|
-| Current Project (default) | Just ask normally | "What did we discuss about caching?" |
-| All Projects | Say "all projects" | "Search all projects for error handling" |
-| Specific Project | Name the project | "Find auth code in MyApp project" |
-
-## Memory Decay
-
+### ⏱️ Memory Decay
 Recent conversations matter more. Old ones fade. Like your brain, but reliable.
 
-Works perfectly out of the box. [Configure if you're particular](docs/memory-decay.md).
-
-## Theoretical Foundation
-
-Claude Self-Reflect addresses the "reality gap" in AI memory systems - the distance between perfect recall expectations and practical utility. Our approach aligns with the SPAR Framework (Sense, Plan, Act, Reflect) for agentic AI systems. [Learn more about our design philosophy](docs/architecture/SPAR-alignment.md).
+### 🚀 Performance
+- **Search**: 200-350ms response time
+- **Import**: 2-second response for new conversations
+- **Memory**: 50MB operational target with smart chunking
 
-## For the Skeptics
-
-**"Just use grep"** - Sure, enjoy your 10,000 matches for "database"  
-**"Overengineered"** - Two functions: store_reflection, reflect_on_past  
-**"Another vector DB"** - Yes, because semantic > string matching
-
-Built by developers tired of re-explaining context every conversation.
-
-## Requirements
-
-- Claude Code or Claude Desktop
-- Docker Desktop (macOS/Windows) or Docker Engine (Linux)
-- Node.js 16+ (for the setup wizard only)
-- 5 minutes for setup
-
-## Upgrading from Earlier Versions
-
-**v2.4.0+ includes major improvements:**
-- **Docker-Only Setup**: No more Python environment issues!
-- **Privacy First**: Local embeddings by default - your data never leaves your machine
-- **Smarter Setup**: Handles existing installations gracefully
-- **Better Security**: Automated vulnerability scanning
-- **Real-Time Import**: Watcher checks for new conversations every 60 seconds
-- **Fixed MCP Server**: Now uses correct server implementation with local embedding support
-
-**To upgrade:**
-```bash
-npm update -g claude-self-reflect
-claude-self-reflect setup  # Re-run setup, it handles everything
-```
-
-The setup wizard now detects and fixes common upgrade issues automatically. Your existing conversations remain searchable.
-
-## Advanced Setup
-
-Want to customize? See [Configuration Guide](docs/installation-guide.md).
-
-## The Technical Stuff
-
-If you must know:
+## The Technical Stack
 
 - **Vector DB**: Qdrant (local, your data stays yours)
 - **Embeddings**: 
-  - Local (Default): FastEmbed with sentence-transformers/all-MiniLM-L6-v2
-  - Cloud (Optional): Voyage AI (200M free tokens/month)
+  - Local (Default): FastEmbed with all-MiniLM-L6-v2
+  - Cloud (Optional): Voyage AI
 - **MCP Server**: Python + FastMCP
 - **Search**: Semantic similarity with time decay
 
-Both embedding options work well. Local mode uses FastEmbed for privacy and offline use. Cloud mode uses Voyage AI for enhanced accuracy when internet is available. We are not affiliated with Voyage AI.
-
-### Want More Details?
-
-- [Architecture Deep Dive](docs/architecture-details.md) - How it actually works
-- [Components Guide](docs/components.md) - Each piece explained
-- [Why We Built This](docs/motivation-and-history.md) - The full story
-- [Advanced Usage](docs/advanced-usage.md) - Power user features
-
-## Security
-
-### Container Security Notice
-⚠️ **Known Vulnerabilities**: Our Docker images are continuously monitored by Snyk and may show vulnerabilities in base system libraries. We want to be transparent about this:
-
-- **Why they exist**: We use official Python Docker images based on Debian stable, which prioritizes stability over latest versions
-- **Actual risk is minimal** because:
-  - Most CVEs are in unused system libraries or require local access
-  - Security patches are backported by Debian (version numbers don't reflect patches)
-  - Our containers run as non-root users with minimal permissions
-  - This is a local-only tool with no network exposure
-- **What we're doing**: Regular updates, security monitoring, and evaluating alternative base images
-
-**For production or security-sensitive environments**, consider:
-- Building your own hardened images
-- Running with additional security constraints (see below)
-- Evaluating if the tool meets your security requirements
-
-For maximum security:
-```bash
-# Run containers with read-only root filesystem
-docker run --read-only --tmpfs /tmp claude-self-reflect
-```
-
-### Privacy & Data Security
-- **Local by default**: Your conversations never leave your machine unless you explicitly enable cloud embeddings
-- **No telemetry**: We don't track usage or collect any data
-- **Secure storage**: All data stored in Docker volumes with proper permissions
-- **API keys**: Stored in .env file with 600 permissions (read/write by owner only)
-
-See our [Security Policy](SECURITY.md) for vulnerability reporting and more details.
-
-## ⚠️ Important Disclaimers
-
-### Tool Operation
-- **Resource Usage**: The import process can be CPU and memory intensive, especially during initial import of large conversation histories
-- **Data Processing**: This tool reads and indexes your Claude conversation files. Ensure you have adequate disk space
-- **No Warranty**: This software is provided "AS IS" under the MIT License, without warranty of any kind
-- **Data Responsibility**: You are responsible for your conversation data and any API keys used
-
-### Limitations
-- **Not Official**: This is a community tool, not officially supported by Anthropic
-- **Experimental Features**: Some features like memory decay are experimental and may change
-- **Import Delays**: Large conversation histories may take significant time to import initially
-- **Docker Dependency**: Requires Docker to be running, which uses system resources
-
-### Best Practices
-- **Backup Your Data**: Always maintain backups of important conversations
-- **Monitor Resources**: Check Docker resource usage if you experience system slowdowns
-- **Test First**: Try with a small subset of conversations before full import
-- **Review Logs**: Check import logs if conversations seem missing
-
-By using this tool, you acknowledge these disclaimers and limitations.
-
 ## Problems?
 
 - [Troubleshooting Guide](docs/troubleshooting.md)
@@ -326,69 +192,30 @@ By using this tool, you acknowledge these disclaimers and limitations.
 
 ## What's New
 
-### Recent Updates
-- **v2.4.5** - 10-40x performance boost, context preservation
-- **v2.4.3** - Project-scoped search (breaking change) 
-- **v2.3.7** - Local embeddings by default for privacy
+- **v2.5.2** - State file compatibility fix
+- **v2.4.5** - 10-40x performance boost
+- **v2.4.3** - Project-scoped search
+- **v2.3.7** - Local embeddings by default
 
-📚 [Full Release History](docs/release-history.md) | 💬 [Discussions](https://github.com/ramakay/claude-self-reflect/discussions)
+[Full changelog](docs/release-history.md)
 
-## Contributing
+## Advanced Topics
 
-See our [Contributing Guide](CONTRIBUTING.md) for development setup and guidelines.
+- [Performance tuning](docs/performance-guide.md)
+- [Security & privacy](docs/security.md)
+- [Windows setup](docs/windows-setup.md)
+- [Architecture details](docs/architecture-details.md)
+- [Contributing](CONTRIBUTING.md)
 
-### Releasing New Versions (Maintainers)
+## Contributors
 
-Since our GitHub Actions automatically publish to npm, the release process is simple:
-
-```bash
-# 1. Ensure you're logged into GitHub CLI
-gh auth login  # Only needed first time
-
-# 2. Create and push a new tag
-git tag v2.3.0  # Use appropriate version number
-git push origin v2.3.0
-
-# 3. Create GitHub release (this triggers npm publish)
-gh release create v2.3.0 \
-  --title "Release v2.3.0" \
-  --notes-file CHANGELOG.md \
-  --draft=false
-
-# The GitHub Action will automatically:
-# - Build the package
-# - Run tests
-# - Publish to npm
-# - Update release assets
-```
-
-Monitor the release at: https://github.com/ramakay/claude-self-reflect/actions
+Special thanks to our contributors:
+- **[@TheGordon](https://github.com/TheGordon)** - Fixed timestamp parsing (#10)
+- **[@akamalov](https://github.com/akamalov)** - Ubuntu WSL insights
+- **[@kylesnowschwartz](https://github.com/kylesnowschwartz)** - Security review (#6)
 
 ---
 
 Stop reading. Start installing. Your future self will thank you.
 
-## Contributors & Acknowledgments
-
-Special thanks to our contributors and security researchers:
-
-- **[@TheGordon](https://github.com/TheGordon)** - Fixed critical timestamp parsing bug for Claude conversation exports (#10)
-- **[@akamalov](https://github.com/akamalov)** - Highlighted Ubuntu WSL bug and helped educate about filesystem nuances
-- **[@kylesnowschwartz](https://github.com/kylesnowschwartz)** - Comprehensive security review leading to v2.3.3+ security improvements (#6)
-
-## Windows Configuration
-
-### Recommended: Use WSL
-For the best experience on Windows, we recommend using WSL (Windows Subsystem for Linux) which provides native Linux compatibility for Docker operations.
-
-### Alternative: Native Windows
-If using Docker Desktop on native Windows, you need to adjust the CONFIG_PATH in your `.env` file to use Docker-compatible paths:
-
-```bash
-# Replace USERNAME with your Windows username
-CONFIG_PATH=/c/Users/USERNAME/.claude-self-reflect/config
-```
-
-This ensures Docker can properly mount the config directory. The setup wizard creates the directory, but Windows users need to update the path format for Docker compatibility.
-
 MIT License. Built with ❤️ for the Claude community.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-self-reflect",
-  "version": "2.5.2",
+  "version": "2.5.3",
   "description": "Give Claude perfect memory of all your conversations - Installation wizard for Python MCP server",
   "keywords": [
     "claude",