claude-mem 3.3.7 → 3.3.9

package/docs/plans/cloud-service-plan.md

@@ -0,0 +1,274 @@
+# Claude-Mem Cloud Service Plan
+
+## Executive Summary
+
+Transform claude-mem into a cloud-hosted SaaS offering at $9.95/month for individual users, with a future team plan at $29.95/month (3-seat minimum). Users will connect multiple Claude Code instances to share a single "brain" (ChromaDB collection) across devices using API keys.
+
+## Service Architecture
+
+### Core Components
+1. **Cloud ChromaDB Cluster** - Dockerized ChromaDB on your Coolify server
+2. **API Gateway** - Node.js/TypeScript authentication & routing service
+3. **Sync Server** - WebSocket-based real-time synchronization
+4. **Billing System** - Stripe integration for subscription management
+5. **Web Dashboard** - Customer portal for key management & usage
+
+## Pricing Structure
+
+### Individual Plan ($9.95/month)
+- 1 API key
+- 100,000 vectors stored
+- 10,000 API requests/month
+- 60 requests/minute rate limit
+- Multi-device sync
+- 30-day data retention
+
+### Team Plan ($29.95/month) - Future
+- 3+ API keys (seats)
+- 500,000 vectors per seat
+- 50,000 API requests/month per seat
+- 120 requests/minute rate limit
+- Admin dashboard
+- 90-day data retention
+- Priority support
+
+## Technical Stack
+
+### Backend Services
+- **Runtime**: Node.js + TypeScript
+- **Framework**: Express.js
+- **Database**: PostgreSQL (users, subscriptions, usage)
+- **Vector DB**: ChromaDB (memory storage)
+- **Cache**: Redis (auth caching, rate limiting)
+- **Queue**: Bull (background jobs)
+- **WebSockets**: Socket.IO (real-time sync)
+
+### Infrastructure (on Coolify)
+- **Deployment**: Docker Compose
+- **SSL**: Traefik (auto Let's Encrypt)
+- **Monitoring**: Prometheus + Grafana
+- **Logging**: Loki + Promtail
+- **Backups**: Automated S3 backups
+
+## API Authentication System
+
+### API Key Structure
+```
+sk_live_[32-byte-random-base64url]
+sk_test_[32-byte-random-base64url]
+```
+
+### Security Features
+- Bcrypt hashed storage
+- Redis-cached validation (5min TTL)
+- Rate limiting per key
+- Automatic revocation on subscription end
+- Audit logging
+
+### Database Schema
+- organizations (subscription entities)
+- subscriptions (billing info)
+- users (team members)
+- api_keys (authentication)
+- api_usage (tracking)
+- api_usage_daily (aggregated metrics)
+
+## Multi-Device Synchronization
+
+### Sync Protocol
+- **Primary**: WebSocket connections
+- **Fallback**: Server-Sent Events
+- **Last Resort**: HTTP polling
+
+### Conflict Resolution
+1. Vector clock tracking for causal ordering
+2. Semantic similarity for auto-merge (>90% similarity)
+3. CRDT for metadata (keywords, tags)
+4. Manual resolution queue for complex conflicts
+
+### Offline Support
+- Local SQLite Write-Ahead Log (WAL)
+- Automatic replay on reconnection
+- Incremental sync from last known state
+
+## Subscription & Billing
+
+### Payment Provider: Stripe
+- 2.9% + $0.30 per transaction
+- Built-in subscription management
+- Smart retry for failed payments
+- Customer portal for self-service
+- Comprehensive webhooks
+
+### Billing Events
+- subscription.created → Provision API key & ChromaDB collection
+- subscription.updated → Adjust limits/seats
+- invoice.payment_failed → Send warning, grace period
+- subscription.deleted → Revoke keys, archive data
+
+### Database Operations
+```sql
+-- Core subscription tracking
+subscriptions (
+  id, organization_id, plan_type, status,
+  seats, api_requests_limit, rate_limit,
+  started_at, ends_at
+)
+```
+
+## Data Isolation & Security
+
+### Collection Naming
+```
+org_${organizationId}_memories
+```
+
+### Access Control
+- API key validates organization ownership
+- Collections prefixed with org ID
+- Row-level security in PostgreSQL
+- Network isolation between tenants
+
+### Compliance
+- GDPR data export/deletion
+- SOC 2 Type I preparation
+- Regular security audits
+- Encrypted data at rest
+
+## Implementation Phases
+
+### Phase 1: Core Infrastructure (Week 1-2)
+- PostgreSQL database setup
+- ChromaDB cluster deployment
+- Redis configuration
+- Basic API server
+
+### Phase 2: Authentication (Week 3)
+- API key generation service
+- Authentication middleware
+- Rate limiting
+- Usage tracking
+
+### Phase 3: Billing Integration (Week 4)
+- Stripe account setup
+- Subscription endpoints
+- Webhook handlers
+- Customer portal
+
+### Phase 4: Synchronization (Week 5-6)
+- WebSocket server
+- Conflict resolution
+- Offline support
+- Device management
+
+### Phase 5: Dashboard (Week 7)
+- Web interface
+- Key management UI
+- Usage analytics
+- Billing management
+
+### Phase 6: Testing & Launch (Week 8)
+- Load testing
+- Security audit
+- Documentation
+- Beta launch
+
+## Monitoring & Operations
+
+### Key Metrics
+- API response time (<100ms p95)
+- Sync latency (<200ms)
+- Uptime (99.9% target)
+- Failed payment rate (<2%)
+- Churn rate (<5% monthly)
+
+### Alerting
+- API errors > 1% (PagerDuty)
+- ChromaDB unreachable (immediate)
+- Payment failures spike (email)
+- Rate limit violations (dashboard)
+
+## Cost Structure
+
+### Infrastructure (Monthly)
+- **Coolify Server**: $0 (existing)
+- **Additional Storage**: ~$20 (100GB)
+- **Bandwidth**: ~$10 (50GB)
+- **Backups (S3)**: ~$5
+- **Monitoring**: ~$0 (self-hosted)
+- **Total**: ~$35/month
+
+### Per Customer Costs
+- **Stripe Fees**: ~$0.59 per customer
+- **Storage**: ~$0.10 per customer
+- **Compute**: ~$0.05 per customer
+- **Total**: ~$0.74 per customer
+
+### Margin Analysis
+- Revenue per customer: $9.95
+- Cost per customer: ~$0.74
+- Gross margin: ~92.5%
+- Break-even: ~4 customers
+
+## Marketing & Growth
+
+### Initial Launch Strategy
+1. Beta with existing claude-mem users
+2. Product Hunt launch
+3. Dev.to/Medium technical articles
+4. Twitter/X developer community
+5. GitHub sponsorship tier
+
+### Pricing Psychology
+- $9.95 under $10 threshold
+- Annual discount (2 months free)
+- Free tier limitations drive upgrades
+- Team pricing encourages expansion
+
+## Risk Mitigation
+
+### Technical Risks
+- **ChromaDB scaling**: Plan sharding strategy
+- **Sync conflicts**: Comprehensive testing
+- **Data loss**: Automated backups, replication
+- **Security breach**: Regular audits, encryption
+
+### Business Risks
+- **Churn**: Focus on activation & retention
+- **Competition**: Unique multi-device sync
+- **Compliance**: Start GDPR/SOC 2 early
+- **Support burden**: Self-service documentation
+
+## Success Metrics
+
+### Month 1 Goals
+- 50 beta users
+- <1% error rate
+- 99% uptime
+- NPS > 50
+
+### Month 6 Goals
+- 500 paying customers
+- $4,975 MRR
+- <5% monthly churn
+- Team plan launch
+
+### Year 1 Goals
+- 2,000 customers
+- $30,000 MRR
+- 20% on team plans
+- Break-even achieved
+
+## Next Steps
+
+1. Deploy ChromaDB cluster on Coolify
+2. Set up PostgreSQL database
+3. Create Stripe account & products
+4. Build authentication service
+5. Implement basic API endpoints
+6. Create landing page
+7. Begin beta testing
+
+## Conclusion
+
+This plan provides a clear path to transform claude-mem into a sustainable SaaS business. The $9.95 price point is accessible while maintaining excellent margins. Multi-device sync provides unique value. Coolify infrastructure keeps costs minimal. Stripe handles complex billing scenarios. The phased approach reduces risk while enabling rapid iteration based on user feedback.

package/docs/plans/fix-response-format-issue.md

@@ -0,0 +1,61 @@
+# Plan: Fix Claude Response Format Issue
+
+## Problem Summary
+The claude-mem system is failing to extract summaries from Claude's responses because:
+- **Expected**: Pipe-separated format with 5 parts: `summary | session_id | keywords | timestamp | archive`
+- **Actual**: Claude returns structured markdown analysis or brief acknowledgments
+- **Result**: 0 summaries extracted, sessions marked as "NO SUMMARIES EXTRACTED"
+
+## Root Cause Analysis
+
+### 1. Prompt Complexity
+The current prompt in `AnalysisTemplates.ts` asks Claude to:
+- Extract entities and create MCP tool calls
+- Create relationships between entities
+- Return pipe-separated summaries
+
+This multi-step process is causing Claude to focus on the analysis portion and not produce the required output format.
+
+### 2. Format Mismatch
+- **Prompt specifies**: 5-part pipe-separated format
+- **Parser checks for**: 3+ parts (mismatch in validation)
+- **Claude returns**: Markdown headers and structured text
+
+### 3. Recent Changes
+A commit on 2025-09-03 updated the output format specification, but the parser wasn't synchronized with the new requirements.
+
+## Solution Approach
+
+### Primary Fix: Simplify the Prompt
+Remove the MCP tool integration from the compression prompt and focus solely on generating pipe-separated summaries. The MCP tools can be called separately if needed.
+
+### Secondary Fix: Align Parser with Format
+Update the parser to validate exactly 5 pipe-separated parts instead of 3+.
+
+## Implementation Steps
+
+1. **Modify `AnalysisTemplates.ts`**
+   - Remove MCP tool instructions from the prompt
+   - Clarify that ONLY pipe-separated lines should be returned
+   - Simplify the prompt to focus on summary generation
+
+2. **Update `TranscriptCompressor.ts`**
+   - Fix parser to validate exactly 5 parts
+   - Add better error logging for format mismatches
+   - Consider fallback parsing for partial matches
+
+3. **Test the Changes**
+   - Run compression on test transcripts
+   - Verify pipe-separated output is generated
+   - Ensure summaries are properly extracted and indexed
+
+## Risk Assessment
+- **Low Risk**: Changes are isolated to prompt and parser
+- **Impact**: Will fix summary extraction for all future sessions
+- **Rollback**: Easy to revert if issues arise
+
+## Success Criteria
+- Claude returns pipe-separated format consistently
+- Parser extracts 3-10 summaries per session
+- No more "NO SUMMARIES EXTRACTED" errors
+- Archive files contain meaningful summaries

package/docs/plans/restructure-session-hook-output.md

@@ -0,0 +1,102 @@
+# Plan: Restructure Session Start Hook Output
+
+## Overview
+Restructure the session start hook to display a cleaner user-facing view with last 3 overviews and time-ago data, while preserving all granular memories in the transcript-level context.
+
+## Requirements
+1. Keep granular memories in transcript level output (hidden from user but still in context)
+2. Show only the last 3 overviews with "time ago" data to user
+3. Preserve all context for Claude while presenting cleaner interface to user
+
+## Current State Analysis
+- Session hook shows both overviews and detailed memory entries
+- All output is visible to user via console.log
+- Memories grouped by session with detailed keywords
+- Single overview shown at top when available
+
+## Target Architecture
+
+### Output Layers
+1. **User-Visible Layer**: Clean display with last 3 overviews + time-ago
+2. **Transcript Layer**: Hidden HTML comments containing granular memories
+3. **Context Preservation**: Both layers remain in Claude's context
+
+### Data Structure
+```
+[User Visible]
+🧠 Recent Context: [Current DateTime]
+====================================================================
+📅 2 hours ago:
+[Overview content from session 1]
+
+📅 5 hours ago:  
+[Overview content from session 2]
+
+📅 Yesterday:
+[Overview content from session 3]
+====================================================================
+
+[Hidden in HTML Comments - Transcript Level]
+<!-- CONTEXT-DATA
+Granular memories for session analysis:
+- Memory 1: [details] (keywords)
+- Memory 2: [details] (keywords)
+...
+END-CONTEXT-DATA -->
+```
+
+## Implementation Steps
+
+### Phase 1: Data Structure Enhancement
+1. Modify `loadContext()` to fetch more overview entries
+2. Create separate data paths for overviews vs memories
+3. Add timestamp metadata to overview objects
+
+### Phase 2: Time-Ago Calculation
+1. Implement relative time formatter utility
+2. Parse overview timestamps 
+3. Calculate time differences from current time
+
+### Phase 3: Template Restructuring
+1. Create new template for user-visible overviews
+2. Add HTML comment wrapper for granular memories
+3. Split rendering logic for dual-layer output
+
+### Phase 4: Output Formatting
+1. Modify `renderSessionStartTemplate()` to handle dual outputs
+2. Implement HTML comment injection for transcript data
+3. Ensure proper escaping and formatting
+
+## Technical Details
+
+### Files to Modify
+1. `/src/commands/load-context.ts` - Data loading logic
+2. `/src/prompts/templates/context/ContextTemplates.ts` - Template definitions
+3. `/src/commands/hooks.ts` - Hook orchestration
+4. `/src/lib/time-utils.ts` - New file for time calculations
+
+### Key Functions to Update
+- `extractOverview()` → `extractOverviews()` (plural, fetch 3)
+- `processMemoryEntries()` - Keep as-is for transcript data
+- `renderSessionStartTemplate()` - Add dual-layer rendering
+- New: `formatRelativeTime()` - Time-ago formatting
+
+### Data Flow Changes
+```
+JSONL → Parse → 
+├── Extract 3 Overviews → Add timestamps → Format time-ago → User display
+└── Extract Memories → Group by session → Wrap in comments → Transcript level
+```
+
+## Risk Mitigation
+- Test HTML comment escaping to prevent injection
+- Ensure backward compatibility with existing memory format
+- Handle missing timestamps gracefully
+- Preserve existing error handling
+
+## Success Criteria
+1. User sees clean display with only 3 recent overviews
+2. Each overview shows relative time (e.g., "2 hours ago")
+3. Granular memories preserved in HTML comments
+4. Claude maintains full context from both layers
+5. No regression in existing functionality

package/docs/plans/session-start-hook-investigation.md

@@ -0,0 +1,45 @@
+# Session Start Hook Investigation Plan
+
+## Problem Statement
+After running `/compact` to compress the latest conversation into Weaviate, the session start hook is not loading the latest updates from the claude-mem-source project. The compaction appears successful but the context retrieval is not working as expected.
+
+## Investigation Areas
+
+### 1. Session Start Hook Implementation
+- Examine `/Users/alexnewman/.claude-mem/hooks/session-start.js`
+- Understand how it queries Weaviate for relevant context
+- Check what search terms it uses to find relevant entities
+
+### 2. Entity Naming Convention 
+- Verify entities are being stored with correct naming pattern: `[project]_[entityType]_[entityName]`
+- Check if project name is correctly identified as "claude-mem-source" vs "claude-mem"
+- Validate entity names in Weaviate match expected format
+
+### 3. Compression Process
+- Review how TranscriptCompressor extracts project name from conversation
+- Verify entities are being created with proper project prefix
+- Check if pre-compact hook is correctly processing the transcript
+
+### 4. Weaviate Data Verification
+- Query Weaviate directly to see what entities exist
+- Check for entities with "claude-mem-source" prefix
+- Verify entities from recent improvements are stored
+
+## Success Criteria
+- Session start hook retrieves context about recent code improvements
+- Entities are correctly prefixed with "claude-mem-source"
+- Context loading works consistently across session restarts
+
+## Parallel Agent Tasks
+
+### Agent 1: Hook Implementation Analysis
+Investigate session-start.js to understand retrieval logic
+
+### Agent 2: Weaviate Data Inspection  
+Check what entities exist in Weaviate and their naming patterns
+
+### Agent 3: Compression Process Verification
+Trace how project names are extracted and used in entity creation
+
+## Expected Outcome
+Identify root cause and implement fix to ensure session start hook properly loads context from the claude-mem-source project.

package/docs/plans/src-reorganization-plan.md

@@ -0,0 +1,181 @@
+# Source Folder Reorganization Plan
+
+## Overview
+
+This plan reorganizes the `src/` folder to improve separation of concerns, eliminate misleading folder names, and create a cleaner, more maintainable architecture.
+
+## Current Issues
+
+1. **Mixed concerns at root level**: Entry points (`cli.ts`, `mcp-server*.ts`) mixed with shared modules
+2. **Misleading "utils" folder**: Contains core business logic instead of utilities
+3. **Large constants.ts file**: 455 lines mixing different concerns
+4. **Unclear prompts organization**: Example files mixed with templates and core logic
+5. **No clear infrastructure/domain separation**
+
+## Target Structure
+
+```
+src/
+├── bin/                          # Entry points only
+│   ├── cli.ts
+│   ├── mcp-server.ts
+│   └── mcp-server-cli.ts
+├── commands/                     # CLI commands (unchanged)
+│   ├── compress.ts
+│   ├── install.ts
+│   ├── load-context.ts
+│   ├── logs.ts
+│   ├── status.ts
+│   └── uninstall.ts
+├── core/                         # Core business logic
+│   ├── compression/
+│   │   └── TranscriptCompressor.ts
+│   └── orchestration/
+│       └── PromptOrchestrator.ts
+├── storage/                      # Data persistence layer
+│   ├── adapters/
+│   │   └── weaviate-mcp-adapter.ts
+│   └── clients/
+│       └── mcp-client.ts
+├── prompts/                      # Template system
+│   ├── templates/
+│   │   ├── analysis/
+│   │   │   └── AnalysisTemplates.ts
+│   │   ├── context/
+│   │   │   └── ContextTemplates.ts
+│   │   └── hooks/
+│   │       ├── HookTemplates.ts
+│   │       └── HookTemplates.test.ts
+│   ├── constants.ts             # Prompt-specific constants (split from main)
+│   └── index.ts                  # Main exports
+├── shared/                       # Shared infrastructure
+│   ├── config.ts
+│   ├── types.ts
+│   ├── paths.ts                 # Renamed from PathResolver
+│   ├── logger.ts
+│   └── error-handler.ts
+└── constants.ts                  # Core app constants only
+```
+
+## Migration Steps
+
+### Phase 1: Create Directory Structure
+
+```bash
+mkdir -p src/bin
+mkdir -p src/core/compression
+mkdir -p src/core/orchestration
+mkdir -p src/storage/adapters
+mkdir -p src/storage/clients
+mkdir -p src/shared
+mkdir -p src/prompts/templates/analysis
+mkdir -p src/prompts/templates/context
+mkdir -p src/prompts/templates/hooks
+```
+
+### Phase 2: Move Files
+
+| From | To | Reason |
+|------|-----|---------|
+| `src/cli.ts` | `src/bin/cli.ts` | Separate entry points |
+| `src/mcp-server.ts` | `src/bin/mcp-server.ts` | Separate entry points |
+| `src/mcp-server-cli.ts` | `src/bin/mcp-server-cli.ts` | Separate entry points |
+| `src/utils/TranscriptCompressor.ts` | `src/core/compression/TranscriptCompressor.ts` | Core business logic |
+| `src/prompts/PromptOrchestrator.ts` | `src/core/orchestration/PromptOrchestrator.ts` | Core business logic |
+| `src/utils/weaviate-mcp-adapter.ts` | `src/storage/adapters/weaviate-mcp-adapter.ts` | Storage layer |
+| `src/utils/mcp-client.ts` | `src/storage/clients/mcp-client.ts` | Storage layer |
+| `src/utils/PathResolver.ts` | `src/shared/paths.ts` | Shared utility, clearer name |
+| `src/utils/logger.ts` | `src/shared/logger.ts` | Shared infrastructure |
+| `src/config.ts` | `src/shared/config.ts` | Shared configuration |
+| `src/types.ts` | `src/shared/types.ts` | Shared types |
+| `src/error-handler.ts` | `src/shared/error-handler.ts` | Shared error handling |
+| `src/prompts/templates/AnalysisTemplates.ts` | `src/prompts/templates/analysis/AnalysisTemplates.ts` | Template organization |
+| `src/prompts/templates/ContextTemplates.ts` | `src/prompts/templates/context/ContextTemplates.ts` | Template organization |
+| `src/prompts/templates/HookTemplates.ts` | `src/prompts/templates/hooks/HookTemplates.ts` | Template organization |
+| `src/prompts/templates/HookTemplates.test.ts` | `src/prompts/templates/hooks/HookTemplates.test.ts` | Keep tests with source |
+
+### Phase 3: Delete Unnecessary Example Files
+
+```bash
+rm src/prompts/examples.ts
+rm src/prompts/templates/context-examples.ts
+rm src/prompts/templates/example-usage.ts
+rm src/prompts/templates/hook-integration-examples.ts
+```
+
+### Phase 4: Split Constants File
+
+The large `constants.ts` file needs to be split:
+- Core constants stay in `src/constants.ts`
+- Prompt-related constants move to `src/prompts/constants.ts`
+
+### Phase 5: Update Imports
+
+All imports will need to be updated to reflect the new structure. TypeScript path aliases should be added to `tsconfig.json`:
+
+```json
+{
+  "compilerOptions": {
+    "baseUrl": "./src",
+    "paths": {
+      "@/bin/*": ["bin/*"],
+      "@/commands/*": ["commands/*"],
+      "@/core/*": ["core/*"],
+      "@/storage/*": ["storage/*"],
+      "@/prompts/*": ["prompts/*"],
+      "@/shared/*": ["shared/*"],
+      "@/*": ["*"]
+    }
+  }
+}
+```
+
+### Phase 6: Clean Up
+
+```bash
+rm -rf src/utils
+```
+
+## Import Update Examples
+
+### Before
+```typescript
+import { TranscriptCompressor } from './utils/TranscriptCompressor.js';
+import { WeaviateMCPAdapter } from './utils/weaviate-mcp-adapter.js';
+import { log } from './utils/logger.js';
+import { CONSTANTS } from './constants.js';
+```
+
+### After
+```typescript
+import { TranscriptCompressor } from '@/core/compression/TranscriptCompressor.js';
+import { WeaviateMCPAdapter } from '@/storage/adapters/weaviate-mcp-adapter.js';
+import { log } from '@/shared/logger.js';
+import { CONSTANTS } from '@/constants.js';
+```
+
+## Benefits
+
+1. **Clear separation of concerns**: Each folder has a single, clear purpose
+2. **No misleading names**: "utils" eliminated, all folders accurately describe contents
+3. **Better scalability**: Room for growth in each module
+4. **Improved maintainability**: Related code is co-located
+5. **Cleaner imports**: Path aliases make dependencies clear
+6. **Easier navigation**: Developers can quickly find what they need
+
+## Risk Assessment
+
+- **Low Risk**: Directory creation, file moves
+- **Medium Risk**: Import updates, constants file splitting
+- **High Risk**: None identified
+
+## Testing Strategy
+
+1. Run TypeScript compiler after each phase to catch import errors
+2. Execute test suite after migration
+3. Test CLI commands manually
+4. Verify MCP server functionality
+
+## Rollback Plan
+
+Keep git commits atomic for each phase to enable easy rollback if needed.

package/docs/plans/terminal-effects-decision.md

@@ -0,0 +1,22 @@
+# Terminal Effects Decision
+
+## Technical Reality
+- `terminaltexteffects` is Python-only
+- Node.js cannot directly execute Python packages
+- Integration would require complex subprocess management
+
+## Current State
+- Installer already has excellent visual feedback via `@clack/prompts`
+- System works reliably and provides clear user guidance
+- No user complaints about current interface
+
+## Decision
+Per the "Make It Work First" manifesto: **No cosmetic enhancements**.
+
+The existing installer does its job. Adding visual effects would be:
+- Defensive coding against a non-problem
+- Complexity that hides core functionality
+- Violation of KISS principles
+
+## Action
+Keep current installer as-is. Focus on functionality, not appearance.