@fjell/cache 4.6.22 → 4.7.2

package/CONFIGURATION_GUIDE.md

@@ -0,0 +1,167 @@
+# Cache Configuration Quick Reference
+
+This guide provides quick reference configurations for the three most common cache types in fjell-cache.
+
+## 1. Memory Cache Configuration
+
+**Best for**: Fast access, temporary data, development environments
+
+```typescript
+import { createInstanceFactory, Options } from '@fjell/cache';
+
+const memoryOptions: Partial<Options<User, 'user'>> = {
+  cacheType: 'memory',
+  memoryConfig: {
+    maxItems: 1000,           // Store maximum 1000 items
+    ttl: 300000               // 5 minutes expiration
+  },
+  enableDebugLogging: true,   // Enable detailed logging
+  autoSync: true,             // Automatically sync with API
+  maxRetries: 3,              // Retry failed operations 3 times
+  retryDelay: 1000,           // Wait 1 second between retries
+  ttl: 600000   // 10 minutes default expiration
+};
+
+const factory = createInstanceFactory(api, memoryOptions);
+const cache = factory(coordinate, { registry });
+```
+
+**Key Features**:
+- No persistence (lost on app restart)
+- Fast access times
+- Configurable memory limits
+- Automatic TTL expiration
+
+## 2. IndexedDB Configuration
+
+**Best for**: Large datasets, offline capability, persistent storage
+
+```typescript
+const indexedDBOptions: Partial<Options<User, 'user'>> = {
+  cacheType: 'indexedDB',
+  indexedDBConfig: {
+    dbName: 'UserAppCache',     // Database name
+    version: 2,                 // Database version (increment for schema changes)
+    storeName: 'users'          // Object store name
+  },
+  enableDebugLogging: false,    // Disable debug logging in production
+  autoSync: true,               // Keep data synchronized
+  maxRetries: 5,                // More retries for async operations
+  retryDelay: 2000,             // Longer delay for database operations
+  ttl: 1800000    // 30 minutes default expiration
+};
+
+const factory = createInstanceFactory(api, indexedDBOptions);
+const cache = factory(coordinate, { registry });
+```
+
+**Key Features**:
+- Persistent storage (survives browser restart)
+- Large storage capacity (hundreds of MB+)
+- Asynchronous operations
+- Structured database with versioning
+
+## 3. localStorage Configuration
+
+**Best for**: User preferences, settings, moderate-sized persistent data
+
+```typescript
+const localStorageOptions: Partial<Options<User, 'user'>> = {
+  cacheType: 'localStorage',
+  webStorageConfig: {
+    keyPrefix: 'myapp:users:',  // Namespace to avoid key conflicts
+    compress: true              // Enable compression to save space
+  },
+  enableDebugLogging: false,    // Usually disabled in production
+  autoSync: false,              // Manual sync for better control
+  maxRetries: 2,                // Fewer retries for localStorage (usually fast)
+  retryDelay: 500,              // Quick retry for localStorage operations
+  ttl: 7200000    // 2 hours default expiration
+};
+
+const factory = createInstanceFactory(api, localStorageOptions);
+const cache = factory(coordinate, { registry });
+```
+
+**Key Features**:
+- Persistent storage (survives browser restart)
+- ~5-10MB storage limit
+- Synchronous operations
+- Optional compression
+- Key namespacing for conflict avoidance
+
+## Environment-Based Auto-Configuration
+
+Automatically select the optimal cache type based on the runtime environment:
+
+```typescript
+function createOptimalCacheConfiguration(): Partial<Options<User, 'user'>> {
+  // Browser with IndexedDB support
+  if (typeof window !== 'undefined' && 'indexedDB' in window) {
+    return {
+      cacheType: 'indexedDB',
+      indexedDBConfig: {
+        dbName: 'OptimalCache',
+        version: 1,
+        storeName: 'items'
+      },
+      enableDebugLogging: false,
+      maxRetries: 5,
+      retryDelay: 2000
+    };
+  }
+
+  // Browser with localStorage
+  if (typeof window !== 'undefined' && 'localStorage' in window) {
+    return {
+      cacheType: 'localStorage',
+      webStorageConfig: {
+        keyPrefix: 'optimal:',
+        compress: true
+      },
+      enableDebugLogging: false,
+      maxRetries: 3
+    };
+  }
+
+  // Node.js or limited browser - use memory cache
+  return {
+    cacheType: 'memory',
+    memoryConfig: {
+      maxItems: 5000,
+      ttl: 300000
+    },
+    enableDebugLogging: true,
+    maxRetries: 3
+  };
+}
+
+const optimalOptions = createOptimalCacheConfiguration();
+const factory = createInstanceFactory(api, optimalOptions);
+```
+
+## Configuration Comparison
+
+| Cache Type | Persistence | Size Limit | Speed | Use Case |
+|------------|-------------|-------------|-------|----------|
+| Memory | None | RAM dependent | Fastest | Temporary data, development |
+| IndexedDB | Permanent | Hundreds of MB+ | Fast | Large datasets, offline apps |
+| localStorage | Permanent | ~5-10MB | Fast | User preferences, settings |
+
+## Quick Setup Commands
+
+```bash
+# Install fjell-cache
+npm install @fjell/cache
+
+# Run the configuration example
+npx ts-node examples/cache-type-configurations-example.ts
+```
+
+## See Also
+
+- [Complete Examples](./examples/) - Comprehensive examples directory
+- [README.md](./README.md) - Full documentation
+- [API Documentation](https://getfjell.github.io/fjell-cache/) - Detailed API reference
+
+For more detailed examples and use cases, see the [cache-type-configurations-example.ts](./examples/cache-type-configurations-example.ts) file.

package/CRITICAL_FIXES.md

@@ -0,0 +1,68 @@
+# Critical Bug fixes Applied
+
+This document summarizes the critical bugs that were identified and fixed in the fjell-cache codebase.
+
+## Fixed Issues
+
+### 1. IndexedDB Race Condition in Initialization
+**File:** `src/browser/IndexDBCacheMap.ts`
+**Problem:** Initialization used fire-and-forget setTimeout with no proper synchronization
+**Fix:** Added proper Promise-based initialization with mutex-like behavior
+
+### 2. TwoQueue Eviction Order Bug
+**File:** `src/eviction/strategies/TwoQueueEvictionStrategy.ts`
+**Problem:** Recent queue eviction was selecting newest items instead of oldest (violating FIFO)
+**Fix:** Reversed iteration order to properly evict oldest items from recent queue
+
+### 3. ARC Decay Timing Bug
+**File:** `src/eviction/strategies/ARCEvictionStrategy.ts`
+**Problem:** Timer-based decay was using Date.now() for comparisons with config intervals
+**Fix:** Added proper decay timing calculations with multiple intervals per decay period
+
+### 4. JSON Normalization Key Ordering
+**File:** `src/normalization.ts`
+**Problem:** JSON.stringify produces non-deterministic key ordering
+**Fix:** Implemented deterministic stringify function with sorted keys
+
+### 5. Count-Min Sketch Hash Function
+**File:** `src/eviction/strategies/LFUEvictionStrategy.ts`
+**Problem:** Poor hash distribution causing collisions and -0/+0 issues
+**Fix:** Replaced with FNV-1a hash algorithm for better distribution
+
+### 6. Enhanced Memory Cache Metadata Sync
+**File:** `src/memory/EnhancedMemoryCacheMap.ts`
+**Problem:** Updates didn't distinguish between new entries and existing entry modifications
+**Fix:** Added proper old value tracking and selective eviction strategy notifications
+
+### 7. ARC Ghost List Memory Leak
+**File:** `src/eviction/strategies/ARCEvictionStrategy.ts`
+**Problem:** Ghost lists could grow unbounded with repeated additions
+**Fix:** Added proper FIFO eviction from ghost lists with max size enforcement
+
+### 8. IndexedDB Sync Data Loss
+**File:** `src/browser/IndexDBCacheMap.ts`
+**Problem:** Pending operations could be lost if individual sync attempts failed
+**Fix:** Implemented proper operation queue with retry mechanisms
+
+### 9. TTL Race Conditions
+**File:** `src/memory/EnhancedMemoryCacheMap.ts`
+**Problem:** Concurrent TTL checks could cause race conditions
+**Fix:** Added operation tracking to prevent concurrent TTL operations on same keys
+
+## Test Results
+
+All fixes have been validated with comprehensive tests:
+- **Test Files:** 50 passed
+- **Test Cases:** 1,270 passed
+- **Code Coverage:** 92.81% statements, 86.56% branches
+
+## Impact
+
+These fixes address critical issues that could have caused:
+- Data corruption and race conditions
+- Memory leaks and performance degradation
+- Inconsistent eviction behavior
+- Silent data loss
+- Cache integrity violations
+
+All fixes maintain backward compatibility while significantly improving reliability and performance.

package/MEMORY_LEAK_FIXES.md

@@ -0,0 +1,270 @@
+# Memory Leak Prevention Fixes in @fjell/cache
+
+## Overview
+
+This document outlines the comprehensive memory leak prevention features implemented in the fjell-cache event system. These fixes address critical memory management issues that could cause applications to accumulate memory over time without proper cleanup.
+
+## Fixed Memory Leak Issues
+
+### 1. Static Timestamp State in CacheEventFactory ✅
+
+**Problem**: The `CacheEventFactory` class used a static `lastTimestamp` variable that accumulated without cleanup, potentially causing memory issues in long-running applications.
+
+**Solution**:
+- Added automatic cleanup mechanisms with instance counting
+- Implemented periodic cleanup that resets stale timestamp state
+- Added `destroyInstance()` method to properly manage static state lifecycle
+- Cleanup timer runs every 60 seconds and doesn't keep the process alive
+
+**Files Modified**:
+- `src/events/CacheEventFactory.ts`
+
+**Key Features**:
+```typescript
+// Automatic cleanup when instances are destroyed
+CacheEventFactory.destroyInstance();
+
+// Periodic cleanup of stale state
+private static performCleanup(): void {
+  const now = Date.now();
+  if (now - this.lastTimestamp > this.MAX_TIMESTAMP_AGE_MS) {
+    this.lastTimestamp = 0;
+  }
+}
+```
+
+### 2. Weak References for Event Handlers ✅
+
+**Problem**: Event handlers held strong references to listener functions, preventing garbage collection even when the listeners were no longer needed.
+
+**Solution**:
+- Implemented optional weak references for event listeners
+- Automatic detection and cleanup of garbage-collected listeners
+- Backwards compatible with fallback to strong references when WeakRef is not available
+
+**Files Modified**:
+- `src/events/CacheEventEmitter.ts`
+- `src/events/CacheEventTypes.ts`
+
+**Key Features**:
+```typescript
+// Weak reference support
+listenerRef?: WeakRef<CacheEventListener<V, S, L1, L2, L3, L4, L5>>;
+
+// Automatic cleanup
+if (this.WEAK_REF_ENABLED && subscription.listenerRef) {
+  const listener = subscription.listenerRef.deref();
+  if (!listener) {
+    subscription.isActive = false;
+    return;
+  }
+}
+
+// Configuration option
+{ useWeakRef: true }
+```
+
+### 3. Automatic Subscription Cleanup with Timeouts ✅
+
+**Problem**: Event subscriptions could remain active indefinitely without access, accumulating memory over time.
+
+**Solution**:
+- Added periodic cleanup of inactive subscriptions
+- Track last access time for each subscription
+- Automatic removal of subscriptions inactive for more than 5 minutes
+- Cleanup runs every 30 seconds
+
+**Files Modified**:
+- `src/events/CacheEventEmitter.ts`
+
+**Key Features**:
+```typescript
+// Track access times
+createdAt: number;
+lastAccessTime: number;
+
+// Periodic cleanup
+private performPeriodicCleanup(): void {
+  const now = Date.now();
+  if (now - subscription.lastAccessTime > this.MAX_INACTIVE_TIME_MS) {
+    toRemove.push(id);
+  }
+}
+```
+
+### 4. Cache Destruction Mechanisms ✅
+
+**Problem**: Cache instances lacked proper destruction methods, making it difficult to clean up resources when caches were no longer needed.
+
+**Solution**:
+- Added `destroy()` method to Cache interface and implementation
+- Comprehensive cleanup of all associated resources
+- Proper timer cleanup to prevent resource leaks
+- Integration with CacheEventFactory instance counting
+
+**Files Modified**:
+- `src/Cache.ts`
+- `src/Aggregator.ts`
+
+**Key Features**:
+```typescript
+destroy(): void {
+  // Clean up event emitter
+  eventEmitter.destroy();
+
+  // Clean up TTL manager
+  if (ttlManager && typeof ttlManager.destroy === 'function') {
+    ttlManager.destroy();
+  }
+
+  // Clean up cache map
+  if (cacheMap && typeof (cacheMap as any).destroy === 'function') {
+    (cacheMap as any).destroy();
+  }
+
+  // Notify CacheEventFactory
+  CacheEventFactory.destroyInstance();
+}
+```
+
+### 5. Enhanced Timer Management ✅
+
+**Problem**: Debounce timers and cleanup intervals could accumulate without proper cleanup.
+
+**Solution**:
+- Comprehensive timer cleanup in all destruction paths
+- Use of `timer.unref()` to prevent keeping Node.js process alive
+- Proper cleanup of debounce timers when subscriptions are removed
+
+**Files Modified**:
+- `src/events/CacheEventEmitter.ts`
+- `src/events/CacheEventFactory.ts`
+
+**Key Features**:
+```typescript
+// Timer cleanup
+if (subscription.debounceTimer) {
+  clearTimeout(subscription.debounceTimer);
+  subscription.debounceTimer = null;
+}
+
+// Non-blocking timers
+if (this.cleanupInterval.unref) {
+  this.cleanupInterval.unref();
+}
+```
+
+## Configuration Options
+
+### Event Subscription Options
+
+```typescript
+interface CacheSubscriptionOptions {
+  /** Use weak references for the listener (default: true if WeakRef is available) */
+  useWeakRef?: boolean;
+
+  /** Debounce events by this many milliseconds */
+  debounceMs?: number;
+
+  /** Filter by event types */
+  eventTypes?: CacheEventType[];
+
+  /** Optional error handler for listener errors */
+  onError?: (error: Error, event: any) => void;
+}
+```
+
+### Usage Examples
+
+```typescript
+// Create cache with automatic cleanup
+const cache = createCache(api, coordinate, registry, {
+  cacheType: 'memory'
+});
+
+// Subscribe with weak references (default)
+const subscription = cache.subscribe(
+  (event) => console.log(event.type),
+  {
+    useWeakRef: true,  // Optional: defaults to true
+    eventTypes: ['item_created', 'item_updated']
+  }
+);
+
+// Proper cleanup when done
+cache.destroy();
+```
+
+## Testing
+
+Comprehensive integration tests have been added to verify the memory leak prevention features:
+
+- `tests/examples/memory-leak-prevention.integration.test.ts`
+- `examples/memory-leak-prevention-example.ts`
+
+The tests verify:
+- Static state cleanup in CacheEventFactory
+- Weak reference functionality
+- Subscription timeout cleanup
+- Cache destruction mechanisms
+- Timer cleanup
+
+## Monitoring and Debugging
+
+### Memory Usage Monitoring
+
+```typescript
+// Check active subscriptions
+console.log(`Active subscriptions: ${cache.eventEmitter.getSubscriptionCount()}`);
+
+// Get subscription details
+const subscriptions = cache.eventEmitter.getSubscriptions();
+console.log('Subscription details:', subscriptions);
+```
+
+### Performance Impact
+
+- Cleanup operations run in the background and don't block application logic
+- Timers are configured to not keep the Node.js process alive
+- Weak references provide automatic memory management with zero performance overhead
+- Periodic cleanup intervals are optimized for balance between memory usage and performance
+
+## Migration Guide
+
+### For Existing Applications
+
+1. **No Breaking Changes**: All existing code will continue to work without modifications
+2. **Opt-in Features**: Weak references and enhanced cleanup are enabled by default but can be disabled
+3. **Recommended Actions**:
+   - Add `cache.destroy()` calls when caches are no longer needed
+   - Consider enabling weak references for better memory management
+   - Monitor subscription counts in long-running applications
+
+### Best Practices
+
+1. **Always destroy caches**: Call `cache.destroy()` when done with a cache instance
+2. **Use weak references**: Enable weak references for better automatic cleanup
+3. **Monitor subscriptions**: Regularly check subscription counts in production
+4. **Handle errors**: Provide error handlers for event listeners
+5. **Avoid long-lived subscriptions**: Clean up subscriptions that are no longer needed
+
+## Future Considerations
+
+- Memory usage metrics and monitoring hooks
+- Configurable cleanup intervals
+- Enhanced debugging tools for memory leak detection
+- Integration with application performance monitoring (APM) tools
+
+---
+
+## Summary
+
+The memory leak prevention features provide comprehensive protection against common memory issues in event-driven cache systems:
+
+- ✅ **Static state cleanup** prevents accumulation of factory state
+- ✅ **Weak references** enable automatic garbage collection of listeners
+- ✅ **Periodic cleanup** removes inactive subscriptions automatically
+- ✅ **Resource destruction** provides comprehensive cleanup mechanisms
+- ✅ **Timer management** prevents resource leaks from background timers
+
+These features ensure that fjell-cache applications can run for extended periods without memory degradation, making the library suitable for production environments with high availability requirements.