@fjell/cache 4.7.0 → 4.7.2

package/CACHE_EVENTS.md

@@ -0,0 +1,306 @@
+# Cache Event System
+
+The fjell-cache event system provides real-time notifications when cache operations occur. This enables reactive patterns where components can subscribe to cache changes and update automatically.
+
+## Overview
+
+The event system is built into the core Cache interface and provides:
+
+- **Event emission** for all cache operations (create, update, remove, query, etc.)
+- **Subscription management** with filtering options
+- **Event types** covering all cache state changes
+- **Debouncing** support for high-frequency updates
+- **Type safety** with full TypeScript support
+
+## Basic Usage
+
+### Subscribing to Events
+
+```typescript
+import { createCache, CacheEventListener } from '@fjell/cache';
+
+// Create cache
+const cache = createCache(api, coordinate, registry);
+
+// Subscribe to all events
+const listener: CacheEventListener<MyItem, 'myType'> = (event) => {
+  console.log('Cache event:', event.type, event);
+};
+
+const subscription = cache.subscribe(listener);
+
+// Later: unsubscribe
+subscription.unsubscribe();
+```
+
+### Filtering Events
+
+```typescript
+// Subscribe only to item creation events
+const subscription = cache.subscribe(listener, {
+  eventTypes: ['item_created', 'item_updated']
+});
+
+// Subscribe only to events for specific keys
+const subscription = cache.subscribe(listener, {
+  keys: [{ pk: 'user-123' }]
+});
+
+// Subscribe only to events in specific locations
+const subscription = cache.subscribe(listener, {
+  locations: [{ lk: 'container-1' }]
+});
+
+// Subscribe with debouncing (useful for UI updates)
+const subscription = cache.subscribe(listener, {
+  debounceMs: 100  // Debounce events by 100ms
+});
+```
+
+## Event Types
+
+### Item Events
+
+These events are emitted when individual items are affected:
+
+- **`item_created`** - Item was created via API and cached
+- **`item_updated`** - Item was updated via API and cache updated
+- **`item_removed`** - Item was removed via API and from cache
+- **`item_retrieved`** - Item was retrieved from API and cached
+- **`item_set`** - Item was set directly in cache (no API call)
+
+```typescript
+interface ItemEvent<V, S, L1, L2, L3, L4, L5> {
+  type: 'item_created' | 'item_updated' | 'item_removed' | 'item_retrieved' | 'item_set';
+  key: ComKey<S, L1, L2, L3, L4, L5> | PriKey<S>;
+  item: V | null;  // null for removed items
+  previousItem?: V | null;  // Previous state before change
+  affectedLocations?: LocKeyArray<L1, L2, L3, L4, L5> | [];
+  timestamp: number;
+  source: 'api' | 'cache' | 'operation';
+}
+```
+
+### Query Events
+
+These events are emitted when multiple items are queried:
+
+- **`items_queried`** - Multiple items were queried and cached
+
+```typescript
+interface QueryEvent<V, S, L1, L2, L3, L4, L5> {
+  type: 'items_queried';
+  query: ItemQuery;
+  locations: LocKeyArray<L1, L2, L3, L4, L5> | [];
+  items: V[];
+  affectedKeys: (ComKey<S, L1, L2, L3, L4, L5> | PriKey<S>)[];
+  timestamp: number;
+  source: 'api' | 'cache' | 'operation';
+}
+```
+
+### Cache Management Events
+
+These events are emitted for cache-wide operations:
+
+- **`cache_cleared`** - Entire cache was cleared
+- **`location_invalidated`** - Specific location(s) were invalidated
+- **`query_invalidated`** - Cached query results were invalidated
+
+## Advanced Usage
+
+### React Integration Example
+
+```typescript
+import { useEffect, useState } from 'react';
+import { Cache, CacheSubscription } from '@fjell/cache';
+
+function useItem<T>(cache: Cache<T, any>, key: any): T | null {
+  const [item, setItem] = useState<T | null>(() =>
+    cache.cacheMap.get(key)
+  );
+
+  useEffect(() => {
+    const subscription = cache.subscribe((event) => {
+      if (event.type === 'item_changed' &&
+          JSON.stringify(event.key) === JSON.stringify(key)) {
+        setItem(event.item);
+      } else if (event.type === 'item_removed' &&
+                 JSON.stringify(event.key) === JSON.stringify(key)) {
+        setItem(null);
+      }
+    }, {
+      keys: [key],
+      eventTypes: ['item_updated', 'item_removed', 'item_set']
+    });
+
+    return () => subscription.unsubscribe();
+  }, [cache, key]);
+
+  return item;
+}
+```
+
+### Listening to Multiple Caches
+
+```typescript
+class CacheEventHub {
+  private subscriptions: CacheSubscription[] = [];
+
+  subscribeTo<T>(cache: Cache<T, any>, listener: CacheEventListener<T, any>) {
+    const subscription = cache.subscribe(listener);
+    this.subscriptions.push(subscription);
+    return subscription;
+  }
+
+  unsubscribeAll() {
+    this.subscriptions.forEach(sub => sub.unsubscribe());
+    this.subscriptions = [];
+  }
+}
+
+const hub = new CacheEventHub();
+
+// Subscribe to multiple caches
+hub.subscribeTo(userCache, (event) => console.log('User event:', event));
+hub.subscribeTo(orderCache, (event) => console.log('Order event:', event));
+
+// Later: clean up all subscriptions
+hub.unsubscribeAll();
+```
+
+### Event Logging and Debugging
+
+```typescript
+// Log all cache events for debugging
+const debugSubscription = cache.subscribe((event) => {
+  console.log(`[${new Date(event.timestamp).toISOString()}] ${event.type}:`, {
+    source: event.source,
+    ...('key' in event ? { key: event.key } : {}),
+    ...('query' in event ? { query: event.query } : {}),
+  });
+});
+
+// Performance monitoring
+let eventCounts = new Map<string, number>();
+
+cache.subscribe((event) => {
+  const count = eventCounts.get(event.type) || 0;
+  eventCounts.set(event.type, count + 1);
+});
+
+setInterval(() => {
+  console.log('Event counts:', Object.fromEntries(eventCounts));
+  eventCounts.clear();
+}, 10000);
+```
+
+## Best Practices
+
+### 1. Use Specific Event Filters
+
+```typescript
+// Good: Filter by specific event types and keys
+cache.subscribe(listener, {
+  eventTypes: ['item_updated'],
+  keys: [userKey]
+});
+
+// Avoid: Subscribing to all events when you only need specific ones
+cache.subscribe(listener); // Too broad
+```
+
+### 2. Debounce High-Frequency Updates
+
+```typescript
+// Good: Debounce UI updates
+cache.subscribe(updateUI, {
+  debounceMs: 100
+});
+
+// Good: Don't debounce critical business logic
+cache.subscribe(saveToDisk, {
+  // No debouncing for critical operations
+});
+```
+
+### 3. Clean Up Subscriptions
+
+```typescript
+useEffect(() => {
+  const subscription = cache.subscribe(listener);
+
+  // Always clean up
+  return () => subscription.unsubscribe();
+}, []);
+```
+
+### 4. Handle Errors in Listeners
+
+```typescript
+cache.subscribe((event) => {
+  try {
+    // Your event handling logic
+    handleEvent(event);
+  } catch (error) {
+    console.error('Error handling cache event:', error);
+    // Don't let errors break other listeners
+  }
+});
+```
+
+## Event Sources
+
+Events include a `source` field indicating where they originated:
+
+- **`api`** - Event from API operation (create, update, remove, etc.)
+- **`cache`** - Event from direct cache operation (set, invalidate)
+- **`operation`** - Event from internal cache operation
+
+## Performance Considerations
+
+- **Subscription count**: Each subscription has minimal overhead, but avoid creating unnecessary subscriptions
+- **Event frequency**: Use debouncing for high-frequency UI updates
+- **Memory leaks**: Always unsubscribe when components unmount
+- **Event filtering**: Use specific filters to reduce unnecessary event processing
+
+## Error Handling
+
+The event system is designed to be resilient:
+
+- Listener errors don't affect other listeners or cache operations
+- Failed subscriptions are automatically cleaned up
+- Event emission continues even if some listeners fail
+
+## Migration from Manual State Management
+
+If you're currently using manual state management patterns:
+
+```typescript
+// Before: Manual state management
+const [items, setItems] = useState([]);
+
+const addItem = async (item) => {
+  const newItem = await cache.operations.create(item);
+  setItems(prev => [...prev, newItem]);  // Manual update
+};
+
+// After: Event-driven updates
+const [items, setItems] = useState([]);
+
+useEffect(() => {
+  const subscription = cache.subscribe((event) => {
+    if (event.type === 'item_created') {
+      setItems(prev => [...prev, event.item]);
+    }
+  }, { eventTypes: ['item_created'] });
+
+  return () => subscription.unsubscribe();
+}, []);
+
+const addItem = async (item) => {
+  await cache.operations.create(item);  // Event automatically updates state
+};
+```
+
+This approach eliminates the need for manual state synchronization and ensures your UI always reflects the current cache state.

package/CACHE_IMPLEMENTATIONS.md

@@ -151,6 +151,123 @@ const cache = new MemoryCacheMap(keyTypeArray);
 - **Small data, persistent**: `LocalStorageCacheMap`
 - **Large data, complex queries**: `AsyncIndexDBCacheMap`
 
+## Cache Information and Capabilities
+
+All cache instances expose metadata about their configuration and capabilities through the `getCacheInfo()` method at the Cache level. This provides client applications with visibility into the cache's behavior and limitations.
+
+### CacheInfo Interface
+
+```typescript
+interface CacheInfo {
+  /** The implementation type in format "<category>/<implementation>" */
+  implementationType: string;
+  /** The eviction policy being used (if any) */
+  evictionPolicy?: string;
+  /** Default TTL in milliseconds (if configured) */
+  defaultTTL?: number;
+  /** Whether TTL is supported by this implementation */
+  supportsTTL: boolean;
+  /** Whether eviction is supported by this implementation */
+  supportsEviction: boolean;
+}
+```
+
+### Getting Cache Information
+
+```typescript
+import { MemoryCacheMap, EnhancedMemoryCacheMap } from '@fjell/cache';
+
+// Create cache instance
+const cache = createCache(api, createCoordinate('item'), registry);
+const cacheInfo = cache.getCacheInfo();
+console.log(cacheInfo);
+// Output: {
+//   implementationType: "memory/memory",
+//   supportsTTL: true,
+//   supportsEviction: false
+// }
+
+// Enhanced memory cache with eviction
+const enhancedCache = createCache(api, createCoordinate('item'), registry, {
+  cacheType: 'memory',
+  memoryConfig: {
+    size: {
+      maxItems: 1000,
+      evictionPolicy: 'lru'
+    }
+  }
+});
+const enhancedInfo = enhancedCache.getCacheInfo();
+console.log(enhancedInfo);
+// Output: {
+//   implementationType: "memory/enhanced",
+//   evictionPolicy: "lru",
+//   supportsTTL: true,
+//   supportsEviction: true
+// }
+```
+
+### Implementation Types
+
+Each cache implementation has a unique type identifier:
+
+- `memory/memory` - Basic MemoryCacheMap
+- `memory/enhanced` - EnhancedMemoryCacheMap with eviction support
+- `browser/localStorage` - LocalStorageCacheMap
+- `browser/sessionStorage` - SessionStorageCacheMap
+- `browser/indexedDB` - IndexDBCacheMap
+
+### Capability Detection
+
+Use the capability flags to adapt your application behavior:
+
+```typescript
+function configureCache<V extends Item<S>, S extends string>(
+  cache: Cache<V, S>
+) {
+  const info = cache.getCacheInfo();
+
+  // Adjust TTL strategy based on support
+  if (info.supportsTTL) {
+    console.log(`TTL operations supported for ${info.implementationType}`);
+  }
+
+  // Show eviction policy if supported
+  if (info.supportsEviction && info.evictionPolicy) {
+    console.log(`Using ${info.evictionPolicy} eviction policy`);
+  }
+
+  // Log implementation details
+  console.log(`Cache implementation: ${info.implementationType}`);
+}
+```
+
+### Display Information for Debugging
+
+The cache info is particularly useful for debugging and monitoring:
+
+```typescript
+function debugCacheStatus<V extends Item<S>, S extends string>(
+  cacheName: string,
+  cache: Cache<V, S>
+) {
+  const info = cache.getCacheInfo();
+
+  console.log(`[${cacheName}] Cache Configuration:`);
+  console.log(`  Type: ${info.implementationType}`);
+  console.log(`  TTL Support: ${info.supportsTTL ? 'Yes' : 'No'}`);
+  console.log(`  Eviction Support: ${info.supportsEviction ? 'Yes' : 'No'}`);
+
+  if (info.evictionPolicy) {
+    console.log(`  Eviction Policy: ${info.evictionPolicy}`);
+  }
+
+  if (info.defaultTTL) {
+    console.log(`  Default TTL: ${info.defaultTTL}ms`);
+  }
+}
+```
+
 ## Common Patterns
 
 ### Factory Pattern

package/MEMORY_LEAK_FIXES.md

@@ -0,0 +1,270 @@
+# Memory Leak Prevention Fixes in @fjell/cache
+
+## Overview
+
+This document outlines the comprehensive memory leak prevention features implemented in the fjell-cache event system. These fixes address critical memory management issues that could cause applications to accumulate memory over time without proper cleanup.
+
+## Fixed Memory Leak Issues
+
+### 1. Static Timestamp State in CacheEventFactory ✅
+
+**Problem**: The `CacheEventFactory` class used a static `lastTimestamp` variable that accumulated without cleanup, potentially causing memory issues in long-running applications.
+
+**Solution**:
+- Added automatic cleanup mechanisms with instance counting
+- Implemented periodic cleanup that resets stale timestamp state
+- Added `destroyInstance()` method to properly manage static state lifecycle
+- Cleanup timer runs every 60 seconds and doesn't keep the process alive
+
+**Files Modified**:
+- `src/events/CacheEventFactory.ts`
+
+**Key Features**:
+```typescript
+// Automatic cleanup when instances are destroyed
+CacheEventFactory.destroyInstance();
+
+// Periodic cleanup of stale state
+private static performCleanup(): void {
+  const now = Date.now();
+  if (now - this.lastTimestamp > this.MAX_TIMESTAMP_AGE_MS) {
+    this.lastTimestamp = 0;
+  }
+}
+```
+
+### 2. Weak References for Event Handlers ✅
+
+**Problem**: Event handlers held strong references to listener functions, preventing garbage collection even when the listeners were no longer needed.
+
+**Solution**:
+- Implemented optional weak references for event listeners
+- Automatic detection and cleanup of garbage-collected listeners
+- Backwards compatible with fallback to strong references when WeakRef is not available
+
+**Files Modified**:
+- `src/events/CacheEventEmitter.ts`
+- `src/events/CacheEventTypes.ts`
+
+**Key Features**:
+```typescript
+// Weak reference support
+listenerRef?: WeakRef<CacheEventListener<V, S, L1, L2, L3, L4, L5>>;
+
+// Automatic cleanup
+if (this.WEAK_REF_ENABLED && subscription.listenerRef) {
+  const listener = subscription.listenerRef.deref();
+  if (!listener) {
+    subscription.isActive = false;
+    return;
+  }
+}
+
+// Configuration option
+{ useWeakRef: true }
+```
+
+### 3. Automatic Subscription Cleanup with Timeouts ✅
+
+**Problem**: Event subscriptions could remain active indefinitely without access, accumulating memory over time.
+
+**Solution**:
+- Added periodic cleanup of inactive subscriptions
+- Track last access time for each subscription
+- Automatic removal of subscriptions inactive for more than 5 minutes
+- Cleanup runs every 30 seconds
+
+**Files Modified**:
+- `src/events/CacheEventEmitter.ts`
+
+**Key Features**:
+```typescript
+// Track access times
+createdAt: number;
+lastAccessTime: number;
+
+// Periodic cleanup
+private performPeriodicCleanup(): void {
+  const now = Date.now();
+  if (now - subscription.lastAccessTime > this.MAX_INACTIVE_TIME_MS) {
+    toRemove.push(id);
+  }
+}
+```
+
+### 4. Cache Destruction Mechanisms ✅
+
+**Problem**: Cache instances lacked proper destruction methods, making it difficult to clean up resources when caches were no longer needed.
+
+**Solution**:
+- Added `destroy()` method to Cache interface and implementation
+- Comprehensive cleanup of all associated resources
+- Proper timer cleanup to prevent resource leaks
+- Integration with CacheEventFactory instance counting
+
+**Files Modified**:
+- `src/Cache.ts`
+- `src/Aggregator.ts`
+
+**Key Features**:
+```typescript
+destroy(): void {
+  // Clean up event emitter
+  eventEmitter.destroy();
+
+  // Clean up TTL manager
+  if (ttlManager && typeof ttlManager.destroy === 'function') {
+    ttlManager.destroy();
+  }
+
+  // Clean up cache map
+  if (cacheMap && typeof (cacheMap as any).destroy === 'function') {
+    (cacheMap as any).destroy();
+  }
+
+  // Notify CacheEventFactory
+  CacheEventFactory.destroyInstance();
+}
+```
+
+### 5. Enhanced Timer Management ✅
+
+**Problem**: Debounce timers and cleanup intervals could accumulate without proper cleanup.
+
+**Solution**:
+- Comprehensive timer cleanup in all destruction paths
+- Use of `timer.unref()` to prevent keeping Node.js process alive
+- Proper cleanup of debounce timers when subscriptions are removed
+
+**Files Modified**:
+- `src/events/CacheEventEmitter.ts`
+- `src/events/CacheEventFactory.ts`
+
+**Key Features**:
+```typescript
+// Timer cleanup
+if (subscription.debounceTimer) {
+  clearTimeout(subscription.debounceTimer);
+  subscription.debounceTimer = null;
+}
+
+// Non-blocking timers
+if (this.cleanupInterval.unref) {
+  this.cleanupInterval.unref();
+}
+```
+
+## Configuration Options
+
+### Event Subscription Options
+
+```typescript
+interface CacheSubscriptionOptions {
+  /** Use weak references for the listener (default: true if WeakRef is available) */
+  useWeakRef?: boolean;
+
+  /** Debounce events by this many milliseconds */
+  debounceMs?: number;
+
+  /** Filter by event types */
+  eventTypes?: CacheEventType[];
+
+  /** Optional error handler for listener errors */
+  onError?: (error: Error, event: any) => void;
+}
+```
+
+### Usage Examples
+
+```typescript
+// Create cache with automatic cleanup
+const cache = createCache(api, coordinate, registry, {
+  cacheType: 'memory'
+});
+
+// Subscribe with weak references (default)
+const subscription = cache.subscribe(
+  (event) => console.log(event.type),
+  {
+    useWeakRef: true,  // Optional: defaults to true
+    eventTypes: ['item_created', 'item_updated']
+  }
+);
+
+// Proper cleanup when done
+cache.destroy();
+```
+
+## Testing
+
+Comprehensive integration tests have been added to verify the memory leak prevention features:
+
+- `tests/examples/memory-leak-prevention.integration.test.ts`
+- `examples/memory-leak-prevention-example.ts`
+
+The tests verify:
+- Static state cleanup in CacheEventFactory
+- Weak reference functionality
+- Subscription timeout cleanup
+- Cache destruction mechanisms
+- Timer cleanup
+
+## Monitoring and Debugging
+
+### Memory Usage Monitoring
+
+```typescript
+// Check active subscriptions
+console.log(`Active subscriptions: ${cache.eventEmitter.getSubscriptionCount()}`);
+
+// Get subscription details
+const subscriptions = cache.eventEmitter.getSubscriptions();
+console.log('Subscription details:', subscriptions);
+```
+
+### Performance Impact
+
+- Cleanup operations run in the background and don't block application logic
+- Timers are configured to not keep the Node.js process alive
+- Weak references provide automatic memory management with zero performance overhead
+- Periodic cleanup intervals are optimized for balance between memory usage and performance
+
+## Migration Guide
+
+### For Existing Applications
+
+1. **No Breaking Changes**: All existing code will continue to work without modifications
+2. **Opt-in Features**: Weak references and enhanced cleanup are enabled by default but can be disabled
+3. **Recommended Actions**:
+   - Add `cache.destroy()` calls when caches are no longer needed
+   - Consider enabling weak references for better memory management
+   - Monitor subscription counts in long-running applications
+
+### Best Practices
+
+1. **Always destroy caches**: Call `cache.destroy()` when done with a cache instance
+2. **Use weak references**: Enable weak references for better automatic cleanup
+3. **Monitor subscriptions**: Regularly check subscription counts in production
+4. **Handle errors**: Provide error handlers for event listeners
+5. **Avoid long-lived subscriptions**: Clean up subscriptions that are no longer needed
+
+## Future Considerations
+
+- Memory usage metrics and monitoring hooks
+- Configurable cleanup intervals
+- Enhanced debugging tools for memory leak detection
+- Integration with application performance monitoring (APM) tools
+
+---
+
+## Summary
+
+The memory leak prevention features provide comprehensive protection against common memory issues in event-driven cache systems:
+
+- ✅ **Static state cleanup** prevents accumulation of factory state
+- ✅ **Weak references** enable automatic garbage collection of listeners
+- ✅ **Periodic cleanup** removes inactive subscriptions automatically
+- ✅ **Resource destruction** provides comprehensive cleanup mechanisms
+- ✅ **Timer management** prevents resource leaks from background timers
+
+These features ensure that fjell-cache applications can run for extended periods without memory degradation, making the library suitable for production environments with high availability requirements.