@fjell/cache 4.7.35 → 4.7.37

package/MEMORY_LEAK_FIXES.md

@@ -1,270 +0,0 @@
-# Memory Leak Prevention Fixes in @fjell/cache
-
-## Overview
-
-This document outlines the comprehensive memory leak prevention features implemented in the fjell-cache event system. These fixes address critical memory management issues that could cause applications to accumulate memory over time without proper cleanup.
-
-## Fixed Memory Leak Issues
-
-### 1. Static Timestamp State in CacheEventFactory ✅
-
-**Problem**: The `CacheEventFactory` class used a static `lastTimestamp` variable that accumulated without cleanup, potentially causing memory issues in long-running applications.
-
-**Solution**:
-- Added automatic cleanup mechanisms with instance counting
-- Implemented periodic cleanup that resets stale timestamp state
-- Added `destroyInstance()` method to properly manage static state lifecycle
-- Cleanup timer runs every 60 seconds and doesn't keep the process alive
-
-**Files Modified**:
-- `src/events/CacheEventFactory.ts`
-
-**Key Features**:
-```typescript
-// Automatic cleanup when instances are destroyed
-CacheEventFactory.destroyInstance();
-
-// Periodic cleanup of stale state
-private static performCleanup(): void {
-  const now = Date.now();
-  if (now - this.lastTimestamp > this.MAX_TIMESTAMP_AGE_MS) {
-    this.lastTimestamp = 0;
-  }
-}
-```
-
-### 2. Weak References for Event Handlers ✅
-
-**Problem**: Event handlers held strong references to listener functions, preventing garbage collection even when the listeners were no longer needed.
-
-**Solution**:
-- Implemented optional weak references for event listeners
-- Automatic detection and cleanup of garbage-collected listeners
-- Backwards compatible with fallback to strong references when WeakRef is not available
-
-**Files Modified**:
-- `src/events/CacheEventEmitter.ts`
-- `src/events/CacheEventTypes.ts`
-
-**Key Features**:
-```typescript
-// Weak reference support
-listenerRef?: WeakRef<CacheEventListener<V, S, L1, L2, L3, L4, L5>>;
-
-// Automatic cleanup
-if (this.WEAK_REF_ENABLED && subscription.listenerRef) {
-  const listener = subscription.listenerRef.deref();
-  if (!listener) {
-    subscription.isActive = false;
-    return;
-  }
-}
-
-// Configuration option
-{ useWeakRef: true }
-```
-
-### 3. Automatic Subscription Cleanup with Timeouts ✅
-
-**Problem**: Event subscriptions could remain active indefinitely without access, accumulating memory over time.
-
-**Solution**:
-- Added periodic cleanup of inactive subscriptions
-- Track last access time for each subscription
-- Automatic removal of subscriptions inactive for more than 5 minutes
-- Cleanup runs every 30 seconds
-
-**Files Modified**:
-- `src/events/CacheEventEmitter.ts`
-
-**Key Features**:
-```typescript
-// Track access times
-createdAt: number;
-lastAccessTime: number;
-
-// Periodic cleanup
-private performPeriodicCleanup(): void {
-  const now = Date.now();
-  if (now - subscription.lastAccessTime > this.MAX_INACTIVE_TIME_MS) {
-    toRemove.push(id);
-  }
-}
-```
-
-### 4. Cache Destruction Mechanisms ✅
-
-**Problem**: Cache instances lacked proper destruction methods, making it difficult to clean up resources when caches were no longer needed.
-
-**Solution**:
-- Added `destroy()` method to Cache interface and implementation
-- Comprehensive cleanup of all associated resources
-- Proper timer cleanup to prevent resource leaks
-- Integration with CacheEventFactory instance counting
-
-**Files Modified**:
-- `src/Cache.ts`
-- `src/Aggregator.ts`
-
-**Key Features**:
-```typescript
-destroy(): void {
-  // Clean up event emitter
-  eventEmitter.destroy();
-
-  // Clean up TTL manager
-  if (ttlManager && typeof ttlManager.destroy === 'function') {
-    ttlManager.destroy();
-  }
-
-  // Clean up cache map
-  if (cacheMap && typeof (cacheMap as any).destroy === 'function') {
-    (cacheMap as any).destroy();
-  }
-
-  // Notify CacheEventFactory
-  CacheEventFactory.destroyInstance();
-}
-```
-
-### 5. Enhanced Timer Management ✅
-
-**Problem**: Debounce timers and cleanup intervals could accumulate without proper cleanup.
-
-**Solution**:
-- Comprehensive timer cleanup in all destruction paths
-- Use of `timer.unref()` to prevent keeping Node.js process alive
-- Proper cleanup of debounce timers when subscriptions are removed
-
-**Files Modified**:
-- `src/events/CacheEventEmitter.ts`
-- `src/events/CacheEventFactory.ts`
-
-**Key Features**:
-```typescript
-// Timer cleanup
-if (subscription.debounceTimer) {
-  clearTimeout(subscription.debounceTimer);
-  subscription.debounceTimer = null;
-}
-
-// Non-blocking timers
-if (this.cleanupInterval.unref) {
-  this.cleanupInterval.unref();
-}
-```
-
-## Configuration Options
-
-### Event Subscription Options
-
-```typescript
-interface CacheSubscriptionOptions {
-  /** Use weak references for the listener (default: true if WeakRef is available) */
-  useWeakRef?: boolean;
-
-  /** Debounce events by this many milliseconds */
-  debounceMs?: number;
-
-  /** Filter by event types */
-  eventTypes?: CacheEventType[];
-
-  /** Optional error handler for listener errors */
-  onError?: (error: Error, event: any) => void;
-}
-```
-
-### Usage Examples
-
-```typescript
-// Create cache with automatic cleanup
-const cache = createCache(api, coordinate, registry, {
-  cacheType: 'memory'
-});
-
-// Subscribe with weak references (default)
-const subscription = cache.subscribe(
-  (event) => console.log(event.type),
-  {
-    useWeakRef: true,  // Optional: defaults to true
-    eventTypes: ['item_created', 'item_updated']
-  }
-);
-
-// Proper cleanup when done
-cache.destroy();
-```
-
-## Testing
-
-Comprehensive integration tests have been added to verify the memory leak prevention features:
-
-- `tests/examples/memory-leak-prevention.integration.test.ts`
-- `examples/memory-leak-prevention-example.ts`
-
-The tests verify:
-- Static state cleanup in CacheEventFactory
-- Weak reference functionality
-- Subscription timeout cleanup
-- Cache destruction mechanisms
-- Timer cleanup
-
-## Monitoring and Debugging
-
-### Memory Usage Monitoring
-
-```typescript
-// Check active subscriptions
-console.log(`Active subscriptions: ${cache.eventEmitter.getSubscriptionCount()}`);
-
-// Get subscription details
-const subscriptions = cache.eventEmitter.getSubscriptions();
-console.log('Subscription details:', subscriptions);
-```
-
-### Performance Impact
-
-- Cleanup operations run in the background and don't block application logic
-- Timers are configured to not keep the Node.js process alive
-- Weak references provide automatic memory management with zero performance overhead
-- Periodic cleanup intervals are optimized for balance between memory usage and performance
-
-## Migration Guide
-
-### For Existing Applications
-
-1. **No Breaking Changes**: All existing code will continue to work without modifications
-2. **Opt-in Features**: Weak references and enhanced cleanup are enabled by default but can be disabled
-3. **Recommended Actions**:
-   - Add `cache.destroy()` calls when caches are no longer needed
-   - Consider enabling weak references for better memory management
-   - Monitor subscription counts in long-running applications
-
-### Best Practices
-
-1. **Always destroy caches**: Call `cache.destroy()` when done with a cache instance
-2. **Use weak references**: Enable weak references for better automatic cleanup
-3. **Monitor subscriptions**: Regularly check subscription counts in production
-4. **Handle errors**: Provide error handlers for event listeners
-5. **Avoid long-lived subscriptions**: Clean up subscriptions that are no longer needed
-
-## Future Considerations
-
-- Memory usage metrics and monitoring hooks
-- Configurable cleanup intervals
-- Enhanced debugging tools for memory leak detection
-- Integration with application performance monitoring (APM) tools
-
----
-
-## Summary
-
-The memory leak prevention features provide comprehensive protection against common memory issues in event-driven cache systems:
-
-- ✅ **Static state cleanup** prevents accumulation of factory state
-- ✅ **Weak references** enable automatic garbage collection of listeners
-- ✅ **Periodic cleanup** removes inactive subscriptions automatically
-- ✅ **Resource destruction** provides comprehensive cleanup mechanisms
-- ✅ **Timer management** prevents resource leaks from background timers
-
-These features ensure that fjell-cache applications can run for extended periods without memory degradation, making the library suitable for production environments with high availability requirements.

package/MIGRATION_v3.md

@@ -1,249 +0,0 @@
-# Migration Guide: v2.x to v3.0
-
-## Overview
-
-Version 3.0 of `@fjell/cache` adopts the centralized Operations interface from `@fjell/core`. This provides a consistent interface across all Fjell packages, better type safety, and shared validation logic.
-
-## Breaking Changes
-
-### 1. Operations Interface Now Extends Core
-
-The `Operations` interface now extends from `@fjell/core/Operations`. This change is mostly transparent to users, but provides better type checking and consistency across the Fjell ecosystem.
-
-### 2. Updated `create` Method Signature
-
-The `create` operation now uses the `CreateOptions` type from `@fjell/core`:
-
-#### Before (v2.x)
-```typescript
-// Create with locations
-await cache.operations.create(itemData, locations);
-
-// Create without locations
-await cache.operations.create(itemData, []);
-await cache.operations.create(itemData);
-```
-
-#### After (v3.0)
-```typescript
-// Create with locations
-await cache.operations.create(itemData, { locations });
-
-// Create without locations (unchanged)
-await cache.operations.create(itemData);
-
-// Create with specific key (new feature)
-await cache.operations.create(itemData, { key: { kt: 'user', pk: 'user-123' } });
-```
-
-### 3. New `upsert` Method
-
-A new `upsert` method has been added to the Operations interface:
-
-```typescript
-// Update if exists, create if doesn't
-const item = await cache.operations.upsert(
-  key,
-  itemData,
-  locations  // Optional, used only for creation
-);
-```
-
-## Migration Steps
-
-### Step 1: Update Dependencies
-
-Update to the latest versions:
-
-```bash
-npm install @fjell/core@latest @fjell/cache@latest
-```
-
-### Step 2: Update `create` Calls
-
-Find all calls to `cache.operations.create` with a locations parameter and wrap the locations in an object:
-
-```typescript
-// Search for patterns like:
-cache.operations.create(item, locations)
-cache.operations.create(item, loc)
-cache.operations.create(itemData, [{kt: 'parent', lk: '...'}])
-
-// Replace with:
-cache.operations.create(item, { locations })
-cache.operations.create(item, { locations: loc })
-cache.operations.create(itemData, { locations: [{kt: 'parent', lk: '...'}] })
-```
-
-### Step 3: Optional - Use New `upsert` Method
-
-If you have code that checks for existence before creating or updating:
-
-```typescript
-// Before (v2.x)
-const existing = await cache.operations.get(key);
-if (existing) {
-  await cache.operations.update(key, updates);
-} else {
-  await cache.operations.create(data, locations);
-}
-
-// After (v3.0) - simpler!
-await cache.operations.upsert(key, data, locations);
-```
-
-## What Stays The Same
-
-- All other cache operations work identically (`get`, `all`, `one`, `update`, `remove`, `find`, `findOne`, `action`, `allAction`, `facet`, `allFacet`)
-- Cache-specific methods unchanged (`retrieve`, `set`, `reset`)
-- Cache events unchanged
-- TTL/eviction unchanged
-- Cache statistics unchanged
-- All cache implementations work the same way
-- Cache configuration options unchanged
-
-## Type Imports
-
-You can now import core types from either package:
-
-```typescript
-// From core (recommended)
-import { OperationParams, AffectedKeys } from '@fjell/core';
-
-// From cache (re-exported for convenience)
-import { OperationParams, AffectedKeys } from '@fjell/cache';
-```
-
-## Benefits
-
-### Smaller Bundle Size
-Shared types between packages reduce overall bundle size.
-
-### Better Type Safety
-Consistent types across packages provide better TypeScript support and catch more errors at compile time.
-
-### Consistent Interface
-The same Operations interface is used across `@fjell/cache`, `@fjell/lib`, `@fjell/providers`, and other packages.
-
-### Enhanced Functionality
-New `upsert` method simplifies common patterns.
-
-## Examples
-
-### Basic Migration Example
-
-```typescript
-import { createCache } from '@fjell/cache';
-import { createCoordinate, createRegistry } from '@fjell/registry';
-
-// Setup (unchanged)
-const registry = createRegistry();
-const cache = await createCache(api, createCoordinate('user'), registry);
-
-// Operations (mostly unchanged)
-await cache.operations.all();                    // ✓ No change
-await cache.operations.get(key);                 // ✓ No change
-await cache.operations.update(key, data);        // ✓ No change
-
-// Create operation (requires update)
-await cache.operations.create(data, locations);  // ✗ v2.x
-await cache.operations.create(data, { locations }); // ✓ v3.0
-
-// New upsert method
-await cache.operations.upsert(key, data);        // ✓ New in v3.0
-```
-
-### Contained Items Migration
-
-```typescript
-// Before (v2.x)
-const commentData = { text: 'Great post!' };
-const locations = [{ kt: 'post', lk: 'post-123' }];
-await commentCache.operations.create(commentData, locations);
-
-// After (v3.0)
-const commentData = { text: 'Great post!' };
-const locations = [{ kt: 'post', lk: 'post-123' }];
-await commentCache.operations.create(commentData, { locations });
-```
-
-### Upsert Pattern
-
-```typescript
-// Before (v2.x) - manual check
-async function saveUser(key, userData) {
-  const existing = await cache.operations.get(key);
-  if (existing) {
-    return await cache.operations.update(key, userData);
-  } else {
-    return await cache.operations.create(userData, []);
-  }
-}
-
-// After (v3.0) - built-in upsert
-async function saveUser(key, userData) {
-  return await cache.operations.upsert(key, userData);
-}
-```
-
-## Troubleshooting
-
-### TypeScript Errors
-
-If you see TypeScript errors after upgrading:
-
-1. **"Argument of type 'LocKeyArray' is not assignable to parameter"**
-   - Wrap the locations array in `{ locations: ... }`
-
-2. **"Type X is not assignable to type CreateOptions"**
-   - Update create calls to use the object syntax
-
-3. **"Property 'upsert' does not exist"**
-   - Ensure you've updated `@fjell/core` and `@fjell/cache` to the latest versions
-
-### Runtime Issues
-
-If the cache still works but you see warnings:
-
-- Check that all `create` calls are using the new signature
-- Verify that dependency versions are compatible
-
-## Testing Your Migration
-
-After migration, verify:
-
-1. All create operations work correctly
-2. No TypeScript compilation errors
-3. Existing tests pass
-4. Cache operations function as expected
-
-```typescript
-// Test create with locations
-const item = await cache.operations.create(data, { locations: [loc1] });
-expect(item).toBeDefined();
-
-// Test upsert
-const upserted = await cache.operations.upsert(key, updates);
-expect(upserted).toBeDefined();
-```
-
-## Getting Help
-
-If you encounter issues during migration:
-
-1. Check this guide for common patterns
-2. Review the [PHASE_3_CACHE_MIGRATION.md](../PHASE_3_CACHE_MIGRATION.md) for implementation details
-3. Open an issue on GitHub with your specific use case
-
-## Summary
-
-The migration from v2.x to v3.0 requires minimal changes:
-
-- ✅ Update `create(item, locations)` to `create(item, { locations })`
-- ✅ Optionally use new `upsert` method for update-or-create patterns
-- ✅ All other operations remain unchanged
-- ✅ Cache behavior and performance unchanged
-- ✅ Better type safety and consistency across packages
-
-The migration should take less than an hour for most codebases, with the majority of time spent finding and updating `create` calls.
-