@cendarsoss/pusher-js 8.4.11

package/DELTA_COMPRESSION.md

@@ -0,0 +1,365 @@
+# Delta Compression in Pusher-JS
+
+This document describes the native delta compression and conflation key support in pusher-js for use with Sockudo servers.
+
+## Overview
+
+Delta compression reduces bandwidth usage by sending only the differences (deltas) between consecutive messages instead of full messages. This is particularly effective for:
+
+- Real-time price updates
+- Market data feeds
+- IoT sensor data
+- Any high-frequency updates with similar structure
+
+### Conflation Keys
+
+Conflation keys group messages by a specific field (e.g., `asset`, `device_id`) so that deltas are computed between messages with the same key value. This dramatically improves compression efficiency.
+
+**Without conflation keys:**
+```
+BTC update → ETH update → BTC update → ETH update
+         ↓           ↓           ↓
+    (comparing unrelated messages = poor compression)
+```
+
+**With conflation keys (asset):**
+```
+BTC updates: BTC₁ → BTC₂ → BTC₃ (71% compression)
+ETH updates: ETH₁ → ETH₂ → ETH₃ (57% compression)
+```
+
+## Features
+
+- ✅ **Multiple algorithms**: Fossil Delta and Xdelta3 (VCDIFF)
+- ✅ **Conflation keys**: Per-entity message caching for optimal compression
+- ✅ **Automatic cache sync**: Server sends cached messages on subscription
+- ✅ **Bandwidth statistics**: Track compression ratios and savings
+- ✅ **Error recovery**: Automatic resync on decode failures
+
+## Installation
+
+### Required Libraries
+
+Include the delta algorithm libraries before pusher-js:
+
+```html
+<!-- Fossil Delta (recommended) -->
+<script src="https://cdn.jsdelivr.net/npm/fossil-delta@1.0.2/fossil-delta.min.js"></script>
+
+<!-- OR Xdelta3 (VCDIFF) -->
+<script src="https://cdn.ably.io/lib/vcdiff-decoder.min-1.js"></script>
+
+<!-- Pusher JS -->
+<script src="path/to/pusher.min.js"></script>
+```
+
+### NPM
+
+```bash
+npm install fossil-delta
+# or
+npm install @ably/vcdiff-decoder
+```
+
+## Usage
+
+### Basic Setup
+
+```javascript
+const pusher = new Pusher('your-app-key', {
+  cluster: 'mt1',
+  wsHost: 'your-sockudo-server.com',
+  wsPort: 6001,
+  forceTLS: false,
+  
+  // Enable delta compression
+  deltaCompression: {
+    enabled: true,
+    algorithms: ['fossil', 'xdelta3'], // Preferred algorithms
+    debug: true, // Enable debug logging
+    
+    // Optional: Track bandwidth savings
+    onStats: (stats) => {
+      console.log(`Bandwidth saved: ${stats.bandwidthSavedPercent.toFixed(1)}%`);
+      console.log(`Delta messages: ${stats.deltaMessages}`);
+      console.log(`Full messages: ${stats.fullMessages}`);
+    },
+    
+    // Optional: Handle errors
+    onError: (error) => {
+      console.error('Delta compression error:', error);
+    }
+  }
+});
+
+// Subscribe to a channel
+const channel = pusher.subscribe('market-data');
+
+// Bind to events - delta decompression is automatic
+channel.bind('price-update', (data) => {
+  console.log('Price update:', data);
+  // Data is automatically reconstructed from deltas
+});
+```
+
+### Get Statistics
+
+```javascript
+// Get current delta compression stats
+const stats = pusher.getDeltaStats();
+
+if (stats) {
+  console.log('Total messages:', stats.totalMessages);
+  console.log('Delta messages:', stats.deltaMessages);
+  console.log('Bandwidth saved:', stats.bandwidthSavedPercent.toFixed(1) + '%');
+  console.log('Channels using delta:', stats.channelCount);
+  
+  // Per-channel statistics
+  stats.channels.forEach(channelStats => {
+    console.log(`Channel ${channelStats.channelName}:`, {
+      conflationKey: channelStats.conflationKey,
+      deltaCount: channelStats.deltaCount,
+      totalMessages: channelStats.totalMessages
+    });
+  });
+}
+
+// Reset statistics
+pusher.resetDeltaStats();
+```
+
+## Server Configuration
+
+### Enable Delta Compression (Sockudo)
+
+On the server side, configure delta compression with conflation keys:
+
+```javascript
+// Sockudo server configuration
+const server = new Sockudo({
+  deltaCompression: {
+    enabled: true,
+    algorithm: 'fossil', // or 'xdelta3'
+    
+    // Channel-specific configuration
+    channels: {
+      'market-data': {
+        conflationKey: 'asset', // Group by 'asset' field
+        maxMessagesPerKey: 100   // Keep last 100 messages per asset
+      }
+    }
+  }
+});
+```
+
+### Message Format
+
+When delta compression is enabled, full messages include metadata:
+
+```javascript
+{
+  "asset": "BTC",
+  "price": "50000.00",
+  "volume": 1234.56,
+  "__delta_seq": 42,           // Sequence number
+  "__conflation_key": "BTC"    // Conflation key value
+}
+```
+
+Delta messages look like:
+
+```javascript
+{
+  "event": "pusher:delta",
+  "channel": "market-data",
+  "data": {
+    "event": "price-update",
+    "delta": "base64-encoded-delta",
+    "seq": 43,
+    "algorithm": "fossil",
+    "conflation_key": "BTC",
+    "base_index": 0
+  }
+}
+```
+
+## Protocol Events
+
+The following Pusher protocol events are handled automatically:
+
+### `pusher:enable_delta_compression`
+
+Sent by the client to enable delta compression:
+
+```javascript
+{
+  "event": "pusher:enable_delta_compression",
+  "data": {
+    "algorithms": ["fossil", "xdelta3"]
+  }
+}
+```
+
+### `pusher:delta_compression_enabled`
+
+Server confirms delta compression is enabled:
+
+```javascript
+{
+  "event": "pusher:delta_compression_enabled",
+  "data": {
+    "enabled": true,
+    "algorithm": "fossil"
+  }
+}
+```
+
+### `pusher:delta_cache_sync`
+
+Server sends cached messages on channel subscription:
+
+```javascript
+{
+  "event": "pusher:delta_cache_sync",
+  "channel": "market-data",
+  "data": {
+    "conflation_key": "asset",
+    "max_messages_per_key": 100,
+    "states": {
+      "BTC": [
+        { "content": "{...}", "seq": 40 },
+        { "content": "{...}", "seq": 41 }
+      ],
+      "ETH": [
+        { "content": "{...}", "seq": 38 }
+      ]
+    }
+  }
+}
+```
+
+### `pusher:delta`
+
+Server sends delta-compressed message:
+
+```javascript
+{
+  "event": "pusher:delta",
+  "channel": "market-data",
+  "data": {
+    "event": "price-update",
+    "delta": "...", // base64 encoded
+    "seq": 42,
+    "algorithm": "fossil",
+    "conflation_key": "BTC",
+    "base_index": 0
+  }
+}
+```
+
+### `pusher:delta_sync_error`
+
+Client requests resync when decode fails:
+
+```javascript
+{
+  "event": "pusher:delta_sync_error",
+  "data": {
+    "channel": "market-data"
+  }
+}
+```
+
+## Compression Results
+
+Based on real-world testing with market data:
+
+### Without Conflation Keys
+- **Compression**: ~32.6% bandwidth savings
+- Messages compared sequentially regardless of content similarity
+
+### With Conflation Keys
+- **Compression**: ~40-71% bandwidth savings per key
+- BTC updates: 71.4% compression (comparing BTC→BTC)
+- ETH updates: 56.8% compression (comparing ETH→ETH)
+
+## TypeScript Support
+
+Full TypeScript definitions are included:
+
+```typescript
+import Pusher from 'pusher-js';
+import { DeltaStats, DeltaOptions } from 'pusher-js/types/src/core/delta/types';
+
+const pusher = new Pusher('key', {
+  deltaCompression: {
+    enabled: true,
+    algorithms: ['fossil'],
+    onStats: (stats: DeltaStats) => {
+      console.log(stats.bandwidthSavedPercent);
+    }
+  }
+});
+```
+
+## Browser Support
+
+- Modern browsers with WebSocket and TextEncoder/TextDecoder support
+- IE11+ (with polyfills)
+- Node.js 12+
+
+## Performance Considerations
+
+1. **Algorithm Choice**:
+   - **Fossil Delta**: Faster, smaller library (~2KB)
+   - **Xdelta3**: Better compression for larger differences
+
+2. **Conflation Keys**:
+   - Choose keys with many repeated updates (e.g., `asset`, `device_id`)
+   - Avoid high-cardinality keys (millions of unique values)
+
+3. **Cache Size**:
+   - Default: 10 messages per key
+   - Increase for better compression (uses more memory)
+   - Server and client must match
+
+## Troubleshooting
+
+### Delta decode failures
+
+If you see decode errors:
+
+1. Ensure delta libraries are loaded before Pusher
+2. Check that server and client algorithms match
+3. Verify cache sync is received before deltas
+4. Enable debug logging: `deltaCompression: { debug: true }`
+
+### No compression
+
+If compression isn't working:
+
+1. Check server has delta compression enabled
+2. Verify `pusher:delta_compression_enabled` event received
+3. Ensure messages have `__delta_seq` field
+4. Check channel is subscribed before messages arrive
+
+### Poor compression ratio
+
+If compression is low:
+
+1. Configure conflation keys on server
+2. Ensure messages have similar structure
+3. Verify conflation key field exists in messages
+4. Consider increasing `maxMessagesPerKey`
+
+## Examples
+
+See the `interactive/` directory for complete working examples:
+
+- `interactive/public/delta-compression.js` - Client-side implementation
+- `interactive/public/conflation-test.js` - Conflation key testing
+- `interactive/public/app.js` - Full dashboard with statistics
+
+## License
+
+Same as pusher-js (MIT)

package/DELTA_USAGE.md

@@ -0,0 +1,179 @@
+# Delta Compression Usage Guide
+
+## Overview
+
+The sockudo-js library now includes built-in support for delta compression with fossil-delta and @ably/vcdiff-decoder packages. Delta messages are **automatically decoded** and delivered to your event handlers seamlessly.
+
+## Installation
+
+The required packages are already installed as dependencies:
+
+```json
+{
+  "dependencies": {
+    "@ably/vcdiff-decoder": "^1.0.6",
+    "fossil-delta": "^2.0.0"
+  }
+}
+```
+
+## Usage
+
+### 1. Enable Delta Compression
+
+```javascript
+const pusher = new Pusher(appKey, {
+  wsHost: 'localhost',
+  wsPort: 6001,
+  forceTLS: false,
+  enabledTransports: ['ws'],
+  delta: {
+    enabled: true,
+    algorithms: ['fossil', 'xdelta3'],
+    debug: true,
+    onStats: (stats) => {
+      console.log('Delta stats:', stats);
+    }
+  }
+});
+```
+
+### 2. Subscribe and Bind to Events (Seamless API)
+
+**The key feature**: You don't need to handle delta messages manually! Just bind to your events normally:
+
+```javascript
+const channel = pusher.subscribe('my-channel');
+
+// This receives BOTH full messages AND automatically decoded delta messages
+channel.bind('price-update', (data) => {
+  console.log('Received price update:', data);
+  // data is already decoded - whether it was a delta or full message,
+  // you get the complete, decoded message here!
+});
+
+// Works with bind_global too
+channel.bind_global((eventName, data) => {
+  console.log(`Event ${eventName}:`, data);
+  // All events (including decoded deltas) come through here
+});
+```
+
+### 3. Automatic Decoding Flow
+
+Here's what happens behind the scenes:
+
+1. **Full Message** arrives:
+   - Library stores it as base for future deltas
+   - Emits to your `bind()` handler normally
+
+2. **Delta Message** arrives:
+   - Library automatically decodes it using the stored base
+   - Reconstructs the full message
+   - Updates the base for next delta
+   - Emits the **decoded** message to your `bind()` handler
+   - Your handler receives the full, decoded data (no difference from full message!)
+
+### 4. What You Should NOT Do
+
+❌ **Don't manually bind to `pusher:delta`**:
+```javascript
+// DON'T DO THIS - it's redundant!
+channel.bind('pusher:delta', (data) => {
+  // Manual decoding...
+});
+```
+
+The library handles delta decoding internally. Binding to `pusher:delta` will not receive events because the library intercepts and decodes them before routing.
+
+### 5. Monitoring Delta Compression Stats
+
+```javascript
+const pusher = new Pusher(appKey, {
+  delta: {
+    enabled: true,
+    onStats: (stats) => {
+      console.log('Total messages:', stats.totalMessages);
+      console.log('Delta messages:', stats.deltaMessages);
+      console.log('Full messages:', stats.fullMessages);
+      console.log('Bandwidth saved:', stats.bandwidthSavedPercent.toFixed(1) + '%');
+    }
+  }
+});
+```
+
+### 6. Advanced: Conflation Keys
+
+For channels using conflation (multiple entity streams on one channel):
+
+```javascript
+// No changes needed in your code!
+// The library automatically handles conflation keys internally
+
+channel.bind('price-update', (data) => {
+  // You get the decoded message for the specific entity
+  // Library manages separate base messages per conflation key
+});
+```
+
+## Example: Complete Integration
+
+```javascript
+// Initialize Pusher with delta compression
+const pusher = new Pusher('your-app-key', {
+  wsHost: 'localhost',
+  wsPort: 6001,
+  forceTLS: false,
+  enabledTransports: ['ws'],
+  delta: {
+    enabled: true,
+    debug: true,
+    onStats: (stats) => {
+      document.getElementById('bandwidth-saved').textContent = 
+        stats.bandwidthSavedPercent.toFixed(1) + '%';
+    }
+  }
+});
+
+// Subscribe to channel
+const channel = pusher.subscribe('stock-prices');
+
+// Bind to events - seamlessly receives full AND decoded delta messages
+channel.bind('price-update', (data) => {
+  // data is always the complete, decoded message
+  updateUI(data.symbol, data.price, data.volume);
+});
+
+// Handle subscription success
+channel.bind('pusher:subscription_succeeded', () => {
+  console.log('Subscribed successfully');
+});
+```
+
+## Benefits
+
+- ✅ **Seamless API**: No changes needed to handle delta vs full messages
+- ✅ **Automatic decoding**: Library handles all complexity
+- ✅ **Bandwidth savings**: 60-90% reduction for similar consecutive messages
+- ✅ **Transparent**: Your event handlers work the same way with or without delta compression
+- ✅ **Conflation support**: Automatically handles multiple entity streams
+- ✅ **Error recovery**: Automatic resync on decode failures
+
+## Troubleshooting
+
+### Delta messages not decoding
+
+1. Check that fossil-delta and @ably/vcdiff-decoder are installed
+2. Enable debug mode: `delta: { debug: true }`
+3. Check browser console for `[DeltaCompression]` logs
+4. Verify server has delta compression enabled
+
+### High error count
+
+Check stats for errors:
+```javascript
+const stats = pusher.deltaCompression?.getStats();
+console.log('Errors:', stats.errors);
+```
+
+If errors are high, the library will automatically request resync from the server.