@push.rocks/smartproxy 19.5.25 → 19.6.0

package/readme.connections.md

@@ -673,4 +673,52 @@ if (!record.connectionClosed && record.outgoing && record.bytesReceived > 0 && r
 - Connection is older than 60 seconds
 - Both sockets are still alive (not destroyed)
 
-This complements the zombie detection by handling cases where sockets remain technically alive but the connection is effectively dead.
+This complements the zombie detection by handling cases where sockets remain technically alive but the connection is effectively dead.
+
+## 🚨 CRITICAL FIX: Cleanup Queue Bug (January 2025)
+
+### Critical Bug Found
+The cleanup queue had a severe bug that caused connection accumulation when more than 100 connections needed cleanup:
+
+```typescript
+// BUG: This cleared the ENTIRE queue after processing only the first batch!
+const toCleanup = Array.from(this.cleanupQueue).slice(0, this.cleanupBatchSize);
+this.cleanupQueue.clear(); // ❌ This discarded all connections beyond the first 100!
+```
+
+### Fix Implemented
+```typescript
+// Now only removes the connections being processed
+const toCleanup = Array.from(this.cleanupQueue).slice(0, this.cleanupBatchSize);
+for (const connectionId of toCleanup) {
+  this.cleanupQueue.delete(connectionId); // ✅ Only remove what we process
+  const record = this.connectionRecords.get(connectionId);
+  if (record) {
+    this.cleanupConnection(record, record.incomingTerminationReason || 'normal');
+  }
+}
+```
+
+### Impact
+- **Before**: If 150 connections needed cleanup, only the first 100 would be processed and the remaining 50 would accumulate forever
+- **After**: All connections are properly cleaned up in batches
+
+### Additional Improvements
+
+1. **Faster Inactivity Checks**: Reduced from 30s to 10s intervals
+   - Zombies and stuck connections are detected 3x faster
+   - Reduces the window for accumulation
+
+2. **Duplicate Prevention**: Added check in queueCleanup to prevent processing already-closed connections
+   - Prevents unnecessary work
+   - Ensures connections are only cleaned up once
+
+### Summary of All Fixes
+
+1. **Connection Timeout** (already documented) - Prevents accumulation when backends are unreachable
+2. **Zombie Detection** - Cleans up connections with destroyed sockets
+3. **Stuck Connection Detection** - Cleans up connections to hanging backends
+4. **Cleanup Queue Bug** - Ensures ALL connections get cleaned up, not just the first 100
+5. **Faster Detection** - Reduced check interval from 30s to 10s
+
+These fixes combined should prevent connection accumulation in all known scenarios.

package/readme.md

@@ -919,6 +919,124 @@ Available helper functions:
     })
     ```
 
+## Metrics and Monitoring
+
+SmartProxy includes a comprehensive metrics collection system that provides real-time insights into proxy performance, connection statistics, and throughput data.
+
+### Getting Metrics
+
+```typescript
+const proxy = new SmartProxy({ /* config */ });
+await proxy.start();
+
+// Access metrics through the getStats() method
+const stats = proxy.getStats();
+
+// Get current active connections
+console.log(`Active connections: ${stats.getActiveConnections()}`);
+
+// Get total connections since start
+console.log(`Total connections: ${stats.getTotalConnections()}`);
+
+// Get requests per second (RPS)
+console.log(`Current RPS: ${stats.getRequestsPerSecond()}`);
+
+// Get throughput data
+const throughput = stats.getThroughput();
+console.log(`Bytes received: ${throughput.bytesIn}`);
+console.log(`Bytes sent: ${throughput.bytesOut}`);
+
+// Get connections by route
+const routeConnections = stats.getConnectionsByRoute();
+for (const [route, count] of routeConnections) {
+  console.log(`Route ${route}: ${count} connections`);
+}
+
+// Get connections by IP address
+const ipConnections = stats.getConnectionsByIP();
+for (const [ip, count] of ipConnections) {
+  console.log(`IP ${ip}: ${count} connections`);
+}
+```
+
+### Available Metrics
+
+The `IProxyStats` interface provides the following methods:
+
+- `getActiveConnections()`: Current number of active connections
+- `getTotalConnections()`: Total connections handled since proxy start
+- `getRequestsPerSecond()`: Current requests per second (1-minute average)
+- `getThroughput()`: Total bytes transferred (in/out)
+- `getConnectionsByRoute()`: Connection count per route
+- `getConnectionsByIP()`: Connection count per client IP
+
+### Monitoring Example
+
+```typescript
+// Create a monitoring loop
+setInterval(() => {
+  const stats = proxy.getStats();
+  
+  // Log key metrics
+  console.log({
+    timestamp: new Date().toISOString(),
+    activeConnections: stats.getActiveConnections(),
+    rps: stats.getRequestsPerSecond(),
+    throughput: stats.getThroughput()
+  });
+  
+  // Check for high connection counts from specific IPs
+  const ipConnections = stats.getConnectionsByIP();
+  for (const [ip, count] of ipConnections) {
+    if (count > 100) {
+      console.warn(`High connection count from ${ip}: ${count}`);
+    }
+  }
+}, 10000); // Every 10 seconds
+```
+
+### Exporting Metrics
+
+You can export metrics in various formats for external monitoring systems:
+
+```typescript
+// Export as JSON
+app.get('/metrics.json', (req, res) => {
+  const stats = proxy.getStats();
+  res.json({
+    activeConnections: stats.getActiveConnections(),
+    totalConnections: stats.getTotalConnections(),
+    requestsPerSecond: stats.getRequestsPerSecond(),
+    throughput: stats.getThroughput(),
+    connectionsByRoute: Object.fromEntries(stats.getConnectionsByRoute()),
+    connectionsByIP: Object.fromEntries(stats.getConnectionsByIP())
+  });
+});
+
+// Export as Prometheus format
+app.get('/metrics', (req, res) => {
+  const stats = proxy.getStats();
+  res.set('Content-Type', 'text/plain');
+  res.send(`
+# HELP smartproxy_active_connections Current active connections
+# TYPE smartproxy_active_connections gauge
+smartproxy_active_connections ${stats.getActiveConnections()}
+
+# HELP smartproxy_requests_per_second Current requests per second
+# TYPE smartproxy_requests_per_second gauge
+smartproxy_requests_per_second ${stats.getRequestsPerSecond()}
+
+# HELP smartproxy_bytes_in Total bytes received
+# TYPE smartproxy_bytes_in counter
+smartproxy_bytes_in ${stats.getThroughput().bytesIn}
+
+# HELP smartproxy_bytes_out Total bytes sent
+# TYPE smartproxy_bytes_out counter
+smartproxy_bytes_out ${stats.getThroughput().bytesOut}
+`);
+});
+```
+
 ## Other Components
 
 While SmartProxy provides a unified API for most needs, you can also use individual components: