@push.rocks/smartproxy 19.6.2 → 19.6.7

package/readme.plan.md

@@ -1,625 +1,364 @@
-# PROXY Protocol Implementation Plan
-
-## ⚠️ CRITICAL: Implementation Order
-
-**Phase 1 (ProxyProtocolSocket/WrappedSocket) MUST be completed first!**
-
-The ProxyProtocolSocket class is the foundation that enables all PROXY protocol functionality. No protocol parsing or integration can happen until this wrapper class is fully implemented and tested.
-
-1. **FIRST**: Implement ProxyProtocolSocket (the WrappedSocket)
-2. **THEN**: Add PROXY protocol parser
-3. **THEN**: Integrate with connection handlers
-4. **FINALLY**: Add security and validation
+# SmartProxy Metrics Improvement Plan
 
 ## Overview
-Implement PROXY protocol support in SmartProxy to preserve client IP information through proxy chains, solving the connection limit accumulation issue where inner proxies see all connections as coming from the outer proxy's IP.
 
-## Problem Statement
-- In proxy chains, the inner proxy sees all connections from the outer proxy's IP
-- This causes the inner proxy to hit per-IP connection limits (default: 100)
-- Results in connection rejections while outer proxy accumulates connections
+The current `getThroughputRate()` implementation calculates cumulative throughput over a 60-second window rather than providing an actual rate, making metrics misleading for monitoring systems. This plan outlines a comprehensive redesign of the metrics system to provide accurate, time-series based metrics suitable for production monitoring.
 
-## Solution Design
+## 1. Core Issues with Current Implementation
 
-### 1. Core Features
+- **Cumulative vs Rate**: Current method accumulates all bytes from connections in the last minute rather than calculating actual throughput rate
+- **No Time-Series Data**: Cannot track throughput changes over time
+- **Inaccurate Estimates**: Attempting to estimate rates for older connections is fundamentally flawed
+- **No Sliding Windows**: Cannot provide different time window views (1s, 10s, 60s, etc.)
+- **Limited Granularity**: Only provides a single 60-second view
 
-#### 1.1 PROXY Protocol Parsing
-- Support PROXY protocol v1 (text format) initially
-- Parse incoming PROXY headers to extract:
-  - Real client IP address
-  - Real client port
-  - Proxy IP address
-  - Proxy port
-  - Protocol (TCP4/TCP6)
+## 2. Proposed Architecture
 
-#### 1.2 PROXY Protocol Generation
-- Add ability to send PROXY protocol headers when forwarding connections
-- Configurable per route or target
-
-#### 1.3 Trusted Proxy IPs
-- New `proxyIPs` array in SmartProxy options
-- Auto-enable PROXY protocol acceptance for connections from these IPs
-- Reject PROXY protocol from untrusted sources (security)
-
-### 2. Configuration Schema
+### A. Time-Series Throughput Tracking
 
 ```typescript
-interface ISmartProxyOptions {
-  // ... existing options
-  
-  // List of trusted proxy IPs that can send PROXY protocol
-  proxyIPs?: string[];
-  
-  // Global option to accept PROXY protocol (defaults based on proxyIPs)
-  acceptProxyProtocol?: boolean;
-  
-  // Global option to send PROXY protocol to all targets
-  sendProxyProtocol?: boolean;
+interface IThroughputSample {
+  timestamp: number;
+  bytesIn: number;
+  bytesOut: number;
 }
 
-interface IRouteAction {
-  // ... existing options
-  
-  // Send PROXY protocol to this specific target
-  sendProxyProtocol?: boolean;
-}
-```
-
-### 3. Implementation Steps
-
-#### IMPORTANT: Phase 1 Must Be Completed First
-The `ProxyProtocolSocket` (WrappedSocket) is the foundation for all PROXY protocol functionality. This wrapper class must be implemented and integrated BEFORE any PROXY protocol parsing can begin.
-
-#### Phase 1: ProxyProtocolSocket (WrappedSocket) Foundation - ✅ COMPLETED (v19.5.19)
-This phase creates the socket wrapper infrastructure that all subsequent phases depend on.
-
-1. **Create WrappedSocket class** in `ts/core/models/wrapped-socket.ts` ✅
-   - Used JavaScript Proxy pattern instead of EventEmitter (avoids infinite loops)
-   - Properties for real client IP and port
-   - Transparent getters that return real or socket IP/port
-   - All socket methods/properties delegated via Proxy
-   
-2. **Implement core wrapper functionality** ✅
-   - Constructor accepts regular socket + optional metadata
-   - `remoteAddress` getter returns real IP or falls back to socket IP
-   - `remotePort` getter returns real port or falls back to socket port
-   - `isFromTrustedProxy` property to check if it has real client info
-   - `setProxyInfo()` method to update real client details
-   
-3. **Update ConnectionManager to handle wrapped sockets** ✅
-   - Accept either `net.Socket` or `WrappedSocket`
-   - Created `getUnderlyingSocket()` helper for socket utilities
-   - All socket utility functions extract underlying socket
-   
-4. **Integration completed** ✅
-   - All incoming sockets wrapped in RouteConnectionHandler
-   - Socket forwarding verified working with wrapped sockets
-   - Type safety maintained with index signature
-
-**Deliverables**: ✅ Working WrappedSocket that can wrap any socket and provide transparent access to client info.
-
-#### Phase 2: PROXY Protocol Parser - ✅ COMPLETED (v19.5.21)
-Only after WrappedSocket is working can we add protocol parsing.
-
-1. ✅ Created `ProxyProtocolParser` class in `ts/core/utils/proxy-protocol.ts`
-2. ✅ Implemented v1 text format parsing with full validation
-3. ✅ Added comprehensive error handling and IP validation
-4. ✅ Integrated parser to work WITH WrappedSocket in RouteConnectionHandler
-
-**Deliverables**: ✅ Working PROXY protocol v1 parser that validates headers, extracts client info, and handles both TCP4 and TCP6 protocols.
-
-#### Phase 3: Connection Handler Integration - ✅ COMPLETED (v19.5.21)
-1. ✅ Modify `RouteConnectionHandler` to create WrappedSocket for all connections
-2. ✅ Check if connection is from trusted proxy IP
-3. ✅ If trusted, attempt to parse PROXY protocol header
-4. ✅ Update wrapped socket with real client info
-5. ✅ Continue normal connection handling with wrapped socket
-
-**Deliverables**: ✅ RouteConnectionHandler now parses PROXY protocol from trusted proxies and updates connection records with real client info.
-
-#### Phase 4: Outbound PROXY Protocol - ✅ COMPLETED (v19.5.21)
-1. ✅ Add PROXY header generation in `setupDirectConnection`
-2. ✅ Make it configurable per route via `sendProxyProtocol` option
-3. ✅ Send header immediately after TCP connection
-4. ✅ Added remotePort tracking to connection records
-
-**Deliverables**: ✅ SmartProxy can now send PROXY protocol headers to backend servers when configured, preserving client IP through proxy chains.
-
-#### Phase 5: Security & Validation - FINAL PHASE
-1. Validate PROXY headers strictly
-2. Reject malformed headers
-3. Only accept from trusted IPs
-4. Add rate limiting for PROXY protocol parsing
-
-### 4. Design Decision: Socket Wrapper Architecture
-
-#### Option A: Minimal Single Socket Wrapper
-- **Scope**: Wraps individual sockets with metadata
-- **Use Case**: PROXY protocol support with minimal refactoring
-- **Pros**: Simple, low risk, easy migration
-- **Cons**: Still need separate connection management
-
-#### Option B: Comprehensive Connection Wrapper
-- **Scope**: Manages socket pairs (incoming + outgoing) with all utilities
-- **Use Case**: Complete connection lifecycle management
-- **Pros**: 
-  - Encapsulates all socket utilities (forwarding, cleanup, backpressure)
-  - Single object represents entire connection
-  - Cleaner API for connection handling
-- **Cons**: 
-  - Major architectural change
-  - Higher implementation risk
-  - More complex migration
-
-#### Recommendation
-Start with **Option A** (ProxyProtocolSocket) for immediate PROXY protocol support, then evaluate Option B based on:
-- Performance impact of additional abstraction
-- Code simplification benefits
-- Team comfort with architectural change
-
-### 5. Code Implementation Details
-
-#### 5.1 ProxyProtocolSocket (WrappedSocket) - PHASE 1 IMPLEMENTATION
-This is the foundational wrapper class that MUST be implemented first. It wraps a regular socket and provides transparent access to the real client IP/port.
-
-```typescript
-// ts/core/models/proxy-protocol-socket.ts
-import { EventEmitter } from 'events';
-import * as plugins from '../../../plugins.js';
-
-/**
- * ProxyProtocolSocket wraps a regular net.Socket to provide transparent access
- * to the real client IP and port when behind a proxy using PROXY protocol.
- * 
- * This is the FOUNDATION for all PROXY protocol support and must be implemented
- * before any protocol parsing can occur.
- */
-export class ProxyProtocolSocket extends EventEmitter {
-  private realClientIP?: string;
-  private realClientPort?: number;
+class ThroughputTracker {
+  private samples: IThroughputSample[] = [];
+  private readonly MAX_SAMPLES = 3600; // 1 hour at 1 sample/second
+  private lastSampleTime: number = 0;
+  private accumulatedBytesIn: number = 0;
+  private accumulatedBytesOut: number = 0;
   
-  constructor(
-    public readonly socket: plugins.net.Socket,
-    realClientIP?: string,
-    realClientPort?: number
-  ) {
-    super();
-    this.realClientIP = realClientIP;
-    this.realClientPort = realClientPort;
-    
-    // Forward all socket events
-    this.forwardSocketEvents();
-  }
-  
-  /**
-   * Returns the real client IP if available, otherwise the socket's remote address
-   */
-  get remoteAddress(): string | undefined {
-    return this.realClientIP || this.socket.remoteAddress;
-  }
-  
-  /**
-   * Returns the real client port if available, otherwise the socket's remote port
-   */
-  get remotePort(): number | undefined {
-    return this.realClientPort || this.socket.remotePort;
-  }
-  
-  /**
-   * Indicates if this connection came through a trusted proxy
-   */
-  get isFromTrustedProxy(): boolean {
-    return !!this.realClientIP;
+  // Called on every data transfer
+  public recordBytes(bytesIn: number, bytesOut: number): void {
+    this.accumulatedBytesIn += bytesIn;
+    this.accumulatedBytesOut += bytesOut;
   }
   
-  /**
-   * Updates the real client information (called after parsing PROXY protocol)
-   */
-  setProxyInfo(ip: string, port: number): void {
-    this.realClientIP = ip;
-    this.realClientPort = port;
-  }
-  
-  // Pass-through all socket methods
-  write(data: any, encoding?: any, callback?: any): boolean {
-    return this.socket.write(data, encoding, callback);
-  }
-  
-  end(data?: any, encoding?: any, callback?: any): this {
-    this.socket.end(data, encoding, callback);
-    return this;
-  }
-  
-  destroy(error?: Error): this {
-    this.socket.destroy(error);
-    return this;
+  // Called periodically (every second)
+  public takeSample(): void {
+    const now = Date.now();
+    
+    // Record accumulated bytes since last sample
+    this.samples.push({
+      timestamp: now,
+      bytesIn: this.accumulatedBytesIn,
+      bytesOut: this.accumulatedBytesOut
+    });
+    
+    // Reset accumulators
+    this.accumulatedBytesIn = 0;
+    this.accumulatedBytesOut = 0;
+    
+    // Trim old samples
+    const cutoff = now - 3600000; // 1 hour
+    this.samples = this.samples.filter(s => s.timestamp > cutoff);
   }
   
-  // ... implement all other socket methods as pass-through
-  
-  /**
-   * Forward all events from the underlying socket
-   */
-  private forwardSocketEvents(): void {
-    const events = ['data', 'end', 'close', 'error', 'drain', 'timeout'];
-    events.forEach(event => {
-      this.socket.on(event, (...args) => {
-        this.emit(event, ...args);
-      });
-    });
+  // Get rate over specified window
+  public getRate(windowSeconds: number): { bytesInPerSec: number; bytesOutPerSec: number } {
+    const now = Date.now();
+    const windowStart = now - (windowSeconds * 1000);
+    
+    const relevantSamples = this.samples.filter(s => s.timestamp > windowStart);
+    
+    if (relevantSamples.length === 0) {
+      return { bytesInPerSec: 0, bytesOutPerSec: 0 };
+    }
+    
+    const totalBytesIn = relevantSamples.reduce((sum, s) => sum + s.bytesIn, 0);
+    const totalBytesOut = relevantSamples.reduce((sum, s) => sum + s.bytesOut, 0);
+    
+    const actualWindow = (now - relevantSamples[0].timestamp) / 1000;
+    
+    return {
+      bytesInPerSec: Math.round(totalBytesIn / actualWindow),
+      bytesOutPerSec: Math.round(totalBytesOut / actualWindow)
+    };
   }
 }
 ```
 
-**KEY POINT**: This wrapper must be fully functional and tested BEFORE moving to Phase 2.
+### B. Connection-Level Byte Tracking
 
-#### 4.2 ProxyProtocolParser (new file)
 ```typescript
-// ts/core/utils/proxy-protocol.ts
-export class ProxyProtocolParser {
-  static readonly PROXY_V1_SIGNATURE = 'PROXY ';
+// In ConnectionRecord, add:
+interface IConnectionRecord {
+  // ... existing fields ...
   
-  static parse(chunk: Buffer): IProxyInfo | null {
-    // Implementation
-  }
+  // Byte counters with timestamps
+  bytesReceivedHistory: Array<{ timestamp: number; bytes: number }>;
+  bytesSentHistory: Array<{ timestamp: number; bytes: number }>;
   
-  static generate(info: IProxyInfo): Buffer {
-    // Implementation
-  }
+  // For efficiency, could use circular buffer
+  lastBytesReceivedUpdate: number;
+  lastBytesSentUpdate: number;
 }
 ```
 
-#### 4.3 Connection Handler Updates
-```typescript
-// In handleConnection method
-let wrappedSocket: ProxyProtocolSocket | plugins.net.Socket = socket;
+### C. Enhanced Metrics Interface
 
-// Wrap socket if from trusted proxy
-if (this.settings.proxyIPs?.includes(socket.remoteAddress)) {
-  wrappedSocket = new ProxyProtocolSocket(socket);
+```typescript
+interface IMetrics {
+  // Connection metrics
+  connections: {
+    active(): number;
+    total(): number;
+    byRoute(): Map<string, number>;
+    byIP(): Map<string, number>;
+    topIPs(limit?: number): Array<{ ip: string; count: number }>;
+  };
+  
+  // Throughput metrics (bytes per second)
+  throughput: {
+    instant(): { in: number; out: number };      // Last 1 second
+    recent(): { in: number; out: number };       // Last 10 seconds  
+    average(): { in: number; out: number };      // Last 60 seconds
+    custom(seconds: number): { in: number; out: number };
+    history(seconds: number): Array<{ timestamp: number; in: number; out: number }>;
+    byRoute(windowSeconds?: number): Map<string, { in: number; out: number }>;
+    byIP(windowSeconds?: number): Map<string, { in: number; out: number }>;
+  };
+  
+  // Request metrics
+  requests: {
+    perSecond(): number;
+    perMinute(): number;
+    total(): number;
+  };
+  
+  // Cumulative totals
+  totals: {
+    bytesIn(): number;
+    bytesOut(): number;
+    connections(): number;
+  };
+  
+  // Performance metrics
+  percentiles: {
+    connectionDuration(): { p50: number; p95: number; p99: number };
+    bytesTransferred(): { 
+      in: { p50: number; p95: number; p99: number };
+      out: { p50: number; p95: number; p99: number };
+    };
+  };
 }
+```
 
-// Create connection record with wrapped socket
-const record = this.connectionManager.createConnection(wrappedSocket);
+## 3. Implementation Plan
+
+### Current Status
+- **Phase 1**: ~90% complete (core functionality implemented, tests need fixing)
+- **Phase 2**: ~60% complete (main features done, percentiles pending)
+- **Phase 3**: ~40% complete (basic optimizations in place)
+- **Phase 4**: 0% complete (export formats not started)
+
+### Phase 1: Core Throughput Tracking (Week 1)
+- [x] Implement `ThroughputTracker` class
+- [x] Integrate byte recording into socket data handlers
+- [x] Add periodic sampling (1-second intervals)
+- [x] Update `getThroughputRate()` to use time-series data (replaced with new clean API)
+- [ ] Add unit tests for throughput tracking
+
+### Phase 2: Enhanced Metrics (Week 2)
+- [x] Add configurable time windows (1s, 10s, 60s, 5m, etc.)
+- [ ] Implement percentile calculations
+- [x] Add route-specific and IP-specific throughput tracking
+- [x] Create historical data access methods
+- [ ] Add integration tests
+
+### Phase 3: Performance Optimization (Week 3)
+- [x] Use circular buffers for efficiency
+- [ ] Implement data aggregation for longer time windows
+- [x] Add configurable retention periods
+- [ ] Optimize memory usage
+- [ ] Add performance benchmarks
+
+### Phase 4: Export Formats (Week 4)
+- [ ] Add Prometheus metric format with proper metric types
+- [ ] Add StatsD format support
+- [ ] Add JSON export with metadata
+- [ ] Create OpenMetrics compatibility
+- [ ] Add documentation and examples
+
+## 4. Key Design Decisions
+
+### A. Sampling Strategy
+- **1-second samples** for fine-grained data
+- **Aggregate to 1-minute** for longer retention
+- **Keep 1 hour** of second-level data
+- **Keep 24 hours** of minute-level data
+
+### B. Memory Management
+- **Circular buffers** for fixed memory usage
+- **Configurable retention** periods
+- **Lazy aggregation** for older data
+- **Efficient data structures** (typed arrays for samples)
+
+### C. Performance Considerations
+- **Batch updates** during high throughput
+- **Debounced calculations** for expensive metrics
+- **Cached results** with TTL
+- **Worker thread** option for heavy calculations
+
+## 5. Configuration Options
 
-// In handleInitialData method
-if (wrappedSocket instanceof ProxyProtocolSocket) {
-  const proxyInfo = await this.checkForProxyProtocol(chunk);
-  if (proxyInfo) {
-    wrappedSocket.setProxyInfo(proxyInfo.sourceIP, proxyInfo.sourcePort);
-    // Continue with remaining data after PROXY header
-  }
+```typescript
+interface IMetricsConfig {
+  enabled: boolean;
+  
+  // Sampling configuration
+  sampleIntervalMs: number;        // Default: 1000 (1 second)
+  retentionSeconds: number;        // Default: 3600 (1 hour)
+  
+  // Performance tuning
+  enableDetailedTracking: boolean; // Per-connection byte history
+  enablePercentiles: boolean;      // Calculate percentiles
+  cacheResultsMs: number;         // Cache expensive calculations
+  
+  // Export configuration
+  prometheusEnabled: boolean;
+  prometheusPath: string;         // Default: /metrics
+  prometheusPrefix: string;       // Default: smartproxy_
 }
 ```
 
-#### 4.4 Security Manager Updates
-- Accept socket or ProxyProtocolSocket
-- Use `socket.remoteAddress` getter for real client IP
-- Transparent handling of both socket types
+## 6. Example Usage
 
-### 5. Configuration Examples
-
-#### Basic Setup (IMPLEMENTED ✅)
 ```typescript
-// Outer proxy - sends PROXY protocol
-const outerProxy = new SmartProxy({
-  routes: [{
-    name: 'to-inner-proxy',
-    match: { ports: 443 },
-    action: {
-      type: 'forward',
-      target: { host: '195.201.98.232', port: 443 },
-      sendProxyProtocol: true  // Enable for this route
-    }
-  }]
+const proxy = new SmartProxy({
+  metrics: {
+    enabled: true,
+    sampleIntervalMs: 1000,
+    enableDetailedTracking: true
+  }
 });
 
-// Inner proxy - accepts PROXY protocol from outer proxy
-const innerProxy = new SmartProxy({
-  proxyIPs: ['212.95.99.130'],  // Outer proxy IP
-  acceptProxyProtocol: true,     // Optional - defaults to true when proxyIPs is set
-  routes: [{
-    name: 'to-backend',
-    match: { ports: 443 },
-    action: {
-      type: 'forward',
-      target: { host: '192.168.5.247', port: 443 }
-    }
-  }]
-});
-```
+// Get metrics instance
+const metrics = proxy.getMetrics();
 
-### 6. Testing Plan
+// Connection metrics
+console.log(`Active connections: ${metrics.connections.active()}`);
+console.log(`Total connections: ${metrics.connections.total()}`);
 
-#### Unit Tests
-- PROXY protocol v1 parsing (valid/invalid formats)
-- Header generation
-- Trusted IP validation
-- Connection record updates
+// Throughput metrics
+const instant = metrics.throughput.instant();
+console.log(`Current: ${instant.in} bytes/sec in, ${instant.out} bytes/sec out`);
 
-#### Integration Tests
-- Single proxy with PROXY protocol
-- Proxy chain with PROXY protocol
-- Security: reject from untrusted IPs
-- Performance: minimal overhead
-- Compatibility: works with TLS passthrough
+const recent = metrics.throughput.recent();   // Last 10 seconds
+const average = metrics.throughput.average(); // Last 60 seconds
 
-#### Test Scenarios
-1. **Connection limit test**: Verify inner proxy sees real client IPs
-2. **Security test**: Ensure PROXY protocol rejected from untrusted sources
-3. **Compatibility test**: Verify no impact on non-PROXY connections
-4. **Performance test**: Measure overhead of PROXY protocol parsing
+// Custom time window
+const custom = metrics.throughput.custom(30); // Last 30 seconds
 
-### 7. Security Considerations
+// Historical data for graphing
+const history = metrics.throughput.history(300); // Last 5 minutes
+history.forEach(point => {
+  console.log(`${new Date(point.timestamp)}: ${point.in} bytes/sec in, ${point.out} bytes/sec out`);
+});
 
-1. **IP Spoofing Prevention**
-   - Only accept PROXY protocol from explicitly trusted IPs
-   - Validate all header fields
-   - Reject malformed headers immediately
+// Top routes by throughput
+const routeThroughput = metrics.throughput.byRoute(60);
+routeThroughput.forEach((stats, route) => {
+  console.log(`Route ${route}: ${stats.in} bytes/sec in, ${stats.out} bytes/sec out`);
+});
 
-2. **Resource Protection**
-   - Limit PROXY header size (107 bytes for v1)
-   - Timeout for incomplete headers
-   - Rate limit connection attempts
+// Request metrics
+console.log(`RPS: ${metrics.requests.perSecond()}`);
+console.log(`RPM: ${metrics.requests.perMinute()}`);
 
-3. **Logging**
-   - Log all PROXY protocol acceptance/rejection
-   - Include real client IP in all connection logs
+// Totals
+console.log(`Total bytes in: ${metrics.totals.bytesIn()}`);
+console.log(`Total bytes out: ${metrics.totals.bytesOut()}`);
+```
 
-### 8. Rollout Strategy
+## 7. Prometheus Export Example
 
-1. **Phase 1**: Deploy parser and acceptance (backward compatible)
-2. **Phase 2**: Enable between controlled proxy pairs
-3. **Phase 3**: Monitor for issues and performance impact
-4. **Phase 4**: Expand to all proxy chains
+```
+# HELP smartproxy_throughput_bytes_per_second Current throughput in bytes per second
+# TYPE smartproxy_throughput_bytes_per_second gauge
+smartproxy_throughput_bytes_per_second{direction="in",window="1s"} 1234567
+smartproxy_throughput_bytes_per_second{direction="out",window="1s"} 987654
+smartproxy_throughput_bytes_per_second{direction="in",window="10s"} 1134567
+smartproxy_throughput_bytes_per_second{direction="out",window="10s"} 887654
+
+# HELP smartproxy_bytes_total Total bytes transferred
+# TYPE smartproxy_bytes_total counter
+smartproxy_bytes_total{direction="in"} 123456789
+smartproxy_bytes_total{direction="out"} 98765432
+
+# HELP smartproxy_active_connections Current number of active connections
+# TYPE smartproxy_active_connections gauge
+smartproxy_active_connections 42
+
+# HELP smartproxy_connection_duration_seconds Connection duration in seconds
+# TYPE smartproxy_connection_duration_seconds histogram
+smartproxy_connection_duration_seconds_bucket{le="0.1"} 100
+smartproxy_connection_duration_seconds_bucket{le="1"} 500
+smartproxy_connection_duration_seconds_bucket{le="10"} 800
+smartproxy_connection_duration_seconds_bucket{le="+Inf"} 850
+smartproxy_connection_duration_seconds_sum 4250
+smartproxy_connection_duration_seconds_count 850
+```
 
-### 9. Success Metrics
+## 8. Migration Strategy
 
-- Inner proxy connection distribution matches outer proxy
-- No more connection limit rejections in proxy chains
-- Accurate client IP logging throughout the chain
-- No performance degradation (<1ms added latency)
+### Breaking Changes
+- Completely replace the old metrics API with the new clean design
+- Remove all `get*` prefixed methods in favor of grouped properties
+- Use simple `{ in, out }` objects instead of verbose property names
+- Provide clear migration guide in documentation
 
-### 10. Future Enhancements
+### Implementation Approach
+1. ✅ Create new `ThroughputTracker` class for time-series data
+2. ✅ Implement new `IMetrics` interface with clean API
+3. ✅ Replace `MetricsCollector` implementation entirely
+4. ✅ Update all references to use new API
+5. ⚠️ Add comprehensive tests for accuracy validation (partial)
 
-- PROXY protocol v2 (binary format) support
-- TLV extensions for additional metadata
-- AWS VPC endpoint ID support
-- Custom metadata fields
+### Additional Refactoring Completed
+- Refactored all SmartProxy components to use cleaner dependency pattern
+- Components now receive only `SmartProxy` instance instead of individual dependencies
+- Access to other components via `this.smartProxy.componentName`
+- Significantly simplified constructor signatures across the codebase
 
-## WrappedSocket Class Design
+## 9. Success Metrics
 
-### Overview
-A WrappedSocket class has been evaluated and recommended to provide cleaner PROXY protocol integration and better socket management architecture.
+- **Accuracy**: Throughput metrics accurate within 1% of actual
+- **Performance**: < 1% CPU overhead for metrics collection
+- **Memory**: < 10MB memory usage for 1 hour of data
+- **Latency**: < 1ms to retrieve any metric
+- **Reliability**: No metrics data loss under load
 
-### Rationale for WrappedSocket
+## 10. Future Enhancements
 
-#### Current Challenges
-- Sockets handled directly as `net.Socket` instances throughout codebase
-- Metadata tracked separately in `IConnectionRecord` objects
-- Socket augmentation via TypeScript module augmentation for TLS properties
-- PROXY protocol would require modifying socket handling in multiple places
+### Phase 5: Advanced Analytics
+- Anomaly detection for traffic patterns
+- Predictive analytics for capacity planning
+- Correlation analysis between routes
+- Real-time alerting integration
 
-#### Benefits
-1. **Clean PROXY Protocol Integration** - Parse and store real client IP/port without modifying existing socket handling
-2. **Better Encapsulation** - Bundle socket + metadata + behavior together
-3. **Type Safety** - No more module augmentation needed
-4. **Future Extensibility** - Easy to add compression, metrics, etc.
-5. **Simplified Testing** - Easier to mock and test socket behavior
+### Phase 6: Distributed Metrics
+- Metrics aggregation across multiple proxies
+- Distributed time-series storage
+- Cross-proxy analytics
+- Global dashboard support
 
-### Implementation Strategy
+## 11. Risks and Mitigations
 
-#### Phase 1: Minimal ProxyProtocolSocket (Immediate)
-Create a minimal wrapper for PROXY protocol support:
+### Risk: Memory Usage
+- **Mitigation**: Circular buffers and configurable retention
+- **Monitoring**: Track memory usage per metric type
 
-```typescript
-class ProxyProtocolSocket {
-  constructor(
-    public socket: net.Socket,
-    public realClientIP?: string,
-    public realClientPort?: number
-  ) {}
-  
-  get remoteAddress(): string {
-    return this.realClientIP || this.socket.remoteAddress || '';
-  }
-  
-  get remotePort(): number {
-    return this.realClientPort || this.socket.remotePort || 0;
-  }
-  
-  get isFromTrustedProxy(): boolean {
-    return !!this.realClientIP;
-  }
-}
-```
-
-Integration points:
-- Use in `RouteConnectionHandler` when receiving from trusted proxy IPs
-- Update `ConnectionManager` to accept wrapped sockets
-- Modify security checks to use `socket.remoteAddress` getter
-
-#### Phase 2: Connection-Aware WrappedSocket (Alternative Design)
-A more comprehensive design that manages both sides of a connection:
+### Risk: Performance Impact
+- **Mitigation**: Efficient data structures and caching
+- **Testing**: Load test with metrics enabled/disabled
 
-```typescript
-// Option A: Single Socket Wrapper (simpler)
-class WrappedSocket extends EventEmitter {
-  private socket: net.Socket;
-  private connectionId: string;
-  private metadata: ISocketMetadata;
-  
-  constructor(socket: net.Socket, metadata?: Partial<ISocketMetadata>) {
-    super();
-    this.socket = socket;
-    this.connectionId = this.generateId();
-    this.metadata = { ...defaultMetadata, ...metadata };
-    this.setupHandlers();
-  }
-  
-  // ... single socket management
-}
+### Risk: Data Accuracy
+- **Mitigation**: Atomic operations and proper synchronization
+- **Validation**: Compare with external monitoring tools
 
-// Option B: Connection Pair Wrapper (comprehensive)
-class WrappedConnection extends EventEmitter {
-  private connectionId: string;
-  private incoming: WrappedSocket;
-  private outgoing?: WrappedSocket;
-  private forwardingActive: boolean = false;
-  
-  constructor(incomingSocket: net.Socket) {
-    super();
-    this.connectionId = this.generateId();
-    this.incoming = new WrappedSocket(incomingSocket);
-  }
-  
-  // Connect to backend and set up forwarding
-  async connectToBackend(target: ITarget): Promise<void> {
-    const outgoingSocket = await this.createOutgoingConnection(target);
-    this.outgoing = new WrappedSocket(outgoingSocket);
-    await this.setupBidirectionalForwarding();
-  }
-  
-  // Built-in forwarding logic from socket-utils
-  private async setupBidirectionalForwarding(): Promise<void> {
-    if (!this.outgoing) throw new Error('No outgoing socket');
-    
-    // Handle data forwarding with backpressure
-    this.incoming.on('data', (chunk) => {
-      this.outgoing!.write(chunk, () => {
-        // Handle backpressure
-      });
-    });
-    
-    this.outgoing.on('data', (chunk) => {
-      this.incoming.write(chunk, () => {
-        // Handle backpressure
-      });
-    });
-    
-    // Handle connection lifecycle
-    const cleanup = (reason: string) => {
-      this.forwardingActive = false;
-      this.incoming.destroy();
-      this.outgoing?.destroy();
-      this.emit('closed', reason);
-    };
-    
-    this.incoming.once('close', () => cleanup('incoming_closed'));
-    this.outgoing.once('close', () => cleanup('outgoing_closed'));
-    
-    this.forwardingActive = true;
-  }
-  
-  // PROXY protocol support
-  async handleProxyProtocol(trustedProxies: string[]): Promise<boolean> {
-    if (trustedProxies.includes(this.incoming.socket.remoteAddress)) {
-      const parsed = await this.incoming.parseProxyProtocol();
-      if (parsed && this.outgoing) {
-        // Forward PROXY protocol to backend if configured
-        await this.outgoing.sendProxyProtocol(this.incoming.realClientIP);
-      }
-      return parsed;
-    }
-    return false;
-  }
-  
-  // Consolidated metrics
-  getMetrics(): IConnectionMetrics {
-    return {
-      connectionId: this.connectionId,
-      duration: Date.now() - this.startTime,
-      incoming: this.incoming.getMetrics(),
-      outgoing: this.outgoing?.getMetrics(),
-      totalBytes: this.getTotalBytes(),
-      state: this.getConnectionState()
-    };
-  }
-}
-```
-
-#### Phase 3: Full Migration (Long-term)
-- Replace all `net.Socket` usage with `WrappedSocket`
-- Remove socket augmentation from `socket-augmentation.ts`
-- Update all socket utilities to work with wrapped sockets
-- Standardize socket handling across all components
-
-### Integration with PROXY Protocol
-
-The WrappedSocket class integrates seamlessly with PROXY protocol:
-
-1. **Connection Acceptance**:
-   ```typescript
-   const wrappedSocket = new ProxyProtocolSocket(socket);
-   if (this.isFromTrustedProxy(socket.remoteAddress)) {
-     await wrappedSocket.parseProxyProtocol(this.settings.proxyIPs);
-   }
-   ```
-
-2. **Security Checks**:
-   ```typescript
-   // Automatically uses real client IP if available
-   const clientIP = wrappedSocket.remoteAddress;
-   if (!this.securityManager.isIPAllowed(clientIP)) {
-     wrappedSocket.destroy();
-   }
-   ```
-
-3. **Connection Records**:
-   ```typescript
-   const record = this.connectionManager.createConnection(wrappedSocket);
-   // ConnectionManager uses wrappedSocket.remoteAddress transparently
-   ```
-
-### Option B Example: How It Would Replace Current Architecture
-
-Instead of current approach with separate components:
-```typescript
-// Current: Multiple separate components
-const record = connectionManager.createConnection(socket);
-const { cleanupClient, cleanupServer } = createIndependentSocketHandlers(
-  clientSocket, serverSocket, onBothClosed
-);
-setupBidirectionalForwarding(clientSocket, serverSocket, handlers);
-```
-
-Option B would consolidate everything:
-```typescript
-// Option B: Single connection object
-const connection = new WrappedConnection(incomingSocket);
-await connection.handleProxyProtocol(trustedProxies);
-await connection.connectToBackend({ host: 'server', port: 443 });
-// Everything is handled internally - forwarding, cleanup, metrics
-
-connection.on('closed', (reason) => {
-  logger.log('Connection closed', connection.getMetrics());
-});
-```
+## Conclusion
 
-This would replace:
-- `IConnectionRecord` - absorbed into WrappedConnection
-- `socket-utils.ts` functions - methods on WrappedConnection
-- Separate incoming/outgoing tracking - unified in one object
-- Manual cleanup coordination - automatic lifecycle management
-
-Additional benefits with Option B:
-- **Connection Pooling Integration**: WrappedConnection could integrate with EnhancedConnectionPool for backend connections
-- **Unified Metrics**: Single point for all connection statistics
-- **Protocol Negotiation**: Handle PROXY, TLS, HTTP/2 upgrade in one place
-- **Resource Management**: Automatic cleanup with LifecycleComponent pattern
-
-### Migration Path
-
-1. **Week 1-2**: Implement minimal ProxyProtocolSocket (Option A)
-2. **Week 3-4**: Test with PROXY protocol implementation
-3. **Month 2**: Prototype WrappedConnection (Option B) if beneficial
-4. **Month 3-6**: Gradual migration if Option B proves valuable
-5. **Future**: Complete adoption in next major version
-
-### Success Criteria
-
-- PROXY protocol works transparently with wrapped sockets
-- No performance regression (<0.1% overhead)
-- Simplified code in connection handlers
-- Better TypeScript type safety
-- Easier to add new socket-level features
+This plan transforms SmartProxy's metrics from a basic cumulative system to a comprehensive, time-series based monitoring solution suitable for production environments. The phased approach ensures minimal disruption while delivering immediate value through accurate throughput measurements.