@push.rocks/smartproxy 19.5.19 → 19.5.20

package/readme.plan.md

@@ -0,0 +1,621 @@
+# PROXY Protocol Implementation Plan
+
+## ⚠️ CRITICAL: Implementation Order
+
+**Phase 1 (ProxyProtocolSocket/WrappedSocket) MUST be completed first!**
+
+The ProxyProtocolSocket class is the foundation that enables all PROXY protocol functionality. No protocol parsing or integration can happen until this wrapper class is fully implemented and tested.
+
+1. **FIRST**: Implement ProxyProtocolSocket (the WrappedSocket)
+2. **THEN**: Add PROXY protocol parser
+3. **THEN**: Integrate with connection handlers
+4. **FINALLY**: Add security and validation
+
+## Overview
+Implement PROXY protocol support in SmartProxy to preserve client IP information through proxy chains, solving the connection limit accumulation issue where inner proxies see all connections as coming from the outer proxy's IP.
+
+## Problem Statement
+- In proxy chains, the inner proxy sees all connections from the outer proxy's IP
+- This causes the inner proxy to hit per-IP connection limits (default: 100)
+- Results in connection rejections while outer proxy accumulates connections
+
+## Solution Design
+
+### 1. Core Features
+
+#### 1.1 PROXY Protocol Parsing
+- Support PROXY protocol v1 (text format) initially
+- Parse incoming PROXY headers to extract:
+  - Real client IP address
+  - Real client port
+  - Proxy IP address
+  - Proxy port
+  - Protocol (TCP4/TCP6)
+
+#### 1.2 PROXY Protocol Generation
+- Add ability to send PROXY protocol headers when forwarding connections
+- Configurable per route or target
+
+#### 1.3 Trusted Proxy IPs
+- New `proxyIPs` array in SmartProxy options
+- Auto-enable PROXY protocol acceptance for connections from these IPs
+- Reject PROXY protocol from untrusted sources (security)
+
+### 2. Configuration Schema
+
+```typescript
+interface ISmartProxyOptions {
+  // ... existing options
+  
+  // List of trusted proxy IPs that can send PROXY protocol
+  proxyIPs?: string[];
+  
+  // Global option to accept PROXY protocol (defaults based on proxyIPs)
+  acceptProxyProtocol?: boolean;
+  
+  // Global option to send PROXY protocol to all targets
+  sendProxyProtocol?: boolean;
+}
+
+interface IRouteAction {
+  // ... existing options
+  
+  // Send PROXY protocol to this specific target
+  sendProxyProtocol?: boolean;
+}
+```
+
+### 3. Implementation Steps
+
+#### IMPORTANT: Phase 1 Must Be Completed First
+The `ProxyProtocolSocket` (WrappedSocket) is the foundation for all PROXY protocol functionality. This wrapper class must be implemented and integrated BEFORE any PROXY protocol parsing can begin.
+
+#### Phase 1: ProxyProtocolSocket (WrappedSocket) Foundation - ✅ COMPLETED (v19.5.19)
+This phase creates the socket wrapper infrastructure that all subsequent phases depend on.
+
+1. **Create WrappedSocket class** in `ts/core/models/wrapped-socket.ts` ✅
+   - Used JavaScript Proxy pattern instead of EventEmitter (avoids infinite loops)
+   - Properties for real client IP and port
+   - Transparent getters that return real or socket IP/port
+   - All socket methods/properties delegated via Proxy
+   
+2. **Implement core wrapper functionality** ✅
+   - Constructor accepts regular socket + optional metadata
+   - `remoteAddress` getter returns real IP or falls back to socket IP
+   - `remotePort` getter returns real port or falls back to socket port
+   - `isFromTrustedProxy` property to check if it has real client info
+   - `setProxyInfo()` method to update real client details
+   
+3. **Update ConnectionManager to handle wrapped sockets** ✅
+   - Accept either `net.Socket` or `WrappedSocket`
+   - Created `getUnderlyingSocket()` helper for socket utilities
+   - All socket utility functions extract underlying socket
+   
+4. **Integration completed** ✅
+   - All incoming sockets wrapped in RouteConnectionHandler
+   - Socket forwarding verified working with wrapped sockets
+   - Type safety maintained with index signature
+
+**Deliverables**: ✅ Working WrappedSocket that can wrap any socket and provide transparent access to client info.
+
+#### Phase 2: PROXY Protocol Parser - DEPENDS ON PHASE 1
+Only after WrappedSocket is working can we add protocol parsing.
+
+1. Create `ProxyProtocolParser` class in `ts/core/utils/proxy-protocol.ts`
+2. Implement v1 text format parsing
+3. Add validation and error handling
+4. Integrate parser to work WITH WrappedSocket (not into it)
+
+#### Phase 3: Connection Handler Integration - DEPENDS ON PHASES 1 & 2
+1. ✅ Modify `RouteConnectionHandler` to create WrappedSocket for all connections
+2. Check if connection is from trusted proxy IP
+3. If trusted, attempt to parse PROXY protocol header
+4. Update wrapped socket with real client info
+5. Continue normal connection handling with wrapped socket
+
+#### Phase 4: Outbound PROXY Protocol - DEPENDS ON PHASES 1-3
+1. Add PROXY header generation in `setupDirectConnection`
+2. Make it configurable per route
+3. Send header immediately after TCP connection
+4. Use ProxyProtocolSocket for outbound connections too
+
+#### Phase 5: Security & Validation - FINAL PHASE
+1. Validate PROXY headers strictly
+2. Reject malformed headers
+3. Only accept from trusted IPs
+4. Add rate limiting for PROXY protocol parsing
+
+### 4. Design Decision: Socket Wrapper Architecture
+
+#### Option A: Minimal Single Socket Wrapper
+- **Scope**: Wraps individual sockets with metadata
+- **Use Case**: PROXY protocol support with minimal refactoring
+- **Pros**: Simple, low risk, easy migration
+- **Cons**: Still need separate connection management
+
+#### Option B: Comprehensive Connection Wrapper
+- **Scope**: Manages socket pairs (incoming + outgoing) with all utilities
+- **Use Case**: Complete connection lifecycle management
+- **Pros**: 
+  - Encapsulates all socket utilities (forwarding, cleanup, backpressure)
+  - Single object represents entire connection
+  - Cleaner API for connection handling
+- **Cons**: 
+  - Major architectural change
+  - Higher implementation risk
+  - More complex migration
+
+#### Recommendation
+Start with **Option A** (ProxyProtocolSocket) for immediate PROXY protocol support, then evaluate Option B based on:
+- Performance impact of additional abstraction
+- Code simplification benefits
+- Team comfort with architectural change
+
+### 5. Code Implementation Details
+
+#### 5.1 ProxyProtocolSocket (WrappedSocket) - PHASE 1 IMPLEMENTATION
+This is the foundational wrapper class that MUST be implemented first. It wraps a regular socket and provides transparent access to the real client IP/port.
+
+```typescript
+// ts/core/models/proxy-protocol-socket.ts
+import { EventEmitter } from 'events';
+import * as plugins from '../../../plugins.js';
+
+/**
+ * ProxyProtocolSocket wraps a regular net.Socket to provide transparent access
+ * to the real client IP and port when behind a proxy using PROXY protocol.
+ * 
+ * This is the FOUNDATION for all PROXY protocol support and must be implemented
+ * before any protocol parsing can occur.
+ */
+export class ProxyProtocolSocket extends EventEmitter {
+  private realClientIP?: string;
+  private realClientPort?: number;
+  
+  constructor(
+    public readonly socket: plugins.net.Socket,
+    realClientIP?: string,
+    realClientPort?: number
+  ) {
+    super();
+    this.realClientIP = realClientIP;
+    this.realClientPort = realClientPort;
+    
+    // Forward all socket events
+    this.forwardSocketEvents();
+  }
+  
+  /**
+   * Returns the real client IP if available, otherwise the socket's remote address
+   */
+  get remoteAddress(): string | undefined {
+    return this.realClientIP || this.socket.remoteAddress;
+  }
+  
+  /**
+   * Returns the real client port if available, otherwise the socket's remote port
+   */
+  get remotePort(): number | undefined {
+    return this.realClientPort || this.socket.remotePort;
+  }
+  
+  /**
+   * Indicates if this connection came through a trusted proxy
+   */
+  get isFromTrustedProxy(): boolean {
+    return !!this.realClientIP;
+  }
+  
+  /**
+   * Updates the real client information (called after parsing PROXY protocol)
+   */
+  setProxyInfo(ip: string, port: number): void {
+    this.realClientIP = ip;
+    this.realClientPort = port;
+  }
+  
+  // Pass-through all socket methods
+  write(data: any, encoding?: any, callback?: any): boolean {
+    return this.socket.write(data, encoding, callback);
+  }
+  
+  end(data?: any, encoding?: any, callback?: any): this {
+    this.socket.end(data, encoding, callback);
+    return this;
+  }
+  
+  destroy(error?: Error): this {
+    this.socket.destroy(error);
+    return this;
+  }
+  
+  // ... implement all other socket methods as pass-through
+  
+  /**
+   * Forward all events from the underlying socket
+   */
+  private forwardSocketEvents(): void {
+    const events = ['data', 'end', 'close', 'error', 'drain', 'timeout'];
+    events.forEach(event => {
+      this.socket.on(event, (...args) => {
+        this.emit(event, ...args);
+      });
+    });
+  }
+}
+```
+
+**KEY POINT**: This wrapper must be fully functional and tested BEFORE moving to Phase 2.
+
+#### 4.2 ProxyProtocolParser (new file)
+```typescript
+// ts/core/utils/proxy-protocol.ts
+export class ProxyProtocolParser {
+  static readonly PROXY_V1_SIGNATURE = 'PROXY ';
+  
+  static parse(chunk: Buffer): IProxyInfo | null {
+    // Implementation
+  }
+  
+  static generate(info: IProxyInfo): Buffer {
+    // Implementation
+  }
+}
+```
+
+#### 4.3 Connection Handler Updates
+```typescript
+// In handleConnection method
+let wrappedSocket: ProxyProtocolSocket | plugins.net.Socket = socket;
+
+// Wrap socket if from trusted proxy
+if (this.settings.proxyIPs?.includes(socket.remoteAddress)) {
+  wrappedSocket = new ProxyProtocolSocket(socket);
+}
+
+// Create connection record with wrapped socket
+const record = this.connectionManager.createConnection(wrappedSocket);
+
+// In handleInitialData method
+if (wrappedSocket instanceof ProxyProtocolSocket) {
+  const proxyInfo = await this.checkForProxyProtocol(chunk);
+  if (proxyInfo) {
+    wrappedSocket.setProxyInfo(proxyInfo.sourceIP, proxyInfo.sourcePort);
+    // Continue with remaining data after PROXY header
+  }
+}
+```
+
+#### 4.4 Security Manager Updates
+- Accept socket or ProxyProtocolSocket
+- Use `socket.remoteAddress` getter for real client IP
+- Transparent handling of both socket types
+
+### 5. Configuration Examples
+
+#### Basic Setup
+```typescript
+// Outer proxy - sends PROXY protocol
+const outerProxy = new SmartProxy({
+  ports: [443],
+  routes: [{
+    name: 'to-inner-proxy',
+    match: { ports: 443 },
+    action: {
+      type: 'forward',
+      target: { host: '195.201.98.232', port: 443 },
+      sendProxyProtocol: true  // Enable for this route
+    }
+  }]
+});
+
+// Inner proxy - accepts PROXY protocol from outer proxy
+const innerProxy = new SmartProxy({
+  ports: [443],
+  proxyIPs: ['212.95.99.130'],  // Outer proxy IP
+  // acceptProxyProtocol: true is automatic for proxyIPs
+  routes: [{
+    name: 'to-backend',
+    match: { ports: 443 },
+    action: {
+      type: 'forward',
+      target: { host: '192.168.5.247', port: 443 }
+    }
+  }]
+});
+```
+
+### 6. Testing Plan
+
+#### Unit Tests
+- PROXY protocol v1 parsing (valid/invalid formats)
+- Header generation
+- Trusted IP validation
+- Connection record updates
+
+#### Integration Tests
+- Single proxy with PROXY protocol
+- Proxy chain with PROXY protocol
+- Security: reject from untrusted IPs
+- Performance: minimal overhead
+- Compatibility: works with TLS passthrough
+
+#### Test Scenarios
+1. **Connection limit test**: Verify inner proxy sees real client IPs
+2. **Security test**: Ensure PROXY protocol rejected from untrusted sources
+3. **Compatibility test**: Verify no impact on non-PROXY connections
+4. **Performance test**: Measure overhead of PROXY protocol parsing
+
+### 7. Security Considerations
+
+1. **IP Spoofing Prevention**
+   - Only accept PROXY protocol from explicitly trusted IPs
+   - Validate all header fields
+   - Reject malformed headers immediately
+
+2. **Resource Protection**
+   - Limit PROXY header size (107 bytes for v1)
+   - Timeout for incomplete headers
+   - Rate limit connection attempts
+
+3. **Logging**
+   - Log all PROXY protocol acceptance/rejection
+   - Include real client IP in all connection logs
+
+### 8. Rollout Strategy
+
+1. **Phase 1**: Deploy parser and acceptance (backward compatible)
+2. **Phase 2**: Enable between controlled proxy pairs
+3. **Phase 3**: Monitor for issues and performance impact
+4. **Phase 4**: Expand to all proxy chains
+
+### 9. Success Metrics
+
+- Inner proxy connection distribution matches outer proxy
+- No more connection limit rejections in proxy chains
+- Accurate client IP logging throughout the chain
+- No performance degradation (<1ms added latency)
+
+### 10. Future Enhancements
+
+- PROXY protocol v2 (binary format) support
+- TLV extensions for additional metadata
+- AWS VPC endpoint ID support
+- Custom metadata fields
+
+## WrappedSocket Class Design
+
+### Overview
+A WrappedSocket class has been evaluated and recommended to provide cleaner PROXY protocol integration and better socket management architecture.
+
+### Rationale for WrappedSocket
+
+#### Current Challenges
+- Sockets handled directly as `net.Socket` instances throughout codebase
+- Metadata tracked separately in `IConnectionRecord` objects
+- Socket augmentation via TypeScript module augmentation for TLS properties
+- PROXY protocol would require modifying socket handling in multiple places
+
+#### Benefits
+1. **Clean PROXY Protocol Integration** - Parse and store real client IP/port without modifying existing socket handling
+2. **Better Encapsulation** - Bundle socket + metadata + behavior together
+3. **Type Safety** - No more module augmentation needed
+4. **Future Extensibility** - Easy to add compression, metrics, etc.
+5. **Simplified Testing** - Easier to mock and test socket behavior
+
+### Implementation Strategy
+
+#### Phase 1: Minimal ProxyProtocolSocket (Immediate)
+Create a minimal wrapper for PROXY protocol support:
+
+```typescript
+class ProxyProtocolSocket {
+  constructor(
+    public socket: net.Socket,
+    public realClientIP?: string,
+    public realClientPort?: number
+  ) {}
+  
+  get remoteAddress(): string {
+    return this.realClientIP || this.socket.remoteAddress || '';
+  }
+  
+  get remotePort(): number {
+    return this.realClientPort || this.socket.remotePort || 0;
+  }
+  
+  get isFromTrustedProxy(): boolean {
+    return !!this.realClientIP;
+  }
+}
+```
+
+Integration points:
+- Use in `RouteConnectionHandler` when receiving from trusted proxy IPs
+- Update `ConnectionManager` to accept wrapped sockets
+- Modify security checks to use `socket.remoteAddress` getter
+
+#### Phase 2: Connection-Aware WrappedSocket (Alternative Design)
+A more comprehensive design that manages both sides of a connection:
+
+```typescript
+// Option A: Single Socket Wrapper (simpler)
+class WrappedSocket extends EventEmitter {
+  private socket: net.Socket;
+  private connectionId: string;
+  private metadata: ISocketMetadata;
+  
+  constructor(socket: net.Socket, metadata?: Partial<ISocketMetadata>) {
+    super();
+    this.socket = socket;
+    this.connectionId = this.generateId();
+    this.metadata = { ...defaultMetadata, ...metadata };
+    this.setupHandlers();
+  }
+  
+  // ... single socket management
+}
+
+// Option B: Connection Pair Wrapper (comprehensive)
+class WrappedConnection extends EventEmitter {
+  private connectionId: string;
+  private incoming: WrappedSocket;
+  private outgoing?: WrappedSocket;
+  private forwardingActive: boolean = false;
+  
+  constructor(incomingSocket: net.Socket) {
+    super();
+    this.connectionId = this.generateId();
+    this.incoming = new WrappedSocket(incomingSocket);
+  }
+  
+  // Connect to backend and set up forwarding
+  async connectToBackend(target: ITarget): Promise<void> {
+    const outgoingSocket = await this.createOutgoingConnection(target);
+    this.outgoing = new WrappedSocket(outgoingSocket);
+    await this.setupBidirectionalForwarding();
+  }
+  
+  // Built-in forwarding logic from socket-utils
+  private async setupBidirectionalForwarding(): Promise<void> {
+    if (!this.outgoing) throw new Error('No outgoing socket');
+    
+    // Handle data forwarding with backpressure
+    this.incoming.on('data', (chunk) => {
+      this.outgoing!.write(chunk, () => {
+        // Handle backpressure
+      });
+    });
+    
+    this.outgoing.on('data', (chunk) => {
+      this.incoming.write(chunk, () => {
+        // Handle backpressure
+      });
+    });
+    
+    // Handle connection lifecycle
+    const cleanup = (reason: string) => {
+      this.forwardingActive = false;
+      this.incoming.destroy();
+      this.outgoing?.destroy();
+      this.emit('closed', reason);
+    };
+    
+    this.incoming.once('close', () => cleanup('incoming_closed'));
+    this.outgoing.once('close', () => cleanup('outgoing_closed'));
+    
+    this.forwardingActive = true;
+  }
+  
+  // PROXY protocol support
+  async handleProxyProtocol(trustedProxies: string[]): Promise<boolean> {
+    if (trustedProxies.includes(this.incoming.socket.remoteAddress)) {
+      const parsed = await this.incoming.parseProxyProtocol();
+      if (parsed && this.outgoing) {
+        // Forward PROXY protocol to backend if configured
+        await this.outgoing.sendProxyProtocol(this.incoming.realClientIP);
+      }
+      return parsed;
+    }
+    return false;
+  }
+  
+  // Consolidated metrics
+  getMetrics(): IConnectionMetrics {
+    return {
+      connectionId: this.connectionId,
+      duration: Date.now() - this.startTime,
+      incoming: this.incoming.getMetrics(),
+      outgoing: this.outgoing?.getMetrics(),
+      totalBytes: this.getTotalBytes(),
+      state: this.getConnectionState()
+    };
+  }
+}
+```
+
+#### Phase 3: Full Migration (Long-term)
+- Replace all `net.Socket` usage with `WrappedSocket`
+- Remove socket augmentation from `socket-augmentation.ts`
+- Update all socket utilities to work with wrapped sockets
+- Standardize socket handling across all components
+
+### Integration with PROXY Protocol
+
+The WrappedSocket class integrates seamlessly with PROXY protocol:
+
+1. **Connection Acceptance**:
+   ```typescript
+   const wrappedSocket = new ProxyProtocolSocket(socket);
+   if (this.isFromTrustedProxy(socket.remoteAddress)) {
+     await wrappedSocket.parseProxyProtocol(this.settings.proxyIPs);
+   }
+   ```
+
+2. **Security Checks**:
+   ```typescript
+   // Automatically uses real client IP if available
+   const clientIP = wrappedSocket.remoteAddress;
+   if (!this.securityManager.isIPAllowed(clientIP)) {
+     wrappedSocket.destroy();
+   }
+   ```
+
+3. **Connection Records**:
+   ```typescript
+   const record = this.connectionManager.createConnection(wrappedSocket);
+   // ConnectionManager uses wrappedSocket.remoteAddress transparently
+   ```
+
+### Option B Example: How It Would Replace Current Architecture
+
+Instead of current approach with separate components:
+```typescript
+// Current: Multiple separate components
+const record = connectionManager.createConnection(socket);
+const { cleanupClient, cleanupServer } = createIndependentSocketHandlers(
+  clientSocket, serverSocket, onBothClosed
+);
+setupBidirectionalForwarding(clientSocket, serverSocket, handlers);
+```
+
+Option B would consolidate everything:
+```typescript
+// Option B: Single connection object
+const connection = new WrappedConnection(incomingSocket);
+await connection.handleProxyProtocol(trustedProxies);
+await connection.connectToBackend({ host: 'server', port: 443 });
+// Everything is handled internally - forwarding, cleanup, metrics
+
+connection.on('closed', (reason) => {
+  logger.log('Connection closed', connection.getMetrics());
+});
+```
+
+This would replace:
+- `IConnectionRecord` - absorbed into WrappedConnection
+- `socket-utils.ts` functions - methods on WrappedConnection
+- Separate incoming/outgoing tracking - unified in one object
+- Manual cleanup coordination - automatic lifecycle management
+
+Additional benefits with Option B:
+- **Connection Pooling Integration**: WrappedConnection could integrate with EnhancedConnectionPool for backend connections
+- **Unified Metrics**: Single point for all connection statistics
+- **Protocol Negotiation**: Handle PROXY, TLS, HTTP/2 upgrade in one place
+- **Resource Management**: Automatic cleanup with LifecycleComponent pattern
+
+### Migration Path
+
+1. **Week 1-2**: Implement minimal ProxyProtocolSocket (Option A)
+2. **Week 3-4**: Test with PROXY protocol implementation
+3. **Month 2**: Prototype WrappedConnection (Option B) if beneficial
+4. **Month 3-6**: Gradual migration if Option B proves valuable
+5. **Future**: Complete adoption in next major version
+
+### Success Criteria
+
+- PROXY protocol works transparently with wrapped sockets
+- No performance regression (<0.1% overhead)
+- Simplified code in connection handlers
+- Better TypeScript type safety
+- Easier to add new socket-level features