@push.rocks/smartproxy 19.5.2 → 19.5.4

package/readme.problems.md

@@ -0,0 +1,86 @@
+# SmartProxy Module Problems
+
+Based on test analysis, the following potential issues have been identified in the SmartProxy module:
+
+## 1. HttpProxy Route Configuration Issue
+**Location**: `ts/proxies/http-proxy/http-proxy.ts:380`
+**Problem**: The HttpProxy is trying to read the 'type' property of an undefined object when updating route configurations.
+**Evidence**: `test.http-forwarding-fix.ts` fails with:
+```
+TypeError: Cannot read properties of undefined (reading 'type')
+    at HttpProxy.updateRouteConfigs (/mnt/data/lossless/push.rocks/smartproxy/ts/proxies/http-proxy/http-proxy.ts:380:24)
+```
+**Impact**: Routes with `useHttpProxy` configuration may not work properly.
+
+## 2. Connection Forwarding Issues
+**Problem**: Basic TCP forwarding appears to not be working correctly after the simplification to just 'forward' and 'socket-handler' action types.
+**Evidence**: Multiple forwarding tests timeout waiting for data to be forwarded:
+- `test.forwarding-fix-verification.ts` - times out waiting for forwarded data
+- `test.connection-forwarding.ts` - times out on SNI-based forwarding
+**Impact**: The 'forward' action type may not be properly forwarding connections to target servers.
+
+## 3. Missing Certificate Manager Methods
+**Problem**: Tests expect `provisionAllCertificates` method on certificate manager but it may not exist or may not be properly initialized.
+**Evidence**: Multiple tests fail with "this.certManager.provisionAllCertificates is not a function"
+**Impact**: Certificate provisioning may not work as expected.
+
+## 4. Route Update Mechanism
+**Problem**: The route update mechanism may have issues preserving certificate manager callbacks and other state.
+**Evidence**: Tests specifically designed to verify callback preservation after route updates.
+**Impact**: Dynamic route updates might break certificate management functionality.
+
+## 5. Route-Specific Security Not Fully Implemented
+**Problem**: While the route definitions support security configurations (ipAllowList, ipBlockList, authentication), these are not being enforced at the route level.
+**Evidence**: 
+- SecurityManager has methods like `isIPAuthorized` for route-specific security
+- Route connection handler only checks global IP validation, not route-specific security rules
+- No evidence of route.action.security being checked when handling connections
+**Impact**: Route-specific security rules defined in configuration are not enforced, potentially allowing unauthorized access.
+**Status**: ✅ FIXED - Route-specific IP allow/block lists are now enforced when a route is matched. Authentication is logged as not enforceable for non-terminated connections.
+**Additional Fix**: Removed security checks from route matching logic - security is now properly enforced AFTER a route is matched, not during matching.
+
+## 6. Security Property Location Consolidation
+**Problem**: Security was defined in two places - route.security and route.action.security - causing confusion.
+**Status**: ✅ FIXED - Consolidated to only route.security. Removed action.security from types and updated all references.
+
+## Recommendations
+
+1. **Verify Forward Action Implementation**: Check that the 'forward' action type properly establishes bidirectional data flow between client and target server. ✅ FIXED - Basic forwarding now works correctly.
+
+2. **Fix HttpProxy Route Handling**: Ensure that route objects passed to HttpProxy.updateRouteConfigs have the expected structure with all required properties. ✅ FIXED - Routes now preserve their structure.
+
+3. **Review Certificate Manager API**: Ensure all expected methods exist and are properly documented.
+
+4. **Add Integration Tests**: Many unit tests are testing internal implementation details. Consider adding more integration tests that test the public API.
+
+5. **Implement Route-Specific Security**: Add security checks when a route is matched to enforce route-specific IP allow/block lists and authentication rules. ✅ FIXED - IP allow/block lists are now enforced at the route level.
+
+6. **Fix TLS Detection Logic**: The connection handler was treating all connections as TLS. This has been partially fixed but needs proper testing for all TLS modes.
+
+## 7. HTTP Domain Matching Issue
+**Problem**: Routes with domain restrictions fail to match HTTP connections because domain information is only available after HTTP headers are received, but route matching happens immediately upon connection.
+**Evidence**: 
+- `test.http-port8080-forwarding.ts` - "No route found for connection on port 8080" despite having a matching route
+- HTTP connections provide domain info via the Host header, which arrives after the initial TCP connection
+- Route matching in `handleInitialData` happens before HTTP headers are parsed
+**Impact**: HTTP routes with domain restrictions cannot be matched, forcing users to remove domain restrictions for HTTP routes.
+**Root Cause**: For non-TLS connections, SmartProxy attempts to match routes immediately, but the domain information needed for matching is only available after parsing HTTP headers.
+**Status**: ✅ FIXED - Added skipDomainCheck parameter to route matching for HTTP proxy ports. When a port is configured with useHttpProxy and the connection is not TLS, domain validation is skipped at the initial route matching stage, allowing the HttpProxy to handle domain-based routing after headers are received.
+
+## 8. HttpProxy Plain HTTP Forwarding Issue
+**Problem**: HttpProxy is an HTTPS server but SmartProxy forwards plain HTTP connections to it via `useHttpProxy` configuration.
+**Evidence**: 
+- `test.http-port8080-forwarding.ts` - Connection immediately closed after forwarding to HttpProxy
+- HttpProxy is created with `http2.createSecureServer` expecting TLS connections
+- SmartProxy forwards raw HTTP data to HttpProxy's HTTPS port
+**Impact**: Plain HTTP connections cannot be handled by HttpProxy, despite `useHttpProxy` configuration suggesting this should work.
+**Root Cause**: Design mismatch - HttpProxy is designed for HTTPS/TLS termination, not plain HTTP forwarding.
+**Status**: Documented. The `useHttpProxy` configuration should only be used for ports that receive TLS connections requiring termination. For plain HTTP forwarding, use direct forwarding without HttpProxy.
+
+## 9. Route Security Configuration Location Issue
+**Problem**: Tests were placing security configuration in `route.action.security` instead of `route.security`.
+**Evidence**: 
+- `test.route-security.ts` - IP block list test failing because security was in wrong location
+- IRouteConfig interface defines security at route level, not inside action
+**Impact**: Security rules defined in action.security were ignored, causing tests to fail.
+**Status**: ✅ FIXED - Updated tests to place security configuration at the correct location (route.security).

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartproxy',
-  version: '19.5.2',
+  version: '19.5.3',
   description: 'A powerful proxy package with unified route-based configuration for high traffic management. Features include SSL/TLS support, flexible routing patterns, WebSocket handling, advanced security options, and automatic ACME certificate management.'
 }

package/ts/proxies/http-proxy/models/types.ts

@@ -153,7 +153,7 @@ export function convertLegacyConfigToRouteConfig(
 
   // Add authentication if present
   if (legacyConfig.authentication) {
-    routeConfig.action.security = {
+    routeConfig.security = {
       authentication: {
         type: 'basic',
         credentials: [{

package/ts/proxies/smart-proxy/http-proxy-bridge.ts

@@ -73,10 +73,7 @@ export class HttpProxyBridge {
     }
     
     return {
-      domain,
-      target: route.action.target,
-      tls: route.action.tls,
-      security: route.action.security,
+      ...route,  // Keep the original route structure
       match: {
         ...route.match,
         domains: domain  // Ensure domains is always set for HttpProxy

package/ts/proxies/smart-proxy/models/route-types.ts

@@ -233,9 +233,6 @@ export interface IRouteAction {
   // Load balancing options
   loadBalancing?: IRouteLoadBalancing;
 
-  // Security options
-  security?: IRouteSecurity;
-
   // Advanced options
   advanced?: IRouteAdvanced;
   

package/ts/proxies/smart-proxy/nftables-manager.ts

@@ -175,13 +175,12 @@ export class NFTablesManager {
     };
     
     // Add security-related options
-    const security = action.security || route.security;
-    if (security?.ipAllowList?.length) {
-      options.ipAllowList = security.ipAllowList;
+    if (route.security?.ipAllowList?.length) {
+      options.ipAllowList = route.security.ipAllowList;
     }
     
-    if (security?.ipBlockList?.length) {
-      options.ipBlockList = security.ipBlockList;
+    if (route.security?.ipBlockList?.length) {
+      options.ipBlockList = route.security.ipBlockList;
     }
     
     // Add QoS options

package/ts/proxies/smart-proxy/route-connection-handler.ts

@@ -146,18 +146,42 @@ export class RouteConnectionHandler {
       );
     }
 
-    // Start TLS SNI handling
-    this.handleTlsConnection(socket, record);
+    // Handle the connection - wait for initial data to determine if it's TLS
+    this.handleInitialData(socket, record);
   }
 
   /**
-   * Handle a connection and wait for TLS handshake for SNI extraction if needed
+   * Handle initial data from a connection to determine routing
    */
-  private handleTlsConnection(socket: plugins.net.Socket, record: IConnectionRecord): void {
+  private handleInitialData(socket: plugins.net.Socket, record: IConnectionRecord): void {
     const connectionId = record.id;
     const localPort = record.localPort;
     let initialDataReceived = false;
 
+    // Check if any routes on this port require TLS handling
+    const allRoutes = this.routeManager.getAllRoutes();
+    const needsTlsHandling = allRoutes.some(route => {
+      // Check if route matches this port
+      const matchesPort = this.routeManager.getRoutesForPort(localPort).includes(route);
+      
+      return matchesPort && 
+             route.action.type === 'forward' && 
+             route.action.tls && 
+             (route.action.tls.mode === 'terminate' || 
+              route.action.tls.mode === 'passthrough');
+    });
+
+    // If no routes require TLS handling and it's not port 443, route immediately
+    if (!needsTlsHandling && localPort !== 443) {
+      // Set up error handler
+      socket.on('error', this.connectionManager.handleError('incoming', record));
+      
+      // Route immediately for non-TLS connections
+      this.routeConnection(socket, record, '', undefined);
+      return;
+    }
+
+    // Otherwise, wait for initial data to check if it's TLS
     // Set an initial timeout for handshake data
     let initialTimeout: NodeJS.Timeout | null = setTimeout(() => {
       if (!initialDataReceived) {
@@ -296,6 +320,12 @@ export class RouteConnectionHandler {
     const localPort = record.localPort;
     const remoteIP = record.remoteIP;
 
+    // Check if this is an HTTP proxy port
+    const isHttpProxyPort = this.settings.useHttpProxy?.includes(localPort);
+    
+    // For HTTP proxy ports without TLS, skip domain check since domain info comes from HTTP headers
+    const skipDomainCheck = isHttpProxyPort && !record.isTLS;
+
     // Find matching route
     const routeMatch = this.routeManager.findMatchingRoute({
       port: localPort,
@@ -303,6 +333,7 @@ export class RouteConnectionHandler {
       clientIp: remoteIP,
       path: undefined, // We don't have path info at this point
       tlsVersion: undefined, // We don't extract TLS version yet
+      skipDomainCheck: skipDomainCheck,
     });
 
     if (!routeMatch) {
@@ -382,6 +413,56 @@ export class RouteConnectionHandler {
       });
     }
 
+    // Apply route-specific security checks
+    if (route.security) {
+      // Check IP allow/block lists
+      if (route.security.ipAllowList || route.security.ipBlockList) {
+        const isIPAllowed = this.securityManager.isIPAuthorized(
+          remoteIP,
+          route.security.ipAllowList || [],
+          route.security.ipBlockList || []
+        );
+
+        if (!isIPAllowed) {
+          logger.log('warn', `IP ${remoteIP} blocked by route security for route ${route.name || 'unnamed'} (connection: ${connectionId})`, {
+            connectionId,
+            remoteIP,
+            routeName: route.name || 'unnamed',
+            component: 'route-handler'
+          });
+          socket.end();
+          this.connectionManager.cleanupConnection(record, 'route_ip_blocked');
+          return;
+        }
+      }
+
+      // Check max connections per route
+      if (route.security.maxConnections !== undefined) {
+        // TODO: Implement per-route connection tracking
+        // For now, log that this feature is not yet implemented
+        if (this.settings.enableDetailedLogging) {
+          logger.log('warn', `Route ${route.name} has maxConnections=${route.security.maxConnections} configured but per-route connection limits are not yet implemented`, {
+            connectionId,
+            routeName: route.name,
+            component: 'route-handler'
+          });
+        }
+      }
+
+      // Check authentication requirements
+      if (route.security.authentication || route.security.basicAuth || route.security.jwtAuth) {
+        // Authentication checks would typically happen at the HTTP layer
+        // For non-HTTP connections or passthrough, we can't enforce authentication
+        if (route.action.type === 'forward' && route.action.tls?.mode !== 'terminate') {
+          logger.log('warn', `Route ${route.name} has authentication configured but it cannot be enforced for non-terminated connections`, {
+            connectionId,
+            routeName: route.name,
+            tlsMode: route.action.tls?.mode || 'none',
+            component: 'route-handler'
+          });
+        }
+      }
+    }
 
     // Handle the route based on its action type
     switch (route.action.type) {
@@ -634,6 +715,18 @@ export class RouteConnectionHandler {
       // No TLS settings - check if this port should use HttpProxy
       const isHttpProxyPort = this.settings.useHttpProxy?.includes(record.localPort);
       
+      // Debug logging
+      if (this.settings.enableDetailedLogging) {
+        logger.log('debug', `Checking HttpProxy forwarding: port=${record.localPort}, useHttpProxy=${JSON.stringify(this.settings.useHttpProxy)}, isHttpProxyPort=${isHttpProxyPort}, hasHttpProxy=${!!this.httpProxyBridge.getHttpProxy()}`, {
+          connectionId,
+          localPort: record.localPort,
+          useHttpProxy: this.settings.useHttpProxy,
+          isHttpProxyPort,
+          hasHttpProxy: !!this.httpProxyBridge.getHttpProxy(),
+          component: 'route-handler'
+        });
+      }
+      
       if (isHttpProxyPort && this.httpProxyBridge.getHttpProxy()) {
         // Forward non-TLS connections to HttpProxy if configured
         if (this.settings.enableDetailedLogging) {

package/ts/proxies/smart-proxy/route-manager.ts

@@ -211,9 +211,10 @@ export class RouteManager extends plugins.EventEmitter {
   
   /**
    * Check if a client IP is allowed by a route's security settings
+   * @deprecated Security is now checked in route-connection-handler.ts after route matching
    */
   private isClientIpAllowed(route: IRouteConfig, clientIp: string): boolean {
-    const security = route.action.security;
+    const security = route.security;
     
     if (!security) {
       return true; // No security settings means allowed
@@ -330,8 +331,9 @@ export class RouteManager extends plugins.EventEmitter {
     clientIp: string;
     path?: string;
     tlsVersion?: string;
+    skipDomainCheck?: boolean;
   }): IRouteMatchResult | null {
-    const { port, domain, clientIp, path, tlsVersion } = options;
+    const { port, domain, clientIp, path, tlsVersion, skipDomainCheck } = options;
     
     // Get all routes for this port
     const routesForPort = this.getRoutesForPort(port);
@@ -340,7 +342,7 @@ export class RouteManager extends plugins.EventEmitter {
     for (const route of routesForPort) {
       // Check domain match
       // If the route has domain restrictions and we have a domain to check
-      if (route.match.domains) {
+      if (route.match.domains && !skipDomainCheck) {
         // If no domain was provided (non-TLS or no SNI), this route doesn't match
         if (!domain) {
           continue;
@@ -351,6 +353,7 @@ export class RouteManager extends plugins.EventEmitter {
         }
       }
       // If route has no domain restrictions, it matches all domains
+      // If skipDomainCheck is true, we skip domain validation for HTTP connections
       
       // Check path match if specified in both route and request
       if (path && route.match.path) {
@@ -371,12 +374,8 @@ export class RouteManager extends plugins.EventEmitter {
         continue;
       }
       
-      // Check security settings
-      if (!this.isClientIpAllowed(route, clientIp)) {
-        continue;
-      }
-      
       // All checks passed, this route matches
+      // NOTE: Security is checked AFTER route matching in route-connection-handler.ts
       return { route };
     }
     

package/ts/proxies/smart-proxy/utils/route-helpers.ts

@@ -625,14 +625,6 @@ export function createNfTablesRoute(
     }
   };
   
-  // Add security if allowed or blocked IPs are specified
-  if (options.ipAllowList?.length || options.ipBlockList?.length) {
-    action.security = {
-      ipAllowList: options.ipAllowList,
-      ipBlockList: options.ipBlockList
-    };
-  }
-  
   // Add TLS options if needed
   if (options.useTls) {
     action.tls = {
@@ -641,11 +633,21 @@ export function createNfTablesRoute(
   }
   
   // Create the route config
-  return {
+  const routeConfig: IRouteConfig = {
     name,
     match,
     action
   };
+  
+  // Add security if allowed or blocked IPs are specified
+  if (options.ipAllowList?.length || options.ipBlockList?.length) {
+    routeConfig.security = {
+      ipAllowList: options.ipAllowList,
+      ipBlockList: options.ipBlockList
+    };
+  }
+  
+  return routeConfig;
 }
 
 /**