@push.rocks/smartproxy 15.0.2 → 16.0.3

package/readme.plan.md

@@ -1,316 +1,103 @@
-# SmartProxy Fully Unified Configuration Plan (Updated)
+# SmartProxy Configuration Troubleshooting
 
-## Project Goal
-Redesign SmartProxy's configuration for a more elegant, unified, and comprehensible approach by:
-1. Creating a single, unified configuration model that covers both "where to listen" and "how to forward"
-2. Eliminating the confusion between domain configs and port forwarding
-3. Providing a clear, declarative API that makes the intent obvious
-4. Enhancing maintainability and extensibility for future features
-5. Completely removing legacy code to eliminate technical debt
+## IPv6/IPv4 Mapping Issue
 
-## Current Issues
+### Problem Identified
+The SmartProxy is failing to match connections for wildcard domains (like `*.lossless.digital`) when IP restrictions are in place. After extensive debugging, the root cause has been identified:
 
-The current approach has several issues:
+When a connection comes in from an IPv4 address (e.g., `212.95.99.130`), the Node.js server receives it as an IPv6-mapped IPv4 address with the format `::ffff:212.95.99.130`. However, the route configuration is expecting the exact string `212.95.99.130`, causing a mismatch.
 
-1. **Dual Configuration Systems**:
-   - Port configuration (`fromPort`, `toPort`, `globalPortRanges`) for "where to listen"
-   - Domain configuration (`domainConfigs`) for "how to forward"
-   - Unclear relationship between these two systems
-
-2. **Mixed Concerns**:
-   - Port management is mixed with forwarding logic
-   - Domain routing is separated from port listening
-   - Security settings defined in multiple places
-
-3. **Complex Logic**:
-   - PortRangeManager must coordinate with domain configuration
-   - Implicit rules for handling connections based on port and domain
-
-4. **Difficult to Understand and Configure**:
-   - Two separate configuration hierarchies that must work together
-   - Unclear which settings take precedence
+From the debug logs:
+```
+[DEBUG] Route rejected: clientIp mismatch. Request: ::ffff:212.95.99.130, Route patterns: ["212.95.99.130"]
+```
 
-## Proposed Solution: Fully Unified Routing Configuration
+### Solution
 
-Replace both port and domain configuration with a single, unified configuration:
+To fix this issue, update the route configurations to include both formats of the IP address. Here's how to modify the affected route:
 
 ```typescript
-// The core unified configuration interface
-interface IRouteConfig {
-  // What to match
+// Wildcard domain route for *.lossless.digital
+{
   match: {
-    // Listen on these ports (required)
-    ports: number | number[] | Array<{ from: number, to: number }>;
-    
-    // Optional domain patterns to match (default: all domains)
-    domains?: string | string[];
-    
-    // Advanced matching criteria
-    path?: string;           // Match specific paths
-    clientIp?: string[];     // Match specific client IPs
-    tlsVersion?: string[];   // Match specific TLS versions
-  };
-  
-  // What to do with matched traffic
+    ports: 443,
+    domains: ['*.lossless.digital'],
+    clientIp: ['212.95.99.130', '::ffff:212.95.99.130'],  // Include both formats
+  },
   action: {
-    // Basic routing
-    type: 'forward' | 'redirect' | 'block';
-    
-    // Target for forwarding
-    target?: {
-      host: string | string[];  // Support single host or round-robin
-      port: number;
-      preservePort?: boolean;   // Use incoming port as target port
-    };
-    
-    // TLS handling
-    tls?: {
-      mode: 'passthrough' | 'terminate' | 'terminate-and-reencrypt';
-      certificate?: 'auto' | {   // Auto = use ACME
-        key: string;
-        cert: string;
-      };
-    };
-    
-    // For redirects
-    redirect?: {
-      to: string;            // URL or template with {domain}, {port}, etc.
-      status: 301 | 302 | 307 | 308;
-    };
-    
-    // Security options
-    security?: {
-      allowedIps?: string[];
-      blockedIps?: string[];
-      maxConnections?: number;
-      authentication?: {
-        type: 'basic' | 'digest' | 'oauth';
-        // Auth-specific options
-      };
-    };
-    
-    // Advanced options
-    advanced?: {
-      timeout?: number;
-      headers?: Record<string, string>;
-      keepAlive?: boolean;
-      // etc.
-    };
-  };
-  
-  // Optional metadata
-  name?: string;             // Human-readable name for this route
-  description?: string;      // Description of the route's purpose
-  priority?: number;         // Controls matching order (higher = matched first)
-  tags?: string[];           // Arbitrary tags for categorization
-}
-
-// Main SmartProxy options
-interface ISmartProxyOptions {
-  // The unified configuration array (required)
-  routes: IRouteConfig[];
-  
-  // Global/default settings
-  defaults?: {
-    target?: {
-      host: string;
-      port: number;
-    };
-    security?: {
-      // Global security defaults
-    };
-    tls?: {
-      // Global TLS defaults
-    };
-    // ...other defaults
-  };
-  
-  // Other global settings remain (acme, etc.)
-  acme?: IAcmeOptions;
-  
-  // Advanced settings remain as well
-  // ...
+    type: 'forward',
+    target: {
+      host: '212.95.99.130',
+      port: 443
+    },
+    tls: {
+      mode: 'passthrough'
+    },
+    security: {
+      allowedIps: ['212.95.99.130', '::ffff:212.95.99.130']  // Include both formats
+    }
+  },
+  name: 'Wildcard lossless.digital route (IP restricted)'
 }
 ```
 
-## Revised Implementation Plan
-
-### Phase 1: Core Design & Interface Definition
-
-1. **Define New Core Interfaces**:
-   - Create `IRouteConfig` interface with `match` and `action` branches
-   - Define all sub-interfaces for matching and actions
-   - Create new `ISmartProxyOptions` to use `routes` array exclusively
-   - Define template variable system for dynamic values
-
-2. **Create Helper Functions**:
-   - `createRoute()` - Basic route creation with reasonable defaults
-   - `createHttpRoute()`, `createHttpsRoute()`, `createRedirect()` - Common scenarios
-   - `createLoadBalancer()` - For multi-target setups
-   - `mergeSecurity()`, `mergeDefaults()` - For combining configs
-
-3. **Design Router**:
-   - Decision tree for route matching algorithm
-   - Priority system for route ordering
-   - Optimized lookup strategy for fast routing
-
-### Phase 2: Core Implementation
-
-1. **Create RouteManager**:
-   - Build a new RouteManager to replace both PortRangeManager and DomainConfigManager
-   - Implement port and domain matching in one unified system
-   - Create efficient route lookup algorithms
-
-2. **Implement New ConnectionHandler**:
-   - Create a new ConnectionHandler built from scratch for routes
-   - Implement the routing logic with the new match/action pattern
-   - Support template processing for headers and other dynamic values
-
-3. **Implement New SmartProxy Core**:
-   - Create new SmartProxy implementation using routes exclusively
-   - Build network servers based on port specifications
-   - Manage TLS contexts and certificates
-
-### Phase 3: Legacy Code Removal
-
-1. **Identify Legacy Components**:
-   - Create an inventory of all files and components to be removed
-   - Document dependencies between legacy components
-   - Create a removal plan that minimizes disruption
+### Alternative Long-Term Fix
 
-2. **Remove Legacy Components**:
-   - Remove PortRangeManager and related code
-   - Remove DomainConfigManager and related code
-   - Remove old ConnectionHandler implementation
-   - Remove other legacy components
+A more robust solution would be to modify the SmartProxy codebase to automatically handle IPv6-mapped IPv4 addresses by normalizing them before comparison. This would involve:
 
-3. **Clean Interface Adaptations**:
-   - Remove all legacy interfaces and types
-   - Update type exports to only expose route-based interfaces
-   - Remove any adapter or backward compatibility code
+1. Modifying the `matchIpPattern` function in `route-manager.ts` to normalize IPv6-mapped IPv4 addresses:
 
-### Phase 4: Updated Documentation & Examples
-
-1. **Update Core Documentation**:
-   - Rewrite README.md with a focus on route-based configuration exclusively
-   - Create interface reference documentation
-   - Document all template variables
-
-2. **Create Example Library**:
-   - Common configuration patterns using the new API
-   - Complex use cases for advanced features
-   - Infrastructure-as-code examples
-
-3. **Add Validation Tools**:
-   - Configuration validator to check for issues
-   - Schema definitions for IDE autocomplete
-   - Runtime validation helpers
-
-### Phase 5: Testing
-
-1. **Unit Tests**:
-   - Test route matching logic
-   - Validate priority handling
-   - Test template processing
-
-2. **Integration Tests**:
-   - Verify full proxy flows with the new system
-   - Test complex routing scenarios
-   - Ensure all features work as expected
-
-3. **Performance Testing**:
-   - Benchmark routing performance
-   - Evaluate memory usage
-   - Test with large numbers of routes
-
-## Implementation Strategy
-
-### Code Organization
-
-1. **New Files**:
-   - `route-config.ts` - Core route interfaces
-   - `route-manager.ts` - Route matching and management
-   - `route-connection-handler.ts` - Connection handling with routes
-   - `route-smart-proxy.ts` - Main SmartProxy implementation
-   - `template-engine.ts` - For variable substitution
-
-2. **File Removal**:
-   - Remove `port-range-manager.ts`
-   - Remove `domain-config-manager.ts`
-   - Remove legacy interfaces and adapter code
-   - Remove backward compatibility shims
-
-### Transition Strategy
-
-1. **Breaking Change Approach**:
-   - This will be a major version update with breaking changes
-   - No backward compatibility will be maintained
-   - Clear migration documentation will guide users to the new API
-
-2. **Package Structure**:
-   - `@push.rocks/smartproxy` package will be updated to v14.0.0
-   - Legacy code fully removed, only route-based API exposed
-   - Support documentation provided for migration
-
-3. **Migration Documentation**:
-   - Provide a migration guide with examples
-   - Show equivalent route configurations for common legacy patterns
-   - Offer code transformation helpers for complex setups
+```typescript
+private matchIpPattern(pattern: string, ip: string): boolean {
+  // Normalize IPv6-mapped IPv4 addresses
+  const normalizedIp = ip.startsWith('::ffff:') ? ip.substring(7) : ip;
+  const normalizedPattern = pattern.startsWith('::ffff:') ? pattern.substring(7) : pattern;
+  
+  // Handle exact match with normalized addresses
+  if (normalizedPattern === normalizedIp) {
+    return true;
+  }
+  
+  // Rest of the existing function...
+}
+```
 
-## Benefits of the Clean Approach
+2. Making similar modifications to other IP-related functions in the codebase.
 
-1. **Reduced Complexity**:
-   - No overlapping or conflicting configuration systems
-   - No dual maintenance of backward compatibility code
-   - Simplified internal architecture
+## Wild Card Domain Matching Issue
 
-2. **Cleaner Code Base**:
-   - Removal of technical debt
-   - Better separation of concerns
-   - More maintainable codebase
+### Explanation
 
-3. **Better User Experience**:
-   - Consistent, predictable API
-   - No confusing overlapping options
-   - Clear documentation of one approach, not two
+The wildcard domain matching in SmartProxy works as follows:
 
-4. **Future-Proof Design**:
-   - Easier to extend with new features
-   - Better performance without legacy overhead
-   - Cleaner foundation for future enhancements
+1. When a pattern like `*.lossless.digital` is specified, it's converted to a regex: `/^.*\.lossless\.digital$/i`
+2. This correctly matches any subdomain like `my.lossless.digital`, `api.lossless.digital`, etc.
+3. However, it does NOT match the apex domain `lossless.digital` (without a subdomain)
 
-## Migration Support
+If you need to match both the apex domain and subdomains, use a list:
+```typescript
+domains: ['lossless.digital', '*.lossless.digital']
+```
 
-While we're removing backward compatibility from the codebase, we'll provide extensive migration support:
+## Debugging SmartProxy
 
-1. **Migration Guide**:
-   - Detailed documentation on moving from legacy to route-based config
-   - Pattern-matching examples for all common use cases
-   - Troubleshooting guide for common migration issues
+To debug routing issues in SmartProxy:
 
-2. **Conversion Tool**:
-   - Provide a standalone one-time conversion tool
-   - Takes legacy configuration and outputs route-based equivalents
-   - Will not be included in the main package to avoid bloat
+1. Add detailed logging to the `route-manager.js` file in the `dist_ts` directory:
+   - `findMatchingRoute` method - to see what criteria are being checked
+   - `matchRouteDomain` method - to see domain matching logic
+   - `matchDomain` method - to see pattern matching
+   - `matchIpPattern` method - to see IP matching logic
 
-3. **Version Policy**:
-   - Maintain the legacy version (13.x) for security updates
-   - Make the route-based version a clear major version change (14.0.0)
-   - Clearly communicate the breaking changes
+2. Run the proxy with debugging enabled:
+   ```
+   pnpm run startNew
+   ```
 
-## Timeline and Versioning
+3. Monitor the logs for detailed information about the routing process and identify where matches are failing.
 
-1. **Development**:
-   - Develop route-based implementation in a separate branch
-   - Complete full test coverage of new implementation
-   - Ensure documentation is complete
+## Priority and Route Order
 
-2. **Release**:
-   - Release as version 14.0.0
-   - Clearly mark as breaking change
-   - Provide migration guide at release time
+Remember that routes are evaluated in priority order (higher priority first). If multiple routes could match the same request, ensure that the more specific routes have higher priority.
 
-3. **Support**:
-   - Offer extended support for migration questions
-   - Consider maintaining security updates for v13.x for 6 months
-   - Focus active development on route-based version only
+When routes have the same priority (or none specified), they're evaluated in the order they're defined in the configuration.

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartproxy',
-  version: '15.0.0',
+  version: '16.0.3',
   description: 'A powerful proxy package with unified route-based configuration for high traffic management. Features include SSL/TLS support, flexible routing patterns, WebSocket handling, advanced security options, and automatic ACME certificate management.'
 }

package/ts/certificate/index.ts

@@ -24,23 +24,31 @@ export * from './storage/file-storage.js';
 
 // Convenience function to create a certificate provisioner with common settings
 import { CertProvisioner } from './providers/cert-provisioner.js';
+import type { TCertProvisionObject } from './providers/cert-provisioner.js';
 import { buildPort80Handler } from './acme/acme-factory.js';
-import type { IAcmeOptions, IDomainForwardConfig } from './models/certificate-types.js';
-import type { IDomainConfig } from '../forwarding/config/domain-config.js';
+import type { IAcmeOptions, IRouteForwardConfig } from './models/certificate-types.js';
+import type { IRouteConfig } from '../proxies/smart-proxy/models/route-types.js';
+
+/**
+ * Interface for NetworkProxyBridge used by CertProvisioner
+ */
+interface ICertNetworkProxyBridge {
+  applyExternalCertificate(certData: any): void;
+}
 
 /**
  * Creates a complete certificate provisioning system with default settings
- * @param domainConfigs Domain configurations
+ * @param routeConfigs Route configurations that may need certificates
  * @param acmeOptions ACME options for certificate provisioning
  * @param networkProxyBridge Bridge to apply certificates to network proxy
  * @param certProvider Optional custom certificate provider
  * @returns Configured CertProvisioner
  */
 export function createCertificateProvisioner(
-  domainConfigs: IDomainConfig[],
+  routeConfigs: IRouteConfig[],
   acmeOptions: IAcmeOptions,
-  networkProxyBridge: any, // Placeholder until NetworkProxyBridge is migrated
-  certProvider?: any // Placeholder until cert provider type is properly defined
+  networkProxyBridge: ICertNetworkProxyBridge,
+  certProvider?: (domain: string) => Promise<TCertProvisionObject>
 ): CertProvisioner {
   // Build the Port80Handler for ACME challenges
   const port80Handler = buildPort80Handler(acmeOptions);
@@ -50,18 +58,18 @@ export function createCertificateProvisioner(
     renewThresholdDays = 30,
     renewCheckIntervalHours = 24,
     autoRenew = true,
-    domainForwards = []
+    routeForwards = []
   } = acmeOptions;
 
   // Create and return the certificate provisioner
   return new CertProvisioner(
-    domainConfigs,
+    routeConfigs,
     port80Handler,
     networkProxyBridge,
     certProvider,
     renewThresholdDays,
     renewCheckIntervalHours,
     autoRenew,
-    domainForwards
+    routeForwards
   );
 }

package/ts/certificate/models/certificate-types.ts

@@ -1,4 +1,5 @@
 import * as plugins from '../../plugins.js';
+import type { IRouteConfig } from '../../proxies/smart-proxy/models/route-types.js';
 
 /**
  * Certificate data structure containing all necessary information
@@ -12,6 +13,11 @@ export interface ICertificateData {
   // Optional source and renewal information for event emissions
   source?: 'static' | 'http01' | 'dns01';
   isRenewal?: boolean;
+  // Reference to the route that requested this certificate (if available)
+  routeReference?: {
+    routeId?: string;
+    routeName?: string;
+  };
 }
 
 /**
@@ -29,6 +35,10 @@ export interface ICertificateFailure {
   domain: string;
   error: string;
   isRenewal: boolean;
+  routeReference?: {
+    routeId?: string;
+    routeName?: string;
+  };
 }
 
 /**
@@ -38,35 +48,46 @@ export interface ICertificateExpiring {
   domain: string;
   expiryDate: Date;
   daysRemaining: number;
+  routeReference?: {
+    routeId?: string;
+    routeName?: string;
+  };
 }
 
 /**
- * Domain forwarding configuration
+ * Route-specific forwarding configuration for ACME challenges
  */
-export interface IForwardConfig {
-  ip: string;
-  port: number;
-}
-
-/**
- * Domain-specific forwarding configuration for ACME challenges
- */
-export interface IDomainForwardConfig {
+export interface IRouteForwardConfig {
   domain: string;
-  forwardConfig?: IForwardConfig;
-  acmeForwardConfig?: IForwardConfig;
+  target: {
+    host: string;
+    port: number;
+  };
   sslRedirect?: boolean;
 }
 
 /**
- * Domain configuration options
+ * Domain configuration options for Port80Handler
+ *
+ * This is used internally by the Port80Handler to manage domains
+ * but will eventually be replaced with route-based options.
  */
 export interface IDomainOptions {
   domainName: string;
   sslRedirect: boolean;   // if true redirects the request to port 443
   acmeMaintenance: boolean; // tries to always have a valid cert for this domain
-  forward?: IForwardConfig; // forwards all http requests to that target
-  acmeForward?: IForwardConfig; // forwards letsencrypt requests to this config
+  forward?: {
+    ip: string;
+    port: number;
+  }; // forwards all http requests to that target
+  acmeForward?: {
+    ip: string;
+    port: number;
+  }; // forwards letsencrypt requests to this config
+  routeReference?: {
+    routeId?: string;
+    routeName?: string;
+  };
 }
 
 /**
@@ -83,6 +104,6 @@ export interface IAcmeOptions {
   autoRenew?: boolean;            // Whether to automatically renew certificates
   certificateStore?: string;      // Directory to store certificates
   skipConfiguredCerts?: boolean;  // Skip domains with existing certificates
-  domainForwards?: IDomainForwardConfig[]; // Domain-specific forwarding configs
+  routeForwards?: IRouteForwardConfig[]; // Route-specific forwarding configs
 }