@push.rocks/smartproxy 10.2.0 → 12.0.0

package/readme.plan.md

@@ -1,42 +1,471 @@
-# Plan: On-Demand Certificate Retrieval in NetworkProxy
-
-When a TLS connection arrives with an SNI for a domain that has no certificate yet, we want to automatically kick off certificate issuance (ACME HTTP-01 or DNS-01) so the domain is provisioned on the fly without prior manual configuration.
-
-## Goals
-- Automatically initiate certificate issuance upon first TLS handshake for an unprovisioned domain.
-- Use Port80Handler (HTTP-01) or custom `certProvisionFunction` (e.g., DNS-01) to retrieve the certificate.
-- Continue the TLS handshake immediately using the default certificate, then swap to the new certificate on subsequent connections.
-- For HTTP traffic on port 80, register the domain for ACME and return a 503 until the challenge is complete.
-
-## Plan
-1. Detect missing certificate in SNI callback:
-   - In `ts/networkproxy/classes.np.networkproxy.ts` (or within `CertificateManager.handleSNI`), after looking up `certificateCache`, if no cert is found:
-     - Call `port80Handler.addDomain({ domainName, sslRedirect: false, acmeMaintenance: true })` to trigger dynamic provisioning.
-     - Emit a `certificateRequested` event for observability.
-     - Immediately call `cb(null, defaultSecureContext)` so the handshake uses the default cert.
-
-2. HTTP-01 fallback on port 80:
-   - In `ts/port80handler/classes.port80handler.ts``, in `handleRequest()`, when a request arrives for a new domain not in `domainCertificates`:
-     - Call `addDomain({ domainName, sslRedirect: false, acmeMaintenance: true })`.
-     - Return HTTP 503 with a message like “Certificate issuance in progress.”
-
-3. CertProvisioner & events:
-   - Ensure `CertProvisioner` is subscribed to `Port80Handler` for newly added domains.
-   - After certificate issuance completes, `Port80Handler` emits `CERTIFICATE_ISSUED`, `CertificateManager` caches and writes disk, and future SNI callbacks will serve the new cert.
-
-4. Metrics and cleanup:
-   - Track dynamic requests count via a `certificateRequested` event or metric.
-   - Handle error paths: if ACME/DNS fails, emit `CERTIFICATE_FAILED` and continue serving default cert.
-
-5. Tests:
-   - Simulate a TLS ClientHello for an unconfigured domain:
-     • Verify `port80Handler.addDomain` is called and `certificateRequested` event emitted.
-     • Confirm handshake completes with default cert context.
-   - Simulate HTTP-01 challenge flow for a new domain:
-     • Verify on first HTTP request, `addDomain` is invoked and 503 returned.
-     • After manually injecting a challenge in `Http01MemoryHandler`, verify 200 with key authorization.
-   - Simulate successful ACME response and ensure SNI now returns the real cert.
-
-6. Final validation:
-   - Run `pnpm test` to ensure all existing tests pass.
-   - Add new unit/integration tests for the dynamic provisioning flow.
+# SmartProxy Unified Forwarding Configuration Plan
+
+## Project Goal
+Create a clean, use-case driven forwarding configuration interface for SmartProxy that elegantly handles all forwarding scenarios: SNI-based forwarding, termination-based forwarding (NetworkProxy), HTTP forwarding, and ACME challenge forwarding.
+
+## Current State
+Currently, SmartProxy has several different forwarding mechanisms configured separately:
+1. **HTTPS/SNI forwarding** via `IDomainConfig` properties
+2. **NetworkProxy forwarding** via `useNetworkProxy` in domain configs
+3. **HTTP forwarding** via Port80Handler's `forward` configuration
+4. **ACME challenge forwarding** via `acmeForward` configuration
+
+This separation creates configuration complexity and reduced cohesion between related settings.
+
+## Proposed Solution: Clean Use-Case Driven Forwarding Interface
+
+### Phase 1: Design Streamlined Forwarding Interface
+
+- [ ] Create a use-case driven `IForwardConfig` interface that simplifies configuration:
+
+```typescript
+export interface IForwardConfig {
+  // Define the primary forwarding type - use-case driven approach
+  type: 'http-only' | 'https-passthrough' | 'https-terminate-to-http' | 'https-terminate-to-https';
+  
+  // Target configuration
+  target: {
+    host: string | string[];  // Support single host or round-robin
+    port: number;
+  };
+  
+  // HTTP-specific options
+  http?: {
+    enabled?: boolean;                 // Defaults to true for http-only, optional for others
+    redirectToHttps?: boolean;         // Redirect HTTP to HTTPS
+    headers?: Record<string, string>;  // Custom headers for HTTP responses
+  };
+  
+  // HTTPS-specific options
+  https?: {
+    customCert?: {                    // Use custom cert instead of auto-provisioned
+      key: string;
+      cert: string;
+    };
+    forwardSni?: boolean;             // Forward SNI info in passthrough mode
+  };
+  
+  // ACME certificate handling
+  acme?: {
+    enabled?: boolean;                // Enable ACME certificate provisioning  
+    maintenance?: boolean;            // Auto-renew certificates
+    production?: boolean;             // Use production ACME servers
+    forwardChallenges?: {             // Forward ACME challenges
+      host: string;
+      port: number;
+      useTls?: boolean;
+    };
+  };
+  
+  // Security options
+  security?: { 
+    allowedIps?: string[];            // IPs allowed to connect
+    blockedIps?: string[];            // IPs blocked from connecting
+    maxConnections?: number;          // Max simultaneous connections
+  };
+  
+  // Advanced options
+  advanced?: {
+    portRanges?: Array<{ from: number; to: number }>; // Allowed port ranges
+    networkProxyPort?: number;        // Custom NetworkProxy port if using terminate mode
+    keepAlive?: boolean;              // Enable TCP keepalive
+    timeout?: number;                 // Connection timeout in ms
+    headers?: Record<string, string>; // Custom headers with support for variables like {sni}
+  };
+}
+```
+
+### Phase 2: Create New Domain Configuration Interface
+
+- [ ] Replace existing `IDomainConfig` interface with a new one using the forwarding pattern:
+
+```typescript
+export interface IDomainConfig {
+  // Core properties
+  domains: string[];  // Domain patterns to match
+  
+  // Unified forwarding configuration
+  forwarding: IForwardConfig;
+}
+```
+
+### Phase 3: Implement Forwarding Handler System
+
+- [ ] Create an implementation strategy focused on the new forwarding types:
+
+```typescript
+/**
+ * Base class for all forwarding handlers
+ */
+abstract class ForwardingHandler {
+  constructor(protected config: IForwardConfig) {}
+  
+  abstract handleConnection(socket: Socket): void;
+  abstract handleHttpRequest(req: IncomingMessage, res: ServerResponse): void;
+}
+
+/**
+ * Factory for creating the appropriate handler based on forwarding type
+ */
+class ForwardingHandlerFactory {
+  public static createHandler(config: IForwardConfig): ForwardingHandler {
+    switch (config.type) {
+      case 'http-only':
+        return new HttpForwardingHandler(config);
+        
+      case 'https-passthrough':
+        return new HttpsPassthroughHandler(config);
+        
+      case 'https-terminate-to-http':
+        return new HttpsTerminateToHttpHandler(config);
+        
+      case 'https-terminate-to-https':
+        return new HttpsTerminateToHttpsHandler(config);
+        
+      default:
+        throw new Error(`Unknown forwarding type: ${config.type}`);
+    }
+  }
+}
+```
+
+## Usage Examples for Common Scenarios
+
+### 1. Basic HTTP Server
+
+```typescript
+{
+  domains: ['example.com'],
+  forwarding: {
+    type: 'http-only',
+    target: {
+      host: 'localhost',
+      port: 3000
+    }
+  }
+}
+```
+
+### 2. HTTPS Termination with HTTP Backend
+
+```typescript
+{
+  domains: ['secure.example.com'],
+  forwarding: {
+    type: 'https-terminate-to-http',
+    target: {
+      host: 'localhost',
+      port: 3000
+    },
+    acme: {
+      production: true  // Use production Let's Encrypt
+    }
+  }
+}
+```
+
+### 3. HTTPS Termination with HTTPS Backend
+
+```typescript
+{
+  domains: ['secure-backend.example.com'],
+  forwarding: {
+    type: 'https-terminate-to-https',
+    target: {
+      host: 'internal-api',
+      port: 8443
+    },
+    http: {
+      redirectToHttps: true  // Redirect HTTP requests to HTTPS
+    }
+  }
+}
+```
+
+### 4. SNI Passthrough
+
+```typescript
+{
+  domains: ['passthrough.example.com'],
+  forwarding: {
+    type: 'https-passthrough',
+    target: {
+      host: '10.0.0.5',
+      port: 443
+    }
+  }
+}
+```
+
+### 5. Mixed HTTP/HTTPS with Custom ACME Forwarding
+
+```typescript
+{
+  domains: ['mixed.example.com'],
+  forwarding: {
+    type: 'https-terminate-to-http',
+    target: {
+      host: 'localhost',
+      port: 3000
+    },
+    http: {
+      redirectToHttps: false  // Allow both HTTP and HTTPS access
+    },
+    acme: {
+      enabled: true,
+      maintenance: true,
+      forwardChallenges: {
+        host: '192.168.1.100',
+        port: 8080
+      }
+    }
+  }
+}
+```
+
+### 6. Load-Balanced Backend
+
+```typescript
+{
+  domains: ['api.example.com'],
+  forwarding: {
+    type: 'https-terminate-to-https',
+    target: {
+      host: ['10.0.0.10', '10.0.0.11', '10.0.0.12'], // Round-robin
+      port: 8443
+    },
+    security: {
+      allowedIps: ['10.0.0.*', '192.168.1.*']  // Restrict access
+    }
+  }
+}
+```
+
+### 7. Advanced Proxy Chain with Custom Headers
+
+```typescript
+{
+  domains: ['secure-chain.example.com'],
+  forwarding: {
+    type: 'https-terminate-to-https',
+    target: {
+      host: 'backend-gateway.internal',
+      port: 443
+    },
+    advanced: {
+      // Pass original client info to backend
+      headers: {
+        'X-Original-SNI': '{sni}',
+        'X-Client-IP': '{clientIp}'
+      }
+    }
+  }
+}
+```
+
+## Implementation Plan
+
+### Task 1: Core Types and Interfaces (Week 1)
+- [ ] Create the new `IForwardConfig` interface in `classes.pp.interfaces.ts`
+- [ ] Design the new `IDomainConfig` interface using the forwarding property
+- [ ] Define the internal data types for expanded configuration
+
+### Task 2: Forwarding Handlers (Week 1-2)
+- [ ] Create abstract `ForwardingHandler` base class
+- [ ] Implement concrete handlers for each forwarding type:
+  - [ ] `HttpForwardingHandler` - For HTTP-only configurations
+  - [ ] `HttpsPassthroughHandler` - For SNI passthrough
+  - [ ] `HttpsTerminateToHttpHandler` - For TLS termination to HTTP backends
+  - [ ] `HttpsTerminateToHttpsHandler` - For TLS termination to HTTPS backends
+- [ ] Implement `ForwardingHandlerFactory` to create the appropriate handler
+
+### Task 3: SmartProxy Integration (Week 2-3)
+- [ ] Update `SmartProxy` class to use the new forwarding system
+- [ ] Modify `ConnectionHandler` to delegate to forwarding handlers
+- [ ] Refactor domain configuration processing to use forwarding types
+- [ ] Update `Port80Handler` integration to work with the new system
+
+### Task 4: Certificate Management (Week 3)
+- [ ] Create a certificate management system that works with forwarding types
+- [ ] Implement automatic ACME provisioning based on forwarding type
+- [ ] Add custom certificate support
+
+### Task 5: Testing & Helper Functions (Week 4)
+- [ ] Create helper functions for common forwarding patterns
+- [ ] Implement comprehensive test suite for each forwarding handler
+- [ ] Add validation for forwarding configurations
+
+### Task 6: Documentation (Week 4)
+- [ ] Create detailed documentation for the new forwarding system
+- [ ] Document the forwarding types and their use cases
+- [ ] Update README with the new configuration examples
+
+## Detailed Type Documentation
+
+### Core Forwarding Types
+
+```typescript
+/**
+ * The primary forwarding types supported by SmartProxy
+ */
+export type ForwardingType = 
+  | 'http-only'                // HTTP forwarding only (no HTTPS)
+  | 'https-passthrough'        // Pass-through TLS traffic (SNI forwarding)
+  | 'https-terminate-to-http'  // Terminate TLS and forward to HTTP backend
+  | 'https-terminate-to-https'; // Terminate TLS and forward to HTTPS backend
+```
+
+### Type-Specific Behavior
+
+Each forwarding type has specific default behavior:
+
+#### HTTP-Only
+- Handles only HTTP traffic
+- No TLS/HTTPS support
+- No certificate management
+
+#### HTTPS Passthrough
+- Forwards raw TLS traffic to backend (no termination)
+- Passes SNI information through
+- No HTTP support (TLS only)
+- No certificate management
+
+#### HTTPS Terminate to HTTP
+- Terminates TLS at SmartProxy
+- Connects to backend using HTTP (non-TLS)
+- Manages certificates automatically (ACME)
+- Supports HTTP requests with option to redirect to HTTPS
+
+#### HTTPS Terminate to HTTPS
+- Terminates client TLS at SmartProxy
+- Creates new TLS connection to backend
+- Manages certificates automatically (ACME)
+- Supports HTTP requests with option to redirect to HTTPS
+
+## Handler Implementation Strategy
+
+```typescript
+/**
+ * Handler for HTTP-only forwarding
+ */
+class HttpForwardingHandler extends ForwardingHandler {
+  public handleConnection(socket: Socket): void {
+    // Process HTTP connection
+    // For HTTP-only, we'll mostly defer to handleHttpRequest
+  }
+  
+  public handleHttpRequest(req: IncomingMessage, res: ServerResponse): void {
+    // Forward HTTP request to target
+    const target = this.getTargetFromConfig();
+    this.proxyRequest(req, res, target);
+  }
+}
+
+/**
+ * Handler for HTTPS passthrough (SNI forwarding)
+ */
+class HttpsPassthroughHandler extends ForwardingHandler {
+  public handleConnection(socket: Socket): void {
+    // Extract SNI from TLS ClientHello if needed
+    // Forward raw TLS traffic to target without termination
+    const target = this.getTargetFromConfig();
+    this.forwardTlsConnection(socket, target);
+  }
+  
+  public handleHttpRequest(req: IncomingMessage, res: ServerResponse): void {
+    // HTTP not supported in SNI passthrough mode
+    res.statusCode = 404;
+    res.end('HTTP not supported for this domain');
+  }
+}
+
+/**
+ * Handler for HTTPS termination with HTTP backend
+ */
+class HttpsTerminateToHttpHandler extends ForwardingHandler {
+  private tlsContext: SecureContext;
+  
+  public async initialize(): Promise<void> {
+    // Set up TLS termination context
+    this.tlsContext = await this.createTlsContext();
+  }
+  
+  public handleConnection(socket: Socket): void {
+    // Terminate TLS
+    const tlsSocket = this.createTlsSocket(socket, this.tlsContext);
+    
+    // Forward to HTTP backend after TLS termination
+    tlsSocket.on('data', (data) => {
+      this.forwardToHttpBackend(data);
+    });
+  }
+  
+  public handleHttpRequest(req: IncomingMessage, res: ServerResponse): void {
+    if (this.config.http?.redirectToHttps) {
+      // Redirect to HTTPS if configured
+      this.redirectToHttps(req, res);
+    } else {
+      // Handle HTTP request
+      const target = this.getTargetFromConfig();
+      this.proxyRequest(req, res, target);
+    }
+  }
+}
+
+/**
+ * Handler for HTTPS termination with HTTPS backend
+ */
+class HttpsTerminateToHttpsHandler extends ForwardingHandler {
+  private tlsContext: SecureContext;
+  
+  public async initialize(): Promise<void> {
+    // Set up TLS termination context
+    this.tlsContext = await this.createTlsContext();
+  }
+  
+  public handleConnection(socket: Socket): void {
+    // Terminate client TLS
+    const tlsSocket = this.createTlsSocket(socket, this.tlsContext);
+    
+    // Create new TLS connection to backend
+    tlsSocket.on('data', (data) => {
+      this.forwardToHttpsBackend(data);
+    });
+  }
+  
+  public handleHttpRequest(req: IncomingMessage, res: ServerResponse): void {
+    if (this.config.http?.redirectToHttps) {
+      // Redirect to HTTPS if configured
+      this.redirectToHttps(req, res);
+    } else {
+      // Handle HTTP request via HTTPS to backend
+      const target = this.getTargetFromConfig();
+      this.proxyRequestOverHttps(req, res, target);
+    }
+  }
+}
+```
+
+## Benefits of This Approach
+
+1. **Clean, Type-Driven Design**
+   - Forwarding types clearly express intent
+   - No backward compatibility compromises
+   - Code structure follows the domain model
+
+2. **Explicit Configuration**
+   - Configuration directly maps to behavior
+   - Reduced chance of unexpected behavior
+
+3. **Modular Implementation**
+   - Each forwarding type handled by dedicated class
+   - Clear separation of concerns
+   - Easier to test and extend
+
+4. **Simplified Mental Model**
+   - Users think in terms of use cases, not low-level settings
+   - Configuration matches mental model
+
+5. **Future-Proof**
+   - Easy to add new forwarding types
+   - Clean extension points for new features

package/ts/00_commitinfo_data.ts

@@ -3,6 +3,6 @@
  */
 export const commitinfo = {
   name: '@push.rocks/smartproxy',
-  version: '10.2.0',
+  version: '12.0.0',
   description: 'A powerful proxy package that effectively handles high traffic, with features such as SSL/TLS support, port proxying, WebSocket handling, dynamic routing with authentication options, and automatic ACME certificate management.'
 }

package/ts/common/port80-adapter.ts

@@ -0,0 +1,87 @@
+import * as plugins from '../plugins.js';
+
+import type {
+  IForwardConfig as ILegacyForwardConfig,
+  IDomainOptions
+} from './types.js';
+
+import type {
+  IForwardConfig
+} from '../smartproxy/types/forwarding.types.js';
+
+/**
+ * Converts a forwarding configuration target to the legacy format
+ * for Port80Handler
+ */
+export function convertToLegacyForwardConfig(
+  forwardConfig: IForwardConfig
+): ILegacyForwardConfig {
+  // Determine host from the target configuration
+  const host = Array.isArray(forwardConfig.target.host)
+    ? forwardConfig.target.host[0]  // Use the first host in the array
+    : forwardConfig.target.host;
+
+  return {
+    ip: host,
+    port: forwardConfig.target.port
+  };
+}
+
+/**
+ * Creates Port80Handler domain options from a domain name and forwarding config
+ */
+export function createPort80HandlerOptions(
+  domain: string,
+  forwardConfig: IForwardConfig
+): IDomainOptions {
+  // Determine if we should redirect HTTP to HTTPS
+  let sslRedirect = false;
+  if (forwardConfig.http?.redirectToHttps) {
+    sslRedirect = true;
+  }
+  
+  // Determine if ACME maintenance should be enabled
+  // Enable by default for termination types, unless explicitly disabled
+  const requiresTls = 
+    forwardConfig.type === 'https-terminate-to-http' || 
+    forwardConfig.type === 'https-terminate-to-https';
+  
+  const acmeMaintenance = 
+    requiresTls && 
+    forwardConfig.acme?.enabled !== false;
+  
+  // Set up forwarding configuration
+  const options: IDomainOptions = {
+    domainName: domain,
+    sslRedirect,
+    acmeMaintenance
+  };
+  
+  // Add ACME challenge forwarding if configured
+  if (forwardConfig.acme?.forwardChallenges) {
+    options.acmeForward = {
+      ip: Array.isArray(forwardConfig.acme.forwardChallenges.host) 
+        ? forwardConfig.acme.forwardChallenges.host[0]
+        : forwardConfig.acme.forwardChallenges.host,
+      port: forwardConfig.acme.forwardChallenges.port
+    };
+  }
+  
+  // Add HTTP forwarding if this is an HTTP-only config or if HTTP is enabled
+  const supportsHttp = 
+    forwardConfig.type === 'http-only' || 
+    (forwardConfig.http?.enabled !== false &&
+     (forwardConfig.type === 'https-terminate-to-http' || 
+      forwardConfig.type === 'https-terminate-to-https'));
+  
+  if (supportsHttp) {
+    options.forward = {
+      ip: Array.isArray(forwardConfig.target.host) 
+        ? forwardConfig.target.host[0]
+        : forwardConfig.target.host,
+      port: forwardConfig.target.port
+    };
+  }
+  
+  return options;
+}

package/ts/index.ts

@@ -7,3 +7,6 @@ export * from './smartproxy/classes.pp.snihandler.js';
 export * from './smartproxy/classes.pp.interfaces.js';
 
 export * from './common/types.js';
+
+// Export forwarding system
+export * as forwarding from './smartproxy/forwarding/index.js';