@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

package/docs/03-PATTERN-GUIDES/error-handling/modules/error-handling-06-retry-strategies.md

@@ -1,1034 +1,1034 @@
-# Module 6: Retry Strategies
-
-**Level:** Advanced
-**Estimated Time:** 25 minutes
-
-## Overview
-
-This module teaches you how to implement effective retry strategies, including how to distinguish retryable from non-retryable errors and how to use exponential backoff to avoid overwhelming systems.
-
-## Learning Objectives
-
-By the end of this module, you will:
-- ✅ Know which errors are retryable and which are permanent
-- ✅ Implement exponential backoff with jitter
-- ✅ Use the SDK's `isRetryable()` method effectively
-- ✅ Handle rate limiting and timeouts correctly
-- ✅ Build production-ready retry logic
-
-## Retryable vs Non-Retryable Errors
-
-### Non-Retryable Errors (Permanent Failures)
-
-These errors indicate problems with data or configuration that won't be fixed by retrying:
-
-```typescript
-const NON_RETRYABLE_CODES = [
-  IngestionErrorCode.PARSE_ERROR,              // Invalid XML/JSON/CSV
-  IngestionErrorCode.VALIDATION_ERROR,         // Data validation failed
-  IngestionErrorCode.REQUIRED_FIELD_MISSING,   // Missing required field
-  IngestionErrorCode.CONFIGURATION_ERROR,      // Invalid config
-  IngestionErrorCode.FIELD_MAPPING_ERROR,      // Mapping config wrong
-  IngestionErrorCode.INVALID_FILE_FORMAT,      // Unsupported file type
-];
-```
-
-**Why not retryable:**
-- Source data is malformed
-- Configuration is incorrect
-- Business logic validation failed
-- Schema mismatch
-
-**What to do:**
-1. Log the error with full context
-2. Alert developers/operators
-3. Fix the underlying issue (data or config)
-4. Re-submit after fix
-
-**Example:**
-
-```typescript
-if (error instanceof FileParsingError) {
-  logger.error('PERMANENT ERROR - Fix source file:', {
-    fileName: error.fileName,
-    lineNumber: error.lineNumber,
-    message: error.message
-  });
-
-  // Don't retry - save to dead letter queue for manual review
-  await saveToDeadLetterQueue(fileName, content, error);
-
-  throw error;  // Fail immediately
-}
-```
-
-### Retryable Errors (Transient Failures)
-
-These errors are likely temporary and may succeed if retried:
-
-```typescript
-const RETRYABLE_CODES = [
-  IngestionErrorCode.NETWORK_ERROR,            // Network connectivity issue
-  IngestionErrorCode.TIMEOUT_ERROR,            // Request timed out
-  IngestionErrorCode.RATE_LIMIT_ERROR,         // API rate limit hit
-  IngestionErrorCode.LOCK_ACQUISITION_FAILED,  // Distributed lock conflict
-];
-
-// Also retryable:
-// - FluentAPIError with statusCode >= 500 (Server Error)
-// - FluentAPIError with statusCode === 429 (Rate Limit)
-// - GraphQLExecutionError with code 'TIMEOUT' or 'NETWORK_ERROR'
-```
-
-**Why retryable:**
-- Network issues are often temporary
-- Timeouts may be due to temporary load
-- Rate limits reset after time period
-- Lock conflicts resolve when other process completes
-
-**What to do:**
-1. Check `error.isRetryable()` returns `true`
-2. Implement exponential backoff
-3. Respect rate limit headers (`retryAfter`)
-4. Set maximum retry limit
-5. Log retry attempts
-
-**Example:**
-
-```typescript
-if (error instanceof IngestionError && error.isRetryable()) {
-  const delay = Math.pow(2, attempt) * 1000;  // Exponential backoff
-  logger.info(`Retryable error - will retry in ${delay}ms`, {
-    code: error.code,
-    attempt,
-    maxRetries
-  });
-
-  await new Promise(resolve => setTimeout(resolve, delay));
-  return retry();
-}
-```
-
-## Using isRetryable()
-
-The SDK's `IngestionError` class provides an `isRetryable()` method:
-
-```typescript
-abstract class IngestionError extends Error {
-  isRetryable(): boolean {
-    const retryableCodes = [
-      IngestionErrorCode.NETWORK_ERROR,
-      IngestionErrorCode.TIMEOUT_ERROR,
-      IngestionErrorCode.RATE_LIMIT_ERROR,
-      IngestionErrorCode.LOCK_ACQUISITION_FAILED,
-    ];
-
-    return retryableCodes.includes(this.code);
-  }
-}
-```
-
-### Basic Usage
-
-```typescript
-import { IngestionError } from '@fluentcommerce/fc-connect-sdk';
-
-try {
-  await processFile(fileName);
-} catch (error) {
-  if (error instanceof IngestionError) {
-    if (error.isRetryable()) {
-      // Retry logic
-      return scheduleRetry();
-    } else {
-      // Permanent failure
-      logger.error('Permanent error - manual intervention required');
-      throw error;
-    }
-  }
-  throw error;
-}
-```
-
-### Handling Non-SDK Errors
-
-Not all errors extend `IngestionError`:
-
-```typescript
-try {
-  await operation();
-} catch (error) {
-  // Check SDK errors first
-  if (error instanceof IngestionError) {
-    if (!error.isRetryable()) {
-      throw error;  // Don't retry
-    }
-  }
-
-  // File parsing errors are never retryable
-  if (error instanceof FileParsingError) {
-    throw error;  // Don't retry
-  }
-
-  // Path resolution errors are config issues
-  if (error instanceof PathResolutionError) {
-    throw error;  // Don't retry
-  }
-
-  // Mapping errors are data/config issues
-  if (error instanceof MappingError) {
-    throw error;  // Don't retry
-  }
-
-  // Unknown errors - retry with caution
-  if (attempt < maxRetries) {
-    logger.warn('Unknown error - attempting retry', { error });
-    await delay(1000 * attempt);
-    return retry();
-  }
-
-  throw error;
-}
-```
-
-## Exponential Backoff
-
-### Why Exponential Backoff?
-
-**Linear backoff** (same delay each time):
-```
-Attempt 1: Wait 1s
-Attempt 2: Wait 1s
-Attempt 3: Wait 1s
-```
-❌ Doesn't give system time to recover from overload
-
-**Exponential backoff** (increasing delay):
-```
-Attempt 1: Wait 1s
-Attempt 2: Wait 2s
-Attempt 3: Wait 4s
-Attempt 4: Wait 8s
-```
-✅ Reduces load on overloaded systems
-✅ Increases chance of success on each retry
-
-### Basic Implementation
-
-```typescript
-async function retryWithBackoff<T>(
-  operation: () => Promise<T>,
-  maxRetries: number = 3,
-  initialDelay: number = 1000
-): Promise<T> {
-  let lastError: Error;
-
-  for (let attempt = 0; attempt < maxRetries; attempt++) {
-    try {
-      return await operation();
-    } catch (error) {
-      lastError = error;
-
-      // Check if error is retryable
-      if (error instanceof IngestionError) {
-        if (!error.isRetryable()) {
-          throw error;  // Don't retry permanent errors
-        }
-      } else if (error instanceof MappingError ||
-                 error instanceof PathResolutionError ||
-                 error instanceof FileParsingError) {
-        throw error;  // These are permanent errors
-      }
-
-      // Last attempt - throw error
-      if (attempt === maxRetries - 1) {
-        throw lastError;
-      }
-
-      // Calculate delay with exponential backoff
-      const delay = initialDelay * Math.pow(2, attempt);
-
-      console.log(`Retry ${attempt + 1}/${maxRetries} after ${delay}ms`);
-      await new Promise(resolve => setTimeout(resolve, delay));
-    }
-  }
-
-  throw lastError!;
-}
-
-// Usage
-const result = await retryWithBackoff(
-  () => processOrder(xmlContent),
-  3,    // max retries
-  2000  // initial delay (2 seconds)
-);
-```
-
-### With Jitter
-
-Adding random jitter prevents thundering herd problem:
-
-```typescript
-async function retryWithBackoffAndJitter<T>(
-  operation: () => Promise<T>,
-  maxRetries: number = 3,
-  initialDelay: number = 1000,
-  maxDelay: number = 60000
-): Promise<T> {
-  let lastError: Error;
-
-  for (let attempt = 0; attempt < maxRetries; attempt++) {
-    try {
-      return await operation();
-    } catch (error) {
-      lastError = error;
-
-      // Check if retryable
-      if (error instanceof IngestionError && !error.isRetryable()) {
-        throw error;
-      }
-      if (error instanceof MappingError ||
-          error instanceof PathResolutionError ||
-          error instanceof FileParsingError) {
-        throw error;
-      }
-
-      if (attempt === maxRetries - 1) {
-        throw lastError;
-      }
-
-      // Exponential backoff with jitter
-      const exponentialDelay = initialDelay * Math.pow(2, attempt);
-      const jitter = Math.random() * 100;  // 0-100ms random jitter
-      const delay = Math.min(exponentialDelay + jitter, maxDelay);
-
-      console.log(`Retry ${attempt + 1}/${maxRetries} after ${delay}ms`);
-      await new Promise(resolve => setTimeout(resolve, delay));
-    }
-  }
-
-  throw lastError!;
-}
-```
-
-**Benefits of jitter:**
-- ✅ Prevents multiple clients retrying at exact same time
-- ✅ Spreads load more evenly
-- ✅ Reduces likelihood of cascading failures
-
-### Full-Jitter Strategy
-
-AWS recommended approach - more aggressive randomization:
-
-```typescript
-function calculateBackoffWithFullJitter(
-  attempt: number,
-  baseDelay: number = 1000,
-  maxDelay: number = 60000
-): number {
-  const exponentialDelay = baseDelay * Math.pow(2, attempt);
-  const cappedDelay = Math.min(exponentialDelay, maxDelay);
-
-  // Full jitter: random value between 0 and cappedDelay
-  return Math.random() * cappedDelay;
-}
-
-// Usage
-const delay = calculateBackoffWithFullJitter(attempt);
-await new Promise(resolve => setTimeout(resolve, delay));
-```
-
-## Rate Limiting
-
-### Handling Rate Limit Errors
-
-Respect the `retryAfter` value from rate limit errors:
-
-```typescript
-import { IngestionError, IngestionErrorCode } from '@fluentcommerce/fc-connect-sdk';
-
-async function handleRateLimit(
-  operation: () => Promise<any>,
-  maxRetries: number = 3
-): Promise<any> {
-  for (let attempt = 0; attempt < maxRetries; attempt++) {
-    try {
-      return await operation();
-    } catch (error) {
-      if (error instanceof IngestionError &&
-          error.code === IngestionErrorCode.RATE_LIMIT_ERROR) {
-
-        // Get retry-after value from error context
-        const retryAfter = error.context?.retryAfter || 60;
-
-        if (attempt < maxRetries - 1) {
-          logger.warn(`Rate limited - waiting ${retryAfter} seconds`, {
-            attempt,
-            maxRetries
-          });
-
-          // Wait for rate limit to reset
-          await new Promise(resolve => setTimeout(resolve, retryAfter * 1000));
-          continue;
-        }
-      }
-
-      // Non-rate-limit error or max retries exceeded
-      throw error;
-    }
-  }
-
-  throw new Error('Max retries exceeded');
-}
-```
-
-### Rate Limit with Exponential Backoff Fallback
-
-If no `retryAfter` provided, use exponential backoff:
-
-```typescript
-async function handleRateLimitSmart(
-  operation: () => Promise<any>,
-  maxRetries: number = 5
-): Promise<any> {
-  for (let attempt = 0; attempt < maxRetries; attempt++) {
-    try {
-      return await operation();
-    } catch (error) {
-      if (error instanceof IngestionError &&
-          error.code === IngestionErrorCode.RATE_LIMIT_ERROR) {
-
-        if (attempt >= maxRetries - 1) {
-          throw error;  // Max retries exceeded
-        }
-
-        // Use retryAfter if provided, else exponential backoff
-        const retryAfter = error.context?.retryAfter;
-        let delay: number;
-
-        if (retryAfter) {
-          delay = retryAfter * 1000;  // Convert to ms
-          logger.info(`Rate limited - retry after ${retryAfter}s`);
-        } else {
-          delay = Math.pow(2, attempt) * 1000;  // Exponential backoff
-          logger.info(`Rate limited - retry after ${delay}ms (no retryAfter)`);
-        }
-
-        await new Promise(resolve => setTimeout(resolve, delay));
-        continue;
-      }
-
-      // Not a rate limit error
-      throw error;
-    }
-  }
-
-  throw new Error('Max retries exceeded');
-}
-```
-
-## Timeout Handling
-
-### Network Timeouts
-
-Handle network timeouts with retry:
-
-```typescript
-async function executeWithTimeout<T>(
-  operation: () => Promise<T>,
-  timeoutMs: number = 30000,
-  maxRetries: number = 3
-): Promise<T> {
-  for (let attempt = 0; attempt < maxRetries; attempt++) {
-    try {
-      // Race between operation and timeout
-      const result = await Promise.race([
-        operation(),
-        new Promise<never>((_, reject) =>
-          setTimeout(() => reject(new Error('Operation timeout')), timeoutMs)
-        )
-      ]);
-
-      return result;
-
-    } catch (error) {
-      if (error.message === 'Operation timeout') {
-        if (attempt < maxRetries - 1) {
-          const delay = Math.pow(2, attempt) * 1000;
-          logger.warn(`Timeout - retrying in ${delay}ms`, { attempt });
-          await new Promise(resolve => setTimeout(resolve, delay));
-          continue;
-        }
-      }
-
-      // Not a timeout or max retries exceeded
-      throw error;
-    }
-  }
-
-  throw new Error('Max retries exceeded');
-}
-```
-
-## SDK Built-In Retry Logic
-
-### S3DataSource Retry
-
-The SDK's S3DataSource uses the shared `RetryHelper` utility for consistent retry behavior:
-
-```typescript
-import { S3DataSource } from '@fluentcommerce/fc-connect-sdk';
-
-const s3Config = {
-  type: 'S3_CSV' as const,
-  s3Config: {
-    bucket: 'my-bucket',
-    region: 'us-east-1',
-    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
-    maxAttempts: 3  // Default: 3 retries
-  }
-};
-
-const s3 = new S3DataSource(s3Config, logger);
-
-// Automatic retry on:
-// - HTTP 429 (rate limit)
-// - HTTP 500, 502, 503, 504 (server errors)
-// - Network errors (ECONNRESET, ETIMEDOUT, etc.)
-const files = await s3.listFiles();  // Retries automatically
-const content = await s3.downloadFile(key);  // Retries automatically
-```
-
-**Retry Configuration:**
-- **Max attempts**: Configurable via `maxAttempts` (default: 3)
-- **Base delay**: 1000ms
-- **Backoff factor**: 2 (exponential)
-- **Max delay**: 8000ms (8 seconds)
-- **Jitter**: Full jitter (AWS recommended)
-- **Retryable status codes**: 429, 500, 502, 503, 504
-- **Retryable network errors**: Connection resets, timeouts, unreachable hosts
-
-**Implementation Note:** S3DataSource uses `RetryHelper.executeWithRetry()` from `src/utils/retry-helper.ts` for consistency across all SDK HTTP operations.
-
-**SFTP Note:** SftpDataSource uses custom retry logic due to connection pooling requirements. See lines 536-645 for SFTP-specific retry patterns.
-
-### SftpDataSource Retry
-
-The SDK's SftpDataSource includes configurable retry logic with connection pooling:
-
-```typescript
-import { SftpDataSource, SftpDataSourceConfig } from '@fluentcommerce/fc-connect-sdk';
-
-const sftpConfig: SftpDataSourceConfig = {
-  type: 'SFTP_CSV',
-  settings: {
-    host: 'sftp.example.com',
-    port: 22,
-    username: 'user',
-    password: 'pass',
-    remotePath: '/data',
-    filePattern: '*.csv',
-
-    // Enhanced retry configuration
-    retry: {
-      maxAttempts: 3,           // Default: 3
-      baseDelayMs: 1000,        // Default: 1000ms (1 second)
-      backoffFactor: 2,         // Default: 2 (exponential)
-      maxDelayMs: 8000,         // Default: 8000ms (8 seconds)
-      jitter: 'full',           // Default: 'full' jitter
-      reconnectOnFailure: true, // Default: true - reconnect on error
-      isRetryable: (error) => {
-        // Custom retry logic (optional)
-        return error.message.includes('timeout');
-      }
-    }
-  }
-};
-
-const sftp = new SftpDataSource(sftpConfig, logger);
-
-// Automatic retry with exponential backoff and connection pooling
-const files = await sftp.listFiles();  // Retries automatically!
-const content = await sftp.downloadFile('file.csv');  // Retries automatically!
-```
-
-**Retry behavior:**
-- **Default strategy**: Exponential backoff with full jitter
-- **Connection pooling**: Reuses connections, creates new ones when needed
-- **Wait queue**: Queues requests when pool is full (prevents connection spam)
-- **Reconnect on failure**: Automatically reconnects on transient errors
-
-**Retryable errors:**
-- Connection resets (`ECONNRESET`)
-- Timeouts (`ETIMEDOUT`)
-- Host unreachable (`EHOSTUNREACH`)
-- Connection refused (`ECONNREFUSED`)
-- Broken pipe (`EPIPE`)
-- Message patterns: "connection lost", "timeout", "network error"
-
-**Non-retryable errors:**
-- Authentication failures
-- Permission denied
-- File not found
-- Algorithm/cipher errors
-- Key exchange failures
-
-**Implementation details:**
-```typescript
-private async executeWithRetry<T>(
-  operation: () => Promise<T>,
-  operationName: string,
-  retryOverride?: Partial<SftpRetryConfig>
-): Promise<T> {
-  const config = { ...this.retryConfig, ...retryOverride };
-  let lastError: Error;
-
-  for (let attempt = 1; attempt <= config.maxAttempts; attempt++) {
-    try {
-      return await operation();
-    } catch (error) {
-      lastError = error as Error;
-      const isRetryable = config.isRetryable(lastError);
-      const isLastAttempt = attempt === config.maxAttempts;
-
-      if (!isRetryable || isLastAttempt) {
-        throw error;
-      }
-
-      // Reconnect on failure if enabled
-      if (config.reconnectOnFailure) {
-        await this.closeConnection(connectionId);
-      }
-
-      // Calculate delay with jitter
-      const delay = this.calculateDelay(attempt);
-      await new Promise(resolve => setTimeout(resolve, delay));
-    }
-  }
-
-  throw lastError!;
-}
-
-private calculateDelay(attempt: number): number {
-  const exponentialDelay =
-    this.retryConfig.baseDelayMs * Math.pow(this.retryConfig.backoffFactor, attempt - 1);
-  const cappedDelay = Math.min(exponentialDelay, this.retryConfig.maxDelayMs);
-
-  if (this.retryConfig.jitter === 'none') {
-    return cappedDelay;
-  }
-
-  // Full jitter: random between 0 and cappedDelay
-  return Math.floor(Math.random() * cappedDelay);
-}
-```
-
-### Connection Pooling (SFTP only)
-
-```typescript
-// Connection pool configuration
-const sftpConfig: SftpDataSourceConfig = {
-  type: 'SFTP_CSV',
-  settings: {
-    // ... other settings
-    maxConnections: 5  // Default: 5 concurrent connections
-  }
-};
-
-// Connection pooling features:
-// - Reuses idle connections
-// - Creates new connections up to maxConnections
-// - Queues requests when pool is full
-// - Automatically closes idle connections after timeout
-// - Handles connection failures gracefully
-```
-
-### When to Use Built-In Retry vs Custom Retry
-
-**Use SDK built-in retry when:**
-- ✅ Working with S3DataSource or SftpDataSource
-- ✅ Default retry behavior is acceptable
-- ✅ You want zero configuration
-- ✅ You need connection pooling (SFTP)
-
-**Use custom retry when:**
-- ✅ Working with FluentClient GraphQL operations
-- ✅ Need custom retry logic beyond data source operations
-- ✅ Need to coordinate retries across multiple services
-- ✅ Want different retry strategies for different operations
-
-**Example: Combining both**
-
-```typescript
-// S3 operations use built-in retry
-const files = await s3.listFiles();  // Automatic retry
-
-// Custom retry for high-level workflow
-await retryOperation(
-  async () => {
-    // Download (automatic retry)
-    const content = await s3.downloadFile(files[0].path);
-
-    // Parse
-    const parsed = await parser.parse(content);
-
-    // Map
-    const payload = await mapper.map(parsed);
-
-    // Submit to Fluent (custom retry)
-    return await client.graphql(payload);
-  },
-  {
-    maxRetries: 5,
-    initialDelay: 2000
-  }
-);
-```
-
-## Partial Batch Recovery
-
-### PartialBatchRecovery Service
-
-The SDK's `PartialBatchRecovery` service handles partial batch failures gracefully by retrying only failed records instead of the entire batch.
-
-**When to use:**
-- Large batch imports where some records may fail
-- Network errors affecting subset of records
-- Validation failures on specific records
-- Want to minimize reprocessing overhead
-
-```typescript
-import {
-  PartialBatchRecovery,
-  createClient,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-
-// Standalone
-const logger = toStructuredLogger(createConsoleLogger(), { service: 'BatchRecovery' });
-
-// Versori Platform
-const { log: logger } = ctx;
-
-const recovery = new PartialBatchRecovery(logger);
-const client = await createClient({ config });
-
-// Process batch with automatic retry of failed records
-const result = await recovery.processBatchWithRecovery(
-  inventoryRecords,
-  async (batch) => {
-    // Your batch processing logic
-    const job = await client.createJob({
-      name: 'Inventory Sync with Recovery',
-      retailerId: 'my-retailer'
-    });
-
-    const batchResult = await client.sendBatch(job.id, {
-      action: 'UPSERT',
-      entityType: 'INVENTORY',
-      entities: batch
-    });
-
-    return batchResult;
-  },
-  {
-    maxRetries: 3,           // Retry each failed record up to 3 times
-    retryOnlyFailed: true,   // Only retry failed records (default: true)
-    batchSize: 100,          // Process in batches of 100
-    retryDelay: 1000,        // Wait 1 second between retries
-    exponentialBackoff: true // Use exponential backoff (default: true)
-  }
-);
-
-// Results
-console.log(`Total records: ${inventoryRecords.length}`);
-console.log(`Successful: ${result.successCount}`);
-console.log(`Failed: ${result.failedCount}`);
-
-if (result.failedCount > 0) {
-  console.log('Failed records:');
-  result.failedRecords.forEach(({ record, error, attemptCount }) => {
-    console.error(`Record ${record.ref} failed after ${attemptCount} attempts:`, error.message);
-  });
-}
-```
-
-### Recovery Result Structure
-
-```typescript
-interface RecoveryResult {
-  successCount: number;      // Total successful records
-  failedCount: number;       // Total failed after all retries
-  failedRecords: Array<{
-    record: any;             // The record that failed
-    error: Error;            // Last error that occurred
-    attemptCount: number;    // Total retry attempts for this record
-  }>;
-}
-```
-
-### Advanced Recovery Options
-
-```typescript
-const result = await recovery.processBatchWithRecovery(
-  records,
-  processFn,
-  {
-    maxRetries: 5,
-    retryOnlyFailed: true,
-    batchSize: 50,
-    retryDelay: 2000,
-    exponentialBackoff: true,
-
-    // Custom retry decision logic
-    shouldRetry: (error, attemptCount) => {
-      // Don't retry validation errors
-      if (error instanceof ValidationError) {
-        return false;
-      }
-      // Retry network/timeout errors
-      if (error instanceof IngestionError) {
-        return error.isRetryable() && attemptCount < 5;
-      }
-      return attemptCount < 3;
-    },
-
-    // Custom backoff calculation
-    calculateDelay: (attempt, baseDelay) => {
-      return Math.min(baseDelay * Math.pow(2, attempt), 30000);
-    },
-
-    // Progress callback
-    onProgress: (current, total) => {
-      logger.info(`Processing: ${current}/${total} records`);
-    },
-
-    // Retry callback
-    onRetry: (record, attempt, error) => {
-      logger.warn(`Retrying record ${record.ref}, attempt ${attempt}`, {
-        error: error.message
-      });
-    }
-  }
-);
-```
-
-### Combining with Built-In Retry
-
-PartialBatchRecovery works well with SDK's built-in retry logic:
-
-```typescript
-// S3 operations have automatic retry
-const files = await s3.listFiles();  // Automatic retry on network errors
-
-// Parse files
-const records = [];
-for (const file of files) {
-  const content = await s3.downloadFile(file.path);  // Automatic retry
-  const parsed = await parser.parse(content);
-  records.push(...parsed);
-}
-
-// Use PartialBatchRecovery for batch submission
-const result = await recovery.processBatchWithRecovery(
-  records,
-  async (batch) => {
-    const job = await client.createJob({ name: 'Sync', retailerId });
-    return await client.sendBatch(job.id, {
-      action: 'UPSERT',
-      entityType: 'INVENTORY',
-      entities: batch
-    });
-  },
-  {
-    maxRetries: 3,
-    retryOnlyFailed: true
-  }
-);
-```
-
-### Benefits of PartialBatchRecovery
-
-**Efficiency:**
-- ✅ Only retries failed records (not entire batch)
-- ✅ Reduces API calls and processing time
-- ✅ Lower resource consumption
-
-**Reliability:**
-- ✅ Tracks each record individually
-- ✅ Configurable retry logic per record
-- ✅ Detailed failure reporting
-
-**Observability:**
-- ✅ Know exactly which records failed
-- ✅ Understand failure patterns
-- ✅ Better error reporting for monitoring
-
-**Example: 1000 records, 10 fail**
-- **Without PartialBatchRecovery:** Retry all 1000 records
-- **With PartialBatchRecovery:** Retry only 10 failed records
-
-## Production-Ready Retry Function
-
-Complete retry implementation with all best practices:
-
-```typescript
-import {
-  IngestionError,
-  IngestionErrorCode,
-  FileParsingError,
-  PathResolutionError,
-  MappingError,
-  FluentAPIError,
-  GraphQLExecutionError
-} from '@fluentcommerce/fc-connect-sdk';
-
-interface RetryOptions {
-  maxRetries?: number;
-  initialDelay?: number;
-  maxDelay?: number;
-  useJitter?: boolean;
-  onRetry?: (attempt: number, error: Error) => void;
-}
-
-async function retryOperation<T>(
-  operation: () => Promise<T>,
-  options: RetryOptions = {}
-): Promise<T> {
-  const {
-    maxRetries = 3,
-    initialDelay = 1000,
-    maxDelay = 60000,
-    useJitter = true,
-    onRetry
-  } = options;
-
-  let lastError: Error;
-
-  for (let attempt = 0; attempt < maxRetries; attempt++) {
-    try {
-      return await operation();
-
-    } catch (error) {
-      lastError = error;
-
-      // 1. Don't retry permanent errors
-      if (error instanceof FileParsingError ||
-          error instanceof PathResolutionError ||
-          error instanceof MappingError) {
-        throw error;
-      }
-
-      // 2. Check FluentAPIError (HTTP 5xx / 429)
-      let isRetryable = false;
-      let retryDelay = 0;
-
-      if (error instanceof FluentAPIError) {
-        if (error.statusCode >= 500 || error.statusCode === 429) {
-          isRetryable = true;
-        } else {
-          throw error; // 4xx are permanent
-        }
-      }
-      
-      // 3. Check IngestionError
-      else if (error instanceof IngestionError) {
-        if (error.isRetryable()) {
-          isRetryable = true;
-           // Special handling for rate limit error context
-          if (error.code === IngestionErrorCode.RATE_LIMIT_ERROR) {
-             retryDelay = (error.context?.retryAfter || 60) * 1000;
-          }
-        } else {
-          throw error;
-        }
-      }
-      
-      // 4. Check GraphQLExecutionError
-      else if (error instanceof GraphQLExecutionError) {
-        const firstError = error.graphqlErrors[0];
-        if (firstError?.extensions?.code === 'TIMEOUT' || 
-            firstError?.extensions?.code === 'NETWORK_ERROR') {
-           isRetryable = true;
-        } else {
-           throw error; // Validation errors are permanent
-        }
-      }
-
-      // Last attempt - throw
-      if (attempt >= maxRetries - 1) {
-        throw error;
-      }
-
-      // Calculate delay if not already set
-      if (!retryDelay) {
-          let delay = initialDelay * Math.pow(2, attempt);
-          if (useJitter) {
-            delay = Math.random() * delay;
-          }
-          retryDelay = Math.min(delay, maxDelay);
-      }
-
-      // Callback
-      if (onRetry) {
-        onRetry(attempt + 1, error);
-      }
-
-      await new Promise(resolve => setTimeout(resolve, retryDelay));
-    }
-  }
-
-  throw lastError!;
-}
-
-// Usage
-const result = await retryOperation(
-  () => processOrder(xmlContent),
-  {
-    maxRetries: 5,
-    initialDelay: 2000,
-    maxDelay: 60000,
-    useJitter: true,
-    onRetry: (attempt, error) => {
-      logger.warn('Retrying operation', { attempt, error: error.message });
-    }
-  }
-);
-```
-
-## Key Takeaways
-
-- 🎯 **SDK built-in retry** - S3DataSource and SftpDataSource have automatic retry
-- 🎯 **Use isRetryable()** - Let SDK determine if error is transient
-- 🎯 **Exponential backoff** - Increases delay between retries (with jitter)
-- 🎯 **Connection pooling** - SftpDataSource reuses connections efficiently
-- 🎯 **Add jitter** - Prevents thundering herd problem (full jitter recommended)
-- 🎯 **Respect rate limits** - Use `retryAfter` when provided
-- 🎯 **Set max retries** - Prevent infinite retry loops (default: 3)
-- 🎯 **Don't retry permanent errors** - Parse, mapping, validation errors won't succeed
-
-## Practice Exercise
-
-Implement a retry function that:
-1. Retries only transient errors
-2. Uses exponential backoff with jitter
-3. Respects rate limit headers
-4. Has a maximum of 5 retries
-
-<details>
-<summary>Click to see solution</summary>
-
-See the "Production-Ready Retry Function" section above for a complete implementation.
-
-</details>
-
-## Next Steps
-
-Continue to [Module 7: Monitoring →](./error-handling-07-monitoring.md) to learn how to effectively log and monitor errors.
-
----
-
-**Previous:** [← Module 5: Calling Patterns](./error-handling-05-calling-patterns.md)
-**Next:** [Module 7: Monitoring →](./error-handling-07-monitoring.md)
+# Module 6: Retry Strategies
+
+**Level:** Advanced
+**Estimated Time:** 25 minutes
+
+## Overview
+
+This module teaches you how to implement effective retry strategies, including how to distinguish retryable from non-retryable errors and how to use exponential backoff to avoid overwhelming systems.
+
+## Learning Objectives
+
+By the end of this module, you will:
+- ✅ Know which errors are retryable and which are permanent
+- ✅ Implement exponential backoff with jitter
+- ✅ Use the SDK's `isRetryable()` method effectively
+- ✅ Handle rate limiting and timeouts correctly
+- ✅ Build production-ready retry logic
+
+## Retryable vs Non-Retryable Errors
+
+### Non-Retryable Errors (Permanent Failures)
+
+These errors indicate problems with data or configuration that won't be fixed by retrying:
+
+```typescript
+const NON_RETRYABLE_CODES = [
+  IngestionErrorCode.PARSE_ERROR,              // Invalid XML/JSON/CSV
+  IngestionErrorCode.VALIDATION_ERROR,         // Data validation failed
+  IngestionErrorCode.REQUIRED_FIELD_MISSING,   // Missing required field
+  IngestionErrorCode.CONFIGURATION_ERROR,      // Invalid config
+  IngestionErrorCode.FIELD_MAPPING_ERROR,      // Mapping config wrong
+  IngestionErrorCode.INVALID_FILE_FORMAT,      // Unsupported file type
+];
+```
+
+**Why not retryable:**
+- Source data is malformed
+- Configuration is incorrect
+- Business logic validation failed
+- Schema mismatch
+
+**What to do:**
+1. Log the error with full context
+2. Alert developers/operators
+3. Fix the underlying issue (data or config)
+4. Re-submit after fix
+
+**Example:**
+
+```typescript
+if (error instanceof FileParsingError) {
+  logger.error('PERMANENT ERROR - Fix source file:', {
+    fileName: error.fileName,
+    lineNumber: error.lineNumber,
+    message: error.message
+  });
+
+  // Don't retry - save to dead letter queue for manual review
+  await saveToDeadLetterQueue(fileName, content, error);
+
+  throw error;  // Fail immediately
+}
+```
+
+### Retryable Errors (Transient Failures)
+
+These errors are likely temporary and may succeed if retried:
+
+```typescript
+const RETRYABLE_CODES = [
+  IngestionErrorCode.NETWORK_ERROR,            // Network connectivity issue
+  IngestionErrorCode.TIMEOUT_ERROR,            // Request timed out
+  IngestionErrorCode.RATE_LIMIT_ERROR,         // API rate limit hit
+  IngestionErrorCode.LOCK_ACQUISITION_FAILED,  // Distributed lock conflict
+];
+
+// Also retryable:
+// - FluentAPIError with statusCode >= 500 (Server Error)
+// - FluentAPIError with statusCode === 429 (Rate Limit)
+// - GraphQLExecutionError with code 'TIMEOUT' or 'NETWORK_ERROR'
+```
+
+**Why retryable:**
+- Network issues are often temporary
+- Timeouts may be due to temporary load
+- Rate limits reset after time period
+- Lock conflicts resolve when other process completes
+
+**What to do:**
+1. Check `error.isRetryable()` returns `true`
+2. Implement exponential backoff
+3. Respect rate limit headers (`retryAfter`)
+4. Set maximum retry limit
+5. Log retry attempts
+
+**Example:**
+
+```typescript
+if (error instanceof IngestionError && error.isRetryable()) {
+  const delay = Math.pow(2, attempt) * 1000;  // Exponential backoff
+  logger.info(`Retryable error - will retry in ${delay}ms`, {
+    code: error.code,
+    attempt,
+    maxRetries
+  });
+
+  await new Promise(resolve => setTimeout(resolve, delay));
+  return retry();
+}
+```
+
+## Using isRetryable()
+
+The SDK's `IngestionError` class provides an `isRetryable()` method:
+
+```typescript
+abstract class IngestionError extends Error {
+  isRetryable(): boolean {
+    const retryableCodes = [
+      IngestionErrorCode.NETWORK_ERROR,
+      IngestionErrorCode.TIMEOUT_ERROR,
+      IngestionErrorCode.RATE_LIMIT_ERROR,
+      IngestionErrorCode.LOCK_ACQUISITION_FAILED,
+    ];
+
+    return retryableCodes.includes(this.code);
+  }
+}
+```
+
+### Basic Usage
+
+```typescript
+import { IngestionError } from '@fluentcommerce/fc-connect-sdk';
+
+try {
+  await processFile(fileName);
+} catch (error) {
+  if (error instanceof IngestionError) {
+    if (error.isRetryable()) {
+      // Retry logic
+      return scheduleRetry();
+    } else {
+      // Permanent failure
+      logger.error('Permanent error - manual intervention required');
+      throw error;
+    }
+  }
+  throw error;
+}
+```
+
+### Handling Non-SDK Errors
+
+Not all errors extend `IngestionError`:
+
+```typescript
+try {
+  await operation();
+} catch (error) {
+  // Check SDK errors first
+  if (error instanceof IngestionError) {
+    if (!error.isRetryable()) {
+      throw error;  // Don't retry
+    }
+  }
+
+  // File parsing errors are never retryable
+  if (error instanceof FileParsingError) {
+    throw error;  // Don't retry
+  }
+
+  // Path resolution errors are config issues
+  if (error instanceof PathResolutionError) {
+    throw error;  // Don't retry
+  }
+
+  // Mapping errors are data/config issues
+  if (error instanceof MappingError) {
+    throw error;  // Don't retry
+  }
+
+  // Unknown errors - retry with caution
+  if (attempt < maxRetries) {
+    logger.warn('Unknown error - attempting retry', { error });
+    await delay(1000 * attempt);
+    return retry();
+  }
+
+  throw error;
+}
+```
+
+## Exponential Backoff
+
+### Why Exponential Backoff?
+
+**Linear backoff** (same delay each time):
+```
+Attempt 1: Wait 1s
+Attempt 2: Wait 1s
+Attempt 3: Wait 1s
+```
+❌ Doesn't give system time to recover from overload
+
+**Exponential backoff** (increasing delay):
+```
+Attempt 1: Wait 1s
+Attempt 2: Wait 2s
+Attempt 3: Wait 4s
+Attempt 4: Wait 8s
+```
+✅ Reduces load on overloaded systems
+✅ Increases chance of success on each retry
+
+### Basic Implementation
+
+```typescript
+async function retryWithBackoff<T>(
+  operation: () => Promise<T>,
+  maxRetries: number = 3,
+  initialDelay: number = 1000
+): Promise<T> {
+  let lastError: Error;
+
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      return await operation();
+    } catch (error) {
+      lastError = error;
+
+      // Check if error is retryable
+      if (error instanceof IngestionError) {
+        if (!error.isRetryable()) {
+          throw error;  // Don't retry permanent errors
+        }
+      } else if (error instanceof MappingError ||
+                 error instanceof PathResolutionError ||
+                 error instanceof FileParsingError) {
+        throw error;  // These are permanent errors
+      }
+
+      // Last attempt - throw error
+      if (attempt === maxRetries - 1) {
+        throw lastError;
+      }
+
+      // Calculate delay with exponential backoff
+      const delay = initialDelay * Math.pow(2, attempt);
+
+      console.log(`Retry ${attempt + 1}/${maxRetries} after ${delay}ms`);
+      await new Promise(resolve => setTimeout(resolve, delay));
+    }
+  }
+
+  throw lastError!;
+}
+
+// Usage
+const result = await retryWithBackoff(
+  () => processOrder(xmlContent),
+  3,    // max retries
+  2000  // initial delay (2 seconds)
+);
+```
+
+### With Jitter
+
+Adding random jitter prevents thundering herd problem:
+
+```typescript
+async function retryWithBackoffAndJitter<T>(
+  operation: () => Promise<T>,
+  maxRetries: number = 3,
+  initialDelay: number = 1000,
+  maxDelay: number = 60000
+): Promise<T> {
+  let lastError: Error;
+
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      return await operation();
+    } catch (error) {
+      lastError = error;
+
+      // Check if retryable
+      if (error instanceof IngestionError && !error.isRetryable()) {
+        throw error;
+      }
+      if (error instanceof MappingError ||
+          error instanceof PathResolutionError ||
+          error instanceof FileParsingError) {
+        throw error;
+      }
+
+      if (attempt === maxRetries - 1) {
+        throw lastError;
+      }
+
+      // Exponential backoff with jitter
+      const exponentialDelay = initialDelay * Math.pow(2, attempt);
+      const jitter = Math.random() * 100;  // 0-100ms random jitter
+      const delay = Math.min(exponentialDelay + jitter, maxDelay);
+
+      console.log(`Retry ${attempt + 1}/${maxRetries} after ${delay}ms`);
+      await new Promise(resolve => setTimeout(resolve, delay));
+    }
+  }
+
+  throw lastError!;
+}
+```
+
+**Benefits of jitter:**
+- ✅ Prevents multiple clients retrying at exact same time
+- ✅ Spreads load more evenly
+- ✅ Reduces likelihood of cascading failures
+
+### Full-Jitter Strategy
+
+AWS recommended approach - more aggressive randomization:
+
+```typescript
+function calculateBackoffWithFullJitter(
+  attempt: number,
+  baseDelay: number = 1000,
+  maxDelay: number = 60000
+): number {
+  const exponentialDelay = baseDelay * Math.pow(2, attempt);
+  const cappedDelay = Math.min(exponentialDelay, maxDelay);
+
+  // Full jitter: random value between 0 and cappedDelay
+  return Math.random() * cappedDelay;
+}
+
+// Usage
+const delay = calculateBackoffWithFullJitter(attempt);
+await new Promise(resolve => setTimeout(resolve, delay));
+```
+
+## Rate Limiting
+
+### Handling Rate Limit Errors
+
+Respect the `retryAfter` value from rate limit errors:
+
+```typescript
+import { IngestionError, IngestionErrorCode } from '@fluentcommerce/fc-connect-sdk';
+
+async function handleRateLimit(
+  operation: () => Promise<any>,
+  maxRetries: number = 3
+): Promise<any> {
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      return await operation();
+    } catch (error) {
+      if (error instanceof IngestionError &&
+          error.code === IngestionErrorCode.RATE_LIMIT_ERROR) {
+
+        // Get retry-after value from error context
+        const retryAfter = error.context?.retryAfter || 60;
+
+        if (attempt < maxRetries - 1) {
+          logger.warn(`Rate limited - waiting ${retryAfter} seconds`, {
+            attempt,
+            maxRetries
+          });
+
+          // Wait for rate limit to reset
+          await new Promise(resolve => setTimeout(resolve, retryAfter * 1000));
+          continue;
+        }
+      }
+
+      // Non-rate-limit error or max retries exceeded
+      throw error;
+    }
+  }
+
+  throw new Error('Max retries exceeded');
+}
+```
+
+### Rate Limit with Exponential Backoff Fallback
+
+If no `retryAfter` provided, use exponential backoff:
+
+```typescript
+async function handleRateLimitSmart(
+  operation: () => Promise<any>,
+  maxRetries: number = 5
+): Promise<any> {
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      return await operation();
+    } catch (error) {
+      if (error instanceof IngestionError &&
+          error.code === IngestionErrorCode.RATE_LIMIT_ERROR) {
+
+        if (attempt >= maxRetries - 1) {
+          throw error;  // Max retries exceeded
+        }
+
+        // Use retryAfter if provided, else exponential backoff
+        const retryAfter = error.context?.retryAfter;
+        let delay: number;
+
+        if (retryAfter) {
+          delay = retryAfter * 1000;  // Convert to ms
+          logger.info(`Rate limited - retry after ${retryAfter}s`);
+        } else {
+          delay = Math.pow(2, attempt) * 1000;  // Exponential backoff
+          logger.info(`Rate limited - retry after ${delay}ms (no retryAfter)`);
+        }
+
+        await new Promise(resolve => setTimeout(resolve, delay));
+        continue;
+      }
+
+      // Not a rate limit error
+      throw error;
+    }
+  }
+
+  throw new Error('Max retries exceeded');
+}
+```
+
+## Timeout Handling
+
+### Network Timeouts
+
+Handle network timeouts with retry:
+
+```typescript
+async function executeWithTimeout<T>(
+  operation: () => Promise<T>,
+  timeoutMs: number = 30000,
+  maxRetries: number = 3
+): Promise<T> {
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      // Race between operation and timeout
+      const result = await Promise.race([
+        operation(),
+        new Promise<never>((_, reject) =>
+          setTimeout(() => reject(new Error('Operation timeout')), timeoutMs)
+        )
+      ]);
+
+      return result;
+
+    } catch (error) {
+      if (error.message === 'Operation timeout') {
+        if (attempt < maxRetries - 1) {
+          const delay = Math.pow(2, attempt) * 1000;
+          logger.warn(`Timeout - retrying in ${delay}ms`, { attempt });
+          await new Promise(resolve => setTimeout(resolve, delay));
+          continue;
+        }
+      }
+
+      // Not a timeout or max retries exceeded
+      throw error;
+    }
+  }
+
+  throw new Error('Max retries exceeded');
+}
+```
+
+## SDK Built-In Retry Logic
+
+### S3DataSource Retry
+
+The SDK's S3DataSource uses the shared `RetryHelper` utility for consistent retry behavior:
+
+```typescript
+import { S3DataSource } from '@fluentcommerce/fc-connect-sdk';
+
+const s3Config = {
+  type: 'S3_CSV' as const,
+  s3Config: {
+    bucket: 'my-bucket',
+    region: 'us-east-1',
+    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+    maxAttempts: 3  // Default: 3 retries
+  }
+};
+
+const s3 = new S3DataSource(s3Config, logger);
+
+// Automatic retry on:
+// - HTTP 429 (rate limit)
+// - HTTP 500, 502, 503, 504 (server errors)
+// - Network errors (ECONNRESET, ETIMEDOUT, etc.)
+const files = await s3.listFiles();  // Retries automatically
+const content = await s3.downloadFile(key);  // Retries automatically
+```
+
+**Retry Configuration:**
+- **Max attempts**: Configurable via `maxAttempts` (default: 3)
+- **Base delay**: 1000ms
+- **Backoff factor**: 2 (exponential)
+- **Max delay**: 8000ms (8 seconds)
+- **Jitter**: Full jitter (AWS recommended)
+- **Retryable status codes**: 429, 500, 502, 503, 504
+- **Retryable network errors**: Connection resets, timeouts, unreachable hosts
+
+**Implementation Note:** S3DataSource uses `RetryHelper.executeWithRetry()` from `src/utils/retry-helper.ts` for consistency across all SDK HTTP operations.
+
+**SFTP Note:** SftpDataSource uses custom retry logic due to connection pooling requirements. See lines 536-645 for SFTP-specific retry patterns.
+
+### SftpDataSource Retry
+
+The SDK's SftpDataSource includes configurable retry logic with connection pooling:
+
+```typescript
+import { SftpDataSource, SftpDataSourceConfig } from '@fluentcommerce/fc-connect-sdk';
+
+const sftpConfig: SftpDataSourceConfig = {
+  type: 'SFTP_CSV',
+  settings: {
+    host: 'sftp.example.com',
+    port: 22,
+    username: 'user',
+    password: 'pass',
+    remotePath: '/data',
+    filePattern: '*.csv',
+
+    // Enhanced retry configuration
+    retry: {
+      maxAttempts: 3,           // Default: 3
+      baseDelayMs: 1000,        // Default: 1000ms (1 second)
+      backoffFactor: 2,         // Default: 2 (exponential)
+      maxDelayMs: 8000,         // Default: 8000ms (8 seconds)
+      jitter: 'full',           // Default: 'full' jitter
+      reconnectOnFailure: true, // Default: true - reconnect on error
+      isRetryable: (error) => {
+        // Custom retry logic (optional)
+        return error.message.includes('timeout');
+      }
+    }
+  }
+};
+
+const sftp = new SftpDataSource(sftpConfig, logger);
+
+// Automatic retry with exponential backoff and connection pooling
+const files = await sftp.listFiles();  // Retries automatically!
+const content = await sftp.downloadFile('file.csv');  // Retries automatically!
+```
+
+**Retry behavior:**
+- **Default strategy**: Exponential backoff with full jitter
+- **Connection pooling**: Reuses connections, creates new ones when needed
+- **Wait queue**: Queues requests when pool is full (prevents connection spam)
+- **Reconnect on failure**: Automatically reconnects on transient errors
+
+**Retryable errors:**
+- Connection resets (`ECONNRESET`)
+- Timeouts (`ETIMEDOUT`)
+- Host unreachable (`EHOSTUNREACH`)
+- Connection refused (`ECONNREFUSED`)
+- Broken pipe (`EPIPE`)
+- Message patterns: "connection lost", "timeout", "network error"
+
+**Non-retryable errors:**
+- Authentication failures
+- Permission denied
+- File not found
+- Algorithm/cipher errors
+- Key exchange failures
+
+**Implementation details:**
+```typescript
+private async executeWithRetry<T>(
+  operation: () => Promise<T>,
+  operationName: string,
+  retryOverride?: Partial<SftpRetryConfig>
+): Promise<T> {
+  const config = { ...this.retryConfig, ...retryOverride };
+  let lastError: Error;
+
+  for (let attempt = 1; attempt <= config.maxAttempts; attempt++) {
+    try {
+      return await operation();
+    } catch (error) {
+      lastError = error as Error;
+      const isRetryable = config.isRetryable(lastError);
+      const isLastAttempt = attempt === config.maxAttempts;
+
+      if (!isRetryable || isLastAttempt) {
+        throw error;
+      }
+
+      // Reconnect on failure if enabled
+      if (config.reconnectOnFailure) {
+        await this.closeConnection(connectionId);
+      }
+
+      // Calculate delay with jitter
+      const delay = this.calculateDelay(attempt);
+      await new Promise(resolve => setTimeout(resolve, delay));
+    }
+  }
+
+  throw lastError!;
+}
+
+private calculateDelay(attempt: number): number {
+  const exponentialDelay =
+    this.retryConfig.baseDelayMs * Math.pow(this.retryConfig.backoffFactor, attempt - 1);
+  const cappedDelay = Math.min(exponentialDelay, this.retryConfig.maxDelayMs);
+
+  if (this.retryConfig.jitter === 'none') {
+    return cappedDelay;
+  }
+
+  // Full jitter: random between 0 and cappedDelay
+  return Math.floor(Math.random() * cappedDelay);
+}
+```
+
+### Connection Pooling (SFTP only)
+
+```typescript
+// Connection pool configuration
+const sftpConfig: SftpDataSourceConfig = {
+  type: 'SFTP_CSV',
+  settings: {
+    // ... other settings
+    maxConnections: 5  // Default: 5 concurrent connections
+  }
+};
+
+// Connection pooling features:
+// - Reuses idle connections
+// - Creates new connections up to maxConnections
+// - Queues requests when pool is full
+// - Automatically closes idle connections after timeout
+// - Handles connection failures gracefully
+```
+
+### When to Use Built-In Retry vs Custom Retry
+
+**Use SDK built-in retry when:**
+- ✅ Working with S3DataSource or SftpDataSource
+- ✅ Default retry behavior is acceptable
+- ✅ You want zero configuration
+- ✅ You need connection pooling (SFTP)
+
+**Use custom retry when:**
+- ✅ Working with FluentClient GraphQL operations
+- ✅ Need custom retry logic beyond data source operations
+- ✅ Need to coordinate retries across multiple services
+- ✅ Want different retry strategies for different operations
+
+**Example: Combining both**
+
+```typescript
+// S3 operations use built-in retry
+const files = await s3.listFiles();  // Automatic retry
+
+// Custom retry for high-level workflow
+await retryOperation(
+  async () => {
+    // Download (automatic retry)
+    const content = await s3.downloadFile(files[0].path);
+
+    // Parse
+    const parsed = await parser.parse(content);
+
+    // Map
+    const payload = await mapper.map(parsed);
+
+    // Submit to Fluent (custom retry)
+    return await client.graphql(payload);
+  },
+  {
+    maxRetries: 5,
+    initialDelay: 2000
+  }
+);
+```
+
+## Partial Batch Recovery
+
+### PartialBatchRecovery Service
+
+The SDK's `PartialBatchRecovery` service handles partial batch failures gracefully by retrying only failed records instead of the entire batch.
+
+**When to use:**
+- Large batch imports where some records may fail
+- Network errors affecting subset of records
+- Validation failures on specific records
+- Want to minimize reprocessing overhead
+
+```typescript
+import {
+  PartialBatchRecovery,
+  createClient,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+
+// Standalone
+const logger = toStructuredLogger(createConsoleLogger(), { service: 'BatchRecovery' });
+
+// Versori Platform
+const { log: logger } = ctx;
+
+const recovery = new PartialBatchRecovery(logger);
+const client = await createClient({ config });
+
+// Process batch with automatic retry of failed records
+const result = await recovery.processBatchWithRecovery(
+  inventoryRecords,
+  async (batch) => {
+    // Your batch processing logic
+    const job = await client.createJob({
+      name: 'Inventory Sync with Recovery',
+      retailerId: 'my-retailer'
+    });
+
+    const batchResult = await client.sendBatch(job.id, {
+      action: 'UPSERT',
+      entityType: 'INVENTORY',
+      entities: batch
+    });
+
+    return batchResult;
+  },
+  {
+    maxRetries: 3,           // Retry each failed record up to 3 times
+    retryOnlyFailed: true,   // Only retry failed records (default: true)
+    batchSize: 100,          // Process in batches of 100
+    retryDelay: 1000,        // Wait 1 second between retries
+    exponentialBackoff: true // Use exponential backoff (default: true)
+  }
+);
+
+// Results
+console.log(`Total records: ${inventoryRecords.length}`);
+console.log(`Successful: ${result.successCount}`);
+console.log(`Failed: ${result.failedCount}`);
+
+if (result.failedCount > 0) {
+  console.log('Failed records:');
+  result.failedRecords.forEach(({ record, error, attemptCount }) => {
+    console.error(`Record ${record.ref} failed after ${attemptCount} attempts:`, error.message);
+  });
+}
+```
+
+### Recovery Result Structure
+
+```typescript
+interface RecoveryResult {
+  successCount: number;      // Total successful records
+  failedCount: number;       // Total failed after all retries
+  failedRecords: Array<{
+    record: any;             // The record that failed
+    error: Error;            // Last error that occurred
+    attemptCount: number;    // Total retry attempts for this record
+  }>;
+}
+```
+
+### Advanced Recovery Options
+
+```typescript
+const result = await recovery.processBatchWithRecovery(
+  records,
+  processFn,
+  {
+    maxRetries: 5,
+    retryOnlyFailed: true,
+    batchSize: 50,
+    retryDelay: 2000,
+    exponentialBackoff: true,
+
+    // Custom retry decision logic
+    shouldRetry: (error, attemptCount) => {
+      // Don't retry validation errors
+      if (error instanceof ValidationError) {
+        return false;
+      }
+      // Retry network/timeout errors
+      if (error instanceof IngestionError) {
+        return error.isRetryable() && attemptCount < 5;
+      }
+      return attemptCount < 3;
+    },
+
+    // Custom backoff calculation
+    calculateDelay: (attempt, baseDelay) => {
+      return Math.min(baseDelay * Math.pow(2, attempt), 30000);
+    },
+
+    // Progress callback
+    onProgress: (current, total) => {
+      logger.info(`Processing: ${current}/${total} records`);
+    },
+
+    // Retry callback
+    onRetry: (record, attempt, error) => {
+      logger.warn(`Retrying record ${record.ref}, attempt ${attempt}`, {
+        error: error.message
+      });
+    }
+  }
+);
+```
+
+### Combining with Built-In Retry
+
+PartialBatchRecovery works well with SDK's built-in retry logic:
+
+```typescript
+// S3 operations have automatic retry
+const files = await s3.listFiles();  // Automatic retry on network errors
+
+// Parse files
+const records = [];
+for (const file of files) {
+  const content = await s3.downloadFile(file.path);  // Automatic retry
+  const parsed = await parser.parse(content);
+  records.push(...parsed);
+}
+
+// Use PartialBatchRecovery for batch submission
+const result = await recovery.processBatchWithRecovery(
+  records,
+  async (batch) => {
+    const job = await client.createJob({ name: 'Sync', retailerId });
+    return await client.sendBatch(job.id, {
+      action: 'UPSERT',
+      entityType: 'INVENTORY',
+      entities: batch
+    });
+  },
+  {
+    maxRetries: 3,
+    retryOnlyFailed: true
+  }
+);
+```
+
+### Benefits of PartialBatchRecovery
+
+**Efficiency:**
+- ✅ Only retries failed records (not entire batch)
+- ✅ Reduces API calls and processing time
+- ✅ Lower resource consumption
+
+**Reliability:**
+- ✅ Tracks each record individually
+- ✅ Configurable retry logic per record
+- ✅ Detailed failure reporting
+
+**Observability:**
+- ✅ Know exactly which records failed
+- ✅ Understand failure patterns
+- ✅ Better error reporting for monitoring
+
+**Example: 1000 records, 10 fail**
+- **Without PartialBatchRecovery:** Retry all 1000 records
+- **With PartialBatchRecovery:** Retry only 10 failed records
+
+## Production-Ready Retry Function
+
+Complete retry implementation with all best practices:
+
+```typescript
+import {
+  IngestionError,
+  IngestionErrorCode,
+  FileParsingError,
+  PathResolutionError,
+  MappingError,
+  FluentAPIError,
+  GraphQLExecutionError
+} from '@fluentcommerce/fc-connect-sdk';
+
+interface RetryOptions {
+  maxRetries?: number;
+  initialDelay?: number;
+  maxDelay?: number;
+  useJitter?: boolean;
+  onRetry?: (attempt: number, error: Error) => void;
+}
+
+async function retryOperation<T>(
+  operation: () => Promise<T>,
+  options: RetryOptions = {}
+): Promise<T> {
+  const {
+    maxRetries = 3,
+    initialDelay = 1000,
+    maxDelay = 60000,
+    useJitter = true,
+    onRetry
+  } = options;
+
+  let lastError: Error;
+
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      return await operation();
+
+    } catch (error) {
+      lastError = error;
+
+      // 1. Don't retry permanent errors
+      if (error instanceof FileParsingError ||
+          error instanceof PathResolutionError ||
+          error instanceof MappingError) {
+        throw error;
+      }
+
+      // 2. Check FluentAPIError (HTTP 5xx / 429)
+      let isRetryable = false;
+      let retryDelay = 0;
+
+      if (error instanceof FluentAPIError) {
+        if (error.statusCode >= 500 || error.statusCode === 429) {
+          isRetryable = true;
+        } else {
+          throw error; // 4xx are permanent
+        }
+      }
+      
+      // 3. Check IngestionError
+      else if (error instanceof IngestionError) {
+        if (error.isRetryable()) {
+          isRetryable = true;
+           // Special handling for rate limit error context
+          if (error.code === IngestionErrorCode.RATE_LIMIT_ERROR) {
+             retryDelay = (error.context?.retryAfter || 60) * 1000;
+          }
+        } else {
+          throw error;
+        }
+      }
+      
+      // 4. Check GraphQLExecutionError
+      else if (error instanceof GraphQLExecutionError) {
+        const firstError = error.graphqlErrors[0];
+        if (firstError?.extensions?.code === 'TIMEOUT' || 
+            firstError?.extensions?.code === 'NETWORK_ERROR') {
+           isRetryable = true;
+        } else {
+           throw error; // Validation errors are permanent
+        }
+      }
+
+      // Last attempt - throw
+      if (attempt >= maxRetries - 1) {
+        throw error;
+      }
+
+      // Calculate delay if not already set
+      if (!retryDelay) {
+          let delay = initialDelay * Math.pow(2, attempt);
+          if (useJitter) {
+            delay = Math.random() * delay;
+          }
+          retryDelay = Math.min(delay, maxDelay);
+      }
+
+      // Callback
+      if (onRetry) {
+        onRetry(attempt + 1, error);
+      }
+
+      await new Promise(resolve => setTimeout(resolve, retryDelay));
+    }
+  }
+
+  throw lastError!;
+}
+
+// Usage
+const result = await retryOperation(
+  () => processOrder(xmlContent),
+  {
+    maxRetries: 5,
+    initialDelay: 2000,
+    maxDelay: 60000,
+    useJitter: true,
+    onRetry: (attempt, error) => {
+      logger.warn('Retrying operation', { attempt, error: error.message });
+    }
+  }
+);
+```
+
+## Key Takeaways
+
+- 🎯 **SDK built-in retry** - S3DataSource and SftpDataSource have automatic retry
+- 🎯 **Use isRetryable()** - Let SDK determine if error is transient
+- 🎯 **Exponential backoff** - Increases delay between retries (with jitter)
+- 🎯 **Connection pooling** - SftpDataSource reuses connections efficiently
+- 🎯 **Add jitter** - Prevents thundering herd problem (full jitter recommended)
+- 🎯 **Respect rate limits** - Use `retryAfter` when provided
+- 🎯 **Set max retries** - Prevent infinite retry loops (default: 3)
+- 🎯 **Don't retry permanent errors** - Parse, mapping, validation errors won't succeed
+
+## Practice Exercise
+
+Implement a retry function that:
+1. Retries only transient errors
+2. Uses exponential backoff with jitter
+3. Respects rate limit headers
+4. Has a maximum of 5 retries
+
+<details>
+<summary>Click to see solution</summary>
+
+See the "Production-Ready Retry Function" section above for a complete implementation.
+
+</details>
+
+## Next Steps
+
+Continue to [Module 7: Monitoring →](./error-handling-07-monitoring.md) to learn how to effectively log and monitor errors.
+
+---
+
+**Previous:** [← Module 5: Calling Patterns](./error-handling-05-calling-patterns.md)
+**Next:** [Module 7: Monitoring →](./error-handling-07-monitoring.md)