@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

package/docs/02-CORE-GUIDES/advanced-services/advanced-services-partial-batch-recovery.md

@@ -1,561 +1,561 @@
-# PartialBatchRecovery
-
-**Level:** Advanced
-**Category:** Resilience & Error Recovery
-**Purpose:** Recover from partial batch failures without losing all data
-
-## Overview
-
-`PartialBatchRecovery` enables per-record tracking in batch operations, allowing you to retry only failed records instead of re-processing the entire batch. This dramatically improves resilience and reduces data loss in production workflows.
-
-## Use Cases
-
-- ✅ Handle transient API failures gracefully
-- ✅ Implement checkpoint/resume functionality
-- ✅ Retry individual failed records
-- ✅ Track detailed error information per record
-- ✅ Reduce duplicate submissions
-- ✅ Minimize data loss on partial failures
-
----
-
-## Installation
-
-```typescript
-import { PartialBatchRecovery } from '@fluentcommerce/fc-connect-sdk';
-```
-
----
-
-## Quick Start
-
-```typescript
-import { PartialBatchRecovery, createClient } from '@fluentcommerce/fc-connect-sdk';
-
-const recovery = new PartialBatchRecovery(logger);
-const client = await createClient(ctx);
-
-// Process with automatic recovery
-const result = await recovery.processBatchWithRecovery(
-  inventoryRecords,
-  async (batch) => await client.sendBatch(jobId, {
-    action: 'UPSERT',
-    entityType: 'INVENTORY',
-    entities: batch
-  }),
-  {
-    maxRetries: 3,
-    retryOnlyFailed: true,
-    checkpointKey: 'inventory-2025-01-30'
-  }
-);
-
-console.log(`✅ Success: ${result.successCount}`);
-console.log(`❌ Failed: ${result.failedCount}`);
-
-if (result.failedRecords.length > 0) {
-  // Handle failed records
-  await sendToDeadLetterQueue(result.failedRecords);
-}
-```
-
----
-
-## API Reference
-
-### Constructor
-
-```typescript
-new PartialBatchRecovery<T>(logger?: StructuredLogger)
-```
-
-**Generic Type `T`:** The type of your records (e.g., `InventoryRecord`)
-
----
-
-### processBatchWithRecovery()
-
-Process a batch with automatic recovery and retry logic.
-
-```typescript
-await recovery.processBatchWithRecovery<T>(
-  records: T[],
-  batchProcessor: (records: T[]) => Promise<any>,
-  options?: BatchRecoveryOptions
-): Promise<BatchRecoveryResult<T>>
-```
-
-**Parameters:**
-
-- `records` - Array of records to process
-- `batchProcessor` - Async function that processes a batch
-- `options` - Recovery configuration
-
-**Returns:** `BatchRecoveryResult<T>` with detailed outcome
-
----
-
-## BatchRecoveryOptions
-
-```typescript
-interface BatchRecoveryOptions {
-  /** Maximum retry attempts per record (default: 3) */
-  maxRetries?: number;
-
-  /** Retry only failed records (true) or entire batch (false) (default: true) */
-  retryOnlyFailed?: boolean;
-
-  /** Checkpoint key for resume functionality */
-  checkpointKey?: string;
-
-  /** Delay between retries in ms (default: 1000) */
-  retryDelayMs?: number;
-
-  /** Custom retry predicate */
-  shouldRetry?: (error: Error, attemptCount: number) => boolean;
-
-  /** Batch size for retry operations (default: 100) */
-  retryBatchSize?: number;
-}
-```
-
----
-
-## BatchRecoveryResult
-
-```typescript
-interface BatchRecoveryResult<T> {
-  /** Total records processed */
-  totalRecords: number;
-
-  /** Successfully processed count */
-  successCount: number;
-
-  /** Failed records after all retries */
-  failedCount: number;
-
-  /** Details of failed records */
-  failedRecords: RecordFailure<T>[];
-
-  /** Checkpoint ID for resuming */
-  checkpointId?: string;
-
-  /** Human-readable summary */
-  summary: string;
-
-  /** Total processing time (ms) */
-  durationMs: number;
-}
-```
-
----
-
-## RecordFailure
-
-```typescript
-interface RecordFailure<T> {
-  /** Original record index */
-  index: number;
-
-  /** The record that failed */
-  record: T;
-
-  /** Error that occurred */
-  error: Error;
-
-  /** Number of retry attempts */
-  attemptCount: number;
-
-  /** Timestamp of failure */
-  timestamp: string;
-}
-```
-
----
-
-## Examples
-
-### Example 1: Basic Recovery
-
-```typescript
-const recovery = new PartialBatchRecovery(logger);
-
-const records = [
-  { ref: 'SKU-001', qty: 100, locationRef: 'DC-001' },
-  { ref: 'SKU-002', qty: 200, locationRef: 'DC-002' },
-  // ... 1000 records
-];
-
-const result = await recovery.processBatchWithRecovery(
-  records,
-  async (batch) => {
-    return await client.sendBatch(jobId, {
-      action: 'UPSERT',
-      entityType: 'INVENTORY',
-      entities: batch
-    });
-  },
-  {
-    maxRetries: 3,
-    retryOnlyFailed: true
-  }
-);
-
-logger.info('Batch processing complete', {
-  total: result.totalRecords,
-  success: result.successCount,
-  failed: result.failedCount,
-  duration: result.durationMs
-});
-```
-
----
-
-### Example 2: Custom Retry Logic
-
-```typescript
-const result = await recovery.processBatchWithRecovery(
-  records,
-  batchProcessor,
-  {
-    maxRetries: 5,
-    retryDelayMs: 2000,
-    shouldRetry: (error, attemptCount) => {
-      // Don't retry validation errors
-      if (error.message.includes('VALIDATION')) {
-        return false;
-      }
-
-      // Retry transient errors
-      if (error.message.includes('TIMEOUT') ||
-          error.message.includes('RATE_LIMIT')) {
-        return attemptCount < 5;
-      }
-
-      return false;
-    }
-  }
-);
-```
-
----
-
-### Example 3: Checkpoint/Resume
-
-```typescript
-const checkpointKey = `inventory-${new Date().toISOString().split('T')[0]}`;
-
-// First attempt
-const result = await recovery.processBatchWithRecovery(
-  records,
-  batchProcessor,
-  {
-    checkpointKey,
-    maxRetries: 3
-  }
-);
-
-// If there were failures, you can resume later
-if (result.failedRecords.length > 0) {
-  logger.warn('Some records failed, saved checkpoint', {
-    checkpointId: result.checkpointId,
-    failedCount: result.failedCount
-  });
-
-  // Later, retry just the failed records
-  const checkpoint = recovery.getCheckpoint(checkpointKey);
-  if (checkpoint) {
-    const retryResult = await recovery.processBatchWithRecovery(
-      checkpoint.failedRecords.map(f => f.record),
-      batchProcessor,
-      { maxRetries: 3 }
-    );
-  }
-}
-```
-
----
-
-### Example 4: Dead Letter Queue Pattern
-
-```typescript
-async function processWithDLQ(records: any[]) {
-  const recovery = new PartialBatchRecovery(logger);
-
-  const result = await recovery.processBatchWithRecovery(
-    records,
-    batchProcessor,
-    {
-      maxRetries: 3,
-      retryOnlyFailed: true
-    }
-  );
-
-  // Send failed records to DLQ for manual review
-  if (result.failedRecords.length > 0) {
-    await s3.uploadFile(
-      `dlq/failed-${Date.now()}.json`,
-      JSON.stringify({
-        timestamp: new Date().toISOString(),
-        totalFailed: result.failedCount,
-        records: result.failedRecords.map(f => ({
-          record: f.record,
-          error: f.error.message,
-          attempts: f.attemptCount,
-          timestamp: f.timestamp
-        }))
-      }, null, 2)
-    );
-
-    logger.error('Records sent to DLQ', {
-      count: result.failedCount,
-      dlqKey: `dlq/failed-${Date.now()}.json`
-    });
-
-    // Optionally send alert
-    await sendAlert({
-      type: 'BATCH_PARTIAL_FAILURE',
-      failedCount: result.failedCount,
-      totalCount: result.totalRecords
-    });
-  }
-
-  return result;
-}
-```
-
----
-
-## Patterns
-
-### Pattern 1: Retry Only Failed
-
-```typescript
-const result = await recovery.processBatchWithRecovery(
-  records,
-  batchProcessor,
-  {
-    retryOnlyFailed: true,
-    maxRetries: 3
-  }
-);
-```
-
-**Benefits:**
-- Faster recovery
-- Reduces duplicate submissions
-- Lower API costs
-
----
-
-### Pattern 2: Retry Entire Batch
-
-```typescript
-// Use when batch must be atomic
-const result = await recovery.processBatchWithRecovery(
-  records,
-  batchProcessor,
-  {
-    retryOnlyFailed: false,  // Retry entire batch
-    maxRetries: 2
-  }
-);
-```
-
-**Use When:**
-- Batch operations must be atomic
-- Records are interdependent
-- Fluent BPP requires full snapshot
-
----
-
-### Pattern 3: Exponential Backoff
-
-```typescript
-let retryDelay = 1000;
-
-for (let attempt = 1; attempt <= 3; attempt++) {
-  const result = await recovery.processBatchWithRecovery(
-    records,
-    batchProcessor,
-    {
-      retryDelayMs: retryDelay,
-      maxRetries: 1  // Manual retry control
-    }
-  );
-
-  if (result.failedCount === 0) break;
-
-  retryDelay *= 2;  // Double delay each attempt
-  records = result.failedRecords.map(f => f.record);
-}
-```
-
----
-
-### Pattern 4: Multi-Stage Recovery
-
-```typescript
-async function multiStageRecovery(records: any[]) {
-  // Stage 1: Fast retries (transient errors)
-  let result = await recovery.processBatchWithRecovery(
-    records,
-    batchProcessor,
-    {
-      maxRetries: 2,
-      retryDelayMs: 500,
-      retryOnlyFailed: true
-    }
-  );
-
-  if (result.failedCount === 0) return result;
-
-  // Stage 2: Slower retries with backoff
-  const stage2Result = await recovery.processBatchWithRecovery(
-    result.failedRecords.map(f => f.record),
-    batchProcessor,
-    {
-      maxRetries: 3,
-      retryDelayMs: 5000,
-      retryOnlyFailed: true
-    }
-  );
-
-  if (stage2Result.failedCount === 0) {
-    return {
-      ...stage2Result,
-      totalRecords: result.totalRecords,
-      successCount: result.totalRecords
-    };
-  }
-
-  // Stage 3: Send to DLQ
-  await sendToDLQ(stage2Result.failedRecords);
-
-  return stage2Result;
-}
-```
-
----
-
-## Patterns
-
-### 1. Always Check Failed Records
-
-```typescript
-const result = await recovery.processBatchWithRecovery(records, processor);
-
-if (result.failedCount > 0) {
-  logger.error('Batch had failures', {
-    failed: result.failedCount,
-    total: result.totalRecords
-  });
-
-  await handleFailures(result.failedRecords);
-}
-```
-
-### 2. Use Appropriate Batch Sizes
-
-```typescript
-const result = await recovery.processBatchWithRecovery(
-  records,
-  batchProcessor,
-  {
-    retryBatchSize: 100
-  }
-);
-```
-
-### 3. Implement Dead Letter Queue
-
-```typescript
-if (result.failedRecords.length > 0) {
-  await saveToDLQ(result.failedRecords);
-  await notifyTeam(result.failedCount);
-}
-```
-
-### 4. Set Realistic Retry Limits
-
-```typescript
-maxRetries: 3
-```
-
----
-
-## Performance Considerations
-
-**Overhead:** ~5-10% for tracking
-**Memory:** O(n) for failed records
-**Network:** Only retries failed records (with `retryOnlyFailed: true`)
-
-**Optimization Tips:**
-```typescript
-// 1. Use retryOnlyFailed for efficiency
-retryOnlyFailed: true
-
-// 2. Reasonable batch sizes
-retryBatchSize: 100
-
-// 3. Short retry delays for transient errors
-retryDelayMs: 1000
-
-// 4. Custom retry predicates to skip non-retryable
-shouldRetry: (error) => !error.message.includes('VALIDATION')
-```
-
----
-
-## Troubleshooting
-
-### All Records Failing
-
-**Problem:** `failedCount === totalRecords`
-
-**Causes:**
-1. Batch processor function has a bug
-2. API credentials invalid
-3. Network connectivity issues
-
-**Solution:**
-```typescript
-// Test processor with small batch
-const testResult = await recovery.processBatchWithRecovery(
-  records.slice(0, 10),
-  batchProcessor,
-  { maxRetries: 1 }
-);
-
-if (testResult.failedCount > 0) {
-  console.error('Processor error:', testResult.failedRecords[0].error);
-}
-```
-
----
-
-### Infinite Retries
-
-**Problem:** Records keep failing and retrying
-
-**Cause:** `shouldRetry` always returns true
-
-**Solution:**
-```typescript
-shouldRetry: (error, attemptCount) => {
-  return attemptCount < 3 && isRetryableError(error);
-}
-```
-
----
-
-## See Also
-
-- [JobTracker](./advanced-services-job-tracker.md) - Track job status
-- [BatchArchivalService](./advanced-services-batch-archival.md) - Archive batches
-- [Error Handling](../api-reference/modules/api-reference-09-error-handling.md) - Error strategies
-- [Ingestion Best Practices](../ingestion/modules/02-core-guides-ingestion-09-best-practices.md)
-
----
-
-[← Back: JobTracker](./advanced-services-job-tracker.md) | [Advanced Services Overview](./advanced-services-readme.md) | [Next: BatchArchival →](./advanced-services-batch-archival.md)
+# PartialBatchRecovery
+
+**Level:** Advanced
+**Category:** Resilience & Error Recovery
+**Purpose:** Recover from partial batch failures without losing all data
+
+## Overview
+
+`PartialBatchRecovery` enables per-record tracking in batch operations, allowing you to retry only failed records instead of re-processing the entire batch. This dramatically improves resilience and reduces data loss in production workflows.
+
+## Use Cases
+
+- ✅ Handle transient API failures gracefully
+- ✅ Implement checkpoint/resume functionality
+- ✅ Retry individual failed records
+- ✅ Track detailed error information per record
+- ✅ Reduce duplicate submissions
+- ✅ Minimize data loss on partial failures
+
+---
+
+## Installation
+
+```typescript
+import { PartialBatchRecovery } from '@fluentcommerce/fc-connect-sdk';
+```
+
+---
+
+## Quick Start
+
+```typescript
+import { PartialBatchRecovery, createClient } from '@fluentcommerce/fc-connect-sdk';
+
+const recovery = new PartialBatchRecovery(logger);
+const client = await createClient(ctx);
+
+// Process with automatic recovery
+const result = await recovery.processBatchWithRecovery(
+  inventoryRecords,
+  async (batch) => await client.sendBatch(jobId, {
+    action: 'UPSERT',
+    entityType: 'INVENTORY',
+    entities: batch
+  }),
+  {
+    maxRetries: 3,
+    retryOnlyFailed: true,
+    checkpointKey: 'inventory-2025-01-30'
+  }
+);
+
+console.log(`✅ Success: ${result.successCount}`);
+console.log(`❌ Failed: ${result.failedCount}`);
+
+if (result.failedRecords.length > 0) {
+  // Handle failed records
+  await sendToDeadLetterQueue(result.failedRecords);
+}
+```
+
+---
+
+## API Reference
+
+### Constructor
+
+```typescript
+new PartialBatchRecovery<T>(logger?: StructuredLogger)
+```
+
+**Generic Type `T`:** The type of your records (e.g., `InventoryRecord`)
+
+---
+
+### processBatchWithRecovery()
+
+Process a batch with automatic recovery and retry logic.
+
+```typescript
+await recovery.processBatchWithRecovery<T>(
+  records: T[],
+  batchProcessor: (records: T[]) => Promise<any>,
+  options?: BatchRecoveryOptions
+): Promise<BatchRecoveryResult<T>>
+```
+
+**Parameters:**
+
+- `records` - Array of records to process
+- `batchProcessor` - Async function that processes a batch
+- `options` - Recovery configuration
+
+**Returns:** `BatchRecoveryResult<T>` with detailed outcome
+
+---
+
+## BatchRecoveryOptions
+
+```typescript
+interface BatchRecoveryOptions {
+  /** Maximum retry attempts per record (default: 3) */
+  maxRetries?: number;
+
+  /** Retry only failed records (true) or entire batch (false) (default: true) */
+  retryOnlyFailed?: boolean;
+
+  /** Checkpoint key for resume functionality */
+  checkpointKey?: string;
+
+  /** Delay between retries in ms (default: 1000) */
+  retryDelayMs?: number;
+
+  /** Custom retry predicate */
+  shouldRetry?: (error: Error, attemptCount: number) => boolean;
+
+  /** Batch size for retry operations (default: 100) */
+  retryBatchSize?: number;
+}
+```
+
+---
+
+## BatchRecoveryResult
+
+```typescript
+interface BatchRecoveryResult<T> {
+  /** Total records processed */
+  totalRecords: number;
+
+  /** Successfully processed count */
+  successCount: number;
+
+  /** Failed records after all retries */
+  failedCount: number;
+
+  /** Details of failed records */
+  failedRecords: RecordFailure<T>[];
+
+  /** Checkpoint ID for resuming */
+  checkpointId?: string;
+
+  /** Human-readable summary */
+  summary: string;
+
+  /** Total processing time (ms) */
+  durationMs: number;
+}
+```
+
+---
+
+## RecordFailure
+
+```typescript
+interface RecordFailure<T> {
+  /** Original record index */
+  index: number;
+
+  /** The record that failed */
+  record: T;
+
+  /** Error that occurred */
+  error: Error;
+
+  /** Number of retry attempts */
+  attemptCount: number;
+
+  /** Timestamp of failure */
+  timestamp: string;
+}
+```
+
+---
+
+## Examples
+
+### Example 1: Basic Recovery
+
+```typescript
+const recovery = new PartialBatchRecovery(logger);
+
+const records = [
+  { ref: 'SKU-001', qty: 100, locationRef: 'DC-001' },
+  { ref: 'SKU-002', qty: 200, locationRef: 'DC-002' },
+  // ... 1000 records
+];
+
+const result = await recovery.processBatchWithRecovery(
+  records,
+  async (batch) => {
+    return await client.sendBatch(jobId, {
+      action: 'UPSERT',
+      entityType: 'INVENTORY',
+      entities: batch
+    });
+  },
+  {
+    maxRetries: 3,
+    retryOnlyFailed: true
+  }
+);
+
+logger.info('Batch processing complete', {
+  total: result.totalRecords,
+  success: result.successCount,
+  failed: result.failedCount,
+  duration: result.durationMs
+});
+```
+
+---
+
+### Example 2: Custom Retry Logic
+
+```typescript
+const result = await recovery.processBatchWithRecovery(
+  records,
+  batchProcessor,
+  {
+    maxRetries: 5,
+    retryDelayMs: 2000,
+    shouldRetry: (error, attemptCount) => {
+      // Don't retry validation errors
+      if (error.message.includes('VALIDATION')) {
+        return false;
+      }
+
+      // Retry transient errors
+      if (error.message.includes('TIMEOUT') ||
+          error.message.includes('RATE_LIMIT')) {
+        return attemptCount < 5;
+      }
+
+      return false;
+    }
+  }
+);
+```
+
+---
+
+### Example 3: Checkpoint/Resume
+
+```typescript
+const checkpointKey = `inventory-${new Date().toISOString().split('T')[0]}`;
+
+// First attempt
+const result = await recovery.processBatchWithRecovery(
+  records,
+  batchProcessor,
+  {
+    checkpointKey,
+    maxRetries: 3
+  }
+);
+
+// If there were failures, you can resume later
+if (result.failedRecords.length > 0) {
+  logger.warn('Some records failed, saved checkpoint', {
+    checkpointId: result.checkpointId,
+    failedCount: result.failedCount
+  });
+
+  // Later, retry just the failed records
+  const checkpoint = recovery.getCheckpoint(checkpointKey);
+  if (checkpoint) {
+    const retryResult = await recovery.processBatchWithRecovery(
+      checkpoint.failedRecords.map(f => f.record),
+      batchProcessor,
+      { maxRetries: 3 }
+    );
+  }
+}
+```
+
+---
+
+### Example 4: Dead Letter Queue Pattern
+
+```typescript
+async function processWithDLQ(records: any[]) {
+  const recovery = new PartialBatchRecovery(logger);
+
+  const result = await recovery.processBatchWithRecovery(
+    records,
+    batchProcessor,
+    {
+      maxRetries: 3,
+      retryOnlyFailed: true
+    }
+  );
+
+  // Send failed records to DLQ for manual review
+  if (result.failedRecords.length > 0) {
+    await s3.uploadFile(
+      `dlq/failed-${Date.now()}.json`,
+      JSON.stringify({
+        timestamp: new Date().toISOString(),
+        totalFailed: result.failedCount,
+        records: result.failedRecords.map(f => ({
+          record: f.record,
+          error: f.error.message,
+          attempts: f.attemptCount,
+          timestamp: f.timestamp
+        }))
+      }, null, 2)
+    );
+
+    logger.error('Records sent to DLQ', {
+      count: result.failedCount,
+      dlqKey: `dlq/failed-${Date.now()}.json`
+    });
+
+    // Optionally send alert
+    await sendAlert({
+      type: 'BATCH_PARTIAL_FAILURE',
+      failedCount: result.failedCount,
+      totalCount: result.totalRecords
+    });
+  }
+
+  return result;
+}
+```
+
+---
+
+## Patterns
+
+### Pattern 1: Retry Only Failed
+
+```typescript
+const result = await recovery.processBatchWithRecovery(
+  records,
+  batchProcessor,
+  {
+    retryOnlyFailed: true,
+    maxRetries: 3
+  }
+);
+```
+
+**Benefits:**
+- Faster recovery
+- Reduces duplicate submissions
+- Lower API costs
+
+---
+
+### Pattern 2: Retry Entire Batch
+
+```typescript
+// Use when batch must be atomic
+const result = await recovery.processBatchWithRecovery(
+  records,
+  batchProcessor,
+  {
+    retryOnlyFailed: false,  // Retry entire batch
+    maxRetries: 2
+  }
+);
+```
+
+**Use When:**
+- Batch operations must be atomic
+- Records are interdependent
+- Fluent BPP requires full snapshot
+
+---
+
+### Pattern 3: Exponential Backoff
+
+```typescript
+let retryDelay = 1000;
+
+for (let attempt = 1; attempt <= 3; attempt++) {
+  const result = await recovery.processBatchWithRecovery(
+    records,
+    batchProcessor,
+    {
+      retryDelayMs: retryDelay,
+      maxRetries: 1  // Manual retry control
+    }
+  );
+
+  if (result.failedCount === 0) break;
+
+  retryDelay *= 2;  // Double delay each attempt
+  records = result.failedRecords.map(f => f.record);
+}
+```
+
+---
+
+### Pattern 4: Multi-Stage Recovery
+
+```typescript
+async function multiStageRecovery(records: any[]) {
+  // Stage 1: Fast retries (transient errors)
+  let result = await recovery.processBatchWithRecovery(
+    records,
+    batchProcessor,
+    {
+      maxRetries: 2,
+      retryDelayMs: 500,
+      retryOnlyFailed: true
+    }
+  );
+
+  if (result.failedCount === 0) return result;
+
+  // Stage 2: Slower retries with backoff
+  const stage2Result = await recovery.processBatchWithRecovery(
+    result.failedRecords.map(f => f.record),
+    batchProcessor,
+    {
+      maxRetries: 3,
+      retryDelayMs: 5000,
+      retryOnlyFailed: true
+    }
+  );
+
+  if (stage2Result.failedCount === 0) {
+    return {
+      ...stage2Result,
+      totalRecords: result.totalRecords,
+      successCount: result.totalRecords
+    };
+  }
+
+  // Stage 3: Send to DLQ
+  await sendToDLQ(stage2Result.failedRecords);
+
+  return stage2Result;
+}
+```
+
+---
+
+## Patterns
+
+### 1. Always Check Failed Records
+
+```typescript
+const result = await recovery.processBatchWithRecovery(records, processor);
+
+if (result.failedCount > 0) {
+  logger.error('Batch had failures', {
+    failed: result.failedCount,
+    total: result.totalRecords
+  });
+
+  await handleFailures(result.failedRecords);
+}
+```
+
+### 2. Use Appropriate Batch Sizes
+
+```typescript
+const result = await recovery.processBatchWithRecovery(
+  records,
+  batchProcessor,
+  {
+    retryBatchSize: 100
+  }
+);
+```
+
+### 3. Implement Dead Letter Queue
+
+```typescript
+if (result.failedRecords.length > 0) {
+  await saveToDLQ(result.failedRecords);
+  await notifyTeam(result.failedCount);
+}
+```
+
+### 4. Set Realistic Retry Limits
+
+```typescript
+maxRetries: 3
+```
+
+---
+
+## Performance Considerations
+
+**Overhead:** ~5-10% for tracking
+**Memory:** O(n) for failed records
+**Network:** Only retries failed records (with `retryOnlyFailed: true`)
+
+**Optimization Tips:**
+```typescript
+// 1. Use retryOnlyFailed for efficiency
+retryOnlyFailed: true
+
+// 2. Reasonable batch sizes
+retryBatchSize: 100
+
+// 3. Short retry delays for transient errors
+retryDelayMs: 1000
+
+// 4. Custom retry predicates to skip non-retryable
+shouldRetry: (error) => !error.message.includes('VALIDATION')
+```
+
+---
+
+## Troubleshooting
+
+### All Records Failing
+
+**Problem:** `failedCount === totalRecords`
+
+**Causes:**
+1. Batch processor function has a bug
+2. API credentials invalid
+3. Network connectivity issues
+
+**Solution:**
+```typescript
+// Test processor with small batch
+const testResult = await recovery.processBatchWithRecovery(
+  records.slice(0, 10),
+  batchProcessor,
+  { maxRetries: 1 }
+);
+
+if (testResult.failedCount > 0) {
+  console.error('Processor error:', testResult.failedRecords[0].error);
+}
+```
+
+---
+
+### Infinite Retries
+
+**Problem:** Records keep failing and retrying
+
+**Cause:** `shouldRetry` always returns true
+
+**Solution:**
+```typescript
+shouldRetry: (error, attemptCount) => {
+  return attemptCount < 3 && isRetryableError(error);
+}
+```
+
+---
+
+## See Also
+
+- [JobTracker](./advanced-services-job-tracker.md) - Track job status
+- [BatchArchivalService](./advanced-services-batch-archival.md) - Archive batches
+- [Error Handling](../api-reference/modules/api-reference-09-error-handling.md) - Error strategies
+- [Ingestion Best Practices](../ingestion/modules/02-core-guides-ingestion-09-best-practices.md)
+
+---
+
+[← Back: JobTracker](./advanced-services-job-tracker.md) | [Advanced Services Overview](./advanced-services-readme.md) | [Next: BatchArchival →](./advanced-services-batch-archival.md)