@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-06-batch-api.md

@@ -1,1295 +1,1295 @@
-# Module 6: Batch API Deep Dive
-
-[← Back to Ingestion Guide](../ingestion-readme.md)
-
-**Module 6 of 9** | **Level**: Intermediate | **Time**: 30 minutes
-
----
-
-## Overview
-
-This module provides complete coverage of the Fluent Commerce Batch API, including job lifecycle, batch operations, status monitoring, and optimization strategies.
-
-## Learning Objectives
-
-By the end of this module, you will:
-
-- ✅ Understand the complete Batch API schema
-- ✅ Master the job creation and management lifecycle
-- ✅ Implement batch size optimization strategies
-- ✅ Monitor job and batch status effectively
-- ✅ Understand job strategies (DAILY vs PER_FILE)
-- ✅ Handle job expiration and recovery
-- ✅ Implement production-ready batch processing
-
----
-
-## Batch API Schema
-
-### Complete GraphQL Schema
-
-The Batch API consists of two main mutations:
-
-#### 1. `createJob` Mutation
-
-```graphql
-mutation CreateJob($input: CreateJobInput!) {
-  createJob(input: $input) {
-    id # Job ID (String!)
-    name # Job name (String)
-    status # Job status (JobStatus enum)
-  }
-}
-```
-
-**Input Schema** (`CreateJobInput`):
-
-```typescript
-{
-  "name": "string",         // ✅ Job name (String! required)
-  "retailerId": "string",   // ✅ Retailer ID (ID! required)
-  "metadata": {             // ✅ JSON (optional) - Custom metadata
-    "source": "S3",
-    "file": "inventory-2025-01-19.csv"
-  }
-}
-```
-
-#### 2. `sendBatch` Mutation
-
-```graphql
-mutation SendBatch($input: BatchInput!) {
-  sendBatch(input: $input) {
-    id # Batch ID (String!)
-    status # Batch status (BatchStatus enum)
-  }
-}
-```
-
-**Input Schema** (`BatchInput`):
-
-```typescript
-{
-  "action": "UPSERT",               // ✅ String (UPSERT only - required)
-  "entityType": "INVENTORY",        // ✅ String (INVENTORY only - required)
-  "source": "S3_CSV_IMPORT",        // ✅ String (required) - Source system identifier
-  "event": "INVENTORY_UPDATE",      // ✅ String (required) - Event type identifier
-  "retailerId": "1",                // ✅ ID! (required)
-  "entities": [                     // ✅ [InventoryPositionInput]! (required array)
-    {
-      "ref": "SKU-WM-001",          // ✅ String! (required)
-      "productRef": "SKU-WM-001",   // ✅ String! (required)
-      "locationRef": "WH-001",      // ✅ String! (required)
-      "qty": 500,                   // ✅ Int! (required)
-      "status": "AVAILABLE"         // ✅ String (optional)
-    }
-  ]
-}
-```
-
-### `InventoryPositionInput` Complete Schema
-
-From `docs/schema/fluent-commerce-schema.json`:
-
-```json
-{
-  "InventoryPositionInput": {
-    "kind": "INPUT_OBJECT",
-    "description": "Input for inventory position in batch",
-    "fields": {
-      "ref": {
-        "type": "String!",
-        "description": "Position reference (Required)",
-        "businessContext": "Unique identifier for this inventory position, typically SKU-LOCATION combination",
-        "example": "SKU-WM-001"
-      },
-      "productRef": {
-        "type": "String!",
-        "description": "Product reference (Required)",
-        "businessContext": "Product SKU that this inventory position represents",
-        "example": "SKU-WM-001"
-      },
-      "locationRef": {
-        "type": "String!",
-        "description": "Location reference (Required)",
-        "businessContext": "Warehouse or location code where inventory is held",
-        "example": "WH-001"
-      },
-      "qty": {
-        "type": "Int!",
-        "description": "Quantity (Required)",
-        "businessContext": "Available quantity at this location",
-        "example": 500,
-        "validationRules": ["qty >= 0"]
-      },
-      "status": {
-        "type": "String",
-        "description": "Status (Optional)",
-        "businessContext": "Inventory position status",
-        "example": "AVAILABLE",
-        "validValues": ["AVAILABLE", "RESERVED", "ON_HOLD", "DISCONTINUED"]
-      },
-      "type": {
-        "type": "String",
-        "description": "Position type (Optional)",
-        "businessContext": "Type of inventory adjustment",
-        "example": "ADJUSTMENT"
-      }
-    }
-  }
-}
-```
-
----
-
-## Complete Job Lifecycle
-
-### Lifecycle Diagram
-
-```mermaid
-graph TD
-    A[Create Job] --> B{Job Created?}
-    B -->|Yes| C[Job PENDING]
-    B -->|No| Z[ERROR]
-
-    C --> D[Send Batch 1]
-    D --> E[Batch PROCESSING]
-
-    C --> F[Send Batch 2]
-    F --> G[Batch PROCESSING]
-
-    E --> H{Batch Complete?}
-    G --> I{Batch Complete?}
-
-    H -->|Success| J[Batch COMPLETED]
-    H -->|Failure| K[Batch FAILED]
-
-    I -->|Success| L[Batch COMPLETED]
-    I -->|Failure| M[Batch FAILED]
-
-    J --> N{More Batches?}
-    L --> N
-
-    N -->|Yes| O[Send Next Batch]
-    N -->|No| P[All Batches Sent]
-
-    O --> E
-
-    P --> Q{Job Expired?}
-    Q -->|No| R[Job COMPLETED]
-    Q -->|Yes| S[Job EXPIRED - Create New Job]
-
-    K --> T[Handle Failed Batch]
-    M --> T
-    T --> U[Retry or Log Error]
-```
-
-### Step 1: Create Job
-
-**Simple job creation:**
-
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-
-const client = await createClient({ config });
-
-const job = await client.createJob({
-  name: 'Daily Inventory Sync',
-  retailerId: '1',
-});
-
-console.log(`Job created: ${job.id}`);
-// Output: Job created: JOB-12345
-```
-
-**Job creation with metadata (for tracking):**
-
-```typescript
-const job = await client.createJob({
-  name: `Inventory Import - ${new Date().toISOString()}`,
-  retailerId: process.env.FLUENT_RETAILER_ID!,
-  metadata: {
-    source: 'S3',
-    bucket: 'inventory-bucket',
-    file: 'data/inventory-2025-01-19.csv',
-    uploadedBy: 'integration-service',
-    triggeredBy: 'scheduled-workflow',
-  },
-});
-```
-
-### Step 2: Send Batches
-
-**Single batch:**
-
-```typescript
-const batch = await client.sendBatch(job.id, {
-  action: 'UPSERT',           // String, not enum
-  entityType: 'INVENTORY',    // String, not enum
-  source: 'S3_CSV_IMPORT',    // REQUIRED - Source system identifier
-  event: 'INVENTORY_UPDATE',  // REQUIRED - Event type identifier
-  entities: [
-    {
-      ref: 'SKU-WM-001',
-      productRef: 'SKU-WM-001',
-      locationRef: 'WH-001',
-      qty: 500,
-      status: 'AVAILABLE',
-    },
-  ],
-});
-
-console.log(`Batch sent: ${batch.id}`);
-```
-
-**Multiple batches (chunked):**
-
-```typescript
-function chunkArray<T>(array: T[], chunkSize: number): T[][] {
-  const chunks: T[][] = [];
-  for (let i = 0; i < array.length; i += chunkSize) {
-    chunks.push(array.slice(i, i + chunkSize));
-  }
-  return chunks;
-}
-
-// Split large dataset into batches
-const batchSize = 1000;
-const batches = chunkArray(inventoryData, batchSize);
-
-const batchResults = [];
-
-for (const [index, batchData] of batches.entries()) {
-  console.log(`Sending batch ${index + 1}/${batches.length}...`);
-
-  const batch = await client.sendBatch(job.id, {
-    action: 'UPSERT',
-    entityType: 'INVENTORY',
-    source: 'S3_CSV_IMPORT',
-    event: 'INVENTORY_UPDATE',
-    entities: batchData,
-  });
-
-  batchResults.push({
-    index,
-    batchId: batch.id,
-    records: batchData.length,
-  });
-}
-
-console.log(`Sent ${batchResults.length} batches with ${inventoryData.length} total records`);
-```
-
-### Step 3: Monitor Status
-
-**Poll batch status:**
-
-```typescript
-async function waitForBatchCompletion(
-  client: FluentClient,
-  jobId: string,
-  batchId: string,
-  pollInterval: number = 5000,
-  timeout: number = 300000
-): Promise<BatchStatus> {
-  const startTime = Date.now();
-
-  while (Date.now() - startTime < timeout) {
-    const status = await client.getBatchStatus(jobId, batchId);
-
-    console.log(`Batch ${batchId}: ${status.status}`);
-
-    if (status.status === 'COMPLETED') {
-      return status;
-    }
-
-    if (status.status === 'FAILED') {
-      throw new Error(`Batch failed: ${JSON.stringify(status.errors)}`);
-    }
-
-    // Wait before next poll
-    await new Promise(resolve => setTimeout(resolve, pollInterval));
-  }
-
-  throw new Error(`Batch monitoring timeout after ${timeout}ms`);
-}
-
-// Usage
-const status = await waitForBatchCompletion(client, job.id, batch.id);
-console.log(`✅ Batch completed successfully`);
-```
-
-**Monitor job status (all batches):**
-
-```typescript
-async function monitorJobProgress(
-  client: FluentClient,
-  jobId: string,
-  onProgress?: (status: JobStatus) => void
-): Promise<JobStatus> {
-  const pollInterval = 5000; // 5 seconds
-  const timeout = 600000; // 10 minutes
-
-  const startTime = Date.now();
-
-  while (Date.now() - startTime < timeout) {
-    const status = await client.getJobStatus(jobId);
-
-    if (onProgress) {
-      onProgress(status);
-    }
-
-    if (status.status === 'COMPLETED' || status.status === 'FAILED') {
-      return status;
-    }
-
-    await new Promise(resolve => setTimeout(resolve, pollInterval));
-  }
-
-  throw new Error(`Job monitoring timeout: ${jobId}`);
-}
-
-// Usage with progress callback
-const jobStatus = await monitorJobProgress(client, job.id, status => {
-  console.log(`Job ${job.id}: ${status.status} - ${status.progress || 0}% complete`);
-});
-```
-
----
-
-## Batch Pre-Processing (BPP) - Change Detection
-
-### Overview
-
-**Batch Pre-Processing (BPP)** is Fluent Commerce's built-in change detection system that **filters out unchanged inventory records** before they reach the workflow engine, significantly improving performance when sending full inventory snapshots.
-
-### How BPP Works
-
-BPP introduces two internal jobs that run **before** your inventory updates hit the workflow:
-
-```
-┌─────────────────────────────────────────────────────┐
-│  WITHOUT BPP (meta.preprocessing: "skip")           │
-│  ────────────────────────────────────────────────   │
-│  Batch API → ALL records → Inventory Queue → Rubix │
-│  100K records → 100K workflow events                │
-└─────────────────────────────────────────────────────┘
-
-┌──────────────────────────────────────────────────────────┐
-│  WITH BPP (default)                                      │
-│  ─────────────────────────────────────────────────────   │
-│  Batch API → Loader Job → Comparison Job →               │
-│  → ONLY CHANGED records → Inventory Queue → Rubix        │
-│  100K records → 5K workflow events (95% filtered)         │
-└──────────────────────────────────────────────────────────┘
-```
-
-#### BPP Components
-
-**1. Loader Job**
-
-- Loads existing inventory data from database
-- Includes current quantities, statuses, and transient inventory
-
-**2. Comparison Job**
-
-- Compares batch records vs existing data
-- Checks for changes in:
-  - **Quantity**: Has qty changed?
-  - **Status**: Has status changed?
-  - **Transient Inventory**: Are there active transient quantities?
-
-**3. Result**
-
-- Only records marked as "changed" proceed to workflow
-- Unchanged records are filtered out (not processed)
-
-### When to Use BPP
-
-#### ✅ Use BPP (Default) When:
-
-**Scenario 1: Full Inventory Snapshots**
-
-```typescript
-// Sending complete inventory file daily
-const job = await client.createJob({
-  name: 'Daily Full Inventory Sync',
-  retailerId: '1',
-  // BPP enabled by default (no meta.preprocessing)
-});
-
-// 100K records sent, only ~5K changed overnight
-// Result: 5K workflow events instead of 100K
-```
-
-**Scenario 2: Unchanged Records in Data Feed**
-
-```typescript
-// WMS sends all SKU-location combinations daily
-// Many haven't changed since yesterday
-const job = await client.createJob({
-  name: 'WMS Daily Extract',
-  retailerId: '1',
-  meta: {
-    preprocessing: 'enabled', // Explicit BPP enable
-    source: 'WMS',
-  },
-});
-```
-
-**Benefits:**
-
-- ✅ Reduces workflow engine load by 80-95%
-- ✅ Faster processing (only changed records)
-- ✅ Automatic duplicate detection
-- ✅ Lower infrastructure costs
-
-#### ❌ Skip BPP When:
-
-**Scenario 1: Delta/Change-Only Feeds**
-
-```typescript
-// Your application already filtered changed records
-// Sending ONLY records that changed
-const job = await client.createJob({
-  name: 'Real-Time Inventory Delta',
-  retailerId: '1',
-  meta: {
-    preprocessing: 'skip', // ← Skip BPP (every record is already changed)
-  },
-});
-
-// 100 records sent, all 100 are changes
-// Result: 100 workflow events (no filtering needed)
-```
-
-**Scenario 2: Real-Time Updates**
-
-```typescript
-// High-frequency updates where every record matters
-// e.g., Point-of-sale inventory decrements
-const job = await client.createJob({
-  name: 'POS Inventory Update',
-  retailerId: '1',
-  meta: {
-    preprocessing: 'skip',
-    source: 'POS',
-    realtime: true,
-  },
-});
-```
-
-**Scenario 3: Performance-Critical Imports**
-
-```typescript
-// Need absolute fastest processing
-// Data is guaranteed to be different
-const job = await client.createJob({
-  name: 'Critical Inventory Adjustment',
-  retailerId: '1',
-  meta: {
-    preprocessing: 'skip',
-    priority: 'high',
-  },
-});
-```
-
-**Benefits:**
-
-- ✅ Faster (no comparison overhead)
-- ✅ No need to load existing data
-- ✅ Simpler processing flow
-
-### Account-Level BPP Configuration
-
-BPP behavior is controlled by **two settings**:
-
-#### 1. Account Setting: `fc.enable.batch.preprocessing`
-
-This is configured at the **Fluent Commerce account level** (contact Fluent support to check/modify):
-
-```typescript
-// Pseudo-code showing the logic
-if (account.settings.fc_enable_batch_preprocessing === false) {
-  // BPP is NEVER used, regardless of job meta
-  processingMode = 'DIRECT';
-} else {
-  // BPP is available, check job-level meta
-  if (job.meta.preprocessing === 'skip') {
-    processingMode = 'DIRECT';
-  } else {
-    processingMode = 'BPP';
-  }
-}
-```
-
-#### 2. Job-Level Setting: `meta.preprocessing`
-
-Set per job when creating:
-
-```typescript
-interface FluentJobMetadata {
-  /**
-   * Controls Batch Pre-Processing (BPP) change detection
-   *
-   * @remarks
-   * - `"skip"`: Bypass BPP (all records processed)
-   * - `"enabled"` or any other value: Use BPP (default)
-   * - Omitted/null: Use BPP (default)
-   *
-   * @important
-   * Only works if account setting `fc.enable.batch.preprocessing` is `true`
-   * If account setting is `false`, BPP is never used regardless of this value
-   */
-  preprocessing?: 'skip' | 'enabled' | string;
-
-  // Other metadata fields
-  source?: string;
-  file?: string;
-  [key: string]: any;
-}
-```
-
-### BPP Behavior Matrix
-
-| Account Setting (`fc.enable.batch.preprocessing`) | Job Meta (`preprocessing`) | Result                            |
-| ------------------------------------------------- | -------------------------- | --------------------------------- |
-| `true`                                            | `"skip"`                   | ❌ **No BPP** (Direct processing) |
-| `true`                                            | `"enabled"`                | ✅ **Use BPP** (Change detection) |
-| `true`                                            | `null` / omitted           | ✅ **Use BPP** (Default)          |
-| `false`                                           | ANY value                  | ❌ **No BPP** (Account override)  |
-
-### Complete Code Examples
-
-#### Example 1: Daily Full Snapshot (Use BPP)
-
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-
-const client = await createClient(ctx);
-
-// Daily full inventory file from WMS
-const job = await client.createJob({
-  name: 'Daily WMS Full Snapshot',
-  retailerId: '1',
-  // No meta.preprocessing = BPP enabled by default
-  meta: {
-    source: 'WMS',
-    feedType: 'FULL_SNAPSHOT',
-    date: new Date().toISOString(),
-  },
-});
-
-// Send 100K records (many unchanged)
-await client.sendBatch(job.id, {
-  action: 'UPSERT',
-  entityType: 'INVENTORY',
-  source: 'WMS_FULL_SNAPSHOT',
-  event: 'INVENTORY_SYNC',
-  entities: allInventoryRecords, // All 100K records
-});
-
-// Result: Only ~5K changed records hit workflow
-console.log('BPP filtered out ~95K unchanged records');
-```
-
-#### Example 2: Real-Time Deltas (Skip BPP)
-
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-
-const client = await createClient(ctx);
-
-// Only sending records that changed in last hour
-const job = await client.createJob({
-  name: 'Hourly Inventory Deltas',
-  retailerId: '1',
-  meta: {
-    preprocessing: 'skip', // ← Skip BPP (already filtered)
-    source: 'CDC', // Change Data Capture
-    feedType: 'DELTA',
-  },
-});
-
-// Send only changed records (already filtered by your system)
-await client.sendBatch(job.id, {
-  action: 'UPSERT',
-  entityType: 'INVENTORY',
-  source: 'CDC_DELTA_FEED',
-  event: 'INVENTORY_CHANGE',
-  entities: changedRecordsOnly, // Only 500 changed records
-});
-
-// Result: All 500 records hit workflow immediately
-console.log('All records processed (no BPP filtering)');
-```
-
-#### Example 3: Hybrid Approach
-
-```typescript
-// Different strategies for different times of day
-async function adaptiveInventorySync(client, records, isFullSync) {
-  const job = await client.createJob({
-    name: isFullSync ? 'Full Sync' : 'Delta Sync',
-    retailerId: '1',
-    meta: {
-      // Use BPP for full syncs, skip for deltas
-      preprocessing: isFullSync ? 'enabled' : 'skip',
-      source: 'Hybrid',
-      syncType: isFullSync ? 'FULL' : 'DELTA',
-    },
-  });
-
-  await client.sendBatch(job.id, {
-    action: 'UPSERT',
-    entityType: 'INVENTORY',
-    source: isFullSync ? 'HYBRID_FULL_SYNC' : 'HYBRID_DELTA_SYNC',
-    event: isFullSync ? 'INVENTORY_FULL_SYNC' : 'INVENTORY_DELTA_SYNC',
-    entities: records,
-  });
-
-  if (isFullSync) {
-    console.log('BPP filtering applied for full sync');
-  } else {
-    console.log('Direct processing for delta sync');
-  }
-}
-
-// Daily full sync at 2 AM
-await adaptiveInventorySync(client, allRecords, true);
-
-// Hourly deltas throughout the day
-await adaptiveInventorySync(client, changedRecords, false);
-```
-
-### Decision Guide: BPP or Skip?
-
-```
-START: What type of data are you sending?
-│
-├─ Full inventory snapshot (all SKUs/locations)
-│  └─ **Use BPP** (default - don't set meta.preprocessing)
-│     → Many unchanged records will be filtered
-│
-├─ Delta feed (only changed records)
-│  └─ **Skip BPP** (set meta.preprocessing: "skip")
-│     → All records are already changes
-│
-├─ Mixed/Unknown
-│  ├─ High percentage unchanged (>50%)
-│  │  └─ **Use BPP** → Better performance
-│  │
-│  └─ Most records changed (>80%)
-│     └─ **Skip BPP** → Faster (no comparison overhead)
-│
-└─ Real-time/urgent updates
-   └─ **Skip BPP** → Lowest latency
-```
-
-### Performance Comparison
-
-#### Scenario: 100K Inventory Records Sent Daily
-
-| Metric                    | With BPP (Default) | BPP Skip (`preprocessing: "skip"`) |
-| ------------------------- | ------------------ | ---------------------------------- |
-| Records sent to API       | 100K               | 100K                               |
-| Records actually changed  | 5K (5%)            | 100K (100% assumed)                |
-| Workflow events generated | 5K                 | 100K                               |
-| Processing time           | ~8 seconds         | ~45 seconds                        |
-| Workflow engine load      | Low                | High                               |
-| **Recommended for**       | Full snapshots     | Delta feeds only                   |
-
-### Common Mistakes
-
-#### ❌ Mistake 1: Using BPP for Delta Feeds
-
-```typescript
-// WRONG: Wasting resources on comparison
-const job = await client.createJob({
-  name: 'Delta Feed',
-  retailerId: '1',
-  // BPP enabled by default - unnecessary for delta!
-});
-
-await client.sendBatch(job.id, {
-  action: 'UPSERT',
-  entityType: 'INVENTORY',
-  source: 'DELTA_FEED',
-  event: 'INVENTORY_DELTA',
-  entities: onlyChangedRecords, // Already filtered!
-});
-// Result: BPP compares all records unnecessarily
-```
-
-**✅ Fix:**
-
-```typescript
-const job = await client.createJob({
-  name: 'Delta Feed',
-  retailerId: '1',
-  meta: {
-    preprocessing: 'skip', // ← Skip BPP for delta feeds
-  },
-});
-```
-
-#### ❌ Mistake 2: Skipping BPP for Full Snapshots
-
-```typescript
-// WRONG: Overwhelming workflow with unchanged records
-const job = await client.createJob({
-  name: 'Full Inventory',
-  retailerId: '1',
-  meta: {
-    preprocessing: 'skip', // ← Bad for full snapshots!
-  },
-});
-
-await client.sendBatch(job.id, {
-  action: 'UPSERT',
-  entityType: 'INVENTORY',
-  source: 'FULL_INVENTORY_SYNC',
-  event: 'INVENTORY_SNAPSHOT',
-  entities: allInventory, // 100K records, 95K unchanged
-});
-// Result: 100K workflow events (95K unnecessary)
-```
-
-**✅ Fix:**
-
-```typescript
-const job = await client.createJob({
-  name: 'Full Inventory',
-  retailerId: '1',
-  // BPP enabled by default - perfect!
-});
-// Result: Only 5K workflow events (95K filtered by BPP)
-```
-
-### Key Takeaways
-
-- 🎯 **BPP = Change Detection**, not data transformation
-- 🎯 **Use BPP** for full inventory snapshots (most common)
-- 🎯 **Skip BPP** for delta feeds (already filtered)
-- 🎯 **Account setting** takes precedence over job setting
-- 🎯 **Default is BPP enabled** (safest for most cases)
-
----
-
-## Batch Size Optimization
-
-### Calculate Optimal Batch Size
-
-```typescript
-interface BatchSizeConfig {
-  maxBatchSize: number; // Maximum records per batch (default: 10000)
-  maxPayloadSize: number; // Maximum payload size in bytes (default: 10MB)
-  avgRecordSize?: number; // Average record size (auto-calculated if not provided)
-}
-
-function calculateOptimalBatchSize(
-  records: any[],
-  config: BatchSizeConfig = {
-    maxBatchSize: 10000,
-    maxPayloadSize: 10 * 1024 * 1024, // 10MB
-  }
-): number {
-  // Calculate average record size if not provided
-  const avgRecordSize = config.avgRecordSize || JSON.stringify(records.slice(0, 100)).length / 100;
-
-  // Calculate records per max payload
-  const recordsPerPayload = Math.floor(config.maxPayloadSize / avgRecordSize);
-
-  // Return minimum of calculated size and max batch size
-  const optimalSize = Math.min(recordsPerPayload, config.maxBatchSize);
-
-  console.log(`Optimal batch size: ${optimalSize} (based on ${avgRecordSize} bytes/record)`);
-
-  return optimalSize;
-}
-
-// Usage
-const batchSize = calculateOptimalBatchSize(inventoryData);
-const batches = chunkArray(inventoryData, batchSize);
-```
-
-### Adaptive Batch Sizing
-
-```typescript
-class AdaptiveBatchProcessor {
-  private currentBatchSize: number;
-  private readonly minBatchSize = 100;
-  private readonly maxBatchSize = 10000;
-
-  constructor(initialBatchSize: number = 1000) {
-    this.currentBatchSize = initialBatchSize;
-  }
-
-  async processBatch(client: FluentClient, jobId: string, data: any[]): Promise<void> {
-    const batches = chunkArray(data, this.currentBatchSize);
-
-    for (const batchData of batches) {
-      const startTime = Date.now();
-
-      try {
-        const batch = await client.sendBatch(jobId, {
-          action: 'UPSERT',
-          entityType: 'INVENTORY',
-          source: 'ADAPTIVE_BATCH',
-          event: 'INVENTORY_BATCH_IMPORT',
-          entities: batchData,
-        });
-
-        const processingTime = Date.now() - startTime;
-
-        // Adjust batch size based on performance
-        this.adjustBatchSize(processingTime, batchData.length);
-      } catch (error) {
-        // Reduce batch size on error
-        this.currentBatchSize = Math.max(this.minBatchSize, Math.floor(this.currentBatchSize / 2));
-        throw error;
-      }
-    }
-  }
-
-  private adjustBatchSize(processingTime: number, recordCount: number): void {
-    const avgTimePerRecord = processingTime / recordCount;
-
-    // If processing is fast, increase batch size
-    if (avgTimePerRecord < 10 && this.currentBatchSize < this.maxBatchSize) {
-      this.currentBatchSize = Math.min(this.maxBatchSize, Math.floor(this.currentBatchSize * 1.2));
-    }
-
-    // If processing is slow, decrease batch size
-    if (avgTimePerRecord > 50 && this.currentBatchSize > this.minBatchSize) {
-      this.currentBatchSize = Math.max(this.minBatchSize, Math.floor(this.currentBatchSize * 0.8));
-    }
-  }
-}
-```
-
----
-
-## Job Strategies
-
-### Strategy 1: DAILY (Recommended for frequent updates)
-
-**Use when:**
-
-- Multiple files processed per day
-- Want to group all updates under one job
-- Need centralized tracking
-
-**Pattern:**
-
-```typescript
-import { FluentClient, StateService, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-
-async function dailyJobStrategy(
-  client: FluentClient,
-  files: string[],
-  kvAdapter: VersoriKVAdapter,
-  state: StateService
-): Promise<void> {
-  const workflowId = 'daily-inventory-sync';
-
-  // Get or create today's job using StateService
-  let job = await state.getDailyJob(kvAdapter, workflowId);
-
-  if (!job) {
-    const dateKey = new Date().toISOString().split('T')[0];
-    const newJob = await client.createJob({
-      name: `Daily Inventory Sync - ${dateKey}`,
-      retailerId: process.env.FLUENT_RETAILER_ID!,
-      meta: { date: dateKey, type: 'daily' },
-    });
-
-    // Store job reference with 24-hour expiration
-    await state.setDailyJob(kvAdapter, workflowId, newJob.id, 24);
-    job = { jobId: newJob.id };
-  }
-
-  // Process all files with same job
-  for (const file of files) {
-    const data = await processFile(file);
-
-    await client.sendBatch(job.jobId, {
-      action: 'UPSERT',
-      entityType: 'INVENTORY',
-      source: 'DAILY_JOB_STRATEGY',
-      event: 'INVENTORY_DAILY_SYNC',
-      entities: data,
-    });
-  }
-}
-```
-
-**Benefits:**
-
-- ✅ Single job for all daily updates
-- ✅ Easier monitoring and reporting
-- ✅ Reduced job creation overhead
-
-**Drawbacks:**
-
-- ❌ Job expiration risk if processing takes too long
-- ❌ All batches fail if job fails
-
-### Strategy 2: PER_FILE (Recommended for large datasets)
-
-**Use when:**
-
-- Processing large files (50k+ records)
-- Want isolated failure handling
-- Need per-file tracking
-
-**Pattern:**
-
-```typescript
-async function perFileJobStrategy(client: FluentClient, files: string[]): Promise<void> {
-  for (const file of files) {
-    // Create new job for each file
-    const job = await client.createJob({
-      name: `Import - ${file}`,
-      retailerId: process.env.FLUENT_RETAILER_ID!,
-      metadata: { file, type: 'per-file' },
-    });
-
-    const data = await processFile(file);
-    const batches = chunkArray(data, 1000);
-
-    for (const batchData of batches) {
-      await client.sendBatch(job.id, {
-        action: 'UPSERT',
-        entityType: 'INVENTORY',
-        source: 'PER_FILE_STRATEGY',
-        event: 'INVENTORY_FILE_IMPORT',
-        entities: batchData,
-      });
-    }
-  }
-}
-```
-
-**Benefits:**
-
-- ✅ Isolated failure handling
-- ✅ No job expiration risk
-- ✅ Clear per-file tracking
-
-**Drawbacks:**
-
-- ❌ More jobs created (overhead)
-- ❌ Harder to aggregate metrics
-
-### Strategy 3: BATCHES_PER_JOB (Advanced)
-
-**Use when:**
-
-- Need balance between DAILY and PER_FILE
-- Want to limit batches per job
-
-**Pattern:**
-
-```typescript
-async function batchesPerJobStrategy(
-  client: FluentClient,
-  data: any[],
-  maxBatchesPerJob: number = 10
-): Promise<void> {
-  const batchSize = 1000;
-  const batches = chunkArray(data, batchSize);
-
-  let currentJob: any = null;
-  let batchCount = 0;
-
-  for (const batchData of batches) {
-    // Create new job if needed
-    if (!currentJob || batchCount >= maxBatchesPerJob) {
-      currentJob = await client.createJob({
-        name: `Inventory Import - ${Date.now()}`,
-        retailerId: process.env.FLUENT_RETAILER_ID!,
-      });
-      batchCount = 0;
-    }
-
-    // Send batch
-    await client.sendBatch(currentJob.id, {
-      action: 'UPSERT',
-      entityType: 'INVENTORY',
-      source: 'BATCHES_PER_JOB',
-      event: 'INVENTORY_BATCH_CHUNK',
-      entities: batchData,
-    });
-
-    batchCount++;
-  }
-}
-```
-
----
-
-## Job Expiration Handling
-
-Jobs expire after a certain time (typically 24 hours). Handle expiration gracefully:
-
-```typescript
-async function sendBatchWithExpirationHandling(
-  client: FluentClient,
-  jobId: string,
-  batchData: any[]
-): Promise<string> {
-  try {
-    const batch = await client.sendBatch(jobId, {
-      action: 'UPSERT',
-      entityType: 'INVENTORY',
-      source: 'EXPIRATION_HANDLING',
-      event: 'INVENTORY_BATCH',
-      entities: batchData,
-    });
-
-    return batch.id;
-  } catch (error) {
-    if (error.code === 'JOB_EXPIRED' || error.message.includes('expired')) {
-      console.log('Job expired, creating new job...');
-
-      // Create new job
-      const newJob = await client.createJob({
-        name: `Recovery Job - ${Date.now()}`,
-        retailerId: process.env.FLUENT_RETAILER_ID!,
-        metadata: {
-          originalJobId: jobId,
-          recoveryReason: 'JOB_EXPIRED',
-        },
-      });
-
-      // Retry with new job
-      const batch = await client.sendBatch(newJob.id, {
-        action: 'UPSERT',
-        entityType: 'INVENTORY',
-        source: 'EXPIRATION_RECOVERY',
-        event: 'INVENTORY_BATCH_RECOVERY',
-        entities: batchData,
-      });
-
-      return batch.id;
-    }
-
-    throw error; // Re-throw if not expiration error
-  }
-}
-```
-
----
-
-## Partial Batch Recovery (NEW)
-
-Use PartialBatchRecovery to automatically analyze batch failures, separate retryable vs non-retryable records, and resend only what can succeed.
-
-### Capabilities
-
-- Analyze batch error responses
-- Identify retryable vs non-retryable records
-- Generate retry payloads and rejection reports
-- Support new-job or append-to-existing strategies
-
-### Example
-
-```typescript
-import { PartialBatchRecovery } from '@fluentcommerce/fc-connect-sdk';
-
-// After a batch completes with failures
-const status = await client.getBatchStatus(job.id, batch.id);
-
-if (status.status === 'FAILED' && Array.isArray(status.errors) && status.errors.length > 0) {
-  const recovery = new PartialBatchRecovery(logger);
-
-  // 'originalEntities' is the array you sent in this batch
-  const analysis = recovery.analyze(status.errors, originalEntities);
-
-  // Retry only retryable records
-  if (analysis.retryable.length > 0) {
-    const recoveryJob = await client.createJob({
-      name: `Recovery - ${job.id}`,
-      retailerId: process.env.FLUENT_RETAILER_ID!,
-      meta: { originalJobId: job.id, recoveryOfBatchId: batch.id },
-    });
-
-    await client.sendBatch(recoveryJob.id, {
-      action: 'UPSERT',
-      entityType: 'INVENTORY',
-      source: 'PARTIAL_BATCH_RECOVERY',
-      event: 'INVENTORY_RECOVERY',
-      entities: analysis.retryable,
-    });
-  }
-
-  // Persist non-retryable errors for investigation
-  if (analysis.nonRetryable.length > 0) {
-    await saveRejectionReport(job.id, batch.id, analysis.nonRetryable);
-  }
-}
-```
-
-Notes:
-
-- Typical retryable categories: transient network errors, rate limits, temporary validation states
-- Typical non-retryable categories: schema violations, permanently invalid data
-- Always store a rejection report for auditing and manual remediation
-
----
-
-## Comprehensive Error Handling Patterns
-
-### Pattern 1: Batch Processing with Rate Limiting
-
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-
-async function processBatchWithRateLimiting(
-  client: any,
-  jobId: string,
-  entities: any[],
-  logger: any
-) {
-  try {
-    const batch = await client.sendBatch(jobId, {
-      action: 'UPSERT',
-      entityType: 'INVENTORY',
-      source: 'BATCH_PROCESSING',
-      event: 'INVENTORY_BATCH',
-      entities
-    });
-
-    logger.info('Batch sent successfully', {
-      batchId: batch.id,
-      recordCount: entities.length
-    });
-
-    return batch.id;
-  } catch (error) {
-    logger.error('Batch submission failed', {
-      jobId,
-      recordCount: entities.length,
-      error: error.message,
-      errorCode: error.code
-    });
-
-    if (error.code === 'RATE_LIMIT_ERROR' || error.message.includes('rate limit')) {
-      logger.warn('Rate limit hit, implementing backoff', { jobId });
-
-      // Exponential backoff retry
-      await new Promise(resolve => setTimeout(resolve, 5000));
-
-      try {
-        const retryBatch = await client.sendBatch(jobId, {
-          action: 'UPSERT',
-          entityType: 'INVENTORY',
-          source: 'BATCH_RETRY',
-          event: 'INVENTORY_BATCH_RETRY',
-          entities
-        });
-
-        logger.info('Batch sent successfully after retry', {
-          batchId: retryBatch.id
-        });
-
-        return retryBatch.id;
-      } catch (retryError) {
-        logger.error('Batch failed after retry', {
-          jobId,
-          error: retryError.message
-        });
-        throw retryError;
-      }
-    } else if (error.code === 'VALIDATION_ERROR') {
-      logger.error('Batch validation failed - data does not match schema', {
-        jobId,
-        error: error.message
-      });
-      throw new Error('Validation error: Check data schema compatibility');
-    } else {
-      throw error;
-    }
-  }
-}
-```
-
-### Pattern 2: Network Error Recovery
-
-```typescript
-async function executeWithNetworkRetry<T>(
-  operation: () => Promise<T>,
-  operationName: string,
-  logger: any,
-  maxRetries: number = 3
-): Promise<T> {
-  let lastError: Error;
-
-  for (let attempt = 0; attempt < maxRetries; attempt++) {
-    try {
-      return await operation();
-    } catch (error) {
-      lastError = error;
-
-      logger.warn(`${operationName} failed`, {
-        attempt: attempt + 1,
-        maxRetries,
-        error: error.message
-      });
-
-      // Check if error is retryable
-      const isRetryable =
-        error.code === 'ECONNRESET' ||
-        error.code === 'ETIMEDOUT' ||
-        error.message.includes('network') ||
-        error.message.includes('timeout');
-
-      if (!isRetryable || attempt === maxRetries - 1) {
-        throw lastError;
-      }
-
-      // Exponential backoff
-      const delay = Math.pow(2, attempt) * 1000;
-      await new Promise(resolve => setTimeout(resolve, delay));
-    }
-  }
-
-  throw lastError!;
-}
-
-// Usage
-const batch = await executeWithNetworkRetry(
-  () => client.sendBatch(jobId, payload),
-  'sendBatch',
-  logger
-);
-```
-
----
-
-## Key Takeaways
-
-- 🎯 **Batch API is INVENTORY only** - No orders, products, or locations
-- 🎯 **Choose right job strategy** - DAILY for frequent updates, PER_FILE for large datasets
-- 🎯 **Optimize batch sizes** - Balance payload size and API limits
-- 🎯 **Handle job expiration** - Implement recovery for long-running processes
-- 🎯 **Monitor status properly** - Poll batch and job status with timeouts
-- 🎯 **Implement error recovery** - Retry transient errors, log permanent failures
-- 🎯 **Use structured logging** - Include context in all error logs
-
----
-
-## Next Steps
-
-Continue to [Module 7: State Management](./02-core-guides-ingestion-07-state-management.md) to learn how to prevent duplicate processing with state tracking.
-
----
-
-[← Previous: Field Mapping](02-core-guides-ingestion-04-field-mapping.md) | [Back to Guide](../ingestion-readme.md) | [Next: State Management →](./02-core-guides-ingestion-07-state-management.md)
-
-## Related Documentation
-
-- [Batch API Reference](../../api-reference/modules/api-reference-01-client-api.md#job-batch-operations) - Complete API documentation
-- [Job Strategies](02-core-guides-ingestion-08-performance-optimization.md#job-strategy-comparison) - Detailed strategy comparison
-- [Schema Validation](../../mapping/mapping-readme.md#04-REFERENCE/schema-validation) - Validate mappings
+# Module 6: Batch API Deep Dive
+
+[← Back to Ingestion Guide](../ingestion-readme.md)
+
+**Module 6 of 9** | **Level**: Intermediate | **Time**: 30 minutes
+
+---
+
+## Overview
+
+This module provides complete coverage of the Fluent Commerce Batch API, including job lifecycle, batch operations, status monitoring, and optimization strategies.
+
+## Learning Objectives
+
+By the end of this module, you will:
+
+- ✅ Understand the complete Batch API schema
+- ✅ Master the job creation and management lifecycle
+- ✅ Implement batch size optimization strategies
+- ✅ Monitor job and batch status effectively
+- ✅ Understand job strategies (DAILY vs PER_FILE)
+- ✅ Handle job expiration and recovery
+- ✅ Implement production-ready batch processing
+
+---
+
+## Batch API Schema
+
+### Complete GraphQL Schema
+
+The Batch API consists of two main mutations:
+
+#### 1. `createJob` Mutation
+
+```graphql
+mutation CreateJob($input: CreateJobInput!) {
+  createJob(input: $input) {
+    id # Job ID (String!)
+    name # Job name (String)
+    status # Job status (JobStatus enum)
+  }
+}
+```
+
+**Input Schema** (`CreateJobInput`):
+
+```typescript
+{
+  "name": "string",         // ✅ Job name (String! required)
+  "retailerId": "string",   // ✅ Retailer ID (ID! required)
+  "metadata": {             // ✅ JSON (optional) - Custom metadata
+    "source": "S3",
+    "file": "inventory-2025-01-19.csv"
+  }
+}
+```
+
+#### 2. `sendBatch` Mutation
+
+```graphql
+mutation SendBatch($input: BatchInput!) {
+  sendBatch(input: $input) {
+    id # Batch ID (String!)
+    status # Batch status (BatchStatus enum)
+  }
+}
+```
+
+**Input Schema** (`BatchInput`):
+
+```typescript
+{
+  "action": "UPSERT",               // ✅ String (UPSERT only - required)
+  "entityType": "INVENTORY",        // ✅ String (INVENTORY only - required)
+  "source": "S3_CSV_IMPORT",        // ✅ String (required) - Source system identifier
+  "event": "INVENTORY_UPDATE",      // ✅ String (required) - Event type identifier
+  "retailerId": "1",                // ✅ ID! (required)
+  "entities": [                     // ✅ [InventoryPositionInput]! (required array)
+    {
+      "ref": "SKU-WM-001",          // ✅ String! (required)
+      "productRef": "SKU-WM-001",   // ✅ String! (required)
+      "locationRef": "WH-001",      // ✅ String! (required)
+      "qty": 500,                   // ✅ Int! (required)
+      "status": "AVAILABLE"         // ✅ String (optional)
+    }
+  ]
+}
+```
+
+### `InventoryPositionInput` Complete Schema
+
+From `docs/schema/fluent-commerce-schema.json`:
+
+```json
+{
+  "InventoryPositionInput": {
+    "kind": "INPUT_OBJECT",
+    "description": "Input for inventory position in batch",
+    "fields": {
+      "ref": {
+        "type": "String!",
+        "description": "Position reference (Required)",
+        "businessContext": "Unique identifier for this inventory position, typically SKU-LOCATION combination",
+        "example": "SKU-WM-001"
+      },
+      "productRef": {
+        "type": "String!",
+        "description": "Product reference (Required)",
+        "businessContext": "Product SKU that this inventory position represents",
+        "example": "SKU-WM-001"
+      },
+      "locationRef": {
+        "type": "String!",
+        "description": "Location reference (Required)",
+        "businessContext": "Warehouse or location code where inventory is held",
+        "example": "WH-001"
+      },
+      "qty": {
+        "type": "Int!",
+        "description": "Quantity (Required)",
+        "businessContext": "Available quantity at this location",
+        "example": 500,
+        "validationRules": ["qty >= 0"]
+      },
+      "status": {
+        "type": "String",
+        "description": "Status (Optional)",
+        "businessContext": "Inventory position status",
+        "example": "AVAILABLE",
+        "validValues": ["AVAILABLE", "RESERVED", "ON_HOLD", "DISCONTINUED"]
+      },
+      "type": {
+        "type": "String",
+        "description": "Position type (Optional)",
+        "businessContext": "Type of inventory adjustment",
+        "example": "ADJUSTMENT"
+      }
+    }
+  }
+}
+```
+
+---
+
+## Complete Job Lifecycle
+
+### Lifecycle Diagram
+
+```mermaid
+graph TD
+    A[Create Job] --> B{Job Created?}
+    B -->|Yes| C[Job PENDING]
+    B -->|No| Z[ERROR]
+
+    C --> D[Send Batch 1]
+    D --> E[Batch PROCESSING]
+
+    C --> F[Send Batch 2]
+    F --> G[Batch PROCESSING]
+
+    E --> H{Batch Complete?}
+    G --> I{Batch Complete?}
+
+    H -->|Success| J[Batch COMPLETED]
+    H -->|Failure| K[Batch FAILED]
+
+    I -->|Success| L[Batch COMPLETED]
+    I -->|Failure| M[Batch FAILED]
+
+    J --> N{More Batches?}
+    L --> N
+
+    N -->|Yes| O[Send Next Batch]
+    N -->|No| P[All Batches Sent]
+
+    O --> E
+
+    P --> Q{Job Expired?}
+    Q -->|No| R[Job COMPLETED]
+    Q -->|Yes| S[Job EXPIRED - Create New Job]
+
+    K --> T[Handle Failed Batch]
+    M --> T
+    T --> U[Retry or Log Error]
+```
+
+### Step 1: Create Job
+
+**Simple job creation:**
+
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+
+const client = await createClient({ config });
+
+const job = await client.createJob({
+  name: 'Daily Inventory Sync',
+  retailerId: '1',
+});
+
+console.log(`Job created: ${job.id}`);
+// Output: Job created: JOB-12345
+```
+
+**Job creation with metadata (for tracking):**
+
+```typescript
+const job = await client.createJob({
+  name: `Inventory Import - ${new Date().toISOString()}`,
+  retailerId: process.env.FLUENT_RETAILER_ID!,
+  metadata: {
+    source: 'S3',
+    bucket: 'inventory-bucket',
+    file: 'data/inventory-2025-01-19.csv',
+    uploadedBy: 'integration-service',
+    triggeredBy: 'scheduled-workflow',
+  },
+});
+```
+
+### Step 2: Send Batches
+
+**Single batch:**
+
+```typescript
+const batch = await client.sendBatch(job.id, {
+  action: 'UPSERT',           // String, not enum
+  entityType: 'INVENTORY',    // String, not enum
+  source: 'S3_CSV_IMPORT',    // REQUIRED - Source system identifier
+  event: 'INVENTORY_UPDATE',  // REQUIRED - Event type identifier
+  entities: [
+    {
+      ref: 'SKU-WM-001',
+      productRef: 'SKU-WM-001',
+      locationRef: 'WH-001',
+      qty: 500,
+      status: 'AVAILABLE',
+    },
+  ],
+});
+
+console.log(`Batch sent: ${batch.id}`);
+```
+
+**Multiple batches (chunked):**
+
+```typescript
+function chunkArray<T>(array: T[], chunkSize: number): T[][] {
+  const chunks: T[][] = [];
+  for (let i = 0; i < array.length; i += chunkSize) {
+    chunks.push(array.slice(i, i + chunkSize));
+  }
+  return chunks;
+}
+
+// Split large dataset into batches
+const batchSize = 1000;
+const batches = chunkArray(inventoryData, batchSize);
+
+const batchResults = [];
+
+for (const [index, batchData] of batches.entries()) {
+  console.log(`Sending batch ${index + 1}/${batches.length}...`);
+
+  const batch = await client.sendBatch(job.id, {
+    action: 'UPSERT',
+    entityType: 'INVENTORY',
+    source: 'S3_CSV_IMPORT',
+    event: 'INVENTORY_UPDATE',
+    entities: batchData,
+  });
+
+  batchResults.push({
+    index,
+    batchId: batch.id,
+    records: batchData.length,
+  });
+}
+
+console.log(`Sent ${batchResults.length} batches with ${inventoryData.length} total records`);
+```
+
+### Step 3: Monitor Status
+
+**Poll batch status:**
+
+```typescript
+async function waitForBatchCompletion(
+  client: FluentClient,
+  jobId: string,
+  batchId: string,
+  pollInterval: number = 5000,
+  timeout: number = 300000
+): Promise<BatchStatus> {
+  const startTime = Date.now();
+
+  while (Date.now() - startTime < timeout) {
+    const status = await client.getBatchStatus(jobId, batchId);
+
+    console.log(`Batch ${batchId}: ${status.status}`);
+
+    if (status.status === 'COMPLETED') {
+      return status;
+    }
+
+    if (status.status === 'FAILED') {
+      throw new Error(`Batch failed: ${JSON.stringify(status.errors)}`);
+    }
+
+    // Wait before next poll
+    await new Promise(resolve => setTimeout(resolve, pollInterval));
+  }
+
+  throw new Error(`Batch monitoring timeout after ${timeout}ms`);
+}
+
+// Usage
+const status = await waitForBatchCompletion(client, job.id, batch.id);
+console.log(`✅ Batch completed successfully`);
+```
+
+**Monitor job status (all batches):**
+
+```typescript
+async function monitorJobProgress(
+  client: FluentClient,
+  jobId: string,
+  onProgress?: (status: JobStatus) => void
+): Promise<JobStatus> {
+  const pollInterval = 5000; // 5 seconds
+  const timeout = 600000; // 10 minutes
+
+  const startTime = Date.now();
+
+  while (Date.now() - startTime < timeout) {
+    const status = await client.getJobStatus(jobId);
+
+    if (onProgress) {
+      onProgress(status);
+    }
+
+    if (status.status === 'COMPLETED' || status.status === 'FAILED') {
+      return status;
+    }
+
+    await new Promise(resolve => setTimeout(resolve, pollInterval));
+  }
+
+  throw new Error(`Job monitoring timeout: ${jobId}`);
+}
+
+// Usage with progress callback
+const jobStatus = await monitorJobProgress(client, job.id, status => {
+  console.log(`Job ${job.id}: ${status.status} - ${status.progress || 0}% complete`);
+});
+```
+
+---
+
+## Batch Pre-Processing (BPP) - Change Detection
+
+### Overview
+
+**Batch Pre-Processing (BPP)** is Fluent Commerce's built-in change detection system that **filters out unchanged inventory records** before they reach the workflow engine, significantly improving performance when sending full inventory snapshots.
+
+### How BPP Works
+
+BPP introduces two internal jobs that run **before** your inventory updates hit the workflow:
+
+```
+┌─────────────────────────────────────────────────────┐
+│  WITHOUT BPP (meta.preprocessing: "skip")           │
+│  ────────────────────────────────────────────────   │
+│  Batch API → ALL records → Inventory Queue → Rubix │
+│  100K records → 100K workflow events                │
+└─────────────────────────────────────────────────────┘
+
+┌──────────────────────────────────────────────────────────┐
+│  WITH BPP (default)                                      │
+│  ─────────────────────────────────────────────────────   │
+│  Batch API → Loader Job → Comparison Job →               │
+│  → ONLY CHANGED records → Inventory Queue → Rubix        │
+│  100K records → 5K workflow events (95% filtered)         │
+└──────────────────────────────────────────────────────────┘
+```
+
+#### BPP Components
+
+**1. Loader Job**
+
+- Loads existing inventory data from database
+- Includes current quantities, statuses, and transient inventory
+
+**2. Comparison Job**
+
+- Compares batch records vs existing data
+- Checks for changes in:
+  - **Quantity**: Has qty changed?
+  - **Status**: Has status changed?
+  - **Transient Inventory**: Are there active transient quantities?
+
+**3. Result**
+
+- Only records marked as "changed" proceed to workflow
+- Unchanged records are filtered out (not processed)
+
+### When to Use BPP
+
+#### ✅ Use BPP (Default) When:
+
+**Scenario 1: Full Inventory Snapshots**
+
+```typescript
+// Sending complete inventory file daily
+const job = await client.createJob({
+  name: 'Daily Full Inventory Sync',
+  retailerId: '1',
+  // BPP enabled by default (no meta.preprocessing)
+});
+
+// 100K records sent, only ~5K changed overnight
+// Result: 5K workflow events instead of 100K
+```
+
+**Scenario 2: Unchanged Records in Data Feed**
+
+```typescript
+// WMS sends all SKU-location combinations daily
+// Many haven't changed since yesterday
+const job = await client.createJob({
+  name: 'WMS Daily Extract',
+  retailerId: '1',
+  meta: {
+    preprocessing: 'enabled', // Explicit BPP enable
+    source: 'WMS',
+  },
+});
+```
+
+**Benefits:**
+
+- ✅ Reduces workflow engine load by 80-95%
+- ✅ Faster processing (only changed records)
+- ✅ Automatic duplicate detection
+- ✅ Lower infrastructure costs
+
+#### ❌ Skip BPP When:
+
+**Scenario 1: Delta/Change-Only Feeds**
+
+```typescript
+// Your application already filtered changed records
+// Sending ONLY records that changed
+const job = await client.createJob({
+  name: 'Real-Time Inventory Delta',
+  retailerId: '1',
+  meta: {
+    preprocessing: 'skip', // ← Skip BPP (every record is already changed)
+  },
+});
+
+// 100 records sent, all 100 are changes
+// Result: 100 workflow events (no filtering needed)
+```
+
+**Scenario 2: Real-Time Updates**
+
+```typescript
+// High-frequency updates where every record matters
+// e.g., Point-of-sale inventory decrements
+const job = await client.createJob({
+  name: 'POS Inventory Update',
+  retailerId: '1',
+  meta: {
+    preprocessing: 'skip',
+    source: 'POS',
+    realtime: true,
+  },
+});
+```
+
+**Scenario 3: Performance-Critical Imports**
+
+```typescript
+// Need absolute fastest processing
+// Data is guaranteed to be different
+const job = await client.createJob({
+  name: 'Critical Inventory Adjustment',
+  retailerId: '1',
+  meta: {
+    preprocessing: 'skip',
+    priority: 'high',
+  },
+});
+```
+
+**Benefits:**
+
+- ✅ Faster (no comparison overhead)
+- ✅ No need to load existing data
+- ✅ Simpler processing flow
+
+### Account-Level BPP Configuration
+
+BPP behavior is controlled by **two settings**:
+
+#### 1. Account Setting: `fc.enable.batch.preprocessing`
+
+This is configured at the **Fluent Commerce account level** (contact Fluent support to check/modify):
+
+```typescript
+// Pseudo-code showing the logic
+if (account.settings.fc_enable_batch_preprocessing === false) {
+  // BPP is NEVER used, regardless of job meta
+  processingMode = 'DIRECT';
+} else {
+  // BPP is available, check job-level meta
+  if (job.meta.preprocessing === 'skip') {
+    processingMode = 'DIRECT';
+  } else {
+    processingMode = 'BPP';
+  }
+}
+```
+
+#### 2. Job-Level Setting: `meta.preprocessing`
+
+Set per job when creating:
+
+```typescript
+interface FluentJobMetadata {
+  /**
+   * Controls Batch Pre-Processing (BPP) change detection
+   *
+   * @remarks
+   * - `"skip"`: Bypass BPP (all records processed)
+   * - `"enabled"` or any other value: Use BPP (default)
+   * - Omitted/null: Use BPP (default)
+   *
+   * @important
+   * Only works if account setting `fc.enable.batch.preprocessing` is `true`
+   * If account setting is `false`, BPP is never used regardless of this value
+   */
+  preprocessing?: 'skip' | 'enabled' | string;
+
+  // Other metadata fields
+  source?: string;
+  file?: string;
+  [key: string]: any;
+}
+```
+
+### BPP Behavior Matrix
+
+| Account Setting (`fc.enable.batch.preprocessing`) | Job Meta (`preprocessing`) | Result                            |
+| ------------------------------------------------- | -------------------------- | --------------------------------- |
+| `true`                                            | `"skip"`                   | ❌ **No BPP** (Direct processing) |
+| `true`                                            | `"enabled"`                | ✅ **Use BPP** (Change detection) |
+| `true`                                            | `null` / omitted           | ✅ **Use BPP** (Default)          |
+| `false`                                           | ANY value                  | ❌ **No BPP** (Account override)  |
+
+### Complete Code Examples
+
+#### Example 1: Daily Full Snapshot (Use BPP)
+
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+
+const client = await createClient(ctx);
+
+// Daily full inventory file from WMS
+const job = await client.createJob({
+  name: 'Daily WMS Full Snapshot',
+  retailerId: '1',
+  // No meta.preprocessing = BPP enabled by default
+  meta: {
+    source: 'WMS',
+    feedType: 'FULL_SNAPSHOT',
+    date: new Date().toISOString(),
+  },
+});
+
+// Send 100K records (many unchanged)
+await client.sendBatch(job.id, {
+  action: 'UPSERT',
+  entityType: 'INVENTORY',
+  source: 'WMS_FULL_SNAPSHOT',
+  event: 'INVENTORY_SYNC',
+  entities: allInventoryRecords, // All 100K records
+});
+
+// Result: Only ~5K changed records hit workflow
+console.log('BPP filtered out ~95K unchanged records');
+```
+
+#### Example 2: Real-Time Deltas (Skip BPP)
+
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+
+const client = await createClient(ctx);
+
+// Only sending records that changed in last hour
+const job = await client.createJob({
+  name: 'Hourly Inventory Deltas',
+  retailerId: '1',
+  meta: {
+    preprocessing: 'skip', // ← Skip BPP (already filtered)
+    source: 'CDC', // Change Data Capture
+    feedType: 'DELTA',
+  },
+});
+
+// Send only changed records (already filtered by your system)
+await client.sendBatch(job.id, {
+  action: 'UPSERT',
+  entityType: 'INVENTORY',
+  source: 'CDC_DELTA_FEED',
+  event: 'INVENTORY_CHANGE',
+  entities: changedRecordsOnly, // Only 500 changed records
+});
+
+// Result: All 500 records hit workflow immediately
+console.log('All records processed (no BPP filtering)');
+```
+
+#### Example 3: Hybrid Approach
+
+```typescript
+// Different strategies for different times of day
+async function adaptiveInventorySync(client, records, isFullSync) {
+  const job = await client.createJob({
+    name: isFullSync ? 'Full Sync' : 'Delta Sync',
+    retailerId: '1',
+    meta: {
+      // Use BPP for full syncs, skip for deltas
+      preprocessing: isFullSync ? 'enabled' : 'skip',
+      source: 'Hybrid',
+      syncType: isFullSync ? 'FULL' : 'DELTA',
+    },
+  });
+
+  await client.sendBatch(job.id, {
+    action: 'UPSERT',
+    entityType: 'INVENTORY',
+    source: isFullSync ? 'HYBRID_FULL_SYNC' : 'HYBRID_DELTA_SYNC',
+    event: isFullSync ? 'INVENTORY_FULL_SYNC' : 'INVENTORY_DELTA_SYNC',
+    entities: records,
+  });
+
+  if (isFullSync) {
+    console.log('BPP filtering applied for full sync');
+  } else {
+    console.log('Direct processing for delta sync');
+  }
+}
+
+// Daily full sync at 2 AM
+await adaptiveInventorySync(client, allRecords, true);
+
+// Hourly deltas throughout the day
+await adaptiveInventorySync(client, changedRecords, false);
+```
+
+### Decision Guide: BPP or Skip?
+
+```
+START: What type of data are you sending?
+│
+├─ Full inventory snapshot (all SKUs/locations)
+│  └─ **Use BPP** (default - don't set meta.preprocessing)
+│     → Many unchanged records will be filtered
+│
+├─ Delta feed (only changed records)
+│  └─ **Skip BPP** (set meta.preprocessing: "skip")
+│     → All records are already changes
+│
+├─ Mixed/Unknown
+│  ├─ High percentage unchanged (>50%)
+│  │  └─ **Use BPP** → Better performance
+│  │
+│  └─ Most records changed (>80%)
+│     └─ **Skip BPP** → Faster (no comparison overhead)
+│
+└─ Real-time/urgent updates
+   └─ **Skip BPP** → Lowest latency
+```
+
+### Performance Comparison
+
+#### Scenario: 100K Inventory Records Sent Daily
+
+| Metric                    | With BPP (Default) | BPP Skip (`preprocessing: "skip"`) |
+| ------------------------- | ------------------ | ---------------------------------- |
+| Records sent to API       | 100K               | 100K                               |
+| Records actually changed  | 5K (5%)            | 100K (100% assumed)                |
+| Workflow events generated | 5K                 | 100K                               |
+| Processing time           | ~8 seconds         | ~45 seconds                        |
+| Workflow engine load      | Low                | High                               |
+| **Recommended for**       | Full snapshots     | Delta feeds only                   |
+
+### Common Mistakes
+
+#### ❌ Mistake 1: Using BPP for Delta Feeds
+
+```typescript
+// WRONG: Wasting resources on comparison
+const job = await client.createJob({
+  name: 'Delta Feed',
+  retailerId: '1',
+  // BPP enabled by default - unnecessary for delta!
+});
+
+await client.sendBatch(job.id, {
+  action: 'UPSERT',
+  entityType: 'INVENTORY',
+  source: 'DELTA_FEED',
+  event: 'INVENTORY_DELTA',
+  entities: onlyChangedRecords, // Already filtered!
+});
+// Result: BPP compares all records unnecessarily
+```
+
+**✅ Fix:**
+
+```typescript
+const job = await client.createJob({
+  name: 'Delta Feed',
+  retailerId: '1',
+  meta: {
+    preprocessing: 'skip', // ← Skip BPP for delta feeds
+  },
+});
+```
+
+#### ❌ Mistake 2: Skipping BPP for Full Snapshots
+
+```typescript
+// WRONG: Overwhelming workflow with unchanged records
+const job = await client.createJob({
+  name: 'Full Inventory',
+  retailerId: '1',
+  meta: {
+    preprocessing: 'skip', // ← Bad for full snapshots!
+  },
+});
+
+await client.sendBatch(job.id, {
+  action: 'UPSERT',
+  entityType: 'INVENTORY',
+  source: 'FULL_INVENTORY_SYNC',
+  event: 'INVENTORY_SNAPSHOT',
+  entities: allInventory, // 100K records, 95K unchanged
+});
+// Result: 100K workflow events (95K unnecessary)
+```
+
+**✅ Fix:**
+
+```typescript
+const job = await client.createJob({
+  name: 'Full Inventory',
+  retailerId: '1',
+  // BPP enabled by default - perfect!
+});
+// Result: Only 5K workflow events (95K filtered by BPP)
+```
+
+### Key Takeaways
+
+- 🎯 **BPP = Change Detection**, not data transformation
+- 🎯 **Use BPP** for full inventory snapshots (most common)
+- 🎯 **Skip BPP** for delta feeds (already filtered)
+- 🎯 **Account setting** takes precedence over job setting
+- 🎯 **Default is BPP enabled** (safest for most cases)
+
+---
+
+## Batch Size Optimization
+
+### Calculate Optimal Batch Size
+
+```typescript
+interface BatchSizeConfig {
+  maxBatchSize: number; // Maximum records per batch (default: 10000)
+  maxPayloadSize: number; // Maximum payload size in bytes (default: 10MB)
+  avgRecordSize?: number; // Average record size (auto-calculated if not provided)
+}
+
+function calculateOptimalBatchSize(
+  records: any[],
+  config: BatchSizeConfig = {
+    maxBatchSize: 10000,
+    maxPayloadSize: 10 * 1024 * 1024, // 10MB
+  }
+): number {
+  // Calculate average record size if not provided
+  const avgRecordSize = config.avgRecordSize || JSON.stringify(records.slice(0, 100)).length / 100;
+
+  // Calculate records per max payload
+  const recordsPerPayload = Math.floor(config.maxPayloadSize / avgRecordSize);
+
+  // Return minimum of calculated size and max batch size
+  const optimalSize = Math.min(recordsPerPayload, config.maxBatchSize);
+
+  console.log(`Optimal batch size: ${optimalSize} (based on ${avgRecordSize} bytes/record)`);
+
+  return optimalSize;
+}
+
+// Usage
+const batchSize = calculateOptimalBatchSize(inventoryData);
+const batches = chunkArray(inventoryData, batchSize);
+```
+
+### Adaptive Batch Sizing
+
+```typescript
+class AdaptiveBatchProcessor {
+  private currentBatchSize: number;
+  private readonly minBatchSize = 100;
+  private readonly maxBatchSize = 10000;
+
+  constructor(initialBatchSize: number = 1000) {
+    this.currentBatchSize = initialBatchSize;
+  }
+
+  async processBatch(client: FluentClient, jobId: string, data: any[]): Promise<void> {
+    const batches = chunkArray(data, this.currentBatchSize);
+
+    for (const batchData of batches) {
+      const startTime = Date.now();
+
+      try {
+        const batch = await client.sendBatch(jobId, {
+          action: 'UPSERT',
+          entityType: 'INVENTORY',
+          source: 'ADAPTIVE_BATCH',
+          event: 'INVENTORY_BATCH_IMPORT',
+          entities: batchData,
+        });
+
+        const processingTime = Date.now() - startTime;
+
+        // Adjust batch size based on performance
+        this.adjustBatchSize(processingTime, batchData.length);
+      } catch (error) {
+        // Reduce batch size on error
+        this.currentBatchSize = Math.max(this.minBatchSize, Math.floor(this.currentBatchSize / 2));
+        throw error;
+      }
+    }
+  }
+
+  private adjustBatchSize(processingTime: number, recordCount: number): void {
+    const avgTimePerRecord = processingTime / recordCount;
+
+    // If processing is fast, increase batch size
+    if (avgTimePerRecord < 10 && this.currentBatchSize < this.maxBatchSize) {
+      this.currentBatchSize = Math.min(this.maxBatchSize, Math.floor(this.currentBatchSize * 1.2));
+    }
+
+    // If processing is slow, decrease batch size
+    if (avgTimePerRecord > 50 && this.currentBatchSize > this.minBatchSize) {
+      this.currentBatchSize = Math.max(this.minBatchSize, Math.floor(this.currentBatchSize * 0.8));
+    }
+  }
+}
+```
+
+---
+
+## Job Strategies
+
+### Strategy 1: DAILY (Recommended for frequent updates)
+
+**Use when:**
+
+- Multiple files processed per day
+- Want to group all updates under one job
+- Need centralized tracking
+
+**Pattern:**
+
+```typescript
+import { FluentClient, StateService, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+
+async function dailyJobStrategy(
+  client: FluentClient,
+  files: string[],
+  kvAdapter: VersoriKVAdapter,
+  state: StateService
+): Promise<void> {
+  const workflowId = 'daily-inventory-sync';
+
+  // Get or create today's job using StateService
+  let job = await state.getDailyJob(kvAdapter, workflowId);
+
+  if (!job) {
+    const dateKey = new Date().toISOString().split('T')[0];
+    const newJob = await client.createJob({
+      name: `Daily Inventory Sync - ${dateKey}`,
+      retailerId: process.env.FLUENT_RETAILER_ID!,
+      meta: { date: dateKey, type: 'daily' },
+    });
+
+    // Store job reference with 24-hour expiration
+    await state.setDailyJob(kvAdapter, workflowId, newJob.id, 24);
+    job = { jobId: newJob.id };
+  }
+
+  // Process all files with same job
+  for (const file of files) {
+    const data = await processFile(file);
+
+    await client.sendBatch(job.jobId, {
+      action: 'UPSERT',
+      entityType: 'INVENTORY',
+      source: 'DAILY_JOB_STRATEGY',
+      event: 'INVENTORY_DAILY_SYNC',
+      entities: data,
+    });
+  }
+}
+```
+
+**Benefits:**
+
+- ✅ Single job for all daily updates
+- ✅ Easier monitoring and reporting
+- ✅ Reduced job creation overhead
+
+**Drawbacks:**
+
+- ❌ Job expiration risk if processing takes too long
+- ❌ All batches fail if job fails
+
+### Strategy 2: PER_FILE (Recommended for large datasets)
+
+**Use when:**
+
+- Processing large files (50k+ records)
+- Want isolated failure handling
+- Need per-file tracking
+
+**Pattern:**
+
+```typescript
+async function perFileJobStrategy(client: FluentClient, files: string[]): Promise<void> {
+  for (const file of files) {
+    // Create new job for each file
+    const job = await client.createJob({
+      name: `Import - ${file}`,
+      retailerId: process.env.FLUENT_RETAILER_ID!,
+      metadata: { file, type: 'per-file' },
+    });
+
+    const data = await processFile(file);
+    const batches = chunkArray(data, 1000);
+
+    for (const batchData of batches) {
+      await client.sendBatch(job.id, {
+        action: 'UPSERT',
+        entityType: 'INVENTORY',
+        source: 'PER_FILE_STRATEGY',
+        event: 'INVENTORY_FILE_IMPORT',
+        entities: batchData,
+      });
+    }
+  }
+}
+```
+
+**Benefits:**
+
+- ✅ Isolated failure handling
+- ✅ No job expiration risk
+- ✅ Clear per-file tracking
+
+**Drawbacks:**
+
+- ❌ More jobs created (overhead)
+- ❌ Harder to aggregate metrics
+
+### Strategy 3: BATCHES_PER_JOB (Advanced)
+
+**Use when:**
+
+- Need balance between DAILY and PER_FILE
+- Want to limit batches per job
+
+**Pattern:**
+
+```typescript
+async function batchesPerJobStrategy(
+  client: FluentClient,
+  data: any[],
+  maxBatchesPerJob: number = 10
+): Promise<void> {
+  const batchSize = 1000;
+  const batches = chunkArray(data, batchSize);
+
+  let currentJob: any = null;
+  let batchCount = 0;
+
+  for (const batchData of batches) {
+    // Create new job if needed
+    if (!currentJob || batchCount >= maxBatchesPerJob) {
+      currentJob = await client.createJob({
+        name: `Inventory Import - ${Date.now()}`,
+        retailerId: process.env.FLUENT_RETAILER_ID!,
+      });
+      batchCount = 0;
+    }
+
+    // Send batch
+    await client.sendBatch(currentJob.id, {
+      action: 'UPSERT',
+      entityType: 'INVENTORY',
+      source: 'BATCHES_PER_JOB',
+      event: 'INVENTORY_BATCH_CHUNK',
+      entities: batchData,
+    });
+
+    batchCount++;
+  }
+}
+```
+
+---
+
+## Job Expiration Handling
+
+Jobs expire after a certain time (typically 24 hours). Handle expiration gracefully:
+
+```typescript
+async function sendBatchWithExpirationHandling(
+  client: FluentClient,
+  jobId: string,
+  batchData: any[]
+): Promise<string> {
+  try {
+    const batch = await client.sendBatch(jobId, {
+      action: 'UPSERT',
+      entityType: 'INVENTORY',
+      source: 'EXPIRATION_HANDLING',
+      event: 'INVENTORY_BATCH',
+      entities: batchData,
+    });
+
+    return batch.id;
+  } catch (error) {
+    if (error.code === 'JOB_EXPIRED' || error.message.includes('expired')) {
+      console.log('Job expired, creating new job...');
+
+      // Create new job
+      const newJob = await client.createJob({
+        name: `Recovery Job - ${Date.now()}`,
+        retailerId: process.env.FLUENT_RETAILER_ID!,
+        metadata: {
+          originalJobId: jobId,
+          recoveryReason: 'JOB_EXPIRED',
+        },
+      });
+
+      // Retry with new job
+      const batch = await client.sendBatch(newJob.id, {
+        action: 'UPSERT',
+        entityType: 'INVENTORY',
+        source: 'EXPIRATION_RECOVERY',
+        event: 'INVENTORY_BATCH_RECOVERY',
+        entities: batchData,
+      });
+
+      return batch.id;
+    }
+
+    throw error; // Re-throw if not expiration error
+  }
+}
+```
+
+---
+
+## Partial Batch Recovery (NEW)
+
+Use PartialBatchRecovery to automatically analyze batch failures, separate retryable vs non-retryable records, and resend only what can succeed.
+
+### Capabilities
+
+- Analyze batch error responses
+- Identify retryable vs non-retryable records
+- Generate retry payloads and rejection reports
+- Support new-job or append-to-existing strategies
+
+### Example
+
+```typescript
+import { PartialBatchRecovery } from '@fluentcommerce/fc-connect-sdk';
+
+// After a batch completes with failures
+const status = await client.getBatchStatus(job.id, batch.id);
+
+if (status.status === 'FAILED' && Array.isArray(status.errors) && status.errors.length > 0) {
+  const recovery = new PartialBatchRecovery(logger);
+
+  // 'originalEntities' is the array you sent in this batch
+  const analysis = recovery.analyze(status.errors, originalEntities);
+
+  // Retry only retryable records
+  if (analysis.retryable.length > 0) {
+    const recoveryJob = await client.createJob({
+      name: `Recovery - ${job.id}`,
+      retailerId: process.env.FLUENT_RETAILER_ID!,
+      meta: { originalJobId: job.id, recoveryOfBatchId: batch.id },
+    });
+
+    await client.sendBatch(recoveryJob.id, {
+      action: 'UPSERT',
+      entityType: 'INVENTORY',
+      source: 'PARTIAL_BATCH_RECOVERY',
+      event: 'INVENTORY_RECOVERY',
+      entities: analysis.retryable,
+    });
+  }
+
+  // Persist non-retryable errors for investigation
+  if (analysis.nonRetryable.length > 0) {
+    await saveRejectionReport(job.id, batch.id, analysis.nonRetryable);
+  }
+}
+```
+
+Notes:
+
+- Typical retryable categories: transient network errors, rate limits, temporary validation states
+- Typical non-retryable categories: schema violations, permanently invalid data
+- Always store a rejection report for auditing and manual remediation
+
+---
+
+## Comprehensive Error Handling Patterns
+
+### Pattern 1: Batch Processing with Rate Limiting
+
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+
+async function processBatchWithRateLimiting(
+  client: any,
+  jobId: string,
+  entities: any[],
+  logger: any
+) {
+  try {
+    const batch = await client.sendBatch(jobId, {
+      action: 'UPSERT',
+      entityType: 'INVENTORY',
+      source: 'BATCH_PROCESSING',
+      event: 'INVENTORY_BATCH',
+      entities
+    });
+
+    logger.info('Batch sent successfully', {
+      batchId: batch.id,
+      recordCount: entities.length
+    });
+
+    return batch.id;
+  } catch (error) {
+    logger.error('Batch submission failed', {
+      jobId,
+      recordCount: entities.length,
+      error: error.message,
+      errorCode: error.code
+    });
+
+    if (error.code === 'RATE_LIMIT_ERROR' || error.message.includes('rate limit')) {
+      logger.warn('Rate limit hit, implementing backoff', { jobId });
+
+      // Exponential backoff retry
+      await new Promise(resolve => setTimeout(resolve, 5000));
+
+      try {
+        const retryBatch = await client.sendBatch(jobId, {
+          action: 'UPSERT',
+          entityType: 'INVENTORY',
+          source: 'BATCH_RETRY',
+          event: 'INVENTORY_BATCH_RETRY',
+          entities
+        });
+
+        logger.info('Batch sent successfully after retry', {
+          batchId: retryBatch.id
+        });
+
+        return retryBatch.id;
+      } catch (retryError) {
+        logger.error('Batch failed after retry', {
+          jobId,
+          error: retryError.message
+        });
+        throw retryError;
+      }
+    } else if (error.code === 'VALIDATION_ERROR') {
+      logger.error('Batch validation failed - data does not match schema', {
+        jobId,
+        error: error.message
+      });
+      throw new Error('Validation error: Check data schema compatibility');
+    } else {
+      throw error;
+    }
+  }
+}
+```
+
+### Pattern 2: Network Error Recovery
+
+```typescript
+async function executeWithNetworkRetry<T>(
+  operation: () => Promise<T>,
+  operationName: string,
+  logger: any,
+  maxRetries: number = 3
+): Promise<T> {
+  let lastError: Error;
+
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      return await operation();
+    } catch (error) {
+      lastError = error;
+
+      logger.warn(`${operationName} failed`, {
+        attempt: attempt + 1,
+        maxRetries,
+        error: error.message
+      });
+
+      // Check if error is retryable
+      const isRetryable =
+        error.code === 'ECONNRESET' ||
+        error.code === 'ETIMEDOUT' ||
+        error.message.includes('network') ||
+        error.message.includes('timeout');
+
+      if (!isRetryable || attempt === maxRetries - 1) {
+        throw lastError;
+      }
+
+      // Exponential backoff
+      const delay = Math.pow(2, attempt) * 1000;
+      await new Promise(resolve => setTimeout(resolve, delay));
+    }
+  }
+
+  throw lastError!;
+}
+
+// Usage
+const batch = await executeWithNetworkRetry(
+  () => client.sendBatch(jobId, payload),
+  'sendBatch',
+  logger
+);
+```
+
+---
+
+## Key Takeaways
+
+- 🎯 **Batch API is INVENTORY only** - No orders, products, or locations
+- 🎯 **Choose right job strategy** - DAILY for frequent updates, PER_FILE for large datasets
+- 🎯 **Optimize batch sizes** - Balance payload size and API limits
+- 🎯 **Handle job expiration** - Implement recovery for long-running processes
+- 🎯 **Monitor status properly** - Poll batch and job status with timeouts
+- 🎯 **Implement error recovery** - Retry transient errors, log permanent failures
+- 🎯 **Use structured logging** - Include context in all error logs
+
+---
+
+## Next Steps
+
+Continue to [Module 7: State Management](./02-core-guides-ingestion-07-state-management.md) to learn how to prevent duplicate processing with state tracking.
+
+---
+
+[← Previous: Field Mapping](02-core-guides-ingestion-04-field-mapping.md) | [Back to Guide](../ingestion-readme.md) | [Next: State Management →](./02-core-guides-ingestion-07-state-management.md)
+
+## Related Documentation
+
+- [Batch API Reference](../../api-reference/modules/api-reference-01-client-api.md#job-batch-operations) - Complete API documentation
+- [Job Strategies](02-core-guides-ingestion-08-performance-optimization.md#job-strategy-comparison) - Detailed strategy comparison
+- [Schema Validation](../../mapping/mapping-readme.md#04-REFERENCE/schema-validation) - Validate mappings