@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-02-batch-processing.md

@@ -1,1547 +1,1547 @@
-# Module 2: Batch Processing
-
-> **Learning Objective:** Master batch processing patterns for high-volume data synchronization using the Fluent Batch API and SDK orchestration services.
->
-> **Level:** Intermediate
-
-## Table of Contents
-
-1. [What is Batch Processing?](#what-is-batch-processing)
-2. [When to Use Batch Processing](#when-to-use-batch-processing)
-3. [Fluent Batch API Overview](#fluent-batch-api-overview)
-4. [SDK Batch Components](#sdk-batch-components)
-5. [Basic Batch Workflow](#basic-batch-workflow)
-6. [Job Creation Strategies](#job-creation-strategies)
-7. [Batch Size Optimization](#batch-size-optimization)
-8. [Status Polling and Completion](#status-polling-and-completion)
-9. [Error Handling in Batches](#error-handling-in-batches)
-10. [Complete Implementation Example](#complete-implementation-example)
-11. [Next Steps](#next-steps)
-
----
-
-## What is Batch Processing?
-
-**Batch processing** means grouping multiple records together and processing them as a single unit, optimized for throughput over latency.
-
-### Key Characteristics
-
-| Characteristic | Description | Example |
-|----------------|-------------|---------|
-| **High Volume** | Process thousands of records | 50,000 inventory positions |
-| **Scheduled** | Runs at specific times | Daily at 2 AM |
-| **Asynchronous** | Submit job, poll for completion | Job completes in 5-10 minutes |
-| **Bulk Operations** | Optimize for throughput | 100 records per API call |
-
-### How It Works
-
-```
-CSV File (50K records) → Parse → Transform → Batch API → Job → Poll Status → Complete
-     ↓                      ↓         ↓            ↓         ↓         ↓           ↓
-  S3 bucket          Parse chunks  Map fields  Create job  Submit  Check every  Archive
-                                                           batches  30 seconds    file
-```
-
-### Visual Flow
-
-```
-┌─────────────────┐
-│ S3 CSV File     │
-│ (50,000 records)│
-└────────┬────────┘
-         │
-         │ ①Download & Parse
-         ▼
-┌─────────────────┐
-│   CSVParserService     │
-│  Stream chunks  │
-└────────┬────────┘
-         │
-         │ ②Transform fields
-         ▼
-┌─────────────────┐
-│ UniversalMapper │
-│  Field mapping  │
-└────────┬────────┘
-         │
-         │ ③Create Batch job
-         ▼
-┌─────────────────┐
-│ FluentClient    │
-│  createJob()    │
-└────────┬────────┘
-         │
-         │ ④Send batches (100 records each)
-         ▼
-┌─────────────────┐
-│  sendBatch()    │ ────┐
-│  500 batches    │     │ Parallel processing
-│  of 100 records │ ────┤ on Fluent servers
-└────────┬────────┘     │
-         │              │
-         │ ⑤Poll status every 30s
-         ▼
-┌─────────────────┐
-│  getJobStatus() │
-│  PROCESSING...  │
-│  COMPLETED ✓    │
-└────────┬────────┘
-         │
-         │ ⑥Archive file
-         ▼
-┌─────────────────┐
-│ S3 Archive      │
-│ Success log     │
-└─────────────────┘
-```
-
-**Total time**: 5-10 minutes for 50,000 records (vs 2-3 hours for sequential GraphQL mutations).
-
----
-
-## When to Use Batch Processing
-
-### ✅ Use Batch Processing When:
-
-| Scenario | Why Batch? | SDK Pattern |
-|----------|------------|-------------|
-| **Large Files** | 1K+ records | Batch API with streaming |
-| **Daily Sync** | Full inventory sync | Scheduled batch job |
-| **High Volume** | > 1,000 events/hour | Batch API (not webhooks) |
-| **Bulk Updates** | Mass price changes | Single job, multiple batches |
-| **CSV/Parquet Files** | Structured file formats | S3DataSource + FluentClient |
-
-### ❌ Avoid Batch Processing When:
-
-| Scenario | Why Not Batch? | Use Instead |
-|----------|----------------|-------------|
-| **Immediate Updates** | Need < 5 second latency | Real-time (Module 1) |
-| **Single Records** | 1-10 records | Direct GraphQL mutation |
-| **Event-Driven** | External webhook triggers | Real-time webhook |
-| **Critical Orders** | Customer waiting for confirmation | Real-time processing |
-
----
-
-## Fluent Batch API Overview
-
-### What is Batch API?
-
-The Fluent Batch API is a specialized GraphQL endpoint for bulk data operations:
-
-- **Asynchronous**: Submit job, get ID, poll for completion
-- **High-throughput**: 10,000+ records in minutes
-- **Fault-tolerant**: Partial failures don't block entire job
-- **Entity-specific**: Currently supports `InventoryQuantity` only
-
-### Supported Operations
-
-| Entity | Operations | Max per Batch |
-|--------|-----------|---------------|
-| `InventoryQuantity` | Create, Update | 100-250 records |
-
-**IMPORTANT**: Batch API currently **only supports InventoryQuantity**. For other entities (Order, Product, Customer), use standard GraphQL mutations.
-
-### Batch API Workflow
-
-```graphql
-# Step 1: Create job
-mutation CreateJob {
-  createJob(input: {
-    name: "Daily Inventory Sync"
-    retailerId: "2"
-  }) {
-    id
-    status
-  }
-}
-
-# Step 2: Send batches
-mutation SendBatch($jobId: ID!, $entities: [InventoryQuantityInput!]!) {
-  sendBatch(jobId: $jobId, entities: $entities) {
-    id
-    status
-    recordCount
-  }
-}
-
-# Step 3: Poll status
-query GetJobStatus($jobId: ID!) {
-  job(id: $jobId) {
-    id
-    status  # PENDING, PROCESSING, COMPLETED, FAILED
-    totalBatches
-    completedBatches
-    errorSummary {
-      totalErrors
-      errorTypes
-    }
-  }
-}
-```
-
-### Job Lifecycle
-
-```
-CREATE_JOB → PENDING → SEND_BATCHES → PROCESSING → COMPLETED
-                ↓                                      ↓
-             (can add more batches)            (can check errors)
-```
-
----
-
-## SDK Batch Components
-
-### Component 1: FluentClient Batch Methods
-
-The SDK provides batch methods directly on `FluentClient` (there is no separate `FluentBatchManager` class):
-
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-
-const client = await createClient({
-  config: {
-    baseUrl: 'https://api.fluentcommerce.com',
-    clientId: process.env.FLUENT_CLIENT_ID,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET,
-    retailerId: process.env.FLUENT_RETAILER_ID
-  }
-});
-
-// Create job
-const job = await client.createJob({
-  name: 'Daily Inventory Sync',
-  retailerId: '2'
-});
-
-// Send batch
-const batch = await client.sendBatch(job.id, {
-  entities: inventoryRecords
-});
-
-// Get status
-const status = await client.getJobStatus(job.id);
-
-// Get detailed status with batches
-const jobDetail = await client.getBatchStatus(job.id, batch.id);
-```
-
-**Available methods**:
-- `createJob(input)` - Create new Batch job
-- `sendBatch(jobId, data)` - Submit batch of records
-- `getJobStatus(jobId)` - Get job status and summary
-- `getBatchStatus(jobId, batchId)` - Get individual batch details
-
-### Component 2: `S3DataSource`
-
-**Purpose**: Read and write files from S3 with streaming support
-
-```typescript
-import { S3DataSource } from '@fluentcommerce/fc-connect-sdk';
-
-const s3 = new S3DataSource({
-  type: 'S3_CSV',
-  connectionId: 'my-s3',
-  name: 'My S3 Source',
-  s3Config: {
-    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
-    region: 'us-east-1',
-    bucket: 'inventory-bucket'
-  }
-}, logger);
-
-// List files
-const files = await s3.listFiles('inventory/updates/');
-
-// Download file
-const fileContent = await s3.downloadFile('inventory/updates/inventory.csv');
-
-// Download large file as Buffer (for files >100MB)
-const buffer = await s3.downloadFile('inventory/updates/large-file.csv', { encoding: 'binary' });
-
-// Upload file
-await s3.uploadFile('inventory/archive/processed.csv', content);
-
-// Move file
-await s3.moveFile('inventory/updates/file.csv', 'inventory/archive/file.csv');
-```
-
-**Key features**:
-- Streaming support for large files (memory-efficient)
-- Automatic retry on network errors
-- Progress callbacks for uploads/downloads
-
-### Component 3: `CSVParserService`
-
-**Purpose**: Parse CSV files with validation and streaming
-
-```typescript
-import { CSVParserService } from '@fluentcommerce/fc-connect-sdk';
-
-const parser = new CSVParserService({
-  headers: true,           // First row is headers
-  skipEmptyLines: true,    // Ignore blank lines
-  delimiter: ',',          // CSV delimiter
-  quote: '"',              // Quote character
-  escape: '\\'             // Escape character
-});
-
-// Parse entire file (for small files < 10MB)
-const records = await parser.parse(csvContent);
-
-// Stream parse (for large files)
-const recordStream = parser.streamParse(csvStream);
-for await (const record of recordStream) {
-  // Process record
-}
-```
-
-**Validation options**:
-- Required columns
-- Type validation
-- Custom validators
-
-### Component 4: `UniversalMapper`
-
-**Purpose**: Transform CSV/JSON fields to Fluent schema
-
-```typescript
-import { UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
-
-const mappingConfig = {
-  fields: {
-    ref: {
-      source: 'sku',
-      resolver: 'custom.buildRef'  // Combine sku + location
-    },
-    type: {
-      value: 'INVENTORY'  // Static value
-    },
-    status: {
-      source: 'status',
-      resolver: 'sdk.uppercase'
-    },
-    productRef: {
-      source: 'sku',
-      required: true
-    },
-    locationRef: {
-      source: 'warehouse_code',
-      required: true
-    },
-    onHand: {
-      source: 'quantity',
-      resolver: 'sdk.parseInt'
-    }
-  }
-};
-
-const mapper = new UniversalMapper(mappingConfig, {
-  customResolvers: {
-    'custom.buildRef': (value, data) => {
-      return `${data.sku}-${data.warehouse_code}`;
-    }
-  }
-});
-
-const result = await mapper.map(csvRecord);
-// result.data = { ref: 'SKU001-WH01', type: 'INVENTORY', ... }
-```
-
-### Workflow Composition Pattern
-
-**Purpose**: Compose SDK services for complete ingestion workflows
-
-Instead of using a single orchestrator, compose the above components into your custom workflow:
-
-```typescript
-import {
-  createClient,
-  S3DataSource,
-  CSVParserService,
-  UniversalMapper,
-  StateService,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-
-async function processInventoryFiles() {
-  const logger = toStructuredLogger(createConsoleLogger(), { logLevel: 'info' });
-
-  // Initialize components
-  const client = await createClient({ config });
-  const s3 = new S3DataSource(s3Config, logger);
-  const parser = new CSVParserService();
-  const mapper = new UniversalMapper(mappingConfig);
-  const stateService = new StateService(logger);
-
-  // List and process files
-  const files = await s3.listFiles({ prefix: 'inventory/updates/' });
-
-  for (const file of files) {
-    if (await stateService.isFileProcessed(file.name)) continue;
-
-    try {
-      // 1. Download and parse
-      const content = await s3.downloadFile(file.name);
-      const records = await parser.parse(content);
-
-      // 2. Map fields
-      const inventory = [];
-      for (const record of records) {
-        const result = await mapper.map(record);
-        if (result.success) inventory.push(result.data);
-      }
-
-      // 3. Create job and send batches
-      const job = await client.createJob({
-        name: `Inventory - ${file.name}`,
-        retailerId: '1'
-      });
-
-      const batches = chunkArray(inventory, 100);
-      for (const batch of batches) {
-        await client.sendBatch(job.id, {
-          action: 'UPSERT',
-          entityType: 'INVENTORY',
-          entities: batch
-        });
-      }
-
-      // 4. Archive and mark processed
-      await s3.moveFile(file.name, `inventory/archive/${file.name}`);
-      await stateService.markFileProcessed(file.name);
-
-    } catch (error) {
-      logger.error(`Failed to process ${file.name}`, error);
-      await s3.moveFile(file.name, `inventory/errors/${file.name}`);
-    }
-  }
-}
-```
-
-**Benefits of building block composition**:
-1. ✅ Full control over workflow logic
-2. ✅ Custom error handling
-3. ✅ Easy to test individual components
-4. ✅ Flexible batch sizing and job strategies
-5. ✅ Works with any entity type (not just INVENTORY)
-6. ✅ Easy to add custom business rules
-
----
-
-## Basic Batch Workflow
-
-### Step-by-Step Pattern
-
-```typescript
-/**
- * Basic Batch Processing Workflow
- *
- * Steps:
- * 1. Create Fluent client
- * 2. Download and parse CSV file
- * 3. Transform records to Fluent schema
- * 4. Create Batch job
- * 5. Send records in batches
- * 6. Poll status until complete
- * 7. Handle results
- */
-
-import { createClient, S3DataSource, CSVParserService, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
-
-async function batchInventorySync() {
-  // Step 1: Create client
-  const client = await createClient({
-    baseUrl: process.env.FLUENT_BASE_URL,
-    clientId: process.env.FLUENT_CLIENT_ID,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET,
-    retailerId: process.env.FLUENT_RETAILER_ID
-  });
-
-  // Step 2: Download CSV
-  const s3 = new S3DataSource({
-    type: 'S3_CSV',
-    connectionId: 'my-s3',
-    name: 'My S3 Source',
-    s3Config: {
-      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
-      region: process.env.AWS_REGION,
-      bucket: process.env.AWS_BUCKET
-    }
-  }, console);
-
-  const csvContent = await s3.downloadFile('inventory/daily-update.csv');
-
-  // Step 3: Parse CSV
-  const parser = new CSVParserService({ headers: true });
-  const records = await parser.parse(csvContent);
-
-  console.log(`Parsed ${records.length} records`);
-
-  // Step 4: Transform records
-  const mapper = new UniversalMapper({
-    fields: {
-      ref: { source: 'sku', resolver: 'custom.buildRef' },
-      type: { value: 'INVENTORY' },
-      productRef: { source: 'sku', required: true },
-      locationRef: { source: 'location', required: true },
-      onHand: { source: 'qty', resolver: 'sdk.parseInt' }
-    }
-  }, {
-    customResolvers: {
-      'custom.buildRef': (value, data) => `${data.sku}-${data.location}`
-    }
-  });
-
-  const mappedRecords = [];
-  for (const record of records) {
-    const result = await mapper.map(record);
-    if (result.success) {
-      mappedRecords.push(result.data);
-    } else {
-      console.error('Mapping error:', result.errors);
-    }
-  }
-
-  console.log(`Mapped ${mappedRecords.length} records`);
-
-  // Step 5: Create Batch job
-  const job = await client.createJob({
-    name: `Inventory Sync ${new Date().toISOString()}`,
-    retailerId: '2'
-  });
-
-  console.log(`Created job ${job.id}`);
-
-  // Step 6: Send batches
-  const BATCH_SIZE = 100;
-  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
-    const chunk = mappedRecords.slice(i, i + BATCH_SIZE);
-
-    const batch = await client.sendBatch(job.id, {
-      entities: chunk
-    });
-
-    console.log(`Sent batch ${batch.id} (${chunk.length} records)`);
-  }
-
-  // Step 7: Poll status
-  let status = await client.getJobStatus(job.id);
-  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-    console.log(`Job status: ${status.status} (${status.completedBatches}/${status.totalBatches} batches)`);
-
-    await new Promise(resolve => setTimeout(resolve, 30000)); // Wait 30 seconds
-
-    status = await client.getJobStatus(job.id);
-  }
-
-  // Step 8: Handle results
-  if (status.status === 'COMPLETED') {
-    console.log('✓ Job completed successfully');
-    console.log(`Total records: ${status.totalRecords}`);
-    console.log(`Errors: ${status.errorSummary?.totalErrors || 0}`);
-
-    // Archive file
-    await s3.moveFile('inventory/daily-update.csv', 'inventory/archive/daily-update.csv');
-  } else {
-    console.error('✗ Job failed:', status.status);
-
-    // Move to error folder
-    await s3.moveFile('inventory/daily-update.csv', 'inventory/errors/daily-update.csv');
-  }
-}
-
-batchInventorySync().catch(console.error);
-```
-
----
-
-## NEW: Job Lifecycle Tracking (v0.1.10+)
-
-**Track job state, metadata, and lifecycle** across your integration workflows.
-
-The SDK provides `JobTracker` service for managing job lifecycle, tracking status, and storing job metadata.
-
-### JobTracker Overview
-
-```typescript
-import { JobTracker, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-// ✅ CORRECT: Access openKv from Versori context
-// import { openKv } from '@versori/run'; // ❌ WRONG - Not a direct export
-
-// In Versori workflow handler:
-const { openKv } = ctx;
-const kvAdapter = new VersoriKVAdapter(openKv(':project:'));
-const tracker = new JobTracker(kvAdapter, logger);
-
-// Create job
-const jobId = `scheduled_${Date.now()}`;
-
-await tracker.createJob(jobId, {
-  triggeredBy: 'schedule',
-  stage: 'initialization',
-  details: {
-    catalogueRef: 'DEFAULT:1',
-    fileName: 'inventory.csv'
-  }
-});
-
-// Update progress
-await tracker.updateJob(jobId, {
-  status: 'processing',
-  stage: 'extraction',
-  message: 'Extracting records from S3'
-});
-
-await tracker.updateJob(jobId, {
-  stage: 'transformation',
-  message: 'Mapping 1000 records',
-  details: { recordCount: 1000 }
-});
-
-// Mark as completed
-await tracker.markCompleted(jobId, {
-  recordCount: 1000,
-  successCount: 998,
-  failedCount: 2
-});
-
-// Or mark as failed
-try {
-  // ... job logic ...
-} catch (error) {
-  await tracker.markFailed(jobId, error);
-}
-```
-
-### Complete Example with Versori
-
-```typescript
-import {
-  createClient,
-  JobTracker,
-  VersoriKVAdapter,
-} from '@fluentcommerce/fc-connect-sdk';
-import { schedule } from '@versori/run';
-
-/**
- * Versori workflow with complete job tracking
- */
-
-export const dailyInventorySync = schedule('daily-inventory', '0 2 * * *')
-  .execute(async ({ log, connections, vars, kv }) => {
-    const jobId = `inventory_${Date.now()}`;
-    const tracker = new JobTracker(new VersoriKVAdapter(kv), log);
-
-    try {
-      // Create job
-      await tracker.createJob(jobId, {
-        triggeredBy: 'schedule',
-        stage: 'start',
-        details: { schedule: 'daily 2am' }
-      });
-
-      // Stage 1: Extraction
-      await tracker.updateJob(jobId, {
-        status: 'processing',
-        stage: 'extraction',
-        message: 'Querying virtual positions'
-      });
-
-      const data = await extractFromFluent();
-
-      // Stage 2: Transformation
-      await tracker.updateJob(jobId, {
-        stage: 'transformation',
-        message: `Processing ${data.length} records`
-      });
-
-      const transformed = await transformData(data);
-
-      // Stage 3: Upload
-      await tracker.updateJob(jobId, {
-        stage: 'upload',
-        message: 'Uploading to SFTP'
-      });
-
-      await uploadToSFTP(transformed);
-
-      // Completed
-      await tracker.markCompleted(jobId, {
-        recordCount: data.length,
-        fileName: `inventory_${jobId}.xml`
-      });
-
-      log.info('Job completed successfully', { jobId });
-
-    } catch (error) {
-      await tracker.markFailed(jobId, error);
-      log.error('Job failed', error);
-      throw error;
-    }
-  });
-```
-
-### Querying Job Status
-
-```typescript
-// Get job status
-const status = await tracker.getJob(jobId);
-
-if (status) {
-  console.log(`Job ${jobId}:`, {
-    status: status.status,
-    stage: status.stage,
-    message: status.message,
-    createdAt: status.createdAt,
-    completedAt: status.completedAt
-  });
-}
-
-// Check if job is still running
-if (status.status === 'processing') {
-  console.log(`Job in progress: ${status.stage}`);
-}
-
-// Check for errors
-if (status.status === 'failed') {
-  console.error('Job failed:', {
-    error: status.error,
-    stack: status.errorStack
-  });
-}
-```
-
-### Custom TTL Configuration
-
-```typescript
-// Default TTL: 7 days
-const tracker = new JobTracker(kvAdapter, logger);
-
-// Custom TTL: 24 hours
-const shortTracker = new JobTracker(
-  kvAdapter,
-  logger,
-  86400 // 24 hours in seconds
-);
-
-// Custom TTL: 30 days
-const longTracker = new JobTracker(
-  kvAdapter,
-  logger,
-  2592000 // 30 days in seconds
-);
-```
-
-### JobTracker API Reference
-
-| Method | Description | Parameters | Example |
-|--------|-------------|------------|---------|
-| `createJob(jobId, metadata)` | Create new job with 'queued' status | jobId, metadata | `await tracker.createJob('job_123', { triggeredBy: 'schedule' })` |
-| `updateJob(jobId, updates)` | Update job metadata/status/stage | jobId, updates | `await tracker.updateJob('job_123', { status: 'processing' })` |
-| `getJob(jobId)` | Get job by ID | jobId | `const job = await tracker.getJob('job_123')` |
-| `markCompleted(jobId, details)` | Mark job complete | jobId, details | `await tracker.markCompleted('job_123', { recordCount: 1000 })` |
-| `markFailed(jobId, error)` | Mark job failed | jobId, error | `await tracker.markFailed('job_123', error)` |
-
-**Constructor:**
-```typescript
-new JobTracker(kvAdapter: KVAdapter, logger: StructuredLogger, ttl?: number)
-```
-
-### When to Use JobTracker
-
-✅ **Use when**:
-- Need job history and audit trail
-- Monitoring multiple concurrent jobs
-- Debugging job failures
-- Tracking job duration and performance
-- Building dashboards or reports
-
-❌ **Skip when**:
-- Simple one-off scripts
-- No state persistence available
-- Memory-constrained environments
-
----
-
-## Job Creation Strategies
-
-### Strategy 1: Single Job Per File (Recommended)
-
-**When to use**: Most common pattern for daily/scheduled files
-
-```typescript
-// Process one file
-const job = await client.createJob({
-  name: `Daily Inventory - ${fileName}`,
-  retailerId: '2'
-});
-
-// Send all batches for this file
-for (const batch of batches) {
-  await client.sendBatch(job.id, { entities: batch });
-}
-
-// Poll until complete
-await pollJobCompletion(job.id);
-```
-
-**Pros**:
-- Simple error tracking (one job = one file)
-- Easy archival (job complete = archive file)
-- Clear audit trail
-
-**Cons**:
-- More jobs in system
-- Can't combine multiple files
-
-### Strategy 2: Single Job Per Day
-
-**When to use**: Multiple small files processed together
-
-```typescript
-// Create one job for entire day
-const job = await client.createJob({
-  name: `Daily Inventory - ${new Date().toISOString().split('T')[0]}`,
-  retailerId: '2'
-});
-
-// Process multiple files
-const files = await s3.listFiles('inventory/updates/');
-for (const file of files) {
-  const records = await processFile(file);
-
-  // Send batches for this file
-  for (const batch of chunkArray(records, 100)) {
-    await client.sendBatch(job.id, { entities: batch });
-  }
-}
-
-// Poll once for entire day's work
-await pollJobCompletion(job.id);
-```
-
-**Pros**:
-- Fewer jobs
-- Combine multiple sources
-- Single status check
-
-**Cons**:
-- Harder to track individual file errors
-- One file failure doesn't block others
-- More complex archival logic
-
-### Strategy 3: Long-Running Job (Advanced)
-
-**When to use**: Continuous processing (hourly micro-batches)
-
-```typescript
-// Create job once
-const job = await client.createJob({
-  name: `Inventory Stream - ${new Date().toISOString().split('T')[0]}`,
-  retailerId: '2'
-});
-
-// Send batches throughout the day
-setInterval(async () => {
-  const newRecords = await fetchLatestUpdates();
-
-  if (newRecords.length > 0) {
-    await client.sendBatch(job.id, { entities: newRecords });
-  }
-}, 3600000); // Every hour
-
-// Check status at end of day
-```
-
-**Pros**:
-- Minimize job creation overhead
-- Continuous processing
-
-**Cons**:
-- Job can stay open for hours
-- Harder to determine "completion"
-- Risk of very large jobs
-
----
-
-## Batch Size Optimization
-
-### Recommended Batch Sizes
-
-| Total Records | Batch Size | Batches | Processing Time |
-|---------------|------------|---------|-----------------|
-| < 1,000 | 100 | 1-10 | 1-2 minutes |
-| 1,000-10,000 | 100-200 | 10-100 | 3-10 minutes |
-| 10,000-50,000 | 200 | 50-250 | 5-15 minutes |
-| > 50,000 | 250 | 200+ | 10-30 minutes |
-
-**Max batch size**: 250 records (Fluent API limit)
-
-### Batch Size Calculator
-
-```typescript
-/**
- * Calculate optimal batch size based on total records
- */
-function calculateBatchSize(totalRecords: number): number {
-  if (totalRecords < 1000) {
-    return 100;
-  } else if (totalRecords < 10000) {
-    return 150;
-  } else {
-    return 250; // Max allowed
-  }
-}
-
-// Usage
-const batchSize = calculateBatchSize(records.length);
-const batches = chunkArray(records, batchSize);
-```
-
-### Chunking Utility
-
-```typescript
-/**
- * Split array into chunks
- */
-function chunkArray<T>(array: T[], size: number): T[][] {
-  const chunks: T[][] = [];
-  for (let i = 0; i < array.length; i += size) {
-    chunks.push(array.slice(i, i + size));
-  }
-  return chunks;
-}
-
-// Usage
-const batches = chunkArray(mappedRecords, 100);
-// [[record1, record2, ...], [record101, record102, ...], ...]
-```
-
-### Parallel Batch Sending (Advanced)
-
-```typescript
-/**
- * Send multiple batches in parallel
- * WARNING: Use with caution - can hit rate limits
- */
-async function sendBatchesParallel(
-  client: FluentClient,
-  jobId: string,
-  batches: any[][],
-  concurrency = 5
-) {
-  const results = [];
-
-  for (let i = 0; i < batches.length; i += concurrency) {
-    const chunk = batches.slice(i, i + concurrency);
-
-    // Send up to 'concurrency' batches at once
-    const promises = chunk.map(batch =>
-      client.sendBatch(jobId, { entities: batch })
-    );
-
-    const chunkResults = await Promise.all(promises);
-    results.push(...chunkResults);
-
-    console.log(`Sent batches ${i + 1} to ${i + chunk.length}`);
-  }
-
-  return results;
-}
-
-// Usage (send 5 batches at a time)
-await sendBatchesParallel(client, job.id, batches, 5);
-```
-
-**Caution**: Parallel sending can trigger rate limits. Start with concurrency=5, monitor, and adjust.
-
----
-
-## Status Polling and Completion
-
-### Basic Polling Pattern
-
-```typescript
-/**
- * Poll job status until complete
- */
-async function pollJobCompletion(
-  client: FluentClient,
-  jobId: string,
-  options = {
-    interval: 30000,      // 30 seconds
-    timeout: 3600000,     // 1 hour
-    onProgress: undefined // Callback
-  }
-) {
-  const startTime = Date.now();
-
-  while (true) {
-    const status = await client.getJobStatus(jobId);
-
-    // Check terminal states
-    if (status.status === 'COMPLETED') {
-      console.log('✓ Job completed successfully');
-      return status;
-    }
-
-    if (status.status === 'FAILED') {
-      throw new Error(`Job failed: ${status.errorSummary?.message || 'Unknown error'}`);
-    }
-
-    // Check timeout
-    if (Date.now() - startTime > options.timeout) {
-      throw new Error(`Job timeout after ${options.timeout}ms`);
-    }
-
-    // Progress callback
-    if (options.onProgress) {
-      options.onProgress(status);
-    }
-
-    // Log progress
-    console.log(
-      `Job ${status.status}: ${status.completedBatches}/${status.totalBatches} batches ` +
-      `(${Math.round((status.completedBatches / status.totalBatches) * 100)}%)`
-    );
-
-    // Wait before next poll
-    await new Promise(resolve => setTimeout(resolve, options.interval));
-  }
-}
-
-// Usage
-const finalStatus = await pollJobCompletion(client, job.id, {
-  interval: 30000,
-  timeout: 3600000,
-  onProgress: (status) => {
-    console.log(`Progress: ${status.completedBatches}/${status.totalBatches}`);
-  }
-});
-```
-
-### Advanced: Adaptive Polling
-
-```typescript
-/**
- * Adjust polling interval based on job size
- */
-function getPollingInterval(totalBatches: number): number {
-  if (totalBatches < 10) {
-    return 10000;  // 10 seconds for small jobs
-  } else if (totalBatches < 100) {
-    return 30000;  // 30 seconds for medium jobs
-  } else {
-    return 60000;  // 1 minute for large jobs
-  }
-}
-
-// Usage
-const interval = getPollingInterval(batches.length);
-await pollJobCompletion(client, job.id, { interval });
-```
-
----
-
-## Error Handling in Batches
-
-### Error Types
-
-| Error Level | Scope | Example | Recovery |
-|-------------|-------|---------|----------|
-| **File Error** | Entire file | Invalid CSV format | Move to error folder |
-| **Record Error** | Single record | Missing required field | Log, continue with others |
-| **Batch Error** | 100-250 records | API validation error | Logged by Fluent, continue |
-| **Job Error** | Entire job | Authentication failure | Retry entire job |
-| **Partial Batch Failure** | Some batches succeed, some fail | Network interruption | **NEW: PartialBatchRecovery** |
-
-### NEW: Partial Batch Recovery (v0.1.10+)
-
-**Problem**: What happens when sending 500 batches and batch #250 fails due to network error? You don't want to resend the first 249 successful batches.
-
-**Solution**: The SDK provides `PartialBatchRecovery` service to track batch progress and resume from failure point.
-
-#### How It Works
-
-```typescript
-import { PartialBatchRecovery, createClient } from '@fluentcommerce/fc-connect-sdk';
-
-/**
- * Partial Batch Recovery Pattern
- *
- * Tracks per-record success/failure in batch operations and enables:
- * - Retrying only failed records instead of entire batch
- * - Checkpoint/resume functionality
- * - Detailed error reporting per record
- */
-
-async function resilientBatchSync() {
-  const client = await createClient({ config });
-  const recovery = new PartialBatchRecovery(logger);
-
-  // Step 1: Create job
-  const job = await client.createJob({
-    name: 'Daily Inventory Sync',
-    retailerId: '2',
-  });
-
-  // Step 2: Process batch with automatic recovery
-  const result = await recovery.processBatchWithRecovery(
-    records,
-    async (batch) => {
-      // Your batch processing logic
-      return await client.sendBatch(job.id, {
-        action: 'UPSERT',
-        entityType: 'INVENTORY',
-        entities: batch
-      });
-    },
-    {
-      maxRetries: 3,
-      retryOnlyFailed: true,  // Only retry failed records
-      retryDelayMs: 1000,     // Start with 1 second
-      retryBatchSize: 100,    // Process 100 at a time
-      checkpointKey: 'inventory-sync-2025-01-24'
-    }
-  );
-
-  console.log(`✓ Success: ${result.successCount}/${result.totalRecords}`);
-  console.log(`✗ Failed: ${result.failedCount} records`);
-
-  if (result.failedCount > 0) {
-    console.error('Failed records:', result.failedRecords);
-    console.log(`Checkpoint saved: ${result.checkpointId}`);
-  }
-}
-```
-
-#### Integration with Batch API
-
-```typescript
-import {
-  createClient,
-  PartialBatchRecovery
-} from '@fluentcommerce/fc-connect-sdk';
-
-async function batchIngestionWithRecovery(records: any[]) {
-  const client = await createClient({ config });
-  const recovery = new PartialBatchRecovery(logger);
-
-  // Create job
-  const job = await client.createJob({
-    name: 'Inventory Ingestion with Recovery',
-    retailerId: 'my-retailer'
-  });
-
-  // Process with recovery
-  const result = await recovery.processBatchWithRecovery(
-    records,
-    async (batch) => {
-      const response = await client.sendBatch(job.id, {
-        action: 'UPSERT',
-        entityType: 'INVENTORY',
-        entities: batch
-      });
-
-      logger.info('Batch sent', {
-        batchId: response.id,
-        recordCount: batch.length
-      });
-
-      return response;
-    },
-    {
-      maxRetries: 3,
-      retryOnlyFailed: true,
-      checkpointKey: `job-${job.id}`
-    }
-  );
-
-  // Check job status
-  const status = await client.getJobStatus(job.id);
-
-  return {
-    jobId: job.id,
-    jobStatus: status.status,
-    ...result
-  };
-}
-```
-
-#### Checkpoint and Resume
-
-```typescript
-// Process batch and save checkpoint
-const result = await recovery.processBatchWithRecovery(
-  records,
-  processBatch,
-  {
-    maxRetries: 3,
-    checkpointKey: 'daily-inventory-sync'
-  }
-);
-
-if (result.failedCount > 0) {
-  console.log(`Checkpoint created: ${result.checkpointId}`);
-  console.log(`Failed records saved for later retry`);
-}
-
-// Later: Resume from checkpoint
-const checkpointId = result.checkpointId;
-const resumeResult = await recovery.resumeFromCheckpoint(
-  checkpointId,
-  processBatch,
-  {
-    maxRetries: 5  // More retries on resume
-  }
-);
-
-console.log(`Resume: ${resumeResult.successCount} recovered`);
-```
-
-#### Custom Retry Logic
-
-```typescript
-const result = await recovery.processBatchWithRecovery(
-  records,
-  processBatch,
-  {
-    maxRetries: 5,
-    retryDelayMs: 2000,
-    // Custom retry decision
-    shouldRetry: (error, attemptCount) => {
-      // Don't retry validation errors
-      if (error.message.includes('validation')) {
-        return false;
-      }
-
-      // Don't retry after 3 attempts for rate limits
-      if (error.message.includes('rate limit') && attemptCount > 3) {
-        return false;
-      }
-
-      // Retry all other errors
-      return true;
-    }
-  }
-);
-```
-
-#### Record Failure Details
-
-```typescript
-// Access detailed failure information
-if (result.failedCount > 0) {
-  result.failedRecords.forEach(failure => {
-    console.error(`Record ${failure.index} failed:`, {
-      record: failure.record,
-      error: failure.error.message,
-      attempts: failure.attemptCount,
-      timestamp: failure.timestamp
-    });
-  });
-
-  // Export failures for manual review
-  await fs.writeFile(
-    'failed-records.json',
-    JSON.stringify(result.failedRecords, null, 2)
-  );
-}
-```
-
-#### Key Features
-
-| Feature | Description | Benefit |
-|---------|-------------|---------|
-| **Per-record tracking** | Tracks each record individually | Know exactly which records failed |
-| **Selective retry** | Retry only failures, not successes | Efficient retry logic |
-| **Checkpoint support** | Resume from failure point | Handle interruptions |
-| **Exponential backoff** | Configurable retry delays | Avoid overwhelming API |
-| **Custom retry logic** | Override retry decisions | Fine-grained control |
-
-#### API Reference
-
-**Constructor:**
-```typescript
-new PartialBatchRecovery(logger?: StructuredLogger)
-```
-
-**Methods:**
-
-| Method | Description | Parameters |
-|--------|-------------|------------|
-| `processBatchWithRecovery()` | Process batch with recovery | records, processor, options |
-| `resumeFromCheckpoint()` | Resume from saved checkpoint | checkpointId, processor, options |
-
-#### When to Use Partial Batch Recovery
-
-✅ **Use when**:
-- Sending 50+ batches (high failure risk)
-- Network is unstable
-- Long-running jobs (> 10 minutes)
-- Scheduled workflows that may be interrupted
-- Critical data that must complete
-
-❌ **Skip when**:
-- < 10 batches (low failure risk)
-- Fast operations (< 2 minutes)
-- Non-critical data (can rerun from scratch)
-- Memory-constrained environments
-
-### File-Level Error Handling
-
-```typescript
-try {
-  // Download file
-  const content = await s3.downloadFile(fileKey);
-
-  // Parse CSV
-  const records = await parser.parse(content);
-
-  // Process batches
-  await processBatches(records);
-
-  // Archive on success
-  await s3.moveFile(fileKey, archiveKey);
-
-} catch (error: any) {
-  console.error(`File processing failed: ${error.message}`);
-
-  // Move to error folder
-  await s3.moveFile(fileKey, errorKey);
-
-  // Log error details
-  await s3.uploadFile(errorKey + '.error.log', JSON.stringify({
-    file: fileKey,
-    error: error.message,
-    stack: error.stack,
-    timestamp: new Date().toISOString()
-  }, null, 2));
-
-  // Continue with next file (don't throw)
-}
-```
-
-### Record-Level Error Handling
-
-```typescript
-const mappedRecords = [];
-const mappingErrors = [];
-
-for (const record of records) {
-  const result = await mapper.map(record);
-
-  if (result.success) {
-    mappedRecords.push(result.data);
-  } else {
-    mappingErrors.push({
-      record,
-      errors: result.errors
-    });
-  }
-}
-
-console.log(`Mapped ${mappedRecords.length} records, ${mappingErrors.length} errors`);
-
-// Write error report
-if (mappingErrors.length > 0) {
-  await s3.uploadFile(
-    'inventory/errors/mapping-errors.json',
-    JSON.stringify(mappingErrors, null, 2)
-  );
-}
-
-// Continue with successful records
-```
-
-### Batch API Error Reporting
-
-```typescript
-// After job completes, check for errors
-const status = await client.getJobStatus(job.id);
-
-if (status.errorSummary && status.errorSummary.totalErrors > 0) {
-  console.warn(`Job completed with ${status.errorSummary.totalErrors} errors`);
-
-  // Get error details
-  const errorDetails = await client.graphql({
-    query: `
-      query GetJobErrors($jobId: ID!) {
-        job(id: $jobId) {
-          batches(first: 100) {
-            edges {
-              node {
-                id
-                status
-                errors {
-                  recordRef
-                  errorType
-                  errorMessage
-                }
-              }
-            }
-          }
-        }
-      }
-    `,
-    variables: { jobId: job.id }
-  });
-
-  // Write error report
-  await s3.uploadFile(
-    'inventory/errors/batch-errors.json',
-    JSON.stringify(errorDetails, null, 2)
-  );
-}
-```
-
----
-
-## Complete Implementation Example
-
-See **[S3 CSV Batch API Guide](../../../01-TEMPLATES/standalone/s3-csv-batch-api.md)** for production-ready implementation with:
-
-- Complete TypeScript code
-- Environment configuration
-- Error handling strategies
-- File archival patterns
-- Scheduling with cron
-- Monitoring and logging
-
-### Quick Example: Scheduled Versori Batch
-
-```typescript
-/**
- * Versori Scheduled Workflow: Daily Inventory Sync
- *
- * Trigger: Cron (daily at 2 AM)
- * Process: Download CSV from S3, send to Batch API
- */
-
-import { createClient, S3DataSource, CSVParserService, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
-
-export default async function dailyInventorySync(activation: any, log: any, connections: any) {
-  try {
-    log.info('Starting daily inventory sync');
-
-    // Create client
-    const client = await createClient({
-      connection: connections.fluent_commerce,
-      logger: log
-    });
-
-    // Create S3 source
-    const s3 = new S3DataSource({
-      connection: connections.aws_s3
-    }, log);
-
-    // List files
-    const files = await s3.listFiles('inventory/daily/');
-    log.info(`Found ${files.length} files to process`);
-
-    // Process each file
-    for (const file of files) {
-      await processFile(client, s3, file.key, log);
-    }
-
-    return { status: 200, body: { success: true, filesProcessed: files.length } };
-
-  } catch (error: any) {
-    log.error('Daily sync failed', error);
-    return { status: 500, body: { success: false, error: error.message } };
-  }
-}
-
-async function processFile(client: any, s3: any, fileKey: string, log: any) {
-  try {
-    // Download
-    const content = await s3.downloadFile(fileKey);
-
-    // Parse
-    const parser = new CSVParserService({ headers: true });
-    const records = await parser.parse(content);
-
-    // Map
-    const mapper = new UniversalMapper({
-      fields: {
-        ref: { source: 'sku', resolver: 'custom.buildRef' },
-        type: { value: 'INVENTORY' },
-        productRef: { source: 'sku', required: true },
-        locationRef: { source: 'location', required: true },
-        onHand: { source: 'qty', resolver: 'sdk.parseInt' }
-      }
-    }, {
-      customResolvers: {
-        'custom.buildRef': (v, d) => `${d.sku}-${d.location}`
-      }
-    });
-
-    const mapped = [];
-    for (const rec of records) {
-      const result = await mapper.map(rec);
-      if (result.success) mapped.push(result.data);
-    }
-
-    // Create job
-    const job = await client.createJob({
-      name: `Daily Inventory - ${fileKey}`,
-      retailerId: '2'
-    });
-
-    // Send batches
-    const BATCH_SIZE = 100;
-    for (let i = 0; i < mapped.length; i += BATCH_SIZE) {
-      await client.sendBatch(job.id, {
-        entities: mapped.slice(i, i + BATCH_SIZE)
-      });
-    }
-
-    // Poll completion
-    let status = await client.getJobStatus(job.id);
-    while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-      await new Promise(r => setTimeout(r, 30000));
-      status = await client.getJobStatus(job.id);
-    }
-
-    if (status.status === 'COMPLETED') {
-      await s3.moveFile(fileKey, fileKey.replace('daily/', 'archive/'));
-      log.info(`✓ Processed ${fileKey}`);
-    } else {
-      throw new Error(`Job failed: ${status.status}`);
-    }
-
-  } catch (error: any) {
-    log.error(`Failed to process ${fileKey}`, error);
-    await s3.moveFile(fileKey, fileKey.replace('daily/', 'errors/'));
-  }
-}
-```
-
----
-
-## Next Steps
-
-Now that you understand batch processing, you're ready to learn delta sync for incremental updates!
-
-**Continue to:** [Module 3: Delta Sync →](./integration-patterns-03-delta-sync.md)
-
-Or explore:
-- [Module 5: Error Handling](./integration-patterns-05-error-handling.md) - Resilience strategies
-- [Complete Example: S3 CSV Batch API](../../../01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md)
-- [Complete Example: Versori Scheduled CSV](../../../01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md)
-
----
-
-## Additional Resources
-
-- [Fluent Batch API Documentation](https://docs.fluentcommerce.com/)
-- [Universal Mapping Guide](../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md)
-- [S3DataSource API Reference](../../../02-CORE-GUIDES/data-sources/modules/data-sources-02-s3-operations.md)
-- [CSVParserService API Reference](../../../02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-02-csv-parser.md)
-
----
-
-[← Back to Index](../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md) | [Previous: Real-Time →](./integration-patterns-01-real-time-processing.md) | [Next: Delta Sync →](./integration-patterns-03-delta-sync.md)
+# Module 2: Batch Processing
+
+> **Learning Objective:** Master batch processing patterns for high-volume data synchronization using the Fluent Batch API and SDK orchestration services.
+>
+> **Level:** Intermediate
+
+## Table of Contents
+
+1. [What is Batch Processing?](#what-is-batch-processing)
+2. [When to Use Batch Processing](#when-to-use-batch-processing)
+3. [Fluent Batch API Overview](#fluent-batch-api-overview)
+4. [SDK Batch Components](#sdk-batch-components)
+5. [Basic Batch Workflow](#basic-batch-workflow)
+6. [Job Creation Strategies](#job-creation-strategies)
+7. [Batch Size Optimization](#batch-size-optimization)
+8. [Status Polling and Completion](#status-polling-and-completion)
+9. [Error Handling in Batches](#error-handling-in-batches)
+10. [Complete Implementation Example](#complete-implementation-example)
+11. [Next Steps](#next-steps)
+
+---
+
+## What is Batch Processing?
+
+**Batch processing** means grouping multiple records together and processing them as a single unit, optimized for throughput over latency.
+
+### Key Characteristics
+
+| Characteristic | Description | Example |
+|----------------|-------------|---------|
+| **High Volume** | Process thousands of records | 50,000 inventory positions |
+| **Scheduled** | Runs at specific times | Daily at 2 AM |
+| **Asynchronous** | Submit job, poll for completion | Job completes in 5-10 minutes |
+| **Bulk Operations** | Optimize for throughput | 100 records per API call |
+
+### How It Works
+
+```
+CSV File (50K records) → Parse → Transform → Batch API → Job → Poll Status → Complete
+     ↓                      ↓         ↓            ↓         ↓         ↓           ↓
+  S3 bucket          Parse chunks  Map fields  Create job  Submit  Check every  Archive
+                                                           batches  30 seconds    file
+```
+
+### Visual Flow
+
+```
+┌─────────────────┐
+│ S3 CSV File     │
+│ (50,000 records)│
+└────────┬────────┘
+         │
+         │ ①Download & Parse
+         ▼
+┌─────────────────┐
+│   CSVParserService     │
+│  Stream chunks  │
+└────────┬────────┘
+         │
+         │ ②Transform fields
+         ▼
+┌─────────────────┐
+│ UniversalMapper │
+│  Field mapping  │
+└────────┬────────┘
+         │
+         │ ③Create Batch job
+         ▼
+┌─────────────────┐
+│ FluentClient    │
+│  createJob()    │
+└────────┬────────┘
+         │
+         │ ④Send batches (100 records each)
+         ▼
+┌─────────────────┐
+│  sendBatch()    │ ────┐
+│  500 batches    │     │ Parallel processing
+│  of 100 records │ ────┤ on Fluent servers
+└────────┬────────┘     │
+         │              │
+         │ ⑤Poll status every 30s
+         ▼
+┌─────────────────┐
+│  getJobStatus() │
+│  PROCESSING...  │
+│  COMPLETED ✓    │
+└────────┬────────┘
+         │
+         │ ⑥Archive file
+         ▼
+┌─────────────────┐
+│ S3 Archive      │
+│ Success log     │
+└─────────────────┘
+```
+
+**Total time**: 5-10 minutes for 50,000 records (vs 2-3 hours for sequential GraphQL mutations).
+
+---
+
+## When to Use Batch Processing
+
+### ✅ Use Batch Processing When:
+
+| Scenario | Why Batch? | SDK Pattern |
+|----------|------------|-------------|
+| **Large Files** | 1K+ records | Batch API with streaming |
+| **Daily Sync** | Full inventory sync | Scheduled batch job |
+| **High Volume** | > 1,000 events/hour | Batch API (not webhooks) |
+| **Bulk Updates** | Mass price changes | Single job, multiple batches |
+| **CSV/Parquet Files** | Structured file formats | S3DataSource + FluentClient |
+
+### ❌ Avoid Batch Processing When:
+
+| Scenario | Why Not Batch? | Use Instead |
+|----------|----------------|-------------|
+| **Immediate Updates** | Need < 5 second latency | Real-time (Module 1) |
+| **Single Records** | 1-10 records | Direct GraphQL mutation |
+| **Event-Driven** | External webhook triggers | Real-time webhook |
+| **Critical Orders** | Customer waiting for confirmation | Real-time processing |
+
+---
+
+## Fluent Batch API Overview
+
+### What is Batch API?
+
+The Fluent Batch API is a specialized GraphQL endpoint for bulk data operations:
+
+- **Asynchronous**: Submit job, get ID, poll for completion
+- **High-throughput**: 10,000+ records in minutes
+- **Fault-tolerant**: Partial failures don't block entire job
+- **Entity-specific**: Currently supports `InventoryQuantity` only
+
+### Supported Operations
+
+| Entity | Operations | Max per Batch |
+|--------|-----------|---------------|
+| `InventoryQuantity` | Create, Update | 100-250 records |
+
+**IMPORTANT**: Batch API currently **only supports InventoryQuantity**. For other entities (Order, Product, Customer), use standard GraphQL mutations.
+
+### Batch API Workflow
+
+```graphql
+# Step 1: Create job
+mutation CreateJob {
+  createJob(input: {
+    name: "Daily Inventory Sync"
+    retailerId: "2"
+  }) {
+    id
+    status
+  }
+}
+
+# Step 2: Send batches
+mutation SendBatch($jobId: ID!, $entities: [InventoryQuantityInput!]!) {
+  sendBatch(jobId: $jobId, entities: $entities) {
+    id
+    status
+    recordCount
+  }
+}
+
+# Step 3: Poll status
+query GetJobStatus($jobId: ID!) {
+  job(id: $jobId) {
+    id
+    status  # PENDING, PROCESSING, COMPLETED, FAILED
+    totalBatches
+    completedBatches
+    errorSummary {
+      totalErrors
+      errorTypes
+    }
+  }
+}
+```
+
+### Job Lifecycle
+
+```
+CREATE_JOB → PENDING → SEND_BATCHES → PROCESSING → COMPLETED
+                ↓                                      ↓
+             (can add more batches)            (can check errors)
+```
+
+---
+
+## SDK Batch Components
+
+### Component 1: FluentClient Batch Methods
+
+The SDK provides batch methods directly on `FluentClient` (there is no separate `FluentBatchManager` class):
+
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+
+const client = await createClient({
+  config: {
+    baseUrl: 'https://api.fluentcommerce.com',
+    clientId: process.env.FLUENT_CLIENT_ID,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET,
+    retailerId: process.env.FLUENT_RETAILER_ID
+  }
+});
+
+// Create job
+const job = await client.createJob({
+  name: 'Daily Inventory Sync',
+  retailerId: '2'
+});
+
+// Send batch
+const batch = await client.sendBatch(job.id, {
+  entities: inventoryRecords
+});
+
+// Get status
+const status = await client.getJobStatus(job.id);
+
+// Get detailed status with batches
+const jobDetail = await client.getBatchStatus(job.id, batch.id);
+```
+
+**Available methods**:
+- `createJob(input)` - Create new Batch job
+- `sendBatch(jobId, data)` - Submit batch of records
+- `getJobStatus(jobId)` - Get job status and summary
+- `getBatchStatus(jobId, batchId)` - Get individual batch details
+
+### Component 2: `S3DataSource`
+
+**Purpose**: Read and write files from S3 with streaming support
+
+```typescript
+import { S3DataSource } from '@fluentcommerce/fc-connect-sdk';
+
+const s3 = new S3DataSource({
+  type: 'S3_CSV',
+  connectionId: 'my-s3',
+  name: 'My S3 Source',
+  s3Config: {
+    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+    region: 'us-east-1',
+    bucket: 'inventory-bucket'
+  }
+}, logger);
+
+// List files
+const files = await s3.listFiles('inventory/updates/');
+
+// Download file
+const fileContent = await s3.downloadFile('inventory/updates/inventory.csv');
+
+// Download large file as Buffer (for files >100MB)
+const buffer = await s3.downloadFile('inventory/updates/large-file.csv', { encoding: 'binary' });
+
+// Upload file
+await s3.uploadFile('inventory/archive/processed.csv', content);
+
+// Move file
+await s3.moveFile('inventory/updates/file.csv', 'inventory/archive/file.csv');
+```
+
+**Key features**:
+- Streaming support for large files (memory-efficient)
+- Automatic retry on network errors
+- Progress callbacks for uploads/downloads
+
+### Component 3: `CSVParserService`
+
+**Purpose**: Parse CSV files with validation and streaming
+
+```typescript
+import { CSVParserService } from '@fluentcommerce/fc-connect-sdk';
+
+const parser = new CSVParserService({
+  headers: true,           // First row is headers
+  skipEmptyLines: true,    // Ignore blank lines
+  delimiter: ',',          // CSV delimiter
+  quote: '"',              // Quote character
+  escape: '\\'             // Escape character
+});
+
+// Parse entire file (for small files < 10MB)
+const records = await parser.parse(csvContent);
+
+// Stream parse (for large files)
+const recordStream = parser.streamParse(csvStream);
+for await (const record of recordStream) {
+  // Process record
+}
+```
+
+**Validation options**:
+- Required columns
+- Type validation
+- Custom validators
+
+### Component 4: `UniversalMapper`
+
+**Purpose**: Transform CSV/JSON fields to Fluent schema
+
+```typescript
+import { UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
+
+const mappingConfig = {
+  fields: {
+    ref: {
+      source: 'sku',
+      resolver: 'custom.buildRef'  // Combine sku + location
+    },
+    type: {
+      value: 'INVENTORY'  // Static value
+    },
+    status: {
+      source: 'status',
+      resolver: 'sdk.uppercase'
+    },
+    productRef: {
+      source: 'sku',
+      required: true
+    },
+    locationRef: {
+      source: 'warehouse_code',
+      required: true
+    },
+    onHand: {
+      source: 'quantity',
+      resolver: 'sdk.parseInt'
+    }
+  }
+};
+
+const mapper = new UniversalMapper(mappingConfig, {
+  customResolvers: {
+    'custom.buildRef': (value, data) => {
+      return `${data.sku}-${data.warehouse_code}`;
+    }
+  }
+});
+
+const result = await mapper.map(csvRecord);
+// result.data = { ref: 'SKU001-WH01', type: 'INVENTORY', ... }
+```
+
+### Workflow Composition Pattern
+
+**Purpose**: Compose SDK services for complete ingestion workflows
+
+Instead of using a single orchestrator, compose the above components into your custom workflow:
+
+```typescript
+import {
+  createClient,
+  S3DataSource,
+  CSVParserService,
+  UniversalMapper,
+  StateService,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+
+async function processInventoryFiles() {
+  const logger = toStructuredLogger(createConsoleLogger(), { logLevel: 'info' });
+
+  // Initialize components
+  const client = await createClient({ config });
+  const s3 = new S3DataSource(s3Config, logger);
+  const parser = new CSVParserService();
+  const mapper = new UniversalMapper(mappingConfig);
+  const stateService = new StateService(logger);
+
+  // List and process files
+  const files = await s3.listFiles({ prefix: 'inventory/updates/' });
+
+  for (const file of files) {
+    if (await stateService.isFileProcessed(file.name)) continue;
+
+    try {
+      // 1. Download and parse
+      const content = await s3.downloadFile(file.name);
+      const records = await parser.parse(content);
+
+      // 2. Map fields
+      const inventory = [];
+      for (const record of records) {
+        const result = await mapper.map(record);
+        if (result.success) inventory.push(result.data);
+      }
+
+      // 3. Create job and send batches
+      const job = await client.createJob({
+        name: `Inventory - ${file.name}`,
+        retailerId: '1'
+      });
+
+      const batches = chunkArray(inventory, 100);
+      for (const batch of batches) {
+        await client.sendBatch(job.id, {
+          action: 'UPSERT',
+          entityType: 'INVENTORY',
+          entities: batch
+        });
+      }
+
+      // 4. Archive and mark processed
+      await s3.moveFile(file.name, `inventory/archive/${file.name}`);
+      await stateService.markFileProcessed(file.name);
+
+    } catch (error) {
+      logger.error(`Failed to process ${file.name}`, error);
+      await s3.moveFile(file.name, `inventory/errors/${file.name}`);
+    }
+  }
+}
+```
+
+**Benefits of building block composition**:
+1. ✅ Full control over workflow logic
+2. ✅ Custom error handling
+3. ✅ Easy to test individual components
+4. ✅ Flexible batch sizing and job strategies
+5. ✅ Works with any entity type (not just INVENTORY)
+6. ✅ Easy to add custom business rules
+
+---
+
+## Basic Batch Workflow
+
+### Step-by-Step Pattern
+
+```typescript
+/**
+ * Basic Batch Processing Workflow
+ *
+ * Steps:
+ * 1. Create Fluent client
+ * 2. Download and parse CSV file
+ * 3. Transform records to Fluent schema
+ * 4. Create Batch job
+ * 5. Send records in batches
+ * 6. Poll status until complete
+ * 7. Handle results
+ */
+
+import { createClient, S3DataSource, CSVParserService, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
+
+async function batchInventorySync() {
+  // Step 1: Create client
+  const client = await createClient({
+    baseUrl: process.env.FLUENT_BASE_URL,
+    clientId: process.env.FLUENT_CLIENT_ID,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET,
+    retailerId: process.env.FLUENT_RETAILER_ID
+  });
+
+  // Step 2: Download CSV
+  const s3 = new S3DataSource({
+    type: 'S3_CSV',
+    connectionId: 'my-s3',
+    name: 'My S3 Source',
+    s3Config: {
+      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+      region: process.env.AWS_REGION,
+      bucket: process.env.AWS_BUCKET
+    }
+  }, console);
+
+  const csvContent = await s3.downloadFile('inventory/daily-update.csv');
+
+  // Step 3: Parse CSV
+  const parser = new CSVParserService({ headers: true });
+  const records = await parser.parse(csvContent);
+
+  console.log(`Parsed ${records.length} records`);
+
+  // Step 4: Transform records
+  const mapper = new UniversalMapper({
+    fields: {
+      ref: { source: 'sku', resolver: 'custom.buildRef' },
+      type: { value: 'INVENTORY' },
+      productRef: { source: 'sku', required: true },
+      locationRef: { source: 'location', required: true },
+      onHand: { source: 'qty', resolver: 'sdk.parseInt' }
+    }
+  }, {
+    customResolvers: {
+      'custom.buildRef': (value, data) => `${data.sku}-${data.location}`
+    }
+  });
+
+  const mappedRecords = [];
+  for (const record of records) {
+    const result = await mapper.map(record);
+    if (result.success) {
+      mappedRecords.push(result.data);
+    } else {
+      console.error('Mapping error:', result.errors);
+    }
+  }
+
+  console.log(`Mapped ${mappedRecords.length} records`);
+
+  // Step 5: Create Batch job
+  const job = await client.createJob({
+    name: `Inventory Sync ${new Date().toISOString()}`,
+    retailerId: '2'
+  });
+
+  console.log(`Created job ${job.id}`);
+
+  // Step 6: Send batches
+  const BATCH_SIZE = 100;
+  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
+    const chunk = mappedRecords.slice(i, i + BATCH_SIZE);
+
+    const batch = await client.sendBatch(job.id, {
+      entities: chunk
+    });
+
+    console.log(`Sent batch ${batch.id} (${chunk.length} records)`);
+  }
+
+  // Step 7: Poll status
+  let status = await client.getJobStatus(job.id);
+  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+    console.log(`Job status: ${status.status} (${status.completedBatches}/${status.totalBatches} batches)`);
+
+    await new Promise(resolve => setTimeout(resolve, 30000)); // Wait 30 seconds
+
+    status = await client.getJobStatus(job.id);
+  }
+
+  // Step 8: Handle results
+  if (status.status === 'COMPLETED') {
+    console.log('✓ Job completed successfully');
+    console.log(`Total records: ${status.totalRecords}`);
+    console.log(`Errors: ${status.errorSummary?.totalErrors || 0}`);
+
+    // Archive file
+    await s3.moveFile('inventory/daily-update.csv', 'inventory/archive/daily-update.csv');
+  } else {
+    console.error('✗ Job failed:', status.status);
+
+    // Move to error folder
+    await s3.moveFile('inventory/daily-update.csv', 'inventory/errors/daily-update.csv');
+  }
+}
+
+batchInventorySync().catch(console.error);
+```
+
+---
+
+## NEW: Job Lifecycle Tracking (v0.1.10+)
+
+**Track job state, metadata, and lifecycle** across your integration workflows.
+
+The SDK provides `JobTracker` service for managing job lifecycle, tracking status, and storing job metadata.
+
+### JobTracker Overview
+
+```typescript
+import { JobTracker, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+// ✅ CORRECT: Access openKv from Versori context
+// import { openKv } from '@versori/run'; // ❌ WRONG - Not a direct export
+
+// In Versori workflow handler:
+const { openKv } = ctx;
+const kvAdapter = new VersoriKVAdapter(openKv(':project:'));
+const tracker = new JobTracker(kvAdapter, logger);
+
+// Create job
+const jobId = `scheduled_${Date.now()}`;
+
+await tracker.createJob(jobId, {
+  triggeredBy: 'schedule',
+  stage: 'initialization',
+  details: {
+    catalogueRef: 'DEFAULT:1',
+    fileName: 'inventory.csv'
+  }
+});
+
+// Update progress
+await tracker.updateJob(jobId, {
+  status: 'processing',
+  stage: 'extraction',
+  message: 'Extracting records from S3'
+});
+
+await tracker.updateJob(jobId, {
+  stage: 'transformation',
+  message: 'Mapping 1000 records',
+  details: { recordCount: 1000 }
+});
+
+// Mark as completed
+await tracker.markCompleted(jobId, {
+  recordCount: 1000,
+  successCount: 998,
+  failedCount: 2
+});
+
+// Or mark as failed
+try {
+  // ... job logic ...
+} catch (error) {
+  await tracker.markFailed(jobId, error);
+}
+```
+
+### Complete Example with Versori
+
+```typescript
+import {
+  createClient,
+  JobTracker,
+  VersoriKVAdapter,
+} from '@fluentcommerce/fc-connect-sdk';
+import { schedule } from '@versori/run';
+
+/**
+ * Versori workflow with complete job tracking
+ */
+
+export const dailyInventorySync = schedule('daily-inventory', '0 2 * * *')
+  .execute(async ({ log, connections, vars, kv }) => {
+    const jobId = `inventory_${Date.now()}`;
+    const tracker = new JobTracker(new VersoriKVAdapter(kv), log);
+
+    try {
+      // Create job
+      await tracker.createJob(jobId, {
+        triggeredBy: 'schedule',
+        stage: 'start',
+        details: { schedule: 'daily 2am' }
+      });
+
+      // Stage 1: Extraction
+      await tracker.updateJob(jobId, {
+        status: 'processing',
+        stage: 'extraction',
+        message: 'Querying virtual positions'
+      });
+
+      const data = await extractFromFluent();
+
+      // Stage 2: Transformation
+      await tracker.updateJob(jobId, {
+        stage: 'transformation',
+        message: `Processing ${data.length} records`
+      });
+
+      const transformed = await transformData(data);
+
+      // Stage 3: Upload
+      await tracker.updateJob(jobId, {
+        stage: 'upload',
+        message: 'Uploading to SFTP'
+      });
+
+      await uploadToSFTP(transformed);
+
+      // Completed
+      await tracker.markCompleted(jobId, {
+        recordCount: data.length,
+        fileName: `inventory_${jobId}.xml`
+      });
+
+      log.info('Job completed successfully', { jobId });
+
+    } catch (error) {
+      await tracker.markFailed(jobId, error);
+      log.error('Job failed', error);
+      throw error;
+    }
+  });
+```
+
+### Querying Job Status
+
+```typescript
+// Get job status
+const status = await tracker.getJob(jobId);
+
+if (status) {
+  console.log(`Job ${jobId}:`, {
+    status: status.status,
+    stage: status.stage,
+    message: status.message,
+    createdAt: status.createdAt,
+    completedAt: status.completedAt
+  });
+}
+
+// Check if job is still running
+if (status.status === 'processing') {
+  console.log(`Job in progress: ${status.stage}`);
+}
+
+// Check for errors
+if (status.status === 'failed') {
+  console.error('Job failed:', {
+    error: status.error,
+    stack: status.errorStack
+  });
+}
+```
+
+### Custom TTL Configuration
+
+```typescript
+// Default TTL: 7 days
+const tracker = new JobTracker(kvAdapter, logger);
+
+// Custom TTL: 24 hours
+const shortTracker = new JobTracker(
+  kvAdapter,
+  logger,
+  86400 // 24 hours in seconds
+);
+
+// Custom TTL: 30 days
+const longTracker = new JobTracker(
+  kvAdapter,
+  logger,
+  2592000 // 30 days in seconds
+);
+```
+
+### JobTracker API Reference
+
+| Method | Description | Parameters | Example |
+|--------|-------------|------------|---------|
+| `createJob(jobId, metadata)` | Create new job with 'queued' status | jobId, metadata | `await tracker.createJob('job_123', { triggeredBy: 'schedule' })` |
+| `updateJob(jobId, updates)` | Update job metadata/status/stage | jobId, updates | `await tracker.updateJob('job_123', { status: 'processing' })` |
+| `getJob(jobId)` | Get job by ID | jobId | `const job = await tracker.getJob('job_123')` |
+| `markCompleted(jobId, details)` | Mark job complete | jobId, details | `await tracker.markCompleted('job_123', { recordCount: 1000 })` |
+| `markFailed(jobId, error)` | Mark job failed | jobId, error | `await tracker.markFailed('job_123', error)` |
+
+**Constructor:**
+```typescript
+new JobTracker(kvAdapter: KVAdapter, logger: StructuredLogger, ttl?: number)
+```
+
+### When to Use JobTracker
+
+✅ **Use when**:
+- Need job history and audit trail
+- Monitoring multiple concurrent jobs
+- Debugging job failures
+- Tracking job duration and performance
+- Building dashboards or reports
+
+❌ **Skip when**:
+- Simple one-off scripts
+- No state persistence available
+- Memory-constrained environments
+
+---
+
+## Job Creation Strategies
+
+### Strategy 1: Single Job Per File (Recommended)
+
+**When to use**: Most common pattern for daily/scheduled files
+
+```typescript
+// Process one file
+const job = await client.createJob({
+  name: `Daily Inventory - ${fileName}`,
+  retailerId: '2'
+});
+
+// Send all batches for this file
+for (const batch of batches) {
+  await client.sendBatch(job.id, { entities: batch });
+}
+
+// Poll until complete
+await pollJobCompletion(job.id);
+```
+
+**Pros**:
+- Simple error tracking (one job = one file)
+- Easy archival (job complete = archive file)
+- Clear audit trail
+
+**Cons**:
+- More jobs in system
+- Can't combine multiple files
+
+### Strategy 2: Single Job Per Day
+
+**When to use**: Multiple small files processed together
+
+```typescript
+// Create one job for entire day
+const job = await client.createJob({
+  name: `Daily Inventory - ${new Date().toISOString().split('T')[0]}`,
+  retailerId: '2'
+});
+
+// Process multiple files
+const files = await s3.listFiles('inventory/updates/');
+for (const file of files) {
+  const records = await processFile(file);
+
+  // Send batches for this file
+  for (const batch of chunkArray(records, 100)) {
+    await client.sendBatch(job.id, { entities: batch });
+  }
+}
+
+// Poll once for entire day's work
+await pollJobCompletion(job.id);
+```
+
+**Pros**:
+- Fewer jobs
+- Combine multiple sources
+- Single status check
+
+**Cons**:
+- Harder to track individual file errors
+- One file failure doesn't block others
+- More complex archival logic
+
+### Strategy 3: Long-Running Job (Advanced)
+
+**When to use**: Continuous processing (hourly micro-batches)
+
+```typescript
+// Create job once
+const job = await client.createJob({
+  name: `Inventory Stream - ${new Date().toISOString().split('T')[0]}`,
+  retailerId: '2'
+});
+
+// Send batches throughout the day
+setInterval(async () => {
+  const newRecords = await fetchLatestUpdates();
+
+  if (newRecords.length > 0) {
+    await client.sendBatch(job.id, { entities: newRecords });
+  }
+}, 3600000); // Every hour
+
+// Check status at end of day
+```
+
+**Pros**:
+- Minimize job creation overhead
+- Continuous processing
+
+**Cons**:
+- Job can stay open for hours
+- Harder to determine "completion"
+- Risk of very large jobs
+
+---
+
+## Batch Size Optimization
+
+### Recommended Batch Sizes
+
+| Total Records | Batch Size | Batches | Processing Time |
+|---------------|------------|---------|-----------------|
+| < 1,000 | 100 | 1-10 | 1-2 minutes |
+| 1,000-10,000 | 100-200 | 10-100 | 3-10 minutes |
+| 10,000-50,000 | 200 | 50-250 | 5-15 minutes |
+| > 50,000 | 250 | 200+ | 10-30 minutes |
+
+**Max batch size**: 250 records (Fluent API limit)
+
+### Batch Size Calculator
+
+```typescript
+/**
+ * Calculate optimal batch size based on total records
+ */
+function calculateBatchSize(totalRecords: number): number {
+  if (totalRecords < 1000) {
+    return 100;
+  } else if (totalRecords < 10000) {
+    return 150;
+  } else {
+    return 250; // Max allowed
+  }
+}
+
+// Usage
+const batchSize = calculateBatchSize(records.length);
+const batches = chunkArray(records, batchSize);
+```
+
+### Chunking Utility
+
+```typescript
+/**
+ * Split array into chunks
+ */
+function chunkArray<T>(array: T[], size: number): T[][] {
+  const chunks: T[][] = [];
+  for (let i = 0; i < array.length; i += size) {
+    chunks.push(array.slice(i, i + size));
+  }
+  return chunks;
+}
+
+// Usage
+const batches = chunkArray(mappedRecords, 100);
+// [[record1, record2, ...], [record101, record102, ...], ...]
+```
+
+### Parallel Batch Sending (Advanced)
+
+```typescript
+/**
+ * Send multiple batches in parallel
+ * WARNING: Use with caution - can hit rate limits
+ */
+async function sendBatchesParallel(
+  client: FluentClient,
+  jobId: string,
+  batches: any[][],
+  concurrency = 5
+) {
+  const results = [];
+
+  for (let i = 0; i < batches.length; i += concurrency) {
+    const chunk = batches.slice(i, i + concurrency);
+
+    // Send up to 'concurrency' batches at once
+    const promises = chunk.map(batch =>
+      client.sendBatch(jobId, { entities: batch })
+    );
+
+    const chunkResults = await Promise.all(promises);
+    results.push(...chunkResults);
+
+    console.log(`Sent batches ${i + 1} to ${i + chunk.length}`);
+  }
+
+  return results;
+}
+
+// Usage (send 5 batches at a time)
+await sendBatchesParallel(client, job.id, batches, 5);
+```
+
+**Caution**: Parallel sending can trigger rate limits. Start with concurrency=5, monitor, and adjust.
+
+---
+
+## Status Polling and Completion
+
+### Basic Polling Pattern
+
+```typescript
+/**
+ * Poll job status until complete
+ */
+async function pollJobCompletion(
+  client: FluentClient,
+  jobId: string,
+  options = {
+    interval: 30000,      // 30 seconds
+    timeout: 3600000,     // 1 hour
+    onProgress: undefined // Callback
+  }
+) {
+  const startTime = Date.now();
+
+  while (true) {
+    const status = await client.getJobStatus(jobId);
+
+    // Check terminal states
+    if (status.status === 'COMPLETED') {
+      console.log('✓ Job completed successfully');
+      return status;
+    }
+
+    if (status.status === 'FAILED') {
+      throw new Error(`Job failed: ${status.errorSummary?.message || 'Unknown error'}`);
+    }
+
+    // Check timeout
+    if (Date.now() - startTime > options.timeout) {
+      throw new Error(`Job timeout after ${options.timeout}ms`);
+    }
+
+    // Progress callback
+    if (options.onProgress) {
+      options.onProgress(status);
+    }
+
+    // Log progress
+    console.log(
+      `Job ${status.status}: ${status.completedBatches}/${status.totalBatches} batches ` +
+      `(${Math.round((status.completedBatches / status.totalBatches) * 100)}%)`
+    );
+
+    // Wait before next poll
+    await new Promise(resolve => setTimeout(resolve, options.interval));
+  }
+}
+
+// Usage
+const finalStatus = await pollJobCompletion(client, job.id, {
+  interval: 30000,
+  timeout: 3600000,
+  onProgress: (status) => {
+    console.log(`Progress: ${status.completedBatches}/${status.totalBatches}`);
+  }
+});
+```
+
+### Advanced: Adaptive Polling
+
+```typescript
+/**
+ * Adjust polling interval based on job size
+ */
+function getPollingInterval(totalBatches: number): number {
+  if (totalBatches < 10) {
+    return 10000;  // 10 seconds for small jobs
+  } else if (totalBatches < 100) {
+    return 30000;  // 30 seconds for medium jobs
+  } else {
+    return 60000;  // 1 minute for large jobs
+  }
+}
+
+// Usage
+const interval = getPollingInterval(batches.length);
+await pollJobCompletion(client, job.id, { interval });
+```
+
+---
+
+## Error Handling in Batches
+
+### Error Types
+
+| Error Level | Scope | Example | Recovery |
+|-------------|-------|---------|----------|
+| **File Error** | Entire file | Invalid CSV format | Move to error folder |
+| **Record Error** | Single record | Missing required field | Log, continue with others |
+| **Batch Error** | 100-250 records | API validation error | Logged by Fluent, continue |
+| **Job Error** | Entire job | Authentication failure | Retry entire job |
+| **Partial Batch Failure** | Some batches succeed, some fail | Network interruption | **NEW: PartialBatchRecovery** |
+
+### NEW: Partial Batch Recovery (v0.1.10+)
+
+**Problem**: What happens when sending 500 batches and batch #250 fails due to network error? You don't want to resend the first 249 successful batches.
+
+**Solution**: The SDK provides `PartialBatchRecovery` service to track batch progress and resume from failure point.
+
+#### How It Works
+
+```typescript
+import { PartialBatchRecovery, createClient } from '@fluentcommerce/fc-connect-sdk';
+
+/**
+ * Partial Batch Recovery Pattern
+ *
+ * Tracks per-record success/failure in batch operations and enables:
+ * - Retrying only failed records instead of entire batch
+ * - Checkpoint/resume functionality
+ * - Detailed error reporting per record
+ */
+
+async function resilientBatchSync() {
+  const client = await createClient({ config });
+  const recovery = new PartialBatchRecovery(logger);
+
+  // Step 1: Create job
+  const job = await client.createJob({
+    name: 'Daily Inventory Sync',
+    retailerId: '2',
+  });
+
+  // Step 2: Process batch with automatic recovery
+  const result = await recovery.processBatchWithRecovery(
+    records,
+    async (batch) => {
+      // Your batch processing logic
+      return await client.sendBatch(job.id, {
+        action: 'UPSERT',
+        entityType: 'INVENTORY',
+        entities: batch
+      });
+    },
+    {
+      maxRetries: 3,
+      retryOnlyFailed: true,  // Only retry failed records
+      retryDelayMs: 1000,     // Start with 1 second
+      retryBatchSize: 100,    // Process 100 at a time
+      checkpointKey: 'inventory-sync-2025-01-24'
+    }
+  );
+
+  console.log(`✓ Success: ${result.successCount}/${result.totalRecords}`);
+  console.log(`✗ Failed: ${result.failedCount} records`);
+
+  if (result.failedCount > 0) {
+    console.error('Failed records:', result.failedRecords);
+    console.log(`Checkpoint saved: ${result.checkpointId}`);
+  }
+}
+```
+
+#### Integration with Batch API
+
+```typescript
+import {
+  createClient,
+  PartialBatchRecovery
+} from '@fluentcommerce/fc-connect-sdk';
+
+async function batchIngestionWithRecovery(records: any[]) {
+  const client = await createClient({ config });
+  const recovery = new PartialBatchRecovery(logger);
+
+  // Create job
+  const job = await client.createJob({
+    name: 'Inventory Ingestion with Recovery',
+    retailerId: 'my-retailer'
+  });
+
+  // Process with recovery
+  const result = await recovery.processBatchWithRecovery(
+    records,
+    async (batch) => {
+      const response = await client.sendBatch(job.id, {
+        action: 'UPSERT',
+        entityType: 'INVENTORY',
+        entities: batch
+      });
+
+      logger.info('Batch sent', {
+        batchId: response.id,
+        recordCount: batch.length
+      });
+
+      return response;
+    },
+    {
+      maxRetries: 3,
+      retryOnlyFailed: true,
+      checkpointKey: `job-${job.id}`
+    }
+  );
+
+  // Check job status
+  const status = await client.getJobStatus(job.id);
+
+  return {
+    jobId: job.id,
+    jobStatus: status.status,
+    ...result
+  };
+}
+```
+
+#### Checkpoint and Resume
+
+```typescript
+// Process batch and save checkpoint
+const result = await recovery.processBatchWithRecovery(
+  records,
+  processBatch,
+  {
+    maxRetries: 3,
+    checkpointKey: 'daily-inventory-sync'
+  }
+);
+
+if (result.failedCount > 0) {
+  console.log(`Checkpoint created: ${result.checkpointId}`);
+  console.log(`Failed records saved for later retry`);
+}
+
+// Later: Resume from checkpoint
+const checkpointId = result.checkpointId;
+const resumeResult = await recovery.resumeFromCheckpoint(
+  checkpointId,
+  processBatch,
+  {
+    maxRetries: 5  // More retries on resume
+  }
+);
+
+console.log(`Resume: ${resumeResult.successCount} recovered`);
+```
+
+#### Custom Retry Logic
+
+```typescript
+const result = await recovery.processBatchWithRecovery(
+  records,
+  processBatch,
+  {
+    maxRetries: 5,
+    retryDelayMs: 2000,
+    // Custom retry decision
+    shouldRetry: (error, attemptCount) => {
+      // Don't retry validation errors
+      if (error.message.includes('validation')) {
+        return false;
+      }
+
+      // Don't retry after 3 attempts for rate limits
+      if (error.message.includes('rate limit') && attemptCount > 3) {
+        return false;
+      }
+
+      // Retry all other errors
+      return true;
+    }
+  }
+);
+```
+
+#### Record Failure Details
+
+```typescript
+// Access detailed failure information
+if (result.failedCount > 0) {
+  result.failedRecords.forEach(failure => {
+    console.error(`Record ${failure.index} failed:`, {
+      record: failure.record,
+      error: failure.error.message,
+      attempts: failure.attemptCount,
+      timestamp: failure.timestamp
+    });
+  });
+
+  // Export failures for manual review
+  await fs.writeFile(
+    'failed-records.json',
+    JSON.stringify(result.failedRecords, null, 2)
+  );
+}
+```
+
+#### Key Features
+
+| Feature | Description | Benefit |
+|---------|-------------|---------|
+| **Per-record tracking** | Tracks each record individually | Know exactly which records failed |
+| **Selective retry** | Retry only failures, not successes | Efficient retry logic |
+| **Checkpoint support** | Resume from failure point | Handle interruptions |
+| **Exponential backoff** | Configurable retry delays | Avoid overwhelming API |
+| **Custom retry logic** | Override retry decisions | Fine-grained control |
+
+#### API Reference
+
+**Constructor:**
+```typescript
+new PartialBatchRecovery(logger?: StructuredLogger)
+```
+
+**Methods:**
+
+| Method | Description | Parameters |
+|--------|-------------|------------|
+| `processBatchWithRecovery()` | Process batch with recovery | records, processor, options |
+| `resumeFromCheckpoint()` | Resume from saved checkpoint | checkpointId, processor, options |
+
+#### When to Use Partial Batch Recovery
+
+✅ **Use when**:
+- Sending 50+ batches (high failure risk)
+- Network is unstable
+- Long-running jobs (> 10 minutes)
+- Scheduled workflows that may be interrupted
+- Critical data that must complete
+
+❌ **Skip when**:
+- < 10 batches (low failure risk)
+- Fast operations (< 2 minutes)
+- Non-critical data (can rerun from scratch)
+- Memory-constrained environments
+
+### File-Level Error Handling
+
+```typescript
+try {
+  // Download file
+  const content = await s3.downloadFile(fileKey);
+
+  // Parse CSV
+  const records = await parser.parse(content);
+
+  // Process batches
+  await processBatches(records);
+
+  // Archive on success
+  await s3.moveFile(fileKey, archiveKey);
+
+} catch (error: any) {
+  console.error(`File processing failed: ${error.message}`);
+
+  // Move to error folder
+  await s3.moveFile(fileKey, errorKey);
+
+  // Log error details
+  await s3.uploadFile(errorKey + '.error.log', JSON.stringify({
+    file: fileKey,
+    error: error.message,
+    stack: error.stack,
+    timestamp: new Date().toISOString()
+  }, null, 2));
+
+  // Continue with next file (don't throw)
+}
+```
+
+### Record-Level Error Handling
+
+```typescript
+const mappedRecords = [];
+const mappingErrors = [];
+
+for (const record of records) {
+  const result = await mapper.map(record);
+
+  if (result.success) {
+    mappedRecords.push(result.data);
+  } else {
+    mappingErrors.push({
+      record,
+      errors: result.errors
+    });
+  }
+}
+
+console.log(`Mapped ${mappedRecords.length} records, ${mappingErrors.length} errors`);
+
+// Write error report
+if (mappingErrors.length > 0) {
+  await s3.uploadFile(
+    'inventory/errors/mapping-errors.json',
+    JSON.stringify(mappingErrors, null, 2)
+  );
+}
+
+// Continue with successful records
+```
+
+### Batch API Error Reporting
+
+```typescript
+// After job completes, check for errors
+const status = await client.getJobStatus(job.id);
+
+if (status.errorSummary && status.errorSummary.totalErrors > 0) {
+  console.warn(`Job completed with ${status.errorSummary.totalErrors} errors`);
+
+  // Get error details
+  const errorDetails = await client.graphql({
+    query: `
+      query GetJobErrors($jobId: ID!) {
+        job(id: $jobId) {
+          batches(first: 100) {
+            edges {
+              node {
+                id
+                status
+                errors {
+                  recordRef
+                  errorType
+                  errorMessage
+                }
+              }
+            }
+          }
+        }
+      }
+    `,
+    variables: { jobId: job.id }
+  });
+
+  // Write error report
+  await s3.uploadFile(
+    'inventory/errors/batch-errors.json',
+    JSON.stringify(errorDetails, null, 2)
+  );
+}
+```
+
+---
+
+## Complete Implementation Example
+
+See **[S3 CSV Batch API Guide](../../../01-TEMPLATES/standalone/s3-csv-batch-api.md)** for production-ready implementation with:
+
+- Complete TypeScript code
+- Environment configuration
+- Error handling strategies
+- File archival patterns
+- Scheduling with cron
+- Monitoring and logging
+
+### Quick Example: Scheduled Versori Batch
+
+```typescript
+/**
+ * Versori Scheduled Workflow: Daily Inventory Sync
+ *
+ * Trigger: Cron (daily at 2 AM)
+ * Process: Download CSV from S3, send to Batch API
+ */
+
+import { createClient, S3DataSource, CSVParserService, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
+
+export default async function dailyInventorySync(activation: any, log: any, connections: any) {
+  try {
+    log.info('Starting daily inventory sync');
+
+    // Create client
+    const client = await createClient({
+      connection: connections.fluent_commerce,
+      logger: log
+    });
+
+    // Create S3 source
+    const s3 = new S3DataSource({
+      connection: connections.aws_s3
+    }, log);
+
+    // List files
+    const files = await s3.listFiles('inventory/daily/');
+    log.info(`Found ${files.length} files to process`);
+
+    // Process each file
+    for (const file of files) {
+      await processFile(client, s3, file.key, log);
+    }
+
+    return { status: 200, body: { success: true, filesProcessed: files.length } };
+
+  } catch (error: any) {
+    log.error('Daily sync failed', error);
+    return { status: 500, body: { success: false, error: error.message } };
+  }
+}
+
+async function processFile(client: any, s3: any, fileKey: string, log: any) {
+  try {
+    // Download
+    const content = await s3.downloadFile(fileKey);
+
+    // Parse
+    const parser = new CSVParserService({ headers: true });
+    const records = await parser.parse(content);
+
+    // Map
+    const mapper = new UniversalMapper({
+      fields: {
+        ref: { source: 'sku', resolver: 'custom.buildRef' },
+        type: { value: 'INVENTORY' },
+        productRef: { source: 'sku', required: true },
+        locationRef: { source: 'location', required: true },
+        onHand: { source: 'qty', resolver: 'sdk.parseInt' }
+      }
+    }, {
+      customResolvers: {
+        'custom.buildRef': (v, d) => `${d.sku}-${d.location}`
+      }
+    });
+
+    const mapped = [];
+    for (const rec of records) {
+      const result = await mapper.map(rec);
+      if (result.success) mapped.push(result.data);
+    }
+
+    // Create job
+    const job = await client.createJob({
+      name: `Daily Inventory - ${fileKey}`,
+      retailerId: '2'
+    });
+
+    // Send batches
+    const BATCH_SIZE = 100;
+    for (let i = 0; i < mapped.length; i += BATCH_SIZE) {
+      await client.sendBatch(job.id, {
+        entities: mapped.slice(i, i + BATCH_SIZE)
+      });
+    }
+
+    // Poll completion
+    let status = await client.getJobStatus(job.id);
+    while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+      await new Promise(r => setTimeout(r, 30000));
+      status = await client.getJobStatus(job.id);
+    }
+
+    if (status.status === 'COMPLETED') {
+      await s3.moveFile(fileKey, fileKey.replace('daily/', 'archive/'));
+      log.info(`✓ Processed ${fileKey}`);
+    } else {
+      throw new Error(`Job failed: ${status.status}`);
+    }
+
+  } catch (error: any) {
+    log.error(`Failed to process ${fileKey}`, error);
+    await s3.moveFile(fileKey, fileKey.replace('daily/', 'errors/'));
+  }
+}
+```
+
+---
+
+## Next Steps
+
+Now that you understand batch processing, you're ready to learn delta sync for incremental updates!
+
+**Continue to:** [Module 3: Delta Sync →](./integration-patterns-03-delta-sync.md)
+
+Or explore:
+- [Module 5: Error Handling](./integration-patterns-05-error-handling.md) - Resilience strategies
+- [Complete Example: S3 CSV Batch API](../../../01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md)
+- [Complete Example: Versori Scheduled CSV](../../../01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md)
+
+---
+
+## Additional Resources
+
+- [Fluent Batch API Documentation](https://docs.fluentcommerce.com/)
+- [Universal Mapping Guide](../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md)
+- [S3DataSource API Reference](../../../02-CORE-GUIDES/data-sources/modules/data-sources-02-s3-operations.md)
+- [CSVParserService API Reference](../../../02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-02-csv-parser.md)
+
+---
+
+[← Back to Index](../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md) | [Previous: Real-Time →](./integration-patterns-01-real-time-processing.md) | [Next: Delta Sync →](./integration-patterns-03-delta-sync.md)