npm - @fluentcommerce/fc-connect-sdk - Versions diffs - 0.1.53 → 0.1.55 - Mend

@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (495) hide show

package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-02-batch-processing.md CHANGED Viewed

@@ -1,1547 +1,1547 @@
-# Module 2: Batch Processing
-> **Learning Objective:** Master batch processing patterns for high-volume data synchronization using the Fluent Batch API and SDK orchestration services.
->
-> **Level:** Intermediate
-## Table of Contents
-1. [What is Batch Processing?](#what-is-batch-processing)
-2. [When to Use Batch Processing](#when-to-use-batch-processing)
-3. [Fluent Batch API Overview](#fluent-batch-api-overview)
-4. [SDK Batch Components](#sdk-batch-components)
-5. [Basic Batch Workflow](#basic-batch-workflow)
-6. [Job Creation Strategies](#job-creation-strategies)
-7. [Batch Size Optimization](#batch-size-optimization)
-8. [Status Polling and Completion](#status-polling-and-completion)
-9. [Error Handling in Batches](#error-handling-in-batches)
-10. [Complete Implementation Example](#complete-implementation-example)
-11. [Next Steps](#next-steps)
----
-## What is Batch Processing?
-**Batch processing** means grouping multiple records together and processing them as a single unit, optimized for throughput over latency.
-### Key Characteristics
-| Characteristic | Description | Example |
-|----------------|-------------|---------|
-| **High Volume** | Process thousands of records | 50,000 inventory positions |
-| **Scheduled** | Runs at specific times | Daily at 2 AM |
-| **Asynchronous** | Submit job, poll for completion | Job completes in 5-10 minutes |
-| **Bulk Operations** | Optimize for throughput | 100 records per API call |
-### How It Works
-```
-CSV File (50K records) → Parse → Transform → Batch API → Job → Poll Status → Complete
-     ↓                      ↓         ↓            ↓         ↓         ↓           ↓
-  S3 bucket          Parse chunks  Map fields  Create job  Submit  Check every  Archive
-                                                           batches  30 seconds    file
-```
-### Visual Flow
-```
-┌─────────────────┐
-│ S3 CSV File     │
-│ (50,000 records)│
-└────────┬────────┘
-         │
-         │ ①Download & Parse
-         ▼
-┌─────────────────┐
-│   CSVParserService     │
-│  Stream chunks  │
-└────────┬────────┘
-         │
-         │ ②Transform fields
-         ▼
-┌─────────────────┐
-│ UniversalMapper │
-│  Field mapping  │
-└────────┬────────┘
-         │
-         │ ③Create Batch job
-         ▼
-┌─────────────────┐
-│ FluentClient    │
-│  createJob()    │
-└────────┬────────┘
-         │
-         │ ④Send batches (100 records each)
-         ▼
-┌─────────────────┐
-│  sendBatch()    │ ────┐
-│  500 batches    │     │ Parallel processing
-│  of 100 records │ ────┤ on Fluent servers
-└────────┬────────┘     │
-         │              │
-         │ ⑤Poll status every 30s
-         ▼
-┌─────────────────┐
-│  getJobStatus() │
-│  PROCESSING...  │
-│  COMPLETED ✓    │
-└────────┬────────┘
-         │
-         │ ⑥Archive file
-         ▼
-┌─────────────────┐
-│ S3 Archive      │
-│ Success log     │
-└─────────────────┘
-```
-**Total time**: 5-10 minutes for 50,000 records (vs 2-3 hours for sequential GraphQL mutations).
----
-## When to Use Batch Processing
-### ✅ Use Batch Processing When:
-| Scenario | Why Batch? | SDK Pattern |
-|----------|------------|-------------|
-| **Large Files** | 1K+ records | Batch API with streaming |
-| **Daily Sync** | Full inventory sync | Scheduled batch job |
-| **High Volume** | > 1,000 events/hour | Batch API (not webhooks) |
-| **Bulk Updates** | Mass price changes | Single job, multiple batches |
-| **CSV/Parquet Files** | Structured file formats | S3DataSource + FluentClient |
-### ❌ Avoid Batch Processing When:
-| Scenario | Why Not Batch? | Use Instead |
-|----------|----------------|-------------|
-| **Immediate Updates** | Need < 5 second latency | Real-time (Module 1) |
-| **Single Records** | 1-10 records | Direct GraphQL mutation |
-| **Event-Driven** | External webhook triggers | Real-time webhook |
-| **Critical Orders** | Customer waiting for confirmation | Real-time processing |
----
-## Fluent Batch API Overview
-### What is Batch API?
-The Fluent Batch API is a specialized GraphQL endpoint for bulk data operations:
-- **Asynchronous**: Submit job, get ID, poll for completion
-- **High-throughput**: 10,000+ records in minutes
-- **Fault-tolerant**: Partial failures don't block entire job
-- **Entity-specific**: Currently supports `InventoryQuantity` only
-### Supported Operations
-| Entity | Operations | Max per Batch |
-|--------|-----------|---------------|
-| `InventoryQuantity` | Create, Update | 100-250 records |
-**IMPORTANT**: Batch API currently **only supports InventoryQuantity**. For other entities (Order, Product, Customer), use standard GraphQL mutations.
-### Batch API Workflow
-```graphql
-# Step 1: Create job
-mutation CreateJob {
-  createJob(input: {
-    name: "Daily Inventory Sync"
-    retailerId: "2"
-  }) {
-    id
-    status
-  }
-}
-# Step 2: Send batches
-mutation SendBatch($jobId: ID!, $entities: [InventoryQuantityInput!]!) {
-  sendBatch(jobId: $jobId, entities: $entities) {
-    id
-    status
-    recordCount
-  }
-}
-# Step 3: Poll status
-query GetJobStatus($jobId: ID!) {
-  job(id: $jobId) {
-    id
-    status  # PENDING, PROCESSING, COMPLETED, FAILED
-    totalBatches
-    completedBatches
-    errorSummary {
-      totalErrors
-      errorTypes
-    }
-  }
-}
-```
-### Job Lifecycle
-```
-CREATE_JOB → PENDING → SEND_BATCHES → PROCESSING → COMPLETED
-                ↓                                      ↓
-             (can add more batches)            (can check errors)
-```
----
-## SDK Batch Components
-### Component 1: FluentClient Batch Methods
-The SDK provides batch methods directly on `FluentClient` (there is no separate `FluentBatchManager` class):
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-const client = await createClient({
-  config: {
-    baseUrl: 'https://api.fluentcommerce.com',
-    clientId: process.env.FLUENT_CLIENT_ID,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET,
-    retailerId: process.env.FLUENT_RETAILER_ID
-  }
-});
-// Create job
-const job = await client.createJob({
-  name: 'Daily Inventory Sync',
-  retailerId: '2'
-});
-// Send batch
-const batch = await client.sendBatch(job.id, {
-  entities: inventoryRecords
-});
-// Get status
-const status = await client.getJobStatus(job.id);
-// Get detailed status with batches
-const jobDetail = await client.getBatchStatus(job.id, batch.id);
-```
-**Available methods**:
-- `createJob(input)` - Create new Batch job
-- `sendBatch(jobId, data)` - Submit batch of records
-- `getJobStatus(jobId)` - Get job status and summary
-- `getBatchStatus(jobId, batchId)` - Get individual batch details
-### Component 2: `S3DataSource`
-**Purpose**: Read and write files from S3 with streaming support
-```typescript
-import { S3DataSource } from '@fluentcommerce/fc-connect-sdk';
-const s3 = new S3DataSource({
-  type: 'S3_CSV',
-  connectionId: 'my-s3',
-  name: 'My S3 Source',
-  s3Config: {
-    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
-    region: 'us-east-1',
-    bucket: 'inventory-bucket'
-  }
-}, logger);
-// List files
-const files = await s3.listFiles('inventory/updates/');
-// Download file
-const fileContent = await s3.downloadFile('inventory/updates/inventory.csv');
-// Download large file as Buffer (for files >100MB)
-const buffer = await s3.downloadFile('inventory/updates/large-file.csv', { encoding: 'binary' });
-// Upload file
-await s3.uploadFile('inventory/archive/processed.csv', content);
-// Move file
-await s3.moveFile('inventory/updates/file.csv', 'inventory/archive/file.csv');
-```
-**Key features**:
-- Streaming support for large files (memory-efficient)
-- Automatic retry on network errors
-- Progress callbacks for uploads/downloads
-### Component 3: `CSVParserService`
-**Purpose**: Parse CSV files with validation and streaming
-```typescript
-import { CSVParserService } from '@fluentcommerce/fc-connect-sdk';
-const parser = new CSVParserService({
-  headers: true,           // First row is headers
-  skipEmptyLines: true,    // Ignore blank lines
-  delimiter: ',',          // CSV delimiter
-  quote: '"',              // Quote character
-  escape: '\\'             // Escape character
-});
-// Parse entire file (for small files < 10MB)
-const records = await parser.parse(csvContent);
-// Stream parse (for large files)
-const recordStream = parser.streamParse(csvStream);
-for await (const record of recordStream) {
-  // Process record
-}
-```
-**Validation options**:
-- Required columns
-- Type validation
-- Custom validators
-### Component 4: `UniversalMapper`
-**Purpose**: Transform CSV/JSON fields to Fluent schema
-```typescript
-import { UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
-const mappingConfig = {
-  fields: {
-    ref: {
-      source: 'sku',
-      resolver: 'custom.buildRef'  // Combine sku + location
-    },
-    type: {
-      value: 'INVENTORY'  // Static value
-    },
-    status: {
-      source: 'status',
-      resolver: 'sdk.uppercase'
-    },
-    productRef: {
-      source: 'sku',
-      required: true
-    },
-    locationRef: {
-      source: 'warehouse_code',
-      required: true
-    },
-    onHand: {
-      source: 'quantity',
-      resolver: 'sdk.parseInt'
-    }
-  }
-};
-const mapper = new UniversalMapper(mappingConfig, {
-  customResolvers: {
-    'custom.buildRef': (value, data) => {
-      return `${data.sku}-${data.warehouse_code}`;
-    }
-  }
-});
-const result = await mapper.map(csvRecord);
-// result.data = { ref: 'SKU001-WH01', type: 'INVENTORY', ... }
-```
-### Workflow Composition Pattern
-**Purpose**: Compose SDK services for complete ingestion workflows
-Instead of using a single orchestrator, compose the above components into your custom workflow:
-```typescript
-import {
-  createClient,
-  S3DataSource,
-  CSVParserService,
-  UniversalMapper,
-  StateService,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-async function processInventoryFiles() {
-  const logger = toStructuredLogger(createConsoleLogger(), { logLevel: 'info' });
-  // Initialize components
-  const client = await createClient({ config });
-  const s3 = new S3DataSource(s3Config, logger);
-  const parser = new CSVParserService();
-  const mapper = new UniversalMapper(mappingConfig);
-  const stateService = new StateService(logger);
-  // List and process files
-  const files = await s3.listFiles({ prefix: 'inventory/updates/' });
-  for (const file of files) {
-    if (await stateService.isFileProcessed(file.name)) continue;
-    try {
-      // 1. Download and parse
-      const content = await s3.downloadFile(file.name);
-      const records = await parser.parse(content);
-      // 2. Map fields
-      const inventory = [];
-      for (const record of records) {
-        const result = await mapper.map(record);
-        if (result.success) inventory.push(result.data);
-      }
-      // 3. Create job and send batches
-      const job = await client.createJob({
-        name: `Inventory - ${file.name}`,
-        retailerId: '1'
-      });
-      const batches = chunkArray(inventory, 100);
-      for (const batch of batches) {
-        await client.sendBatch(job.id, {
-          action: 'UPSERT',
-          entityType: 'INVENTORY',
-          entities: batch
-        });
-      }
-      // 4. Archive and mark processed
-      await s3.moveFile(file.name, `inventory/archive/${file.name}`);
-      await stateService.markFileProcessed(file.name);
-    } catch (error) {
-      logger.error(`Failed to process ${file.name}`, error);
-      await s3.moveFile(file.name, `inventory/errors/${file.name}`);
-    }
-  }
-}
-```
-**Benefits of building block composition**:
-1. ✅ Full control over workflow logic
-2. ✅ Custom error handling
-3. ✅ Easy to test individual components
-4. ✅ Flexible batch sizing and job strategies
-5. ✅ Works with any entity type (not just INVENTORY)
-6. ✅ Easy to add custom business rules
----
-## Basic Batch Workflow
-### Step-by-Step Pattern
-```typescript
-/**
- * Basic Batch Processing Workflow
- *
- * Steps:
- * 1. Create Fluent client
- * 2. Download and parse CSV file
- * 3. Transform records to Fluent schema
- * 4. Create Batch job
- * 5. Send records in batches
- * 6. Poll status until complete
- * 7. Handle results
- */
-import { createClient, S3DataSource, CSVParserService, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
-async function batchInventorySync() {
-  // Step 1: Create client
-  const client = await createClient({
-    baseUrl: process.env.FLUENT_BASE_URL,
-    clientId: process.env.FLUENT_CLIENT_ID,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET,
-    retailerId: process.env.FLUENT_RETAILER_ID
-  });
-  // Step 2: Download CSV
-  const s3 = new S3DataSource({
-    type: 'S3_CSV',
-    connectionId: 'my-s3',
-    name: 'My S3 Source',
-    s3Config: {
-      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
-      region: process.env.AWS_REGION,
-      bucket: process.env.AWS_BUCKET
-    }
-  }, console);
-  const csvContent = await s3.downloadFile('inventory/daily-update.csv');
-  // Step 3: Parse CSV
-  const parser = new CSVParserService({ headers: true });
-  const records = await parser.parse(csvContent);
-  console.log(`Parsed ${records.length} records`);
-  // Step 4: Transform records
-  const mapper = new UniversalMapper({
-    fields: {
-      ref: { source: 'sku', resolver: 'custom.buildRef' },
-      type: { value: 'INVENTORY' },
-      productRef: { source: 'sku', required: true },
-      locationRef: { source: 'location', required: true },
-      onHand: { source: 'qty', resolver: 'sdk.parseInt' }
-    }
-  }, {
-    customResolvers: {
-      'custom.buildRef': (value, data) => `${data.sku}-${data.location}`
-    }
-  });
-  const mappedRecords = [];
-  for (const record of records) {
-    const result = await mapper.map(record);
-    if (result.success) {
-      mappedRecords.push(result.data);
-    } else {
-      console.error('Mapping error:', result.errors);
-    }
-  }
-  console.log(`Mapped ${mappedRecords.length} records`);
-  // Step 5: Create Batch job
-  const job = await client.createJob({
-    name: `Inventory Sync ${new Date().toISOString()}`,
-    retailerId: '2'
-  });
-  console.log(`Created job ${job.id}`);
-  // Step 6: Send batches
-  const BATCH_SIZE = 100;
-  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
-    const chunk = mappedRecords.slice(i, i + BATCH_SIZE);
-    const batch = await client.sendBatch(job.id, {
-      entities: chunk
-    });
-    console.log(`Sent batch ${batch.id} (${chunk.length} records)`);
-  }
-  // Step 7: Poll status
-  let status = await client.getJobStatus(job.id);
-  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-    console.log(`Job status: ${status.status} (${status.completedBatches}/${status.totalBatches} batches)`);
-    await new Promise(resolve => setTimeout(resolve, 30000)); // Wait 30 seconds
-    status = await client.getJobStatus(job.id);
-  }
-  // Step 8: Handle results
-  if (status.status === 'COMPLETED') {
-    console.log('✓ Job completed successfully');
-    console.log(`Total records: ${status.totalRecords}`);
-    console.log(`Errors: ${status.errorSummary?.totalErrors || 0}`);
-    // Archive file
-    await s3.moveFile('inventory/daily-update.csv', 'inventory/archive/daily-update.csv');
-  } else {
-    console.error('✗ Job failed:', status.status);
-    // Move to error folder
-    await s3.moveFile('inventory/daily-update.csv', 'inventory/errors/daily-update.csv');
-  }
-}
-batchInventorySync().catch(console.error);
-```
----
-## NEW: Job Lifecycle Tracking (v0.1.10+)
-**Track job state, metadata, and lifecycle** across your integration workflows.
-The SDK provides `JobTracker` service for managing job lifecycle, tracking status, and storing job metadata.
-### JobTracker Overview
-```typescript
-import { JobTracker, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-// ✅ CORRECT: Access openKv from Versori context
-// import { openKv } from '@versori/run'; // ❌ WRONG - Not a direct export
-// In Versori workflow handler:
-const { openKv } = ctx;
-const kvAdapter = new VersoriKVAdapter(openKv(':project:'));
-const tracker = new JobTracker(kvAdapter, logger);
-// Create job
-const jobId = `scheduled_${Date.now()}`;
-await tracker.createJob(jobId, {
-  triggeredBy: 'schedule',
-  stage: 'initialization',
-  details: {
-    catalogueRef: 'DEFAULT:1',
-    fileName: 'inventory.csv'
-  }
-});
-// Update progress
-await tracker.updateJob(jobId, {
-  status: 'processing',
-  stage: 'extraction',
-  message: 'Extracting records from S3'
-});
-await tracker.updateJob(jobId, {
-  stage: 'transformation',
-  message: 'Mapping 1000 records',
-  details: { recordCount: 1000 }
-});
-// Mark as completed
-await tracker.markCompleted(jobId, {
-  recordCount: 1000,
-  successCount: 998,
-  failedCount: 2
-});
-// Or mark as failed
-try {
-  // ... job logic ...
-} catch (error) {
-  await tracker.markFailed(jobId, error);
-}
-```
-### Complete Example with Versori
-```typescript
-import {
-  createClient,
-  JobTracker,
-  VersoriKVAdapter,
-} from '@fluentcommerce/fc-connect-sdk';
-import { schedule } from '@versori/run';
-/**
- * Versori workflow with complete job tracking
- */
-export const dailyInventorySync = schedule('daily-inventory', '0 2 * * *')
-  .execute(async ({ log, connections, vars, kv }) => {
-    const jobId = `inventory_${Date.now()}`;
-    const tracker = new JobTracker(new VersoriKVAdapter(kv), log);
-    try {
-      // Create job
-      await tracker.createJob(jobId, {
-        triggeredBy: 'schedule',
-        stage: 'start',
-        details: { schedule: 'daily 2am' }
-      });
-      // Stage 1: Extraction
-      await tracker.updateJob(jobId, {
-        status: 'processing',
-        stage: 'extraction',
-        message: 'Querying virtual positions'
-      });
-      const data = await extractFromFluent();
-      // Stage 2: Transformation
-      await tracker.updateJob(jobId, {
-        stage: 'transformation',
-        message: `Processing ${data.length} records`
-      });
-      const transformed = await transformData(data);
-      // Stage 3: Upload
-      await tracker.updateJob(jobId, {
-        stage: 'upload',
-        message: 'Uploading to SFTP'
-      });
-      await uploadToSFTP(transformed);
-      // Completed
-      await tracker.markCompleted(jobId, {
-        recordCount: data.length,
-        fileName: `inventory_${jobId}.xml`
-      });
-      log.info('Job completed successfully', { jobId });
-    } catch (error) {
-      await tracker.markFailed(jobId, error);
-      log.error('Job failed', error);
-      throw error;
-    }
-  });
-```
-### Querying Job Status
-```typescript
-// Get job status
-const status = await tracker.getJob(jobId);
-if (status) {
-  console.log(`Job ${jobId}:`, {
-    status: status.status,
-    stage: status.stage,
-    message: status.message,
-    createdAt: status.createdAt,
-    completedAt: status.completedAt
-  });
-}
-// Check if job is still running
-if (status.status === 'processing') {
-  console.log(`Job in progress: ${status.stage}`);
-}
-// Check for errors
-if (status.status === 'failed') {
-  console.error('Job failed:', {
-    error: status.error,
-    stack: status.errorStack
-  });
-}
-```
-### Custom TTL Configuration
-```typescript
-// Default TTL: 7 days
-const tracker = new JobTracker(kvAdapter, logger);
-// Custom TTL: 24 hours
-const shortTracker = new JobTracker(
-  kvAdapter,
-  logger,
-  86400 // 24 hours in seconds
-);
-// Custom TTL: 30 days
-const longTracker = new JobTracker(
-  kvAdapter,
-  logger,
-  2592000 // 30 days in seconds
-);
-```
-### JobTracker API Reference
-| Method | Description | Parameters | Example |
-|--------|-------------|------------|---------|
-| `createJob(jobId, metadata)` | Create new job with 'queued' status | jobId, metadata | `await tracker.createJob('job_123', { triggeredBy: 'schedule' })` |
-| `updateJob(jobId, updates)` | Update job metadata/status/stage | jobId, updates | `await tracker.updateJob('job_123', { status: 'processing' })` |
-| `getJob(jobId)` | Get job by ID | jobId | `const job = await tracker.getJob('job_123')` |
-| `markCompleted(jobId, details)` | Mark job complete | jobId, details | `await tracker.markCompleted('job_123', { recordCount: 1000 })` |
-| `markFailed(jobId, error)` | Mark job failed | jobId, error | `await tracker.markFailed('job_123', error)` |
-**Constructor:**
-```typescript
-new JobTracker(kvAdapter: KVAdapter, logger: StructuredLogger, ttl?: number)
-```
-### When to Use JobTracker
-✅ **Use when**:
-- Need job history and audit trail
-- Monitoring multiple concurrent jobs
-- Debugging job failures
-- Tracking job duration and performance
-- Building dashboards or reports
-❌ **Skip when**:
-- Simple one-off scripts
-- No state persistence available
-- Memory-constrained environments
----
-## Job Creation Strategies
-### Strategy 1: Single Job Per File (Recommended)
-**When to use**: Most common pattern for daily/scheduled files
-```typescript
-// Process one file
-const job = await client.createJob({
-  name: `Daily Inventory - ${fileName}`,
-  retailerId: '2'
-});
-// Send all batches for this file
-for (const batch of batches) {
-  await client.sendBatch(job.id, { entities: batch });
-}
-// Poll until complete
-await pollJobCompletion(job.id);
-```
-**Pros**:
-- Simple error tracking (one job = one file)
-- Easy archival (job complete = archive file)
-- Clear audit trail
-**Cons**:
-- More jobs in system
-- Can't combine multiple files
-### Strategy 2: Single Job Per Day
-**When to use**: Multiple small files processed together
-```typescript
-// Create one job for entire day
-const job = await client.createJob({
-  name: `Daily Inventory - ${new Date().toISOString().split('T')[0]}`,
-  retailerId: '2'
-});
-// Process multiple files
-const files = await s3.listFiles('inventory/updates/');
-for (const file of files) {
-  const records = await processFile(file);
-  // Send batches for this file
-  for (const batch of chunkArray(records, 100)) {
-    await client.sendBatch(job.id, { entities: batch });
-  }
-}
-// Poll once for entire day's work
-await pollJobCompletion(job.id);
-```
-**Pros**:
-- Fewer jobs
-- Combine multiple sources
-- Single status check
-**Cons**:
-- Harder to track individual file errors
-- One file failure doesn't block others
-- More complex archival logic
-### Strategy 3: Long-Running Job (Advanced)
-**When to use**: Continuous processing (hourly micro-batches)
-```typescript
-// Create job once
-const job = await client.createJob({
-  name: `Inventory Stream - ${new Date().toISOString().split('T')[0]}`,
-  retailerId: '2'
-});
-// Send batches throughout the day
-setInterval(async () => {
-  const newRecords = await fetchLatestUpdates();
-  if (newRecords.length > 0) {
-    await client.sendBatch(job.id, { entities: newRecords });
-  }
-}, 3600000); // Every hour
-// Check status at end of day
-```
-**Pros**:
-- Minimize job creation overhead
-- Continuous processing
-**Cons**:
-- Job can stay open for hours
-- Harder to determine "completion"
-- Risk of very large jobs
----
-## Batch Size Optimization
-### Recommended Batch Sizes
-| Total Records | Batch Size | Batches | Processing Time |
-|---------------|------------|---------|-----------------|
-| < 1,000 | 100 | 1-10 | 1-2 minutes |
-| 1,000-10,000 | 100-200 | 10-100 | 3-10 minutes |
-| 10,000-50,000 | 200 | 50-250 | 5-15 minutes |
-| > 50,000 | 250 | 200+ | 10-30 minutes |
-**Max batch size**: 250 records (Fluent API limit)
-### Batch Size Calculator
-```typescript
-/**
- * Calculate optimal batch size based on total records
- */
-function calculateBatchSize(totalRecords: number): number {
-  if (totalRecords < 1000) {
-    return 100;
-  } else if (totalRecords < 10000) {
-    return 150;
-  } else {
-    return 250; // Max allowed
-  }
-}
-// Usage
-const batchSize = calculateBatchSize(records.length);
-const batches = chunkArray(records, batchSize);
-```
-### Chunking Utility
-```typescript
-/**
- * Split array into chunks
- */
-function chunkArray<T>(array: T[], size: number): T[][] {
-  const chunks: T[][] = [];
-  for (let i = 0; i < array.length; i += size) {
-    chunks.push(array.slice(i, i + size));
-  }
-  return chunks;
-}
-// Usage
-const batches = chunkArray(mappedRecords, 100);
-// [[record1, record2, ...], [record101, record102, ...], ...]
-```
-### Parallel Batch Sending (Advanced)
-```typescript
-/**
- * Send multiple batches in parallel
- * WARNING: Use with caution - can hit rate limits
- */
-async function sendBatchesParallel(
-  client: FluentClient,
-  jobId: string,
-  batches: any[][],
-  concurrency = 5
-) {
-  const results = [];
-  for (let i = 0; i < batches.length; i += concurrency) {
-    const chunk = batches.slice(i, i + concurrency);
-    // Send up to 'concurrency' batches at once
-    const promises = chunk.map(batch =>
-      client.sendBatch(jobId, { entities: batch })
-    );
-    const chunkResults = await Promise.all(promises);
-    results.push(...chunkResults);
-    console.log(`Sent batches ${i + 1} to ${i + chunk.length}`);
-  }
-  return results;
-}
-// Usage (send 5 batches at a time)
-await sendBatchesParallel(client, job.id, batches, 5);
-```
-**Caution**: Parallel sending can trigger rate limits. Start with concurrency=5, monitor, and adjust.
----
-## Status Polling and Completion
-### Basic Polling Pattern
-```typescript
-/**
- * Poll job status until complete
- */
-async function pollJobCompletion(
-  client: FluentClient,
-  jobId: string,
-  options = {
-    interval: 30000,      // 30 seconds
-    timeout: 3600000,     // 1 hour
-    onProgress: undefined // Callback
-  }
-) {
-  const startTime = Date.now();
-  while (true) {
-    const status = await client.getJobStatus(jobId);
-    // Check terminal states
-    if (status.status === 'COMPLETED') {
-      console.log('✓ Job completed successfully');
-      return status;
-    }
-    if (status.status === 'FAILED') {
-      throw new Error(`Job failed: ${status.errorSummary?.message || 'Unknown error'}`);
-    }
-    // Check timeout
-    if (Date.now() - startTime > options.timeout) {
-      throw new Error(`Job timeout after ${options.timeout}ms`);
-    }
-    // Progress callback
-    if (options.onProgress) {
-      options.onProgress(status);
-    }
-    // Log progress
-    console.log(
-      `Job ${status.status}: ${status.completedBatches}/${status.totalBatches} batches ` +
-      `(${Math.round((status.completedBatches / status.totalBatches) * 100)}%)`
-    );
-    // Wait before next poll
-    await new Promise(resolve => setTimeout(resolve, options.interval));
-  }
-}
-// Usage
-const finalStatus = await pollJobCompletion(client, job.id, {
-  interval: 30000,
-  timeout: 3600000,
-  onProgress: (status) => {
-    console.log(`Progress: ${status.completedBatches}/${status.totalBatches}`);
-  }
-});
-```
-### Advanced: Adaptive Polling
-```typescript
-/**
- * Adjust polling interval based on job size
- */
-function getPollingInterval(totalBatches: number): number {
-  if (totalBatches < 10) {
-    return 10000;  // 10 seconds for small jobs
-  } else if (totalBatches < 100) {
-    return 30000;  // 30 seconds for medium jobs
-  } else {
-    return 60000;  // 1 minute for large jobs
-  }
-}
-// Usage
-const interval = getPollingInterval(batches.length);
-await pollJobCompletion(client, job.id, { interval });
-```
----
-## Error Handling in Batches
-### Error Types
-| Error Level | Scope | Example | Recovery |
-|-------------|-------|---------|----------|
-| **File Error** | Entire file | Invalid CSV format | Move to error folder |
-| **Record Error** | Single record | Missing required field | Log, continue with others |
-| **Batch Error** | 100-250 records | API validation error | Logged by Fluent, continue |
-| **Job Error** | Entire job | Authentication failure | Retry entire job |
-| **Partial Batch Failure** | Some batches succeed, some fail | Network interruption | **NEW: PartialBatchRecovery** |
-### NEW: Partial Batch Recovery (v0.1.10+)
-**Problem**: What happens when sending 500 batches and batch #250 fails due to network error? You don't want to resend the first 249 successful batches.
-**Solution**: The SDK provides `PartialBatchRecovery` service to track batch progress and resume from failure point.
-#### How It Works
-```typescript
-import { PartialBatchRecovery, createClient } from '@fluentcommerce/fc-connect-sdk';
-/**
- * Partial Batch Recovery Pattern
- *
- * Tracks per-record success/failure in batch operations and enables:
- * - Retrying only failed records instead of entire batch
- * - Checkpoint/resume functionality
- * - Detailed error reporting per record
- */
-async function resilientBatchSync() {
-  const client = await createClient({ config });
-  const recovery = new PartialBatchRecovery(logger);
-  // Step 1: Create job
-  const job = await client.createJob({
-    name: 'Daily Inventory Sync',
-    retailerId: '2',
-  });
-  // Step 2: Process batch with automatic recovery
-  const result = await recovery.processBatchWithRecovery(
-    records,
-    async (batch) => {
-      // Your batch processing logic
-      return await client.sendBatch(job.id, {
-        action: 'UPSERT',
-        entityType: 'INVENTORY',
-        entities: batch
-      });
-    },
-    {
-      maxRetries: 3,
-      retryOnlyFailed: true,  // Only retry failed records
-      retryDelayMs: 1000,     // Start with 1 second
-      retryBatchSize: 100,    // Process 100 at a time
-      checkpointKey: 'inventory-sync-2025-01-24'
-    }
-  );
-  console.log(`✓ Success: ${result.successCount}/${result.totalRecords}`);
-  console.log(`✗ Failed: ${result.failedCount} records`);
-  if (result.failedCount > 0) {
-    console.error('Failed records:', result.failedRecords);
-    console.log(`Checkpoint saved: ${result.checkpointId}`);
-  }
-}
-```
-#### Integration with Batch API
-```typescript
-import {
-  createClient,
-  PartialBatchRecovery
-} from '@fluentcommerce/fc-connect-sdk';
-async function batchIngestionWithRecovery(records: any[]) {
-  const client = await createClient({ config });
-  const recovery = new PartialBatchRecovery(logger);
-  // Create job
-  const job = await client.createJob({
-    name: 'Inventory Ingestion with Recovery',
-    retailerId: 'my-retailer'
-  });
-  // Process with recovery
-  const result = await recovery.processBatchWithRecovery(
-    records,
-    async (batch) => {
-      const response = await client.sendBatch(job.id, {
-        action: 'UPSERT',
-        entityType: 'INVENTORY',
-        entities: batch
-      });
-      logger.info('Batch sent', {
-        batchId: response.id,
-        recordCount: batch.length
-      });
-      return response;
-    },
-    {
-      maxRetries: 3,
-      retryOnlyFailed: true,
-      checkpointKey: `job-${job.id}`
-    }
-  );
-  // Check job status
-  const status = await client.getJobStatus(job.id);
-  return {
-    jobId: job.id,
-    jobStatus: status.status,
-    ...result
-  };
-}
-```
-#### Checkpoint and Resume
-```typescript
-// Process batch and save checkpoint
-const result = await recovery.processBatchWithRecovery(
-  records,
-  processBatch,
-  {
-    maxRetries: 3,
-    checkpointKey: 'daily-inventory-sync'
-  }
-);
-if (result.failedCount > 0) {
-  console.log(`Checkpoint created: ${result.checkpointId}`);
-  console.log(`Failed records saved for later retry`);
-}
-// Later: Resume from checkpoint
-const checkpointId = result.checkpointId;
-const resumeResult = await recovery.resumeFromCheckpoint(
-  checkpointId,
-  processBatch,
-  {
-    maxRetries: 5  // More retries on resume
-  }
-);
-console.log(`Resume: ${resumeResult.successCount} recovered`);
-```
-#### Custom Retry Logic
-```typescript
-const result = await recovery.processBatchWithRecovery(
-  records,
-  processBatch,
-  {
-    maxRetries: 5,
-    retryDelayMs: 2000,
-    // Custom retry decision
-    shouldRetry: (error, attemptCount) => {
-      // Don't retry validation errors
-      if (error.message.includes('validation')) {
-        return false;
-      }
-      // Don't retry after 3 attempts for rate limits
-      if (error.message.includes('rate limit') && attemptCount > 3) {
-        return false;
-      }
-      // Retry all other errors
-      return true;
-    }
-  }
-);
-```
-#### Record Failure Details
-```typescript
-// Access detailed failure information
-if (result.failedCount > 0) {
-  result.failedRecords.forEach(failure => {
-    console.error(`Record ${failure.index} failed:`, {
-      record: failure.record,
-      error: failure.error.message,
-      attempts: failure.attemptCount,
-      timestamp: failure.timestamp
-    });
-  });
-  // Export failures for manual review
-  await fs.writeFile(
-    'failed-records.json',
-    JSON.stringify(result.failedRecords, null, 2)
-  );
-}
-```
-#### Key Features
-| Feature | Description | Benefit |
-|---------|-------------|---------|
-| **Per-record tracking** | Tracks each record individually | Know exactly which records failed |
-| **Selective retry** | Retry only failures, not successes | Efficient retry logic |
-| **Checkpoint support** | Resume from failure point | Handle interruptions |
-| **Exponential backoff** | Configurable retry delays | Avoid overwhelming API |
-| **Custom retry logic** | Override retry decisions | Fine-grained control |
-#### API Reference
-**Constructor:**
-```typescript
-new PartialBatchRecovery(logger?: StructuredLogger)
-```
-**Methods:**
-| Method | Description | Parameters |
-|--------|-------------|------------|
-| `processBatchWithRecovery()` | Process batch with recovery | records, processor, options |
-| `resumeFromCheckpoint()` | Resume from saved checkpoint | checkpointId, processor, options |
-#### When to Use Partial Batch Recovery
-✅ **Use when**:
-- Sending 50+ batches (high failure risk)
-- Network is unstable
-- Long-running jobs (> 10 minutes)
-- Scheduled workflows that may be interrupted
-- Critical data that must complete
-❌ **Skip when**:
-- < 10 batches (low failure risk)
-- Fast operations (< 2 minutes)
-- Non-critical data (can rerun from scratch)
-- Memory-constrained environments
-### File-Level Error Handling
-```typescript
-try {
-  // Download file
-  const content = await s3.downloadFile(fileKey);
-  // Parse CSV
-  const records = await parser.parse(content);
-  // Process batches
-  await processBatches(records);
-  // Archive on success
-  await s3.moveFile(fileKey, archiveKey);
-} catch (error: any) {
-  console.error(`File processing failed: ${error.message}`);
-  // Move to error folder
-  await s3.moveFile(fileKey, errorKey);
-  // Log error details
-  await s3.uploadFile(errorKey + '.error.log', JSON.stringify({
-    file: fileKey,
-    error: error.message,
-    stack: error.stack,
-    timestamp: new Date().toISOString()
-  }, null, 2));
-  // Continue with next file (don't throw)
-}
-```
-### Record-Level Error Handling
-```typescript
-const mappedRecords = [];
-const mappingErrors = [];
-for (const record of records) {
-  const result = await mapper.map(record);
-  if (result.success) {
-    mappedRecords.push(result.data);
-  } else {
-    mappingErrors.push({
-      record,
-      errors: result.errors
-    });
-  }
-}
-console.log(`Mapped ${mappedRecords.length} records, ${mappingErrors.length} errors`);
-// Write error report
-if (mappingErrors.length > 0) {
-  await s3.uploadFile(
-    'inventory/errors/mapping-errors.json',
-    JSON.stringify(mappingErrors, null, 2)
-  );
-}
-// Continue with successful records
-```
-### Batch API Error Reporting
-```typescript
-// After job completes, check for errors
-const status = await client.getJobStatus(job.id);
-if (status.errorSummary && status.errorSummary.totalErrors > 0) {
-  console.warn(`Job completed with ${status.errorSummary.totalErrors} errors`);
-  // Get error details
-  const errorDetails = await client.graphql({
-    query: `
-      query GetJobErrors($jobId: ID!) {
-        job(id: $jobId) {
-          batches(first: 100) {
-            edges {
-              node {
-                id
-                status
-                errors {
-                  recordRef
-                  errorType
-                  errorMessage
-                }
-              }
-            }
-          }
-        }
-      }
-    `,
-    variables: { jobId: job.id }
-  });
-  // Write error report
-  await s3.uploadFile(
-    'inventory/errors/batch-errors.json',
-    JSON.stringify(errorDetails, null, 2)
-  );
-}
-```
----
-## Complete Implementation Example
-See **[S3 CSV Batch API Guide](../../../01-TEMPLATES/standalone/s3-csv-batch-api.md)** for production-ready implementation with:
-- Complete TypeScript code
-- Environment configuration
-- Error handling strategies
-- File archival patterns
-- Scheduling with cron
-- Monitoring and logging
-### Quick Example: Scheduled Versori Batch
-```typescript
-/**
- * Versori Scheduled Workflow: Daily Inventory Sync
- *
- * Trigger: Cron (daily at 2 AM)
- * Process: Download CSV from S3, send to Batch API
- */
-import { createClient, S3DataSource, CSVParserService, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
-export default async function dailyInventorySync(activation: any, log: any, connections: any) {
-  try {
-    log.info('Starting daily inventory sync');
-    // Create client
-    const client = await createClient({
-      connection: connections.fluent_commerce,
-      logger: log
-    });
-    // Create S3 source
-    const s3 = new S3DataSource({
-      connection: connections.aws_s3
-    }, log);
-    // List files
-    const files = await s3.listFiles('inventory/daily/');
-    log.info(`Found ${files.length} files to process`);
-    // Process each file
-    for (const file of files) {
-      await processFile(client, s3, file.key, log);
-    }
-    return { status: 200, body: { success: true, filesProcessed: files.length } };
-  } catch (error: any) {
-    log.error('Daily sync failed', error);
-    return { status: 500, body: { success: false, error: error.message } };
-  }
-}
-async function processFile(client: any, s3: any, fileKey: string, log: any) {
-  try {
-    // Download
-    const content = await s3.downloadFile(fileKey);
-    // Parse
-    const parser = new CSVParserService({ headers: true });
-    const records = await parser.parse(content);
-    // Map
-    const mapper = new UniversalMapper({
-      fields: {
-        ref: { source: 'sku', resolver: 'custom.buildRef' },
-        type: { value: 'INVENTORY' },
-        productRef: { source: 'sku', required: true },
-        locationRef: { source: 'location', required: true },
-        onHand: { source: 'qty', resolver: 'sdk.parseInt' }
-      }
-    }, {
-      customResolvers: {
-        'custom.buildRef': (v, d) => `${d.sku}-${d.location}`
-      }
-    });
-    const mapped = [];
-    for (const rec of records) {
-      const result = await mapper.map(rec);
-      if (result.success) mapped.push(result.data);
-    }
-    // Create job
-    const job = await client.createJob({
-      name: `Daily Inventory - ${fileKey}`,
-      retailerId: '2'
-    });
-    // Send batches
-    const BATCH_SIZE = 100;
-    for (let i = 0; i < mapped.length; i += BATCH_SIZE) {
-      await client.sendBatch(job.id, {
-        entities: mapped.slice(i, i + BATCH_SIZE)
-      });
-    }
-    // Poll completion
-    let status = await client.getJobStatus(job.id);
-    while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-      await new Promise(r => setTimeout(r, 30000));
-      status = await client.getJobStatus(job.id);
-    }
-    if (status.status === 'COMPLETED') {
-      await s3.moveFile(fileKey, fileKey.replace('daily/', 'archive/'));
-      log.info(`✓ Processed ${fileKey}`);
-    } else {
-      throw new Error(`Job failed: ${status.status}`);
-    }
-  } catch (error: any) {
-    log.error(`Failed to process ${fileKey}`, error);
-    await s3.moveFile(fileKey, fileKey.replace('daily/', 'errors/'));
-  }
-}
-```
----
-## Next Steps
-Now that you understand batch processing, you're ready to learn delta sync for incremental updates!
-**Continue to:** [Module 3: Delta Sync →](./integration-patterns-03-delta-sync.md)
-Or explore:
-- [Module 5: Error Handling](./integration-patterns-05-error-handling.md) - Resilience strategies
-- [Complete Example: S3 CSV Batch API](../../../01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md)
-- [Complete Example: Versori Scheduled CSV](../../../01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md)
----
-## Additional Resources
-- [Fluent Batch API Documentation](https://docs.fluentcommerce.com/)
-- [Universal Mapping Guide](../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md)
-- [S3DataSource API Reference](../../../02-CORE-GUIDES/data-sources/modules/data-sources-02-s3-operations.md)
-- [CSVParserService API Reference](../../../02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-02-csv-parser.md)
----
-[← Back to Index](../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md) | [Previous: Real-Time →](./integration-patterns-01-real-time-processing.md) | [Next: Delta Sync →](./integration-patterns-03-delta-sync.md)
+# Module 2: Batch Processing
+> **Learning Objective:** Master batch processing patterns for high-volume data synchronization using the Fluent Batch API and SDK orchestration services.
+>
+> **Level:** Intermediate
+## Table of Contents
+1. [What is Batch Processing?](#what-is-batch-processing)
+2. [When to Use Batch Processing](#when-to-use-batch-processing)
+3. [Fluent Batch API Overview](#fluent-batch-api-overview)
+4. [SDK Batch Components](#sdk-batch-components)
+5. [Basic Batch Workflow](#basic-batch-workflow)
+6. [Job Creation Strategies](#job-creation-strategies)
+7. [Batch Size Optimization](#batch-size-optimization)
+8. [Status Polling and Completion](#status-polling-and-completion)
+9. [Error Handling in Batches](#error-handling-in-batches)
+10. [Complete Implementation Example](#complete-implementation-example)
+11. [Next Steps](#next-steps)
+---
+## What is Batch Processing?
+**Batch processing** means grouping multiple records together and processing them as a single unit, optimized for throughput over latency.
+### Key Characteristics
+| Characteristic | Description | Example |
+|----------------|-------------|---------|
+| **High Volume** | Process thousands of records | 50,000 inventory positions |
+| **Scheduled** | Runs at specific times | Daily at 2 AM |
+| **Asynchronous** | Submit job, poll for completion | Job completes in 5-10 minutes |
+| **Bulk Operations** | Optimize for throughput | 100 records per API call |
+### How It Works
+```
+CSV File (50K records) → Parse → Transform → Batch API → Job → Poll Status → Complete
+     ↓                      ↓         ↓            ↓         ↓         ↓           ↓
+  S3 bucket          Parse chunks  Map fields  Create job  Submit  Check every  Archive
+                                                           batches  30 seconds    file
+```
+### Visual Flow
+```
+┌─────────────────┐
+│ S3 CSV File     │
+│ (50,000 records)│
+└────────┬────────┘
+         │
+         │ ①Download & Parse
+         ▼
+┌─────────────────┐
+│   CSVParserService     │
+│  Stream chunks  │
+└────────┬────────┘
+         │
+         │ ②Transform fields
+         ▼
+┌─────────────────┐
+│ UniversalMapper │
+│  Field mapping  │
+└────────┬────────┘
+         │
+         │ ③Create Batch job
+         ▼
+┌─────────────────┐
+│ FluentClient    │
+│  createJob()    │
+└────────┬────────┘
+         │
+         │ ④Send batches (100 records each)
+         ▼
+┌─────────────────┐
+│  sendBatch()    │ ────┐
+│  500 batches    │     │ Parallel processing
+│  of 100 records │ ────┤ on Fluent servers
+└────────┬────────┘     │
+         │              │
+         │ ⑤Poll status every 30s
+         ▼
+┌─────────────────┐
+│  getJobStatus() │
+│  PROCESSING...  │
+│  COMPLETED ✓    │
+└────────┬────────┘
+         │
+         │ ⑥Archive file
+         ▼
+┌─────────────────┐
+│ S3 Archive      │
+│ Success log     │
+└─────────────────┘
+```
+**Total time**: 5-10 minutes for 50,000 records (vs 2-3 hours for sequential GraphQL mutations).
+---
+## When to Use Batch Processing
+### ✅ Use Batch Processing When:
+| Scenario | Why Batch? | SDK Pattern |
+|----------|------------|-------------|
+| **Large Files** | 1K+ records | Batch API with streaming |
+| **Daily Sync** | Full inventory sync | Scheduled batch job |
+| **High Volume** | > 1,000 events/hour | Batch API (not webhooks) |
+| **Bulk Updates** | Mass price changes | Single job, multiple batches |
+| **CSV/Parquet Files** | Structured file formats | S3DataSource + FluentClient |
+### ❌ Avoid Batch Processing When:
+| Scenario | Why Not Batch? | Use Instead |
+|----------|----------------|-------------|
+| **Immediate Updates** | Need < 5 second latency | Real-time (Module 1) |
+| **Single Records** | 1-10 records | Direct GraphQL mutation |
+| **Event-Driven** | External webhook triggers | Real-time webhook |
+| **Critical Orders** | Customer waiting for confirmation | Real-time processing |
+---
+## Fluent Batch API Overview
+### What is Batch API?
+The Fluent Batch API is a specialized GraphQL endpoint for bulk data operations:
+- **Asynchronous**: Submit job, get ID, poll for completion
+- **High-throughput**: 10,000+ records in minutes
+- **Fault-tolerant**: Partial failures don't block entire job
+- **Entity-specific**: Currently supports `InventoryQuantity` only
+### Supported Operations
+| Entity | Operations | Max per Batch |
+|--------|-----------|---------------|
+| `InventoryQuantity` | Create, Update | 100-250 records |
+**IMPORTANT**: Batch API currently **only supports InventoryQuantity**. For other entities (Order, Product, Customer), use standard GraphQL mutations.
+### Batch API Workflow
+```graphql
+# Step 1: Create job
+mutation CreateJob {
+  createJob(input: {
+    name: "Daily Inventory Sync"
+    retailerId: "2"
+  }) {
+    id
+    status
+  }
+}
+# Step 2: Send batches
+mutation SendBatch($jobId: ID!, $entities: [InventoryQuantityInput!]!) {
+  sendBatch(jobId: $jobId, entities: $entities) {
+    id
+    status
+    recordCount
+  }
+}
+# Step 3: Poll status
+query GetJobStatus($jobId: ID!) {
+  job(id: $jobId) {
+    id
+    status  # PENDING, PROCESSING, COMPLETED, FAILED
+    totalBatches
+    completedBatches
+    errorSummary {
+      totalErrors
+      errorTypes
+    }
+  }
+}
+```
+### Job Lifecycle
+```
+CREATE_JOB → PENDING → SEND_BATCHES → PROCESSING → COMPLETED
+                ↓                                      ↓
+             (can add more batches)            (can check errors)
+```
+---
+## SDK Batch Components
+### Component 1: FluentClient Batch Methods
+The SDK provides batch methods directly on `FluentClient` (there is no separate `FluentBatchManager` class):
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+const client = await createClient({
+  config: {
+    baseUrl: 'https://api.fluentcommerce.com',
+    clientId: process.env.FLUENT_CLIENT_ID,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET,
+    retailerId: process.env.FLUENT_RETAILER_ID
+  }
+});
+// Create job
+const job = await client.createJob({
+  name: 'Daily Inventory Sync',
+  retailerId: '2'
+});
+// Send batch
+const batch = await client.sendBatch(job.id, {
+  entities: inventoryRecords
+});
+// Get status
+const status = await client.getJobStatus(job.id);
+// Get detailed status with batches
+const jobDetail = await client.getBatchStatus(job.id, batch.id);
+```
+**Available methods**:
+- `createJob(input)` - Create new Batch job
+- `sendBatch(jobId, data)` - Submit batch of records
+- `getJobStatus(jobId)` - Get job status and summary
+- `getBatchStatus(jobId, batchId)` - Get individual batch details
+### Component 2: `S3DataSource`
+**Purpose**: Read and write files from S3 with streaming support
+```typescript
+import { S3DataSource } from '@fluentcommerce/fc-connect-sdk';
+const s3 = new S3DataSource({
+  type: 'S3_CSV',
+  connectionId: 'my-s3',
+  name: 'My S3 Source',
+  s3Config: {
+    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+    region: 'us-east-1',
+    bucket: 'inventory-bucket'
+  }
+}, logger);
+// List files
+const files = await s3.listFiles('inventory/updates/');
+// Download file
+const fileContent = await s3.downloadFile('inventory/updates/inventory.csv');
+// Download large file as Buffer (for files >100MB)
+const buffer = await s3.downloadFile('inventory/updates/large-file.csv', { encoding: 'binary' });
+// Upload file
+await s3.uploadFile('inventory/archive/processed.csv', content);
+// Move file
+await s3.moveFile('inventory/updates/file.csv', 'inventory/archive/file.csv');
+```
+**Key features**:
+- Streaming support for large files (memory-efficient)
+- Automatic retry on network errors
+- Progress callbacks for uploads/downloads
+### Component 3: `CSVParserService`
+**Purpose**: Parse CSV files with validation and streaming
+```typescript
+import { CSVParserService } from '@fluentcommerce/fc-connect-sdk';
+const parser = new CSVParserService({
+  headers: true,           // First row is headers
+  skipEmptyLines: true,    // Ignore blank lines
+  delimiter: ',',          // CSV delimiter
+  quote: '"',              // Quote character
+  escape: '\\'             // Escape character
+});
+// Parse entire file (for small files < 10MB)
+const records = await parser.parse(csvContent);
+// Stream parse (for large files)
+const recordStream = parser.streamParse(csvStream);
+for await (const record of recordStream) {
+  // Process record
+}
+```
+**Validation options**:
+- Required columns
+- Type validation
+- Custom validators
+### Component 4: `UniversalMapper`
+**Purpose**: Transform CSV/JSON fields to Fluent schema
+```typescript
+import { UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
+const mappingConfig = {
+  fields: {
+    ref: {
+      source: 'sku',
+      resolver: 'custom.buildRef'  // Combine sku + location
+    },
+    type: {
+      value: 'INVENTORY'  // Static value
+    },
+    status: {
+      source: 'status',
+      resolver: 'sdk.uppercase'
+    },
+    productRef: {
+      source: 'sku',
+      required: true
+    },
+    locationRef: {
+      source: 'warehouse_code',
+      required: true
+    },
+    onHand: {
+      source: 'quantity',
+      resolver: 'sdk.parseInt'
+    }
+  }
+};
+const mapper = new UniversalMapper(mappingConfig, {
+  customResolvers: {
+    'custom.buildRef': (value, data) => {
+      return `${data.sku}-${data.warehouse_code}`;
+    }
+  }
+});
+const result = await mapper.map(csvRecord);
+// result.data = { ref: 'SKU001-WH01', type: 'INVENTORY', ... }
+```
+### Workflow Composition Pattern
+**Purpose**: Compose SDK services for complete ingestion workflows
+Instead of using a single orchestrator, compose the above components into your custom workflow:
+```typescript
+import {
+  createClient,
+  S3DataSource,
+  CSVParserService,
+  UniversalMapper,
+  StateService,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+async function processInventoryFiles() {
+  const logger = toStructuredLogger(createConsoleLogger(), { logLevel: 'info' });
+  // Initialize components
+  const client = await createClient({ config });
+  const s3 = new S3DataSource(s3Config, logger);
+  const parser = new CSVParserService();
+  const mapper = new UniversalMapper(mappingConfig);
+  const stateService = new StateService(logger);
+  // List and process files
+  const files = await s3.listFiles({ prefix: 'inventory/updates/' });
+  for (const file of files) {
+    if (await stateService.isFileProcessed(file.name)) continue;
+    try {
+      // 1. Download and parse
+      const content = await s3.downloadFile(file.name);
+      const records = await parser.parse(content);
+      // 2. Map fields
+      const inventory = [];
+      for (const record of records) {
+        const result = await mapper.map(record);
+        if (result.success) inventory.push(result.data);
+      }
+      // 3. Create job and send batches
+      const job = await client.createJob({
+        name: `Inventory - ${file.name}`,
+        retailerId: '1'
+      });
+      const batches = chunkArray(inventory, 100);
+      for (const batch of batches) {
+        await client.sendBatch(job.id, {
+          action: 'UPSERT',
+          entityType: 'INVENTORY',
+          entities: batch
+        });
+      }
+      // 4. Archive and mark processed
+      await s3.moveFile(file.name, `inventory/archive/${file.name}`);
+      await stateService.markFileProcessed(file.name);
+    } catch (error) {
+      logger.error(`Failed to process ${file.name}`, error);
+      await s3.moveFile(file.name, `inventory/errors/${file.name}`);
+    }
+  }
+}
+```
+**Benefits of building block composition**:
+1. ✅ Full control over workflow logic
+2. ✅ Custom error handling
+3. ✅ Easy to test individual components
+4. ✅ Flexible batch sizing and job strategies
+5. ✅ Works with any entity type (not just INVENTORY)
+6. ✅ Easy to add custom business rules
+---
+## Basic Batch Workflow
+### Step-by-Step Pattern
+```typescript
+/**
+ * Basic Batch Processing Workflow
+ *
+ * Steps:
+ * 1. Create Fluent client
+ * 2. Download and parse CSV file
+ * 3. Transform records to Fluent schema
+ * 4. Create Batch job
+ * 5. Send records in batches
+ * 6. Poll status until complete
+ * 7. Handle results
+ */
+import { createClient, S3DataSource, CSVParserService, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
+async function batchInventorySync() {
+  // Step 1: Create client
+  const client = await createClient({
+    baseUrl: process.env.FLUENT_BASE_URL,
+    clientId: process.env.FLUENT_CLIENT_ID,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET,
+    retailerId: process.env.FLUENT_RETAILER_ID
+  });
+  // Step 2: Download CSV
+  const s3 = new S3DataSource({
+    type: 'S3_CSV',
+    connectionId: 'my-s3',
+    name: 'My S3 Source',
+    s3Config: {
+      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+      region: process.env.AWS_REGION,
+      bucket: process.env.AWS_BUCKET
+    }
+  }, console);
+  const csvContent = await s3.downloadFile('inventory/daily-update.csv');
+  // Step 3: Parse CSV
+  const parser = new CSVParserService({ headers: true });
+  const records = await parser.parse(csvContent);
+  console.log(`Parsed ${records.length} records`);
+  // Step 4: Transform records
+  const mapper = new UniversalMapper({
+    fields: {
+      ref: { source: 'sku', resolver: 'custom.buildRef' },
+      type: { value: 'INVENTORY' },
+      productRef: { source: 'sku', required: true },
+      locationRef: { source: 'location', required: true },
+      onHand: { source: 'qty', resolver: 'sdk.parseInt' }
+    }
+  }, {
+    customResolvers: {
+      'custom.buildRef': (value, data) => `${data.sku}-${data.location}`
+    }
+  });
+  const mappedRecords = [];
+  for (const record of records) {
+    const result = await mapper.map(record);
+    if (result.success) {
+      mappedRecords.push(result.data);
+    } else {
+      console.error('Mapping error:', result.errors);
+    }
+  }
+  console.log(`Mapped ${mappedRecords.length} records`);
+  // Step 5: Create Batch job
+  const job = await client.createJob({
+    name: `Inventory Sync ${new Date().toISOString()}`,
+    retailerId: '2'
+  });
+  console.log(`Created job ${job.id}`);
+  // Step 6: Send batches
+  const BATCH_SIZE = 100;
+  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
+    const chunk = mappedRecords.slice(i, i + BATCH_SIZE);
+    const batch = await client.sendBatch(job.id, {
+      entities: chunk
+    });
+    console.log(`Sent batch ${batch.id} (${chunk.length} records)`);
+  }
+  // Step 7: Poll status
+  let status = await client.getJobStatus(job.id);
+  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+    console.log(`Job status: ${status.status} (${status.completedBatches}/${status.totalBatches} batches)`);
+    await new Promise(resolve => setTimeout(resolve, 30000)); // Wait 30 seconds
+    status = await client.getJobStatus(job.id);
+  }
+  // Step 8: Handle results
+  if (status.status === 'COMPLETED') {
+    console.log('✓ Job completed successfully');
+    console.log(`Total records: ${status.totalRecords}`);
+    console.log(`Errors: ${status.errorSummary?.totalErrors || 0}`);
+    // Archive file
+    await s3.moveFile('inventory/daily-update.csv', 'inventory/archive/daily-update.csv');
+  } else {
+    console.error('✗ Job failed:', status.status);
+    // Move to error folder
+    await s3.moveFile('inventory/daily-update.csv', 'inventory/errors/daily-update.csv');
+  }
+}
+batchInventorySync().catch(console.error);
+```
+---
+## NEW: Job Lifecycle Tracking (v0.1.10+)
+**Track job state, metadata, and lifecycle** across your integration workflows.
+The SDK provides `JobTracker` service for managing job lifecycle, tracking status, and storing job metadata.
+### JobTracker Overview
+```typescript
+import { JobTracker, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+// ✅ CORRECT: Access openKv from Versori context
+// import { openKv } from '@versori/run'; // ❌ WRONG - Not a direct export
+// In Versori workflow handler:
+const { openKv } = ctx;
+const kvAdapter = new VersoriKVAdapter(openKv(':project:'));
+const tracker = new JobTracker(kvAdapter, logger);
+// Create job
+const jobId = `scheduled_${Date.now()}`;
+await tracker.createJob(jobId, {
+  triggeredBy: 'schedule',
+  stage: 'initialization',
+  details: {
+    catalogueRef: 'DEFAULT:1',
+    fileName: 'inventory.csv'
+  }
+});
+// Update progress
+await tracker.updateJob(jobId, {
+  status: 'processing',
+  stage: 'extraction',
+  message: 'Extracting records from S3'
+});
+await tracker.updateJob(jobId, {
+  stage: 'transformation',
+  message: 'Mapping 1000 records',
+  details: { recordCount: 1000 }
+});
+// Mark as completed
+await tracker.markCompleted(jobId, {
+  recordCount: 1000,
+  successCount: 998,
+  failedCount: 2
+});
+// Or mark as failed
+try {
+  // ... job logic ...
+} catch (error) {
+  await tracker.markFailed(jobId, error);
+}
+```
+### Complete Example with Versori
+```typescript
+import {
+  createClient,
+  JobTracker,
+  VersoriKVAdapter,
+} from '@fluentcommerce/fc-connect-sdk';
+import { schedule } from '@versori/run';
+/**
+ * Versori workflow with complete job tracking
+ */
+export const dailyInventorySync = schedule('daily-inventory', '0 2 * * *')
+  .execute(async ({ log, connections, vars, kv }) => {
+    const jobId = `inventory_${Date.now()}`;
+    const tracker = new JobTracker(new VersoriKVAdapter(kv), log);
+    try {
+      // Create job
+      await tracker.createJob(jobId, {
+        triggeredBy: 'schedule',
+        stage: 'start',
+        details: { schedule: 'daily 2am' }
+      });
+      // Stage 1: Extraction
+      await tracker.updateJob(jobId, {
+        status: 'processing',
+        stage: 'extraction',
+        message: 'Querying virtual positions'
+      });
+      const data = await extractFromFluent();
+      // Stage 2: Transformation
+      await tracker.updateJob(jobId, {
+        stage: 'transformation',
+        message: `Processing ${data.length} records`
+      });
+      const transformed = await transformData(data);
+      // Stage 3: Upload
+      await tracker.updateJob(jobId, {
+        stage: 'upload',
+        message: 'Uploading to SFTP'
+      });
+      await uploadToSFTP(transformed);
+      // Completed
+      await tracker.markCompleted(jobId, {
+        recordCount: data.length,
+        fileName: `inventory_${jobId}.xml`
+      });
+      log.info('Job completed successfully', { jobId });
+    } catch (error) {
+      await tracker.markFailed(jobId, error);
+      log.error('Job failed', error);
+      throw error;
+    }
+  });
+```
+### Querying Job Status
+```typescript
+// Get job status
+const status = await tracker.getJob(jobId);
+if (status) {
+  console.log(`Job ${jobId}:`, {
+    status: status.status,
+    stage: status.stage,
+    message: status.message,
+    createdAt: status.createdAt,
+    completedAt: status.completedAt
+  });
+}
+// Check if job is still running
+if (status.status === 'processing') {
+  console.log(`Job in progress: ${status.stage}`);
+}
+// Check for errors
+if (status.status === 'failed') {
+  console.error('Job failed:', {
+    error: status.error,
+    stack: status.errorStack
+  });
+}
+```
+### Custom TTL Configuration
+```typescript
+// Default TTL: 7 days
+const tracker = new JobTracker(kvAdapter, logger);
+// Custom TTL: 24 hours
+const shortTracker = new JobTracker(
+  kvAdapter,
+  logger,
+  86400 // 24 hours in seconds
+);
+// Custom TTL: 30 days
+const longTracker = new JobTracker(
+  kvAdapter,
+  logger,
+  2592000 // 30 days in seconds
+);
+```
+### JobTracker API Reference
+| Method | Description | Parameters | Example |
+|--------|-------------|------------|---------|
+| `createJob(jobId, metadata)` | Create new job with 'queued' status | jobId, metadata | `await tracker.createJob('job_123', { triggeredBy: 'schedule' })` |
+| `updateJob(jobId, updates)` | Update job metadata/status/stage | jobId, updates | `await tracker.updateJob('job_123', { status: 'processing' })` |
+| `getJob(jobId)` | Get job by ID | jobId | `const job = await tracker.getJob('job_123')` |
+| `markCompleted(jobId, details)` | Mark job complete | jobId, details | `await tracker.markCompleted('job_123', { recordCount: 1000 })` |
+| `markFailed(jobId, error)` | Mark job failed | jobId, error | `await tracker.markFailed('job_123', error)` |
+**Constructor:**
+```typescript
+new JobTracker(kvAdapter: KVAdapter, logger: StructuredLogger, ttl?: number)
+```
+### When to Use JobTracker
+✅ **Use when**:
+- Need job history and audit trail
+- Monitoring multiple concurrent jobs
+- Debugging job failures
+- Tracking job duration and performance
+- Building dashboards or reports
+❌ **Skip when**:
+- Simple one-off scripts
+- No state persistence available
+- Memory-constrained environments
+---
+## Job Creation Strategies
+### Strategy 1: Single Job Per File (Recommended)
+**When to use**: Most common pattern for daily/scheduled files
+```typescript
+// Process one file
+const job = await client.createJob({
+  name: `Daily Inventory - ${fileName}`,
+  retailerId: '2'
+});
+// Send all batches for this file
+for (const batch of batches) {
+  await client.sendBatch(job.id, { entities: batch });
+}
+// Poll until complete
+await pollJobCompletion(job.id);
+```
+**Pros**:
+- Simple error tracking (one job = one file)
+- Easy archival (job complete = archive file)
+- Clear audit trail
+**Cons**:
+- More jobs in system
+- Can't combine multiple files
+### Strategy 2: Single Job Per Day
+**When to use**: Multiple small files processed together
+```typescript
+// Create one job for entire day
+const job = await client.createJob({
+  name: `Daily Inventory - ${new Date().toISOString().split('T')[0]}`,
+  retailerId: '2'
+});
+// Process multiple files
+const files = await s3.listFiles('inventory/updates/');
+for (const file of files) {
+  const records = await processFile(file);
+  // Send batches for this file
+  for (const batch of chunkArray(records, 100)) {
+    await client.sendBatch(job.id, { entities: batch });
+  }
+}
+// Poll once for entire day's work
+await pollJobCompletion(job.id);
+```
+**Pros**:
+- Fewer jobs
+- Combine multiple sources
+- Single status check
+**Cons**:
+- Harder to track individual file errors
+- One file failure doesn't block others
+- More complex archival logic
+### Strategy 3: Long-Running Job (Advanced)
+**When to use**: Continuous processing (hourly micro-batches)
+```typescript
+// Create job once
+const job = await client.createJob({
+  name: `Inventory Stream - ${new Date().toISOString().split('T')[0]}`,
+  retailerId: '2'
+});
+// Send batches throughout the day
+setInterval(async () => {
+  const newRecords = await fetchLatestUpdates();
+  if (newRecords.length > 0) {
+    await client.sendBatch(job.id, { entities: newRecords });
+  }
+}, 3600000); // Every hour
+// Check status at end of day
+```
+**Pros**:
+- Minimize job creation overhead
+- Continuous processing
+**Cons**:
+- Job can stay open for hours
+- Harder to determine "completion"
+- Risk of very large jobs
+---
+## Batch Size Optimization
+### Recommended Batch Sizes
+| Total Records | Batch Size | Batches | Processing Time |
+|---------------|------------|---------|-----------------|
+| < 1,000 | 100 | 1-10 | 1-2 minutes |
+| 1,000-10,000 | 100-200 | 10-100 | 3-10 minutes |
+| 10,000-50,000 | 200 | 50-250 | 5-15 minutes |
+| > 50,000 | 250 | 200+ | 10-30 minutes |
+**Max batch size**: 250 records (Fluent API limit)
+### Batch Size Calculator
+```typescript
+/**
+ * Calculate optimal batch size based on total records
+ */
+function calculateBatchSize(totalRecords: number): number {
+  if (totalRecords < 1000) {
+    return 100;
+  } else if (totalRecords < 10000) {
+    return 150;
+  } else {
+    return 250; // Max allowed
+  }
+}
+// Usage
+const batchSize = calculateBatchSize(records.length);
+const batches = chunkArray(records, batchSize);
+```
+### Chunking Utility
+```typescript
+/**
+ * Split array into chunks
+ */
+function chunkArray<T>(array: T[], size: number): T[][] {
+  const chunks: T[][] = [];
+  for (let i = 0; i < array.length; i += size) {
+    chunks.push(array.slice(i, i + size));
+  }
+  return chunks;
+}
+// Usage
+const batches = chunkArray(mappedRecords, 100);
+// [[record1, record2, ...], [record101, record102, ...], ...]
+```
+### Parallel Batch Sending (Advanced)
+```typescript
+/**
+ * Send multiple batches in parallel
+ * WARNING: Use with caution - can hit rate limits
+ */
+async function sendBatchesParallel(
+  client: FluentClient,
+  jobId: string,
+  batches: any[][],
+  concurrency = 5
+) {
+  const results = [];
+  for (let i = 0; i < batches.length; i += concurrency) {
+    const chunk = batches.slice(i, i + concurrency);
+    // Send up to 'concurrency' batches at once
+    const promises = chunk.map(batch =>
+      client.sendBatch(jobId, { entities: batch })
+    );
+    const chunkResults = await Promise.all(promises);
+    results.push(...chunkResults);
+    console.log(`Sent batches ${i + 1} to ${i + chunk.length}`);
+  }
+  return results;
+}
+// Usage (send 5 batches at a time)
+await sendBatchesParallel(client, job.id, batches, 5);
+```
+**Caution**: Parallel sending can trigger rate limits. Start with concurrency=5, monitor, and adjust.
+---
+## Status Polling and Completion
+### Basic Polling Pattern
+```typescript
+/**
+ * Poll job status until complete
+ */
+async function pollJobCompletion(
+  client: FluentClient,
+  jobId: string,
+  options = {
+    interval: 30000,      // 30 seconds
+    timeout: 3600000,     // 1 hour
+    onProgress: undefined // Callback
+  }
+) {
+  const startTime = Date.now();
+  while (true) {
+    const status = await client.getJobStatus(jobId);
+    // Check terminal states
+    if (status.status === 'COMPLETED') {
+      console.log('✓ Job completed successfully');
+      return status;
+    }
+    if (status.status === 'FAILED') {
+      throw new Error(`Job failed: ${status.errorSummary?.message || 'Unknown error'}`);
+    }
+    // Check timeout
+    if (Date.now() - startTime > options.timeout) {
+      throw new Error(`Job timeout after ${options.timeout}ms`);
+    }
+    // Progress callback
+    if (options.onProgress) {
+      options.onProgress(status);
+    }
+    // Log progress
+    console.log(
+      `Job ${status.status}: ${status.completedBatches}/${status.totalBatches} batches ` +
+      `(${Math.round((status.completedBatches / status.totalBatches) * 100)}%)`
+    );
+    // Wait before next poll
+    await new Promise(resolve => setTimeout(resolve, options.interval));
+  }
+}
+// Usage
+const finalStatus = await pollJobCompletion(client, job.id, {
+  interval: 30000,
+  timeout: 3600000,
+  onProgress: (status) => {
+    console.log(`Progress: ${status.completedBatches}/${status.totalBatches}`);
+  }
+});
+```
+### Advanced: Adaptive Polling
+```typescript
+/**
+ * Adjust polling interval based on job size
+ */
+function getPollingInterval(totalBatches: number): number {
+  if (totalBatches < 10) {
+    return 10000;  // 10 seconds for small jobs
+  } else if (totalBatches < 100) {
+    return 30000;  // 30 seconds for medium jobs
+  } else {
+    return 60000;  // 1 minute for large jobs
+  }
+}
+// Usage
+const interval = getPollingInterval(batches.length);
+await pollJobCompletion(client, job.id, { interval });
+```
+---
+## Error Handling in Batches
+### Error Types
+| Error Level | Scope | Example | Recovery |
+|-------------|-------|---------|----------|
+| **File Error** | Entire file | Invalid CSV format | Move to error folder |
+| **Record Error** | Single record | Missing required field | Log, continue with others |
+| **Batch Error** | 100-250 records | API validation error | Logged by Fluent, continue |
+| **Job Error** | Entire job | Authentication failure | Retry entire job |
+| **Partial Batch Failure** | Some batches succeed, some fail | Network interruption | **NEW: PartialBatchRecovery** |
+### NEW: Partial Batch Recovery (v0.1.10+)
+**Problem**: What happens when sending 500 batches and batch #250 fails due to network error? You don't want to resend the first 249 successful batches.
+**Solution**: The SDK provides `PartialBatchRecovery` service to track batch progress and resume from failure point.
+#### How It Works
+```typescript
+import { PartialBatchRecovery, createClient } from '@fluentcommerce/fc-connect-sdk';
+/**
+ * Partial Batch Recovery Pattern
+ *
+ * Tracks per-record success/failure in batch operations and enables:
+ * - Retrying only failed records instead of entire batch
+ * - Checkpoint/resume functionality
+ * - Detailed error reporting per record
+ */
+async function resilientBatchSync() {
+  const client = await createClient({ config });
+  const recovery = new PartialBatchRecovery(logger);
+  // Step 1: Create job
+  const job = await client.createJob({
+    name: 'Daily Inventory Sync',
+    retailerId: '2',
+  });
+  // Step 2: Process batch with automatic recovery
+  const result = await recovery.processBatchWithRecovery(
+    records,
+    async (batch) => {
+      // Your batch processing logic
+      return await client.sendBatch(job.id, {
+        action: 'UPSERT',
+        entityType: 'INVENTORY',
+        entities: batch
+      });
+    },
+    {
+      maxRetries: 3,
+      retryOnlyFailed: true,  // Only retry failed records
+      retryDelayMs: 1000,     // Start with 1 second
+      retryBatchSize: 100,    // Process 100 at a time
+      checkpointKey: 'inventory-sync-2025-01-24'
+    }
+  );
+  console.log(`✓ Success: ${result.successCount}/${result.totalRecords}`);
+  console.log(`✗ Failed: ${result.failedCount} records`);
+  if (result.failedCount > 0) {
+    console.error('Failed records:', result.failedRecords);
+    console.log(`Checkpoint saved: ${result.checkpointId}`);
+  }
+}
+```
+#### Integration with Batch API
+```typescript
+import {
+  createClient,
+  PartialBatchRecovery
+} from '@fluentcommerce/fc-connect-sdk';
+async function batchIngestionWithRecovery(records: any[]) {
+  const client = await createClient({ config });
+  const recovery = new PartialBatchRecovery(logger);
+  // Create job
+  const job = await client.createJob({
+    name: 'Inventory Ingestion with Recovery',
+    retailerId: 'my-retailer'
+  });
+  // Process with recovery
+  const result = await recovery.processBatchWithRecovery(
+    records,
+    async (batch) => {
+      const response = await client.sendBatch(job.id, {
+        action: 'UPSERT',
+        entityType: 'INVENTORY',
+        entities: batch
+      });
+      logger.info('Batch sent', {
+        batchId: response.id,
+        recordCount: batch.length
+      });
+      return response;
+    },
+    {
+      maxRetries: 3,
+      retryOnlyFailed: true,
+      checkpointKey: `job-${job.id}`
+    }
+  );
+  // Check job status
+  const status = await client.getJobStatus(job.id);
+  return {
+    jobId: job.id,
+    jobStatus: status.status,
+    ...result
+  };
+}
+```
+#### Checkpoint and Resume
+```typescript
+// Process batch and save checkpoint
+const result = await recovery.processBatchWithRecovery(
+  records,
+  processBatch,
+  {
+    maxRetries: 3,
+    checkpointKey: 'daily-inventory-sync'
+  }
+);
+if (result.failedCount > 0) {
+  console.log(`Checkpoint created: ${result.checkpointId}`);
+  console.log(`Failed records saved for later retry`);
+}
+// Later: Resume from checkpoint
+const checkpointId = result.checkpointId;
+const resumeResult = await recovery.resumeFromCheckpoint(
+  checkpointId,
+  processBatch,
+  {
+    maxRetries: 5  // More retries on resume
+  }
+);
+console.log(`Resume: ${resumeResult.successCount} recovered`);
+```
+#### Custom Retry Logic
+```typescript
+const result = await recovery.processBatchWithRecovery(
+  records,
+  processBatch,
+  {
+    maxRetries: 5,
+    retryDelayMs: 2000,
+    // Custom retry decision
+    shouldRetry: (error, attemptCount) => {
+      // Don't retry validation errors
+      if (error.message.includes('validation')) {
+        return false;
+      }
+      // Don't retry after 3 attempts for rate limits
+      if (error.message.includes('rate limit') && attemptCount > 3) {
+        return false;
+      }
+      // Retry all other errors
+      return true;
+    }
+  }
+);
+```
+#### Record Failure Details
+```typescript
+// Access detailed failure information
+if (result.failedCount > 0) {
+  result.failedRecords.forEach(failure => {
+    console.error(`Record ${failure.index} failed:`, {
+      record: failure.record,
+      error: failure.error.message,
+      attempts: failure.attemptCount,
+      timestamp: failure.timestamp
+    });
+  });
+  // Export failures for manual review
+  await fs.writeFile(
+    'failed-records.json',
+    JSON.stringify(result.failedRecords, null, 2)
+  );
+}
+```
+#### Key Features
+| Feature | Description | Benefit |
+|---------|-------------|---------|
+| **Per-record tracking** | Tracks each record individually | Know exactly which records failed |
+| **Selective retry** | Retry only failures, not successes | Efficient retry logic |
+| **Checkpoint support** | Resume from failure point | Handle interruptions |
+| **Exponential backoff** | Configurable retry delays | Avoid overwhelming API |
+| **Custom retry logic** | Override retry decisions | Fine-grained control |
+#### API Reference
+**Constructor:**
+```typescript
+new PartialBatchRecovery(logger?: StructuredLogger)
+```
+**Methods:**
+| Method | Description | Parameters |
+|--------|-------------|------------|
+| `processBatchWithRecovery()` | Process batch with recovery | records, processor, options |
+| `resumeFromCheckpoint()` | Resume from saved checkpoint | checkpointId, processor, options |
+#### When to Use Partial Batch Recovery
+✅ **Use when**:
+- Sending 50+ batches (high failure risk)
+- Network is unstable
+- Long-running jobs (> 10 minutes)
+- Scheduled workflows that may be interrupted
+- Critical data that must complete
+❌ **Skip when**:
+- < 10 batches (low failure risk)
+- Fast operations (< 2 minutes)
+- Non-critical data (can rerun from scratch)
+- Memory-constrained environments
+### File-Level Error Handling
+```typescript
+try {
+  // Download file
+  const content = await s3.downloadFile(fileKey);
+  // Parse CSV
+  const records = await parser.parse(content);
+  // Process batches
+  await processBatches(records);
+  // Archive on success
+  await s3.moveFile(fileKey, archiveKey);
+} catch (error: any) {
+  console.error(`File processing failed: ${error.message}`);
+  // Move to error folder
+  await s3.moveFile(fileKey, errorKey);
+  // Log error details
+  await s3.uploadFile(errorKey + '.error.log', JSON.stringify({
+    file: fileKey,
+    error: error.message,
+    stack: error.stack,
+    timestamp: new Date().toISOString()
+  }, null, 2));
+  // Continue with next file (don't throw)
+}
+```
+### Record-Level Error Handling
+```typescript
+const mappedRecords = [];
+const mappingErrors = [];
+for (const record of records) {
+  const result = await mapper.map(record);
+  if (result.success) {
+    mappedRecords.push(result.data);
+  } else {
+    mappingErrors.push({
+      record,
+      errors: result.errors
+    });
+  }
+}
+console.log(`Mapped ${mappedRecords.length} records, ${mappingErrors.length} errors`);
+// Write error report
+if (mappingErrors.length > 0) {
+  await s3.uploadFile(
+    'inventory/errors/mapping-errors.json',
+    JSON.stringify(mappingErrors, null, 2)
+  );
+}
+// Continue with successful records
+```
+### Batch API Error Reporting
+```typescript
+// After job completes, check for errors
+const status = await client.getJobStatus(job.id);
+if (status.errorSummary && status.errorSummary.totalErrors > 0) {
+  console.warn(`Job completed with ${status.errorSummary.totalErrors} errors`);
+  // Get error details
+  const errorDetails = await client.graphql({
+    query: `
+      query GetJobErrors($jobId: ID!) {
+        job(id: $jobId) {
+          batches(first: 100) {
+            edges {
+              node {
+                id
+                status
+                errors {
+                  recordRef
+                  errorType
+                  errorMessage
+                }
+              }
+            }
+          }
+        }
+      }
+    `,
+    variables: { jobId: job.id }
+  });
+  // Write error report
+  await s3.uploadFile(
+    'inventory/errors/batch-errors.json',
+    JSON.stringify(errorDetails, null, 2)
+  );
+}
+```
+---
+## Complete Implementation Example
+See **[S3 CSV Batch API Guide](../../../01-TEMPLATES/standalone/s3-csv-batch-api.md)** for production-ready implementation with:
+- Complete TypeScript code
+- Environment configuration
+- Error handling strategies
+- File archival patterns
+- Scheduling with cron
+- Monitoring and logging
+### Quick Example: Scheduled Versori Batch
+```typescript
+/**
+ * Versori Scheduled Workflow: Daily Inventory Sync
+ *
+ * Trigger: Cron (daily at 2 AM)
+ * Process: Download CSV from S3, send to Batch API
+ */
+import { createClient, S3DataSource, CSVParserService, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
+export default async function dailyInventorySync(activation: any, log: any, connections: any) {
+  try {
+    log.info('Starting daily inventory sync');
+    // Create client
+    const client = await createClient({
+      connection: connections.fluent_commerce,
+      logger: log
+    });
+    // Create S3 source
+    const s3 = new S3DataSource({
+      connection: connections.aws_s3
+    }, log);
+    // List files
+    const files = await s3.listFiles('inventory/daily/');
+    log.info(`Found ${files.length} files to process`);
+    // Process each file
+    for (const file of files) {
+      await processFile(client, s3, file.key, log);
+    }
+    return { status: 200, body: { success: true, filesProcessed: files.length } };
+  } catch (error: any) {
+    log.error('Daily sync failed', error);
+    return { status: 500, body: { success: false, error: error.message } };
+  }
+}
+async function processFile(client: any, s3: any, fileKey: string, log: any) {
+  try {
+    // Download
+    const content = await s3.downloadFile(fileKey);
+    // Parse
+    const parser = new CSVParserService({ headers: true });
+    const records = await parser.parse(content);
+    // Map
+    const mapper = new UniversalMapper({
+      fields: {
+        ref: { source: 'sku', resolver: 'custom.buildRef' },
+        type: { value: 'INVENTORY' },
+        productRef: { source: 'sku', required: true },
+        locationRef: { source: 'location', required: true },
+        onHand: { source: 'qty', resolver: 'sdk.parseInt' }
+      }
+    }, {
+      customResolvers: {
+        'custom.buildRef': (v, d) => `${d.sku}-${d.location}`
+      }
+    });
+    const mapped = [];
+    for (const rec of records) {
+      const result = await mapper.map(rec);
+      if (result.success) mapped.push(result.data);
+    }
+    // Create job
+    const job = await client.createJob({
+      name: `Daily Inventory - ${fileKey}`,
+      retailerId: '2'
+    });
+    // Send batches
+    const BATCH_SIZE = 100;
+    for (let i = 0; i < mapped.length; i += BATCH_SIZE) {
+      await client.sendBatch(job.id, {
+        entities: mapped.slice(i, i + BATCH_SIZE)
+      });
+    }
+    // Poll completion
+    let status = await client.getJobStatus(job.id);
+    while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+      await new Promise(r => setTimeout(r, 30000));
+      status = await client.getJobStatus(job.id);
+    }
+    if (status.status === 'COMPLETED') {
+      await s3.moveFile(fileKey, fileKey.replace('daily/', 'archive/'));
+      log.info(`✓ Processed ${fileKey}`);
+    } else {
+      throw new Error(`Job failed: ${status.status}`);
+    }
+  } catch (error: any) {
+    log.error(`Failed to process ${fileKey}`, error);
+    await s3.moveFile(fileKey, fileKey.replace('daily/', 'errors/'));
+  }
+}
+```
+---
+## Next Steps
+Now that you understand batch processing, you're ready to learn delta sync for incremental updates!
+**Continue to:** [Module 3: Delta Sync →](./integration-patterns-03-delta-sync.md)
+Or explore:
+- [Module 5: Error Handling](./integration-patterns-05-error-handling.md) - Resilience strategies
+- [Complete Example: S3 CSV Batch API](../../../01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md)
+- [Complete Example: Versori Scheduled CSV](../../../01-TEMPLATES/versori/workflows/ingestion/batch-api/template-ingestion-s3-csv-inventory-batch.md)
+---
+## Additional Resources
+- [Fluent Batch API Documentation](https://docs.fluentcommerce.com/)
+- [Universal Mapping Guide](../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md)
+- [S3DataSource API Reference](../../../02-CORE-GUIDES/data-sources/modules/data-sources-02-s3-operations.md)
+- [CSVParserService API Reference](../../../02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-02-csv-parser.md)
+---
+[← Back to Index](../../../02-CORE-GUIDES/advanced-services/advanced-services-readme.md) | [Previous: Real-Time →](./integration-patterns-01-real-time-processing.md) | [Next: Delta Sync →](./integration-patterns-03-delta-sync.md)