@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

package/docs/03-PATTERN-GUIDES/integration-patterns/modules/integration-patterns-03-delta-sync.md

@@ -1,1108 +1,1108 @@
-# Module 3: Delta Sync (Change Detection)
-
-> **Learning Objective:** Implement efficient delta synchronization to process only changed records, reducing processing time and API calls by 90%+.
->
-> **Level:** Advanced
-
-## Table of Contents
-
-1. [What is Delta Sync?](#what-is-delta-sync)
-2. [Why Delta Sync Matters](#why-delta-sync-matters)
-3. [Change Detection Strategies](#change-detection-strategies)
-4. [SDK State Management](#sdk-state-management)
-5. [Pattern 1: Timestamp-Based Detection](#pattern-1-timestamp-based-detection)
-6. [Pattern 2: Hash-Based Detection](#pattern-2-hash-based-detection)
-7. [Pattern 3: Source System Change Tracking](#pattern-3-source-system-change-tracking)
-8. [State Storage Options](#state-storage-options)
-9. [Complete Delta Sync Implementation](#complete-delta-sync-implementation)
-10. [Performance Optimization](#performance-optimization)
-11. [Next Steps](#next-steps)
-
----
-
-## What is Delta Sync?
-
-**Delta sync** (delta synchronization) means processing only the records that have **changed** since the last sync, rather than reprocessing the entire dataset.
-
-### Full Sync vs Delta Sync
-
-```
-FULL SYNC (every run):
-Day 1: Process 50,000 records → 10 minutes
-Day 2: Process 50,000 records → 10 minutes (99% duplicates!)
-Day 3: Process 50,000 records → 10 minutes (99% duplicates!)
-
-DELTA SYNC (only changes):
-Day 1: Process 50,000 records → 10 minutes (initial load)
-Day 2: Process 50 changed records → 10 seconds
-Day 3: Process 120 changed records → 15 seconds
-```
-
-### Visual Comparison
-
-```
-FULL SYNC:
-┌──────────────────────────────────────┐
-│ Source: 50,000 inventory records     │
-└──────────────┬───────────────────────┘
-               │
-               │ Process ALL 50K records every time
-               ▼
-┌──────────────────────────────────────┐
-│ Fluent Batch API: 500 batches       │
-│ Processing Time: 10 minutes          │
-│ API Calls: 500                       │
-└──────────────────────────────────────┘
-
-DELTA SYNC:
-┌──────────────────────────────────────┐
-│ Source: 50,000 inventory records     │
-│ Changed: 50 records (0.1%)           │
-└──────────────┬───────────────────────┘
-               │
-               │ Process ONLY 50 changed records
-               ▼
-┌──────────────────────────────────────┐
-│ Fluent Batch API: 1 batch            │
-│ Processing Time: 10 seconds          │
-│ API Calls: 1                         │
-└──────────────────────────────────────┘
-
-Performance Improvement: 99% reduction in processing time
-```
-
----
-
-## Why Delta Sync Matters
-
-### Benefits
-
-| Benefit | Impact | Example |
-|---------|--------|---------|
-| **Faster Processing** | 90-99% time reduction | 10 min → 10 sec |
-| **Lower API Costs** | Fewer API calls | 500 calls → 1 call |
-| **Reduced Load** | Less database/network strain | Minimal impact on source system |
-| **Real-Time Capable** | Frequent syncs (hourly/15min) | Near real-time updates |
-| **Error Recovery** | Smaller failure blast radius | 50 records vs 50K records |
-
-### When Delta Sync is Critical
-
-| Scenario | Why? | Change Rate |
-|----------|------|-------------|
-| **High-Frequency Sync** | Hourly/15-minute syncs | < 1% change rate |
-| **Large Datasets** | 100K+ records | Any change rate |
-| **Real-Time Requirements** | Must sync frequently | < 5% change rate |
-| **API Rate Limits** | Limited API quota | Any change rate |
-| **Cost Optimization** | Pay-per-API-call pricing | Any change rate |
-
-### Cost Example
-
-```
-Scenario: 100,000 inventory records, 0.5% daily change rate
-
-FULL SYNC (daily):
-- Records processed: 100,000
-- Batches: 1,000
-- API calls: 1,000
-- Monthly API calls: 30,000
-- Processing time: 20 minutes/day
-
-DELTA SYNC (daily):
-- Records processed: 500 (0.5% of 100K)
-- Batches: 5
-- API calls: 5
-- Monthly API calls: 150
-- Processing time: 30 seconds/day
-
-Savings:
-- API calls: 99.5% reduction
-- Processing time: 97.5% reduction
-- Infrastructure cost: 95%+ reduction
-```
-
----
-
-## Change Detection Strategies
-
-### Strategy Comparison
-
-| Strategy | Pros | Cons | Best For |
-|----------|------|------|----------|
-| **Timestamp** | Simple, reliable | Requires lastModified field | Most systems |
-| **Hash** | Detects any change | CPU overhead for hashing | Systems without timestamps |
-| **Source System** | Most accurate | Requires source system support | Modern APIs |
-| **Hybrid** | Best accuracy | More complex | Production systems |
-
-### Decision Matrix
-
-```
-Does source have lastModified/updatedAt field?
-├── YES → Use Timestamp-Based Detection
-│   └── Does source support "changes since" query?
-│       ├── YES → Use Source System Change Tracking (best)
-│       └── NO → Use Timestamp Comparison
-│
-└── NO → Use Hash-Based Detection
-    └── Can you add lastModified to source?
-        ├── YES → Add timestamp, use Timestamp-Based
-        └── NO → Hash-Based (required)
-```
-
----
-
-## SDK State Management
-
-### StateService Overview
-
-The SDK provides `StateService` for tracking processed files and records:
-
-```typescript
-import { StateService, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-
-// Versori platform (KV storage)
-const kvAdapter = new VersoriKVAdapter(openKv());
-const stateService = new StateService(logger);
-
-// Standalone (file-based storage)
-import { FileKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-const kvAdapter = new FileKVAdapter('./state');
-const stateService = new StateService(logger);
-```
-
-### StateService Methods
-
-```typescript
-// Check if file was processed
-const processed = await stateService.isFileProcessed('file-key');
-
-// Mark file as processed
-await stateService.markFileProcessed('file-key', {
-  processedAt: new Date(),
-  recordCount: 5000,
-  jobId: 'job-123'
-});
-
-// Get file processing metadata
-const metadata = await stateService.getFileMetadata('file-key');
-
-// Store custom state
-await stateService.setState('last-sync-timestamp', Date.now());
-
-// Retrieve custom state
-const lastSync = await stateService.getState('last-sync-timestamp');
-
-// Clear state (for testing/reset)
-await stateService.clearFileState('file-key');
-```
-
-### KV Storage Adapters
-
-**VersoriKVAdapter** (for Versori platform):
-```typescript
-import { VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-
-const kvAdapter = new VersoriKVAdapter(openKv());
-```
-
-**FileKVAdapter** (for standalone Node.js/Deno):
-```typescript
-import { FileKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-
-const kvAdapter = new FileKVAdapter('./state', {
-  encoding: 'json',
-  pretty: true
-});
-```
-
----
-
-## Pattern 1: Timestamp-Based Detection
-
-### Concept
-
-Compare `lastModified` timestamp from source with last sync timestamp.
-
-### How It Works
-
-```
-Step 1: Get last sync timestamp from state
-  └── lastSync = 2025-01-15T10:00:00Z
-
-Step 2: Query source for records modified after lastSync
-  └── SELECT * FROM inventory WHERE lastModified > '2025-01-15T10:00:00Z'
-
-Step 3: Process only those changed records
-  └── 50 records (instead of 50,000)
-
-Step 4: Update last sync timestamp
-  └── lastSync = 2025-01-15T14:00:00Z (current time)
-```
-
-### Implementation
-
-```typescript
-/**
- * Timestamp-Based Delta Sync
- *
- * Requirements:
- * - Source data has lastModified/updatedAt field
- * - Timestamps are reliable and monotonic
- */
-
-import { createClient, S3DataSource, CSVParserService, UniversalMapper, StateService, FileKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-
-async function deltaSync() {
-  // Initialize state service
-  const kvAdapter = new FileKVAdapter('./state');
-  const stateService = new StateService(logger);
-
-  // Get last sync timestamp
-  let lastSyncTime = await stateService.getState('last-sync-timestamp');
-
-  if (!lastSyncTime) {
-    console.log('First run - performing full sync');
-    lastSyncTime = new Date(0).toISOString(); // Epoch (all records)
-  } else {
-    console.log(`Last sync: ${lastSyncTime}`);
-  }
-
-  // Download CSV
-  const s3 = new S3DataSource({
-    type: 'S3_CSV',
-    connectionId: 'my-s3',
-    name: 'My S3 Source',
-    s3Config: {
-      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
-      region: process.env.AWS_REGION,
-      bucket: process.env.AWS_BUCKET
-    }
-  }, console);
-
-  const csvContent = await s3.downloadFile('inventory/current.csv');
-
-  // Parse CSV
-  const parser = new CSVParserService({ headers: true });
-  const allRecords = await parser.parse(csvContent);
-
-  console.log(`Total records in source: ${allRecords.length}`);
-
-  // Filter changed records
-  const changedRecords = allRecords.filter(record => {
-    const recordTimestamp = new Date(record.lastModified).toISOString();
-    return recordTimestamp > lastSyncTime;
-  });
-
-  console.log(`Changed records: ${changedRecords.length} (${((changedRecords.length / allRecords.length) * 100).toFixed(2)}%)`);
-
-  if (changedRecords.length === 0) {
-    console.log('No changes detected - skipping sync');
-    return;
-  }
-
-  // Map records
-  const mapper = new UniversalMapper({
-    fields: {
-      ref: { source: 'sku', resolver: 'custom.buildRef' },
-      type: { value: 'INVENTORY' },
-      productRef: { source: 'sku', required: true },
-      locationRef: { source: 'location', required: true },
-      onHand: { source: 'qty', resolver: 'sdk.parseInt' }
-    }
-  }, {
-    customResolvers: {
-      'custom.buildRef': (v, d) => `${d.sku}-${d.location}`
-    }
-  });
-
-  const mappedRecords = [];
-  for (const record of changedRecords) {
-    const result = await mapper.map(record);
-    if (result.success) {
-      mappedRecords.push(result.data);
-    }
-  }
-
-  // Create client and job
-  const client = await createClient({
-    config: {
-      baseUrl: process.env.FLUENT_BASE_URL,
-      clientId: process.env.FLUENT_CLIENT_ID,
-      clientSecret: process.env.FLUENT_CLIENT_SECRET,
-      retailerId: process.env.FLUENT_RETAILER_ID
-    }
-  });
-
-  const job = await client.createJob({
-    name: `Delta Sync - ${new Date().toISOString()}`,
-    retailerId: '2'
-  });
-
-  // Send batches
-  const BATCH_SIZE = 100;
-  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
-    const batch = mappedRecords.slice(i, i + BATCH_SIZE);
-    await client.sendBatch(job.id, { entities: batch });
-  }
-
-  // Poll completion
-  let status = await client.getJobStatus(job.id);
-  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-    await new Promise(r => setTimeout(r, 30000));
-    status = await client.getJobStatus(job.id);
-  }
-
-  if (status.status === 'COMPLETED') {
-    // Update last sync timestamp
-    const currentTime = new Date().toISOString();
-    await stateService.setState('last-sync-timestamp', currentTime);
-
-    console.log(`✓ Delta sync completed - ${mappedRecords.length} records processed`);
-    console.log(`Next sync will process changes after ${currentTime}`);
-  } else {
-    throw new Error(`Job failed: ${status.status}`);
-  }
-}
-
-deltaSync().catch(console.error);
-```
-
-### Timestamp Format Handling
-
-```typescript
-/**
- * Normalize various timestamp formats
- */
-function normalizeTimestamp(timestamp: string | number | Date): string {
-  // Handle different input types
-  if (timestamp instanceof Date) {
-    return timestamp.toISOString();
-  }
-
-  if (typeof timestamp === 'number') {
-    return new Date(timestamp).toISOString();
-  }
-
-  // Parse string timestamp
-  const date = new Date(timestamp);
-
-  if (isNaN(date.getTime())) {
-    throw new Error(`Invalid timestamp: ${timestamp}`);
-  }
-
-  return date.toISOString();
-}
-
-// Usage
-const recordTimestamp = normalizeTimestamp(record.lastModified);
-const isChanged = recordTimestamp > lastSyncTime;
-```
-
----
-
-## Pattern 2: Hash-Based Detection
-
-### Concept
-
-Calculate hash of record content and compare with previous hash.
-
-### How It Works
-
-```
-Step 1: Load previous hashes from state
-  └── { "SKU001-WH01": "a1b2c3d4", "SKU002-WH01": "e5f6g7h8" }
-
-Step 2: For each record, calculate current hash
-  └── currentHash = hash(sku + location + qty + ...)
-
-Step 3: Compare with previous hash
-  └── If hash changed → process record
-  └── If hash same → skip record
-
-Step 4: Store current hashes for next run
-  └── { "SKU001-WH01": "a1b2c3d4", "SKU002-WH01": "x9y8z7w6" }
-```
-
-### Implementation
-
-```typescript
-/**
- * Hash-Based Delta Sync
- *
- * Use when:
- * - Source data has NO lastModified field
- * - Need to detect ANY field changes
- */
-
-import crypto from 'crypto';
-
-/**
- * Calculate hash of record
- */
-function calculateRecordHash(record: any, fields: string[]): string {
-  // Extract relevant fields in consistent order
-  const values = fields.map(field => String(record[field] || ''));
-
-  // Join and hash
-  const combined = values.join('|');
-  return crypto.createHash('md5').update(combined).digest('hex');
-}
-
-async function hashBasedDeltaSync() {
-  const kvAdapter = new FileKVAdapter('./state');
-  const stateService = new StateService(logger);
-
-  // Load previous hashes
-  const previousHashes = await stateService.getState('record-hashes') || {};
-
-  console.log(`Loaded ${Object.keys(previousHashes).length} previous hashes`);
-
-  // Download and parse
-  const s3 = new S3DataSource({ /* config */ }, console);
-  const csvContent = await s3.downloadFile('inventory/current.csv');
-
-  const parser = new CSVParserService({ headers: true });
-  const allRecords = await parser.parse(csvContent);
-
-  console.log(`Total records: ${allRecords.length}`);
-
-  // Detect changes
-  const changedRecords = [];
-  const currentHashes: Record<string, string> = {};
-
-  // Fields to include in hash (all fields that matter)
-  const hashFields = ['sku', 'location', 'qty', 'status'];
-
-  for (const record of allRecords) {
-    const recordKey = `${record.sku}-${record.location}`;
-    const currentHash = calculateRecordHash(record, hashFields);
-
-    currentHashes[recordKey] = currentHash;
-
-    const previousHash = previousHashes[recordKey];
-
-    if (!previousHash) {
-      // New record
-      console.log(`New record: ${recordKey}`);
-      changedRecords.push(record);
-    } else if (currentHash !== previousHash) {
-      // Changed record
-      console.log(`Changed record: ${recordKey} (hash: ${previousHash} → ${currentHash})`);
-      changedRecords.push(record);
-    }
-    // else: unchanged, skip
-  }
-
-  console.log(`Changed records: ${changedRecords.length} (${((changedRecords.length / allRecords.length) * 100).toFixed(2)}%)`);
-
-  if (changedRecords.length === 0) {
-    console.log('No changes detected - skipping sync');
-    return;
-  }
-
-  // Process changed records (same as timestamp-based)
-  const mapper = new UniversalMapper({ /* config */ });
-  const mappedRecords = [];
-
-  for (const record of changedRecords) {
-    const result = await mapper.map(record);
-    if (result.success) {
-      mappedRecords.push(result.data);
-    }
-  }
-
-  // Send to Batch API
-  const client = await createClient({ /* config */ });
-  const job = await client.createJob({ name: 'Hash-Based Delta Sync', retailerId: '2' });
-
-  const BATCH_SIZE = 100;
-  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
-    await client.sendBatch(job.id, {
-      entities: mappedRecords.slice(i, i + BATCH_SIZE)
-    });
-  }
-
-  // Poll completion
-  let status = await client.getJobStatus(job.id);
-  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-    await new Promise(r => setTimeout(r, 30000));
-    status = await client.getJobStatus(job.id);
-  }
-
-  if (status.status === 'COMPLETED') {
-    // Store current hashes for next run
-    await stateService.setState('record-hashes', currentHashes);
-
-    console.log(`✓ Delta sync completed - ${mappedRecords.length} records processed`);
-    console.log(`Stored ${Object.keys(currentHashes).length} hashes for next run`);
-  } else {
-    throw new Error(`Job failed: ${status.status}`);
-  }
-}
-
-hashBasedDeltaSync().catch(console.error);
-```
-
-### Hash Performance Optimization
-
-```typescript
-/**
- * Fast hash using Node.js crypto (faster than MD5)
- */
-function fastHash(data: string): string {
-  return crypto.createHash('sha1').update(data).digest('hex');
-}
-
-/**
- * Calculate hash for large datasets efficiently
- */
-async function calculateHashesBatch(
-  records: any[],
-  fields: string[],
-  batchSize = 1000
-): Promise<Map<string, string>> {
-  const hashes = new Map<string, string>();
-
-  for (let i = 0; i < records.length; i += batchSize) {
-    const batch = records.slice(i, i + batchSize);
-
-    for (const record of batch) {
-      const key = `${record.sku}-${record.location}`;
-      const values = fields.map(f => String(record[f] || ''));
-      const hash = fastHash(values.join('|'));
-      hashes.set(key, hash);
-    }
-
-    if (i % 10000 === 0) {
-      console.log(`Processed ${i} records...`);
-    }
-  }
-
-  return hashes;
-}
-```
-
----
-
-## Pattern 3: Source System Change Tracking
-
-### Concept
-
-Use source system's native change tracking (change logs, CDC, delta exports).
-
-### How It Works
-
-```
-Source System with Change Tracking:
-┌─────────────────────────────────┐
-│ Inventory Table                 │
-│ - Changes logged automatically  │
-│ - Delta export API available    │
-└──────────────┬──────────────────┘
-               │
-               │ Query: GET /inventory/changes?since=2025-01-15T10:00:00Z
-               ▼
-┌─────────────────────────────────┐
-│ API Response: Only changed      │
-│ - 50 records (not 50,000)       │
-│ - Source filtered, not client   │
-└─────────────────────────────────┘
-```
-
-### Implementation (API-Based)
-
-```typescript
-/**
- * Source System Change Tracking
- *
- * Use when:
- * - Source system provides change tracking API
- * - Most efficient - source does the filtering
- */
-
-interface SourceSystemConfig {
-  apiUrl: string;
-  apiKey: string;
-}
-
-/**
- * Fetch only changed records from source system
- */
-async function fetchChangedRecords(
-  config: SourceSystemConfig,
-  lastSyncTime: string
-): Promise<any[]> {
-  const response = await fetch(
-    `${config.apiUrl}/inventory/changes?since=${encodeURIComponent(lastSyncTime)}`,
-    {
-      headers: {
-        'Authorization': `Bearer ${config.apiKey}`,
-        'Content-Type': 'application/json'
-      }
-    }
-  );
-
-  if (!response.ok) {
-    throw new Error(`Source API error: ${response.statusText}`);
-  }
-
-  const data = await response.json();
-  return data.changes;
-}
-
-async function sourceSystemDeltaSync() {
-  const kvAdapter = new FileKVAdapter('./state');
-  const stateService = new StateService(logger);
-
-  // Get last sync timestamp
-  let lastSyncTime = await stateService.getState('last-sync-timestamp');
-
-  if (!lastSyncTime) {
-    lastSyncTime = new Date(Date.now() - 24 * 60 * 60 * 1000).toISOString(); // Last 24 hours
-  }
-
-  console.log(`Fetching changes since ${lastSyncTime}`);
-
-  // Fetch only changed records from source
-  const changedRecords = await fetchChangedRecords({
-    apiUrl: process.env.SOURCE_API_URL!,
-    apiKey: process.env.SOURCE_API_KEY!
-  }, lastSyncTime);
-
-  console.log(`Source returned ${changedRecords.length} changed records`);
-
-  if (changedRecords.length === 0) {
-    console.log('No changes detected');
-    return;
-  }
-
-  // Process changed records (same as other patterns)
-  const mapper = new UniversalMapper({ /* config */ });
-  const mappedRecords = [];
-
-  for (const record of changedRecords) {
-    const result = await mapper.map(record);
-    if (result.success) {
-      mappedRecords.push(result.data);
-    }
-  }
-
-  // Send to Batch API
-  const client = await createClient({ /* config */ });
-  const job = await client.createJob({
-    name: `Source Delta Sync - ${new Date().toISOString()}`,
-    retailerId: '2'
-  });
-
-  const BATCH_SIZE = 100;
-  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
-    await client.sendBatch(job.id, {
-      entities: mappedRecords.slice(i, i + BATCH_SIZE)
-    });
-  }
-
-  // Poll completion
-  let status = await client.getJobStatus(job.id);
-  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-    await new Promise(r => setTimeout(r, 30000));
-    status = await client.getJobStatus(job.id);
-  }
-
-  if (status.status === 'COMPLETED') {
-    // Update last sync timestamp
-    await stateService.setState('last-sync-timestamp', new Date().toISOString());
-    console.log(`✓ Delta sync completed - ${mappedRecords.length} records processed`);
-  }
-}
-```
-
-### Implementation (File-Based Delta)
-
-```typescript
-/**
- * Source generates delta files (only changed records)
- */
-
-async function fileDeltaSync() {
-  const s3 = new S3DataSource({ /* config */ }, console);
-
-  // Source system uploads delta files to specific prefix
-  const deltaFiles = await s3.listFiles('inventory/deltas/');
-
-  console.log(`Found ${deltaFiles.length} delta files to process`);
-
-  for (const file of deltaFiles) {
-    // Download delta file (contains only changes)
-    const csvContent = await s3.downloadFile(file.key);
-
-    const parser = new CSVParserService({ headers: true });
-    const changedRecords = await parser.parse(csvContent);
-
-    console.log(`Processing ${changedRecords.length} changes from ${file.key}`);
-
-    // Process records (same as other patterns)
-    // ...
-
-    // Archive after processing
-    await s3.moveFile(file.key, file.key.replace('deltas/', 'deltas/processed/'));
-  }
-}
-```
-
----
-
-## State Storage Options
-
-### Option 1: File-Based (Standalone)
-
-**Best for**: Node.js/Deno standalone scripts
-
-```typescript
-import { FileKVAdapter, StateService } from '@fluentcommerce/fc-connect-sdk';
-
-const kvAdapter = new FileKVAdapter('./state', {
-  encoding: 'json',
-  pretty: true
-});
-
-const stateService = new StateService(logger);
-
-// State stored in: ./state/last-sync-timestamp.json
-await stateService.setState('last-sync-timestamp', '2025-01-15T10:00:00Z');
-```
-
-**File structure**:
-```
-./state/
-  ├── last-sync-timestamp.json
-  ├── record-hashes.json
-  └── processed-files.json
-```
-
-### Option 2: Versori KV Storage
-
-**Best for**: Versori platform workflows
-
-```typescript
-import { VersoriKVAdapter, StateService } from '@fluentcommerce/fc-connect-sdk';
-
-const kvAdapter = new VersoriKVAdapter(openKv());
-const stateService = new StateService(logger);
-
-// State stored in Versori KV storage (persistent across workflow runs)
-await stateService.setState('last-sync-timestamp', '2025-01-15T10:00:00Z');
-```
-
-### Option 3: S3 State Storage (Custom)
-
-**Best for**: Distributed processing, shared state
-
-```typescript
-/**
- * Custom S3-based state storage
- */
-class S3StateStore {
-  constructor(
-    private s3: S3DataSource,
-    private statePrefix: string = 'state/'
-  ) {}
-
-  async getState(key: string): Promise<any> {
-    try {
-      const content = await this.s3.downloadFile(`${this.statePrefix}${key}.json`);
-      return JSON.parse(content);
-    } catch (error) {
-      return null; // State doesn't exist
-    }
-  }
-
-  async setState(key: string, value: any): Promise<void> {
-    await this.s3.uploadFile(
-      `${this.statePrefix}${key}.json`,
-      JSON.stringify(value, null, 2)
-    );
-  }
-}
-
-// Usage
-const stateStore = new S3StateStore(s3, 'inventory-sync/state/');
-const lastSync = await stateStore.getState('last-sync-timestamp');
-await stateStore.setState('last-sync-timestamp', new Date().toISOString());
-```
-
----
-
-## Complete Delta Sync Implementation
-
-### Production-Ready Pattern (Versori Scheduled)
-
-```typescript
-/**
- * Complete Delta Sync - Versori Scheduled Workflow
- *
- * Features:
- * - Timestamp-based change detection
- * - State management with Versori KV
- * - Batch API processing
- * - Error handling and logging
- * - File archival
- */
-
-import { createClient, S3DataSource, CSVParserService, UniversalMapper, StateService, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-
-export default async function scheduledDeltaSync(activation: any, log: any, connections: any) {
-  try {
-    log.info('Starting delta sync workflow');
-
-    // Initialize state service
-const kvAdapter = new VersoriKVAdapter(openKv());
-const stateService = new StateService(logger);
-
-    // Get last sync timestamp
-    let lastSyncTime = await stateService.getState('last-sync-timestamp');
-
-    if (!lastSyncTime) {
-      log.info('First run - performing full sync');
-      lastSyncTime = new Date(0).toISOString();
-    } else {
-      log.info('Last sync timestamp', { lastSyncTime });
-    }
-
-    // Create S3 data source
-    const s3 = new S3DataSource({
-      connection: connections.aws_s3
-    }, log);
-
-    // Download current inventory file
-    const csvContent = await s3.downloadFile('inventory/current/inventory.csv');
-
-    // Parse CSV
-    const parser = new CSVParserService({ headers: true, skipEmptyLines: true });
-    const allRecords = await parser.parse(csvContent);
-
-    log.info('Parsed CSV', { totalRecords: allRecords.length });
-
-    // Filter changed records
-    const changedRecords = allRecords.filter(record => {
-      if (!record.lastModified) {
-        log.warn('Record missing lastModified', { sku: record.sku });
-        return true; // Include if no timestamp (safer)
-      }
-
-      const recordTimestamp = new Date(record.lastModified).toISOString();
-      return recordTimestamp > lastSyncTime;
-    });
-
-    log.info('Change detection complete', {
-      totalRecords: allRecords.length,
-      changedRecords: changedRecords.length,
-      changePercentage: ((changedRecords.length / allRecords.length) * 100).toFixed(2) + '%'
-    });
-
-    if (changedRecords.length === 0) {
-      log.info('No changes detected - skipping sync');
-      return { status: 200, body: { message: 'No changes', totalRecords: allRecords.length } };
-    }
-
-    // Map records
-    const mapper = new UniversalMapper({
-      fields: {
-        ref: { source: 'sku', resolver: 'custom.buildRef' },
-        type: { value: 'INVENTORY' },
-        productRef: { source: 'sku', required: true },
-        locationRef: { source: 'location', required: true },
-        onHand: { source: 'qty', resolver: 'sdk.parseInt' },
-        status: { source: 'status', resolver: 'sdk.uppercase' }
-      }
-    }, {
-      customResolvers: {
-        'custom.buildRef': (value, data) => `${data.sku}-${data.location}`
-      }
-    });
-
-    const mappedRecords = [];
-    const mappingErrors = [];
-
-    for (const record of changedRecords) {
-      const result = await mapper.map(record);
-      if (result.success) {
-        mappedRecords.push(result.data);
-      } else {
-        mappingErrors.push({ record, errors: result.errors });
-      }
-    }
-
-    if (mappingErrors.length > 0) {
-      log.warn('Mapping errors encountered', { errorCount: mappingErrors.length });
-    }
-
-    log.info('Mapping complete', { mappedRecords: mappedRecords.length });
-
-    // Create Fluent client
-    const client = await createClient({
-      connection: connections.fluent_commerce,
-      logger: log
-    });
-
-    // Create Batch job
-    const job = await client.createJob({
-      name: `Delta Sync - ${new Date().toISOString()}`,
-      retailerId: '2'
-    });
-
-    log.info('Created Batch job', { jobId: job.id });
-
-    // Send batches
-    const BATCH_SIZE = 100;
-    let batchCount = 0;
-
-    for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
-      const batch = mappedRecords.slice(i, i + BATCH_SIZE);
-
-      await client.sendBatch(job.id, { entities: batch });
-      batchCount++;
-
-      log.info('Sent batch', { batchNumber: batchCount, recordCount: batch.length });
-    }
-
-    // Poll status
-    let status = await client.getJobStatus(job.id);
-    let pollCount = 0;
-
-    while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-      pollCount++;
-
-      log.info('Polling job status', {
-        status: status.status,
-        completedBatches: status.completedBatches,
-        totalBatches: status.totalBatches,
-        pollCount
-      });
-
-      await new Promise(resolve => setTimeout(resolve, 30000)); // 30 seconds
-
-      status = await client.getJobStatus(job.id);
-    }
-
-    if (status.status === 'COMPLETED') {
-      // Update last sync timestamp
-      const currentTime = new Date().toISOString();
-      await stateService.setState('last-sync-timestamp', currentTime);
-
-      log.info('Delta sync completed successfully', {
-        recordsProcessed: mappedRecords.length,
-        errorCount: status.errorSummary?.totalErrors || 0,
-        nextSyncAfter: currentTime
-      });
-
-      return {
-        status: 200,
-        body: {
-          success: true,
-          totalRecords: allRecords.length,
-          changedRecords: changedRecords.length,
-          processedRecords: mappedRecords.length,
-          jobId: job.id,
-          nextSyncAfter: currentTime
-        }
-      };
-    } else {
-      throw new Error(`Job failed with status: ${status.status}`);
-    }
-
-  } catch (error: any) {
-    log.error('Delta sync failed', error);
-
-    return {
-      status: 500,
-      body: {
-        success: false,
-        error: error.message
-      }
-    };
-  }
-}
-```
-
----
-
-## Performance Optimization
-
-### Memory-Efficient Hash Calculation
-
-```typescript
-/**
- * Stream-based hash calculation for large datasets
- */
-async function streamHashCalculation(
-  csvStream: ReadableStream,
-  hashFields: string[]
-): Promise<Map<string, { hash: string; record: any }>> {
-  const parser = new CSVParserService({ headers: true });
-  const hashes = new Map();
-
-  for await (const record of parser.streamParse(csvStream)) {
-    const key = `${record.sku}-${record.location}`;
-    const values = hashFields.map(f => String(record[f] || ''));
-    const hash = fastHash(values.join('|'));
-
-    hashes.set(key, { hash, record });
-  }
-
-  return hashes;
-}
-```
-
-### Incremental State Updates
-
-```typescript
-/**
- * Update state incrementally instead of all at once
- */
-async function updateStateIncremental(
-  stateService: StateService,
-  records: any[],
-  batchSize = 1000
-) {
-  const hashes: Record<string, string> = {};
-
-  for (let i = 0; i < records.length; i += batchSize) {
-    const batch = records.slice(i, i + batchSize);
-
-    for (const record of batch) {
-      const key = `${record.sku}-${record.location}`;
-      hashes[key] = calculateRecordHash(record, ['sku', 'location', 'qty']);
-    }
-
-    // Update state every batch (more resilient to failures)
-    await stateService.setState('record-hashes', hashes);
-
-    console.log(`Updated state for ${i + batch.length} records`);
-  }
-}
-```
-
----
-
-## Next Steps
-
-Now that you understand delta sync, you're ready to explore webhook patterns for event-driven architectures!
-
-**Continue to:** [Module 4: Webhook Patterns →](./integration-patterns-04-webhook-patterns.md)
-
-Or explore:
-- [Module 5: Error Handling](./integration-patterns-05-error-handling.md) - Resilience strategies
-- [Complete Example: Delta Sync](../examples/delta-sync.ts)
-- [Complete Example: Versori Delta Sync](../../../01-TEMPLATES/versori/workflows/readme.md) - See delta sync patterns
-
----
-
-## Additional Resources
-
-- [StateService API Reference](../../../02-CORE-GUIDES/api-reference/modules/api-reference-05-services.md#state-management-service)
-- [File Tracking Patterns](../../../02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-07-state-management.md)
-- [Performance Optimization Guide](../../../02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-08-performance-optimization.md)
-
----
-
-[← Back to Index](../integration-patterns-readme.md) | [Previous: Batch Processing →](./integration-patterns-02-batch-processing.md) | [Next: Webhook Patterns →](./integration-patterns-04-webhook-patterns.md)
+# Module 3: Delta Sync (Change Detection)
+
+> **Learning Objective:** Implement efficient delta synchronization to process only changed records, reducing processing time and API calls by 90%+.
+>
+> **Level:** Advanced
+
+## Table of Contents
+
+1. [What is Delta Sync?](#what-is-delta-sync)
+2. [Why Delta Sync Matters](#why-delta-sync-matters)
+3. [Change Detection Strategies](#change-detection-strategies)
+4. [SDK State Management](#sdk-state-management)
+5. [Pattern 1: Timestamp-Based Detection](#pattern-1-timestamp-based-detection)
+6. [Pattern 2: Hash-Based Detection](#pattern-2-hash-based-detection)
+7. [Pattern 3: Source System Change Tracking](#pattern-3-source-system-change-tracking)
+8. [State Storage Options](#state-storage-options)
+9. [Complete Delta Sync Implementation](#complete-delta-sync-implementation)
+10. [Performance Optimization](#performance-optimization)
+11. [Next Steps](#next-steps)
+
+---
+
+## What is Delta Sync?
+
+**Delta sync** (delta synchronization) means processing only the records that have **changed** since the last sync, rather than reprocessing the entire dataset.
+
+### Full Sync vs Delta Sync
+
+```
+FULL SYNC (every run):
+Day 1: Process 50,000 records → 10 minutes
+Day 2: Process 50,000 records → 10 minutes (99% duplicates!)
+Day 3: Process 50,000 records → 10 minutes (99% duplicates!)
+
+DELTA SYNC (only changes):
+Day 1: Process 50,000 records → 10 minutes (initial load)
+Day 2: Process 50 changed records → 10 seconds
+Day 3: Process 120 changed records → 15 seconds
+```
+
+### Visual Comparison
+
+```
+FULL SYNC:
+┌──────────────────────────────────────┐
+│ Source: 50,000 inventory records     │
+└──────────────┬───────────────────────┘
+               │
+               │ Process ALL 50K records every time
+               ▼
+┌──────────────────────────────────────┐
+│ Fluent Batch API: 500 batches       │
+│ Processing Time: 10 minutes          │
+│ API Calls: 500                       │
+└──────────────────────────────────────┘
+
+DELTA SYNC:
+┌──────────────────────────────────────┐
+│ Source: 50,000 inventory records     │
+│ Changed: 50 records (0.1%)           │
+└──────────────┬───────────────────────┘
+               │
+               │ Process ONLY 50 changed records
+               ▼
+┌──────────────────────────────────────┐
+│ Fluent Batch API: 1 batch            │
+│ Processing Time: 10 seconds          │
+│ API Calls: 1                         │
+└──────────────────────────────────────┘
+
+Performance Improvement: 99% reduction in processing time
+```
+
+---
+
+## Why Delta Sync Matters
+
+### Benefits
+
+| Benefit | Impact | Example |
+|---------|--------|---------|
+| **Faster Processing** | 90-99% time reduction | 10 min → 10 sec |
+| **Lower API Costs** | Fewer API calls | 500 calls → 1 call |
+| **Reduced Load** | Less database/network strain | Minimal impact on source system |
+| **Real-Time Capable** | Frequent syncs (hourly/15min) | Near real-time updates |
+| **Error Recovery** | Smaller failure blast radius | 50 records vs 50K records |
+
+### When Delta Sync is Critical
+
+| Scenario | Why? | Change Rate |
+|----------|------|-------------|
+| **High-Frequency Sync** | Hourly/15-minute syncs | < 1% change rate |
+| **Large Datasets** | 100K+ records | Any change rate |
+| **Real-Time Requirements** | Must sync frequently | < 5% change rate |
+| **API Rate Limits** | Limited API quota | Any change rate |
+| **Cost Optimization** | Pay-per-API-call pricing | Any change rate |
+
+### Cost Example
+
+```
+Scenario: 100,000 inventory records, 0.5% daily change rate
+
+FULL SYNC (daily):
+- Records processed: 100,000
+- Batches: 1,000
+- API calls: 1,000
+- Monthly API calls: 30,000
+- Processing time: 20 minutes/day
+
+DELTA SYNC (daily):
+- Records processed: 500 (0.5% of 100K)
+- Batches: 5
+- API calls: 5
+- Monthly API calls: 150
+- Processing time: 30 seconds/day
+
+Savings:
+- API calls: 99.5% reduction
+- Processing time: 97.5% reduction
+- Infrastructure cost: 95%+ reduction
+```
+
+---
+
+## Change Detection Strategies
+
+### Strategy Comparison
+
+| Strategy | Pros | Cons | Best For |
+|----------|------|------|----------|
+| **Timestamp** | Simple, reliable | Requires lastModified field | Most systems |
+| **Hash** | Detects any change | CPU overhead for hashing | Systems without timestamps |
+| **Source System** | Most accurate | Requires source system support | Modern APIs |
+| **Hybrid** | Best accuracy | More complex | Production systems |
+
+### Decision Matrix
+
+```
+Does source have lastModified/updatedAt field?
+├── YES → Use Timestamp-Based Detection
+│   └── Does source support "changes since" query?
+│       ├── YES → Use Source System Change Tracking (best)
+│       └── NO → Use Timestamp Comparison
+│
+└── NO → Use Hash-Based Detection
+    └── Can you add lastModified to source?
+        ├── YES → Add timestamp, use Timestamp-Based
+        └── NO → Hash-Based (required)
+```
+
+---
+
+## SDK State Management
+
+### StateService Overview
+
+The SDK provides `StateService` for tracking processed files and records:
+
+```typescript
+import { StateService, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+
+// Versori platform (KV storage)
+const kvAdapter = new VersoriKVAdapter(openKv());
+const stateService = new StateService(logger);
+
+// Standalone (file-based storage)
+import { FileKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+const kvAdapter = new FileKVAdapter('./state');
+const stateService = new StateService(logger);
+```
+
+### StateService Methods
+
+```typescript
+// Check if file was processed
+const processed = await stateService.isFileProcessed('file-key');
+
+// Mark file as processed
+await stateService.markFileProcessed('file-key', {
+  processedAt: new Date(),
+  recordCount: 5000,
+  jobId: 'job-123'
+});
+
+// Get file processing metadata
+const metadata = await stateService.getFileMetadata('file-key');
+
+// Store custom state
+await stateService.setState('last-sync-timestamp', Date.now());
+
+// Retrieve custom state
+const lastSync = await stateService.getState('last-sync-timestamp');
+
+// Clear state (for testing/reset)
+await stateService.clearFileState('file-key');
+```
+
+### KV Storage Adapters
+
+**VersoriKVAdapter** (for Versori platform):
+```typescript
+import { VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+
+const kvAdapter = new VersoriKVAdapter(openKv());
+```
+
+**FileKVAdapter** (for standalone Node.js/Deno):
+```typescript
+import { FileKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+
+const kvAdapter = new FileKVAdapter('./state', {
+  encoding: 'json',
+  pretty: true
+});
+```
+
+---
+
+## Pattern 1: Timestamp-Based Detection
+
+### Concept
+
+Compare `lastModified` timestamp from source with last sync timestamp.
+
+### How It Works
+
+```
+Step 1: Get last sync timestamp from state
+  └── lastSync = 2025-01-15T10:00:00Z
+
+Step 2: Query source for records modified after lastSync
+  └── SELECT * FROM inventory WHERE lastModified > '2025-01-15T10:00:00Z'
+
+Step 3: Process only those changed records
+  └── 50 records (instead of 50,000)
+
+Step 4: Update last sync timestamp
+  └── lastSync = 2025-01-15T14:00:00Z (current time)
+```
+
+### Implementation
+
+```typescript
+/**
+ * Timestamp-Based Delta Sync
+ *
+ * Requirements:
+ * - Source data has lastModified/updatedAt field
+ * - Timestamps are reliable and monotonic
+ */
+
+import { createClient, S3DataSource, CSVParserService, UniversalMapper, StateService, FileKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+
+async function deltaSync() {
+  // Initialize state service
+  const kvAdapter = new FileKVAdapter('./state');
+  const stateService = new StateService(logger);
+
+  // Get last sync timestamp
+  let lastSyncTime = await stateService.getState('last-sync-timestamp');
+
+  if (!lastSyncTime) {
+    console.log('First run - performing full sync');
+    lastSyncTime = new Date(0).toISOString(); // Epoch (all records)
+  } else {
+    console.log(`Last sync: ${lastSyncTime}`);
+  }
+
+  // Download CSV
+  const s3 = new S3DataSource({
+    type: 'S3_CSV',
+    connectionId: 'my-s3',
+    name: 'My S3 Source',
+    s3Config: {
+      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY,
+      region: process.env.AWS_REGION,
+      bucket: process.env.AWS_BUCKET
+    }
+  }, console);
+
+  const csvContent = await s3.downloadFile('inventory/current.csv');
+
+  // Parse CSV
+  const parser = new CSVParserService({ headers: true });
+  const allRecords = await parser.parse(csvContent);
+
+  console.log(`Total records in source: ${allRecords.length}`);
+
+  // Filter changed records
+  const changedRecords = allRecords.filter(record => {
+    const recordTimestamp = new Date(record.lastModified).toISOString();
+    return recordTimestamp > lastSyncTime;
+  });
+
+  console.log(`Changed records: ${changedRecords.length} (${((changedRecords.length / allRecords.length) * 100).toFixed(2)}%)`);
+
+  if (changedRecords.length === 0) {
+    console.log('No changes detected - skipping sync');
+    return;
+  }
+
+  // Map records
+  const mapper = new UniversalMapper({
+    fields: {
+      ref: { source: 'sku', resolver: 'custom.buildRef' },
+      type: { value: 'INVENTORY' },
+      productRef: { source: 'sku', required: true },
+      locationRef: { source: 'location', required: true },
+      onHand: { source: 'qty', resolver: 'sdk.parseInt' }
+    }
+  }, {
+    customResolvers: {
+      'custom.buildRef': (v, d) => `${d.sku}-${d.location}`
+    }
+  });
+
+  const mappedRecords = [];
+  for (const record of changedRecords) {
+    const result = await mapper.map(record);
+    if (result.success) {
+      mappedRecords.push(result.data);
+    }
+  }
+
+  // Create client and job
+  const client = await createClient({
+    config: {
+      baseUrl: process.env.FLUENT_BASE_URL,
+      clientId: process.env.FLUENT_CLIENT_ID,
+      clientSecret: process.env.FLUENT_CLIENT_SECRET,
+      retailerId: process.env.FLUENT_RETAILER_ID
+    }
+  });
+
+  const job = await client.createJob({
+    name: `Delta Sync - ${new Date().toISOString()}`,
+    retailerId: '2'
+  });
+
+  // Send batches
+  const BATCH_SIZE = 100;
+  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
+    const batch = mappedRecords.slice(i, i + BATCH_SIZE);
+    await client.sendBatch(job.id, { entities: batch });
+  }
+
+  // Poll completion
+  let status = await client.getJobStatus(job.id);
+  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+    await new Promise(r => setTimeout(r, 30000));
+    status = await client.getJobStatus(job.id);
+  }
+
+  if (status.status === 'COMPLETED') {
+    // Update last sync timestamp
+    const currentTime = new Date().toISOString();
+    await stateService.setState('last-sync-timestamp', currentTime);
+
+    console.log(`✓ Delta sync completed - ${mappedRecords.length} records processed`);
+    console.log(`Next sync will process changes after ${currentTime}`);
+  } else {
+    throw new Error(`Job failed: ${status.status}`);
+  }
+}
+
+deltaSync().catch(console.error);
+```
+
+### Timestamp Format Handling
+
+```typescript
+/**
+ * Normalize various timestamp formats
+ */
+function normalizeTimestamp(timestamp: string | number | Date): string {
+  // Handle different input types
+  if (timestamp instanceof Date) {
+    return timestamp.toISOString();
+  }
+
+  if (typeof timestamp === 'number') {
+    return new Date(timestamp).toISOString();
+  }
+
+  // Parse string timestamp
+  const date = new Date(timestamp);
+
+  if (isNaN(date.getTime())) {
+    throw new Error(`Invalid timestamp: ${timestamp}`);
+  }
+
+  return date.toISOString();
+}
+
+// Usage
+const recordTimestamp = normalizeTimestamp(record.lastModified);
+const isChanged = recordTimestamp > lastSyncTime;
+```
+
+---
+
+## Pattern 2: Hash-Based Detection
+
+### Concept
+
+Calculate hash of record content and compare with previous hash.
+
+### How It Works
+
+```
+Step 1: Load previous hashes from state
+  └── { "SKU001-WH01": "a1b2c3d4", "SKU002-WH01": "e5f6g7h8" }
+
+Step 2: For each record, calculate current hash
+  └── currentHash = hash(sku + location + qty + ...)
+
+Step 3: Compare with previous hash
+  └── If hash changed → process record
+  └── If hash same → skip record
+
+Step 4: Store current hashes for next run
+  └── { "SKU001-WH01": "a1b2c3d4", "SKU002-WH01": "x9y8z7w6" }
+```
+
+### Implementation
+
+```typescript
+/**
+ * Hash-Based Delta Sync
+ *
+ * Use when:
+ * - Source data has NO lastModified field
+ * - Need to detect ANY field changes
+ */
+
+import crypto from 'crypto';
+
+/**
+ * Calculate hash of record
+ */
+function calculateRecordHash(record: any, fields: string[]): string {
+  // Extract relevant fields in consistent order
+  const values = fields.map(field => String(record[field] || ''));
+
+  // Join and hash
+  const combined = values.join('|');
+  return crypto.createHash('md5').update(combined).digest('hex');
+}
+
+async function hashBasedDeltaSync() {
+  const kvAdapter = new FileKVAdapter('./state');
+  const stateService = new StateService(logger);
+
+  // Load previous hashes
+  const previousHashes = await stateService.getState('record-hashes') || {};
+
+  console.log(`Loaded ${Object.keys(previousHashes).length} previous hashes`);
+
+  // Download and parse
+  const s3 = new S3DataSource({ /* config */ }, console);
+  const csvContent = await s3.downloadFile('inventory/current.csv');
+
+  const parser = new CSVParserService({ headers: true });
+  const allRecords = await parser.parse(csvContent);
+
+  console.log(`Total records: ${allRecords.length}`);
+
+  // Detect changes
+  const changedRecords = [];
+  const currentHashes: Record<string, string> = {};
+
+  // Fields to include in hash (all fields that matter)
+  const hashFields = ['sku', 'location', 'qty', 'status'];
+
+  for (const record of allRecords) {
+    const recordKey = `${record.sku}-${record.location}`;
+    const currentHash = calculateRecordHash(record, hashFields);
+
+    currentHashes[recordKey] = currentHash;
+
+    const previousHash = previousHashes[recordKey];
+
+    if (!previousHash) {
+      // New record
+      console.log(`New record: ${recordKey}`);
+      changedRecords.push(record);
+    } else if (currentHash !== previousHash) {
+      // Changed record
+      console.log(`Changed record: ${recordKey} (hash: ${previousHash} → ${currentHash})`);
+      changedRecords.push(record);
+    }
+    // else: unchanged, skip
+  }
+
+  console.log(`Changed records: ${changedRecords.length} (${((changedRecords.length / allRecords.length) * 100).toFixed(2)}%)`);
+
+  if (changedRecords.length === 0) {
+    console.log('No changes detected - skipping sync');
+    return;
+  }
+
+  // Process changed records (same as timestamp-based)
+  const mapper = new UniversalMapper({ /* config */ });
+  const mappedRecords = [];
+
+  for (const record of changedRecords) {
+    const result = await mapper.map(record);
+    if (result.success) {
+      mappedRecords.push(result.data);
+    }
+  }
+
+  // Send to Batch API
+  const client = await createClient({ /* config */ });
+  const job = await client.createJob({ name: 'Hash-Based Delta Sync', retailerId: '2' });
+
+  const BATCH_SIZE = 100;
+  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
+    await client.sendBatch(job.id, {
+      entities: mappedRecords.slice(i, i + BATCH_SIZE)
+    });
+  }
+
+  // Poll completion
+  let status = await client.getJobStatus(job.id);
+  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+    await new Promise(r => setTimeout(r, 30000));
+    status = await client.getJobStatus(job.id);
+  }
+
+  if (status.status === 'COMPLETED') {
+    // Store current hashes for next run
+    await stateService.setState('record-hashes', currentHashes);
+
+    console.log(`✓ Delta sync completed - ${mappedRecords.length} records processed`);
+    console.log(`Stored ${Object.keys(currentHashes).length} hashes for next run`);
+  } else {
+    throw new Error(`Job failed: ${status.status}`);
+  }
+}
+
+hashBasedDeltaSync().catch(console.error);
+```
+
+### Hash Performance Optimization
+
+```typescript
+/**
+ * Fast hash using Node.js crypto (faster than MD5)
+ */
+function fastHash(data: string): string {
+  return crypto.createHash('sha1').update(data).digest('hex');
+}
+
+/**
+ * Calculate hash for large datasets efficiently
+ */
+async function calculateHashesBatch(
+  records: any[],
+  fields: string[],
+  batchSize = 1000
+): Promise<Map<string, string>> {
+  const hashes = new Map<string, string>();
+
+  for (let i = 0; i < records.length; i += batchSize) {
+    const batch = records.slice(i, i + batchSize);
+
+    for (const record of batch) {
+      const key = `${record.sku}-${record.location}`;
+      const values = fields.map(f => String(record[f] || ''));
+      const hash = fastHash(values.join('|'));
+      hashes.set(key, hash);
+    }
+
+    if (i % 10000 === 0) {
+      console.log(`Processed ${i} records...`);
+    }
+  }
+
+  return hashes;
+}
+```
+
+---
+
+## Pattern 3: Source System Change Tracking
+
+### Concept
+
+Use source system's native change tracking (change logs, CDC, delta exports).
+
+### How It Works
+
+```
+Source System with Change Tracking:
+┌─────────────────────────────────┐
+│ Inventory Table                 │
+│ - Changes logged automatically  │
+│ - Delta export API available    │
+└──────────────┬──────────────────┘
+               │
+               │ Query: GET /inventory/changes?since=2025-01-15T10:00:00Z
+               ▼
+┌─────────────────────────────────┐
+│ API Response: Only changed      │
+│ - 50 records (not 50,000)       │
+│ - Source filtered, not client   │
+└─────────────────────────────────┘
+```
+
+### Implementation (API-Based)
+
+```typescript
+/**
+ * Source System Change Tracking
+ *
+ * Use when:
+ * - Source system provides change tracking API
+ * - Most efficient - source does the filtering
+ */
+
+interface SourceSystemConfig {
+  apiUrl: string;
+  apiKey: string;
+}
+
+/**
+ * Fetch only changed records from source system
+ */
+async function fetchChangedRecords(
+  config: SourceSystemConfig,
+  lastSyncTime: string
+): Promise<any[]> {
+  const response = await fetch(
+    `${config.apiUrl}/inventory/changes?since=${encodeURIComponent(lastSyncTime)}`,
+    {
+      headers: {
+        'Authorization': `Bearer ${config.apiKey}`,
+        'Content-Type': 'application/json'
+      }
+    }
+  );
+
+  if (!response.ok) {
+    throw new Error(`Source API error: ${response.statusText}`);
+  }
+
+  const data = await response.json();
+  return data.changes;
+}
+
+async function sourceSystemDeltaSync() {
+  const kvAdapter = new FileKVAdapter('./state');
+  const stateService = new StateService(logger);
+
+  // Get last sync timestamp
+  let lastSyncTime = await stateService.getState('last-sync-timestamp');
+
+  if (!lastSyncTime) {
+    lastSyncTime = new Date(Date.now() - 24 * 60 * 60 * 1000).toISOString(); // Last 24 hours
+  }
+
+  console.log(`Fetching changes since ${lastSyncTime}`);
+
+  // Fetch only changed records from source
+  const changedRecords = await fetchChangedRecords({
+    apiUrl: process.env.SOURCE_API_URL!,
+    apiKey: process.env.SOURCE_API_KEY!
+  }, lastSyncTime);
+
+  console.log(`Source returned ${changedRecords.length} changed records`);
+
+  if (changedRecords.length === 0) {
+    console.log('No changes detected');
+    return;
+  }
+
+  // Process changed records (same as other patterns)
+  const mapper = new UniversalMapper({ /* config */ });
+  const mappedRecords = [];
+
+  for (const record of changedRecords) {
+    const result = await mapper.map(record);
+    if (result.success) {
+      mappedRecords.push(result.data);
+    }
+  }
+
+  // Send to Batch API
+  const client = await createClient({ /* config */ });
+  const job = await client.createJob({
+    name: `Source Delta Sync - ${new Date().toISOString()}`,
+    retailerId: '2'
+  });
+
+  const BATCH_SIZE = 100;
+  for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
+    await client.sendBatch(job.id, {
+      entities: mappedRecords.slice(i, i + BATCH_SIZE)
+    });
+  }
+
+  // Poll completion
+  let status = await client.getJobStatus(job.id);
+  while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+    await new Promise(r => setTimeout(r, 30000));
+    status = await client.getJobStatus(job.id);
+  }
+
+  if (status.status === 'COMPLETED') {
+    // Update last sync timestamp
+    await stateService.setState('last-sync-timestamp', new Date().toISOString());
+    console.log(`✓ Delta sync completed - ${mappedRecords.length} records processed`);
+  }
+}
+```
+
+### Implementation (File-Based Delta)
+
+```typescript
+/**
+ * Source generates delta files (only changed records)
+ */
+
+async function fileDeltaSync() {
+  const s3 = new S3DataSource({ /* config */ }, console);
+
+  // Source system uploads delta files to specific prefix
+  const deltaFiles = await s3.listFiles('inventory/deltas/');
+
+  console.log(`Found ${deltaFiles.length} delta files to process`);
+
+  for (const file of deltaFiles) {
+    // Download delta file (contains only changes)
+    const csvContent = await s3.downloadFile(file.key);
+
+    const parser = new CSVParserService({ headers: true });
+    const changedRecords = await parser.parse(csvContent);
+
+    console.log(`Processing ${changedRecords.length} changes from ${file.key}`);
+
+    // Process records (same as other patterns)
+    // ...
+
+    // Archive after processing
+    await s3.moveFile(file.key, file.key.replace('deltas/', 'deltas/processed/'));
+  }
+}
+```
+
+---
+
+## State Storage Options
+
+### Option 1: File-Based (Standalone)
+
+**Best for**: Node.js/Deno standalone scripts
+
+```typescript
+import { FileKVAdapter, StateService } from '@fluentcommerce/fc-connect-sdk';
+
+const kvAdapter = new FileKVAdapter('./state', {
+  encoding: 'json',
+  pretty: true
+});
+
+const stateService = new StateService(logger);
+
+// State stored in: ./state/last-sync-timestamp.json
+await stateService.setState('last-sync-timestamp', '2025-01-15T10:00:00Z');
+```
+
+**File structure**:
+```
+./state/
+  ├── last-sync-timestamp.json
+  ├── record-hashes.json
+  └── processed-files.json
+```
+
+### Option 2: Versori KV Storage
+
+**Best for**: Versori platform workflows
+
+```typescript
+import { VersoriKVAdapter, StateService } from '@fluentcommerce/fc-connect-sdk';
+
+const kvAdapter = new VersoriKVAdapter(openKv());
+const stateService = new StateService(logger);
+
+// State stored in Versori KV storage (persistent across workflow runs)
+await stateService.setState('last-sync-timestamp', '2025-01-15T10:00:00Z');
+```
+
+### Option 3: S3 State Storage (Custom)
+
+**Best for**: Distributed processing, shared state
+
+```typescript
+/**
+ * Custom S3-based state storage
+ */
+class S3StateStore {
+  constructor(
+    private s3: S3DataSource,
+    private statePrefix: string = 'state/'
+  ) {}
+
+  async getState(key: string): Promise<any> {
+    try {
+      const content = await this.s3.downloadFile(`${this.statePrefix}${key}.json`);
+      return JSON.parse(content);
+    } catch (error) {
+      return null; // State doesn't exist
+    }
+  }
+
+  async setState(key: string, value: any): Promise<void> {
+    await this.s3.uploadFile(
+      `${this.statePrefix}${key}.json`,
+      JSON.stringify(value, null, 2)
+    );
+  }
+}
+
+// Usage
+const stateStore = new S3StateStore(s3, 'inventory-sync/state/');
+const lastSync = await stateStore.getState('last-sync-timestamp');
+await stateStore.setState('last-sync-timestamp', new Date().toISOString());
+```
+
+---
+
+## Complete Delta Sync Implementation
+
+### Production-Ready Pattern (Versori Scheduled)
+
+```typescript
+/**
+ * Complete Delta Sync - Versori Scheduled Workflow
+ *
+ * Features:
+ * - Timestamp-based change detection
+ * - State management with Versori KV
+ * - Batch API processing
+ * - Error handling and logging
+ * - File archival
+ */
+
+import { createClient, S3DataSource, CSVParserService, UniversalMapper, StateService, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+
+export default async function scheduledDeltaSync(activation: any, log: any, connections: any) {
+  try {
+    log.info('Starting delta sync workflow');
+
+    // Initialize state service
+const kvAdapter = new VersoriKVAdapter(openKv());
+const stateService = new StateService(logger);
+
+    // Get last sync timestamp
+    let lastSyncTime = await stateService.getState('last-sync-timestamp');
+
+    if (!lastSyncTime) {
+      log.info('First run - performing full sync');
+      lastSyncTime = new Date(0).toISOString();
+    } else {
+      log.info('Last sync timestamp', { lastSyncTime });
+    }
+
+    // Create S3 data source
+    const s3 = new S3DataSource({
+      connection: connections.aws_s3
+    }, log);
+
+    // Download current inventory file
+    const csvContent = await s3.downloadFile('inventory/current/inventory.csv');
+
+    // Parse CSV
+    const parser = new CSVParserService({ headers: true, skipEmptyLines: true });
+    const allRecords = await parser.parse(csvContent);
+
+    log.info('Parsed CSV', { totalRecords: allRecords.length });
+
+    // Filter changed records
+    const changedRecords = allRecords.filter(record => {
+      if (!record.lastModified) {
+        log.warn('Record missing lastModified', { sku: record.sku });
+        return true; // Include if no timestamp (safer)
+      }
+
+      const recordTimestamp = new Date(record.lastModified).toISOString();
+      return recordTimestamp > lastSyncTime;
+    });
+
+    log.info('Change detection complete', {
+      totalRecords: allRecords.length,
+      changedRecords: changedRecords.length,
+      changePercentage: ((changedRecords.length / allRecords.length) * 100).toFixed(2) + '%'
+    });
+
+    if (changedRecords.length === 0) {
+      log.info('No changes detected - skipping sync');
+      return { status: 200, body: { message: 'No changes', totalRecords: allRecords.length } };
+    }
+
+    // Map records
+    const mapper = new UniversalMapper({
+      fields: {
+        ref: { source: 'sku', resolver: 'custom.buildRef' },
+        type: { value: 'INVENTORY' },
+        productRef: { source: 'sku', required: true },
+        locationRef: { source: 'location', required: true },
+        onHand: { source: 'qty', resolver: 'sdk.parseInt' },
+        status: { source: 'status', resolver: 'sdk.uppercase' }
+      }
+    }, {
+      customResolvers: {
+        'custom.buildRef': (value, data) => `${data.sku}-${data.location}`
+      }
+    });
+
+    const mappedRecords = [];
+    const mappingErrors = [];
+
+    for (const record of changedRecords) {
+      const result = await mapper.map(record);
+      if (result.success) {
+        mappedRecords.push(result.data);
+      } else {
+        mappingErrors.push({ record, errors: result.errors });
+      }
+    }
+
+    if (mappingErrors.length > 0) {
+      log.warn('Mapping errors encountered', { errorCount: mappingErrors.length });
+    }
+
+    log.info('Mapping complete', { mappedRecords: mappedRecords.length });
+
+    // Create Fluent client
+    const client = await createClient({
+      connection: connections.fluent_commerce,
+      logger: log
+    });
+
+    // Create Batch job
+    const job = await client.createJob({
+      name: `Delta Sync - ${new Date().toISOString()}`,
+      retailerId: '2'
+    });
+
+    log.info('Created Batch job', { jobId: job.id });
+
+    // Send batches
+    const BATCH_SIZE = 100;
+    let batchCount = 0;
+
+    for (let i = 0; i < mappedRecords.length; i += BATCH_SIZE) {
+      const batch = mappedRecords.slice(i, i + BATCH_SIZE);
+
+      await client.sendBatch(job.id, { entities: batch });
+      batchCount++;
+
+      log.info('Sent batch', { batchNumber: batchCount, recordCount: batch.length });
+    }
+
+    // Poll status
+    let status = await client.getJobStatus(job.id);
+    let pollCount = 0;
+
+    while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+      pollCount++;
+
+      log.info('Polling job status', {
+        status: status.status,
+        completedBatches: status.completedBatches,
+        totalBatches: status.totalBatches,
+        pollCount
+      });
+
+      await new Promise(resolve => setTimeout(resolve, 30000)); // 30 seconds
+
+      status = await client.getJobStatus(job.id);
+    }
+
+    if (status.status === 'COMPLETED') {
+      // Update last sync timestamp
+      const currentTime = new Date().toISOString();
+      await stateService.setState('last-sync-timestamp', currentTime);
+
+      log.info('Delta sync completed successfully', {
+        recordsProcessed: mappedRecords.length,
+        errorCount: status.errorSummary?.totalErrors || 0,
+        nextSyncAfter: currentTime
+      });
+
+      return {
+        status: 200,
+        body: {
+          success: true,
+          totalRecords: allRecords.length,
+          changedRecords: changedRecords.length,
+          processedRecords: mappedRecords.length,
+          jobId: job.id,
+          nextSyncAfter: currentTime
+        }
+      };
+    } else {
+      throw new Error(`Job failed with status: ${status.status}`);
+    }
+
+  } catch (error: any) {
+    log.error('Delta sync failed', error);
+
+    return {
+      status: 500,
+      body: {
+        success: false,
+        error: error.message
+      }
+    };
+  }
+}
+```
+
+---
+
+## Performance Optimization
+
+### Memory-Efficient Hash Calculation
+
+```typescript
+/**
+ * Stream-based hash calculation for large datasets
+ */
+async function streamHashCalculation(
+  csvStream: ReadableStream,
+  hashFields: string[]
+): Promise<Map<string, { hash: string; record: any }>> {
+  const parser = new CSVParserService({ headers: true });
+  const hashes = new Map();
+
+  for await (const record of parser.streamParse(csvStream)) {
+    const key = `${record.sku}-${record.location}`;
+    const values = hashFields.map(f => String(record[f] || ''));
+    const hash = fastHash(values.join('|'));
+
+    hashes.set(key, { hash, record });
+  }
+
+  return hashes;
+}
+```
+
+### Incremental State Updates
+
+```typescript
+/**
+ * Update state incrementally instead of all at once
+ */
+async function updateStateIncremental(
+  stateService: StateService,
+  records: any[],
+  batchSize = 1000
+) {
+  const hashes: Record<string, string> = {};
+
+  for (let i = 0; i < records.length; i += batchSize) {
+    const batch = records.slice(i, i + batchSize);
+
+    for (const record of batch) {
+      const key = `${record.sku}-${record.location}`;
+      hashes[key] = calculateRecordHash(record, ['sku', 'location', 'qty']);
+    }
+
+    // Update state every batch (more resilient to failures)
+    await stateService.setState('record-hashes', hashes);
+
+    console.log(`Updated state for ${i + batch.length} records`);
+  }
+}
+```
+
+---
+
+## Next Steps
+
+Now that you understand delta sync, you're ready to explore webhook patterns for event-driven architectures!
+
+**Continue to:** [Module 4: Webhook Patterns →](./integration-patterns-04-webhook-patterns.md)
+
+Or explore:
+- [Module 5: Error Handling](./integration-patterns-05-error-handling.md) - Resilience strategies
+- [Complete Example: Delta Sync](../examples/delta-sync.ts)
+- [Complete Example: Versori Delta Sync](../../../01-TEMPLATES/versori/workflows/readme.md) - See delta sync patterns
+
+---
+
+## Additional Resources
+
+- [StateService API Reference](../../../02-CORE-GUIDES/api-reference/modules/api-reference-05-services.md#state-management-service)
+- [File Tracking Patterns](../../../02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-07-state-management.md)
+- [Performance Optimization Guide](../../../02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-08-performance-optimization.md)
+
+---
+
+[← Back to Index](../integration-patterns-readme.md) | [Previous: Batch Processing →](./integration-patterns-02-batch-processing.md) | [Next: Webhook Patterns →](./integration-patterns-04-webhook-patterns.md)