@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-02-simple-scenarios.md

@@ -1,681 +1,681 @@
-# Module 2: Simple Scenarios
-
-**Level:** Beginner
-**Estimated Time:** 30 minutes
-
-## Overview
-
-This module covers simple connector scenarios focusing on SFTP CSV ingestion. You'll build a complete inventory update connector from scratch, learning the fundamental patterns that apply to all connector types.
-
-## Learning Objectives
-
-By the end of this module, you will:
-
-- ✅ Build a complete SFTP CSV to Fluent inventory connector
-- ✅ Configure data sources and parsers
-- ✅ Implement field mapping and validation
-- ✅ Send data via Fluent Batch API
-- ✅ Handle file archival and error management
-- ✅ Understand the complete end-to-end flow
-
-## Scenario 1: SFTP CSV to Fluent Inventory Update
-
-**Goal:** Read inventory CSV files from SFTP, validate, transform, and update inventory in Fluent Commerce.
-
-**Complexity:** Low
-**Components:** SftpDataSource, CSVParserService, UniversalMapper, FluentClient
-**Estimated Lines:** ~250 lines
-
-### Problem Statement
-
-Your warehouse partner sends daily inventory updates via SFTP as CSV files. Each file contains:
-
-- SKU identifier
-- Warehouse location code
-- Available quantity
-- Status (available, unavailable, reserved)
-
-You need to:
-
-1. Connect to SFTP daily
-2. Process all new CSV files
-3. Validate data quality
-4. Transform to Fluent schema
-5. Update inventory positions
-6. Archive processed files
-7. Move failed files to error folder
-
-### CSV File Format
-
-```csv
-sku,location,quantity,status
-SKU-001,WAREHOUSE-A,100,available
-SKU-002,WAREHOUSE-A,50,available
-SKU-003,WAREHOUSE-B,25,reserved
-SKU-001,WAREHOUSE-B,75,available
-```
-
-### Complete Implementation
-
-```typescript
-import {
-  createClient,
-  SftpDataSource,
-  UniversalMapper,
-  StateService,
-  CSVParserService,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-
-// ====================
-// 1. CONFIGURATION
-// ====================
-
-/**
- * IMPORTANT: Credential Access Methods
- *
- * This example shows standalone Node.js configuration using environment variables.
- *
- * For Versori Platform deployments, use secure credential access methods:
- *
- * ✅ RECOMMENDED: activation.connections (already decoded, works everywhere)
- * ⚠️ ALTERNATIVE: credentials().get() (requires base64 decoding)
- * ❌ NEVER: Hardcoded credentials or unsecured environment variables
- *
- * See: docs/02-CORE-GUIDES/data-sources/sftp-credential-access-security.md
- * For complete credential access patterns including:
- * - Method comparison (credentials().get vs activation.connections)
- * - Connection configuration in Versori UI
- * - Security best practices
- * - Error handling examples
- * - Debug utilities
- */
-const config = {
-  sftp: {
-    host: process.env.SFTP_HOST!,
-    username: process.env.SFTP_USERNAME!,
-    password: process.env.SFTP_PASSWORD!,
-    remotePath: '/inventory/updates',
-    filePattern: 'INV_*.csv',
-  },
-  fluent: {
-    baseUrl: process.env.FLUENT_BASE_URL!,
-    clientId: process.env.FLUENT_CLIENT_ID!,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
-    username: process.env.FLUENT_USERNAME!,
-    password: process.env.FLUENT_PASSWORD!,
-    retailerId: process.env.FLUENT_RETAILER_ID!,
-  },
-};
-
-// ====================
-// 2. INITIALIZE SERVICES
-// ====================
-
-const logger = toStructuredLogger(createConsoleLogger(), {
-  logLevel: 'info'
-});
-
-const stateService = new StateService(logger);
-
-// ====================
-// 3. SETUP SFTP DATA SOURCE
-// ====================
-
-const sftpSource = new SftpDataSource(
-  {
-    type: 'SFTP_CSV',
-    connectionId: 'inventory-sftp',
-    name: 'Inventory SFTP',
-    settings: {
-      ...config.sftp,
-      csvDelimiter: ',',
-      csvHeaders: ['sku', 'location', 'quantity', 'status'],
-      csvSkipEmptyLines: true,
-      csvTrimValues: true,
-    },
-  },
-  logger
-);
-
-// ====================
-// 4. SETUP FIELD MAPPING
-// ====================
-
-const fieldMapper = new UniversalMapper({
-  fields: {
-    // Map 'sku' to 'ref', convert to uppercase
-    ref: {
-      source: 'sku',
-      resolver: 'sdk.uppercase',
-    },
-
-    // Map 'location' to 'locationRef' with prefix
-    locationRef: {
-      source: 'location',
-      resolver: value => `LOC:${value}`,
-    },
-
-    // Parse quantity as integer
-    qty: {
-      source: 'quantity',
-      resolver: 'sdk.parseInt',
-    },
-
-    // Transform status values
-    status: {
-      source: 'status',
-      resolver: value => {
-        const statusMap: Record<string, string> = {
-          available: 'ACTIVE',
-          unavailable: 'INACTIVE',
-          reserved: 'RESERVED',
-        };
-        return statusMap[String(value).toLowerCase()] || 'INACTIVE';
-      },
-    },
-  },
-});
-
-// ====================
-// 5. CREATE FLUENT CLIENT
-// ====================
-
-const fluentClient = await createClient({ config: config.fluent });
-
-// ====================
-// 6. MAIN PROCESSING FUNCTION
-// ====================
-
-async function processInventoryUpdates() {
-  logger.info('Starting inventory update process');
-
-  try {
-    // 7. LIST FILES FROM SFTP
-    // Get last processed timestamp from sync state
-    // Initialize KV store (implementation depends on runtime)
-    // const kv = openKv(); // For Deno or Versori: ctx.openKv(':project:')
-    // const syncState = await stateService.getSyncState(kv, 'inventory-sync');
-    // const lastProcessedTimestamp = syncState.lastProcessedTimestamp;
-
-    const files = await sftpSource.listFiles({
-      filePattern: 'INV_*.csv'
-    });
-
-    logger.info(`Found ${files.length} new files to process`);
-
-    // 8. PROCESS EACH FILE
-    for (const file of files) {
-      logger.info(`Processing file: ${file.name}`);
-
-      try {
-        // 9. DOWNLOAD FILE
-        const content = await sftpSource.downloadFile(file.name);
-
-        // 10. PARSE CSV
-        const parser = new CSVParserService();
-        const records = await parser.parse(content as string);
-        logger.info(`Parsed ${records.length} records`);
-
-        // 11. VALIDATION
-        const { validRecords, validationErrors } = validateRecords(records);
-
-        logger.info(
-          `Validation complete: ${validRecords.length} valid, ${validationErrors.length} errors`
-        );
-
-        if (validationErrors.length > 0) {
-          logger.warn('Validation errors found:', validationErrors.slice(0, 5));
-        }
-
-        // 12. TRANSFORM DATA
-        const transformedRecords: any[] = [];
-        for (const record of validRecords) {
-          const result = await fieldMapper.map(record);
-          if (!result.success) {
-            logger.warn('Mapping errors', result.errors);
-            continue;
-          }
-          transformedRecords.push(result.data);
-        }
-
-        // 13. CREATE FLUENT JOB
-        const job = await fluentClient.createJob({
-          name: `Inventory Update - ${file.name}`,
-          retailerId: config.fluent.retailerId,
-        });
-
-        // 14. SEND TO FLUENT VIA BATCH API
-        const batchResult = await fluentClient.sendBatch(job.id, {
-          action: 'UPSERT',
-          entityType: 'INVENTORY',
-          entities: transformedRecords.map(record => ({
-            ref: record.ref,
-            productRef: record.ref,
-            locationRef: record.locationRef,
-            qty: record.qty,
-            status: record.status,
-            type: 'ADJUSTMENT',
-          })),
-        });
-
-        logger.info('Batch sent to Fluent', {
-          batchId: batchResult.id,
-          recordCount: transformedRecords.length,
-          jobId: job.id,
-        });
-
-        // 15. MONITOR BATCH STATUS
-        await waitForBatchCompletion(fluentClient, job.id, batchResult.id);
-
-        // 16. ARCHIVE PROCESSED FILE
-        await sftpSource.moveFile(
-          `/inventory/updates/${file.name}`,
-          `/inventory/archive/${new Date().toISOString().split('T')[0]}/${file.name}`,
-          false
-        );
-
-        // 17. UPDATE STATE
-        // Update sync state using KV adapter (tracks file automatically)
-        // Initialize KV store (implementation depends on runtime)
-        // const kv = openKv(); // For Deno or Versori: ctx.openKv(':project:')
-        // await stateService.updateSyncState(kv, [{
-        //   fileName: file.name,
-        //   lastModified: new Date().toISOString(),
-        //   recordCount: transformedRecords.length
-        // }], 'inventory-sync');
-
-        logger.info(`Successfully processed ${file.name}`);
-      } catch (error) {
-        logger.error(`Failed to process file ${file.name}:`, error);
-
-        // Move to error folder
-        await sftpSource.moveFile(
-          `/inventory/updates/${file.name}`,
-          `/inventory/errors/${file.name}`,
-          true
-        );
-
-        // Log error details (use KV directly for error tracking)
-        // const kv = openKv(); // Initialize KV store
-        // await kv.set(['error-log', 'inventory', file.name], {
-        //   error: (error as Error).message,
-        //   timestamp: new Date().toISOString(),
-        //   stack: (error as Error).stack
-        // });
-      }
-    }
-
-    logger.info('Inventory update process completed');
-  } catch (error) {
-    logger.error('Process failed:', error);
-    throw error;
-  }
-}
-
-// ====================
-// HELPER FUNCTIONS
-// ====================
-
-/**
- * Validate CSV records
- */
-function validateRecords(records: any[]) {
-  const validationErrors: any[] = [];
-  const validRecords = records.filter(record => {
-    // Required field validation
-    if (!record.sku || !record.location || record.quantity === undefined) {
-      validationErrors.push({
-        record,
-        error: 'Missing required fields',
-      });
-      return false;
-    }
-
-    // Numeric validation
-    if (isNaN(Number(record.quantity))) {
-      validationErrors.push({
-        record,
-        error: 'Invalid quantity',
-      });
-      return false;
-    }
-
-    // SKU format validation
-    if (!/^[A-Z0-9-]+$/i.test(record.sku)) {
-      validationErrors.push({
-        record,
-        error: 'Invalid SKU format',
-      });
-      return false;
-    }
-
-    return true;
-  });
-
-  return { validRecords, validationErrors };
-}
-
-/**
- * Wait for batch processing to complete
- */
-async function waitForBatchCompletion(client: any, jobId: string, batchId: string) {
-  let batchStatus = 'PENDING';
-  let attempts = 0;
-  const maxAttempts = 60; // 5 minutes max
-
-  while (batchStatus !== 'COMPLETED' && batchStatus !== 'FAILED' && attempts < maxAttempts) {
-    await new Promise(resolve => setTimeout(resolve, 5000)); // Wait 5 seconds
-
-    const statusResult = await client.getBatchStatus(jobId, batchId);
-    batchStatus = statusResult.status;
-
-    logger.debug(`Batch status: ${batchStatus}`);
-    attempts++;
-  }
-
-  if (batchStatus === 'COMPLETED') {
-    logger.info('Batch processing completed successfully');
-  } else if (batchStatus === 'FAILED') {
-    throw new Error('Batch processing failed');
-  } else {
-    throw new Error('Batch processing timeout');
-  }
-}
-
-// ====================
-// EXECUTE
-// ====================
-
-processInventoryUpdates()
-  .then(() => console.log('✅ Inventory sync complete'))
-  .catch(err => console.error('❌ Sync failed:', err));
-```
-
-### Code Walkthrough
-
-#### Step 1-2: Configuration and Service Setup
-
-```typescript
-// Configuration separates environment-specific values
-const config = {
-  sftp: { host, username, password, remotePath },
-  fluent: { baseUrl, clientId, clientSecret, retailerId },
-};
-
-// Initialize core services
-const logger = toStructuredLogger(createConsoleLogger(), {
-  logLevel: 'info'
-});
-const stateService = new StateService(logger);
-```
-
-**Why this matters:**
-
-- Configuration centralization makes deployment easier
-- Logger provides structured output for debugging
-- StateService prevents re-processing the same file
-
-**IMPORTANT - Credential Access:**
-
-This example shows **standalone Node.js** configuration using environment variables. For **Versori Platform** deployments:
-
-- ✅ **Use `activation.connections`** for secure credential access (recommended)
-- ⚠️ **Or `credentials().get()`** if needed (requires base64 decoding)
-- ❌ **Never hardcode** credentials in code
-
-**See:** [SFTP Credential Access & Security](../../../02-CORE-GUIDES/data-sources/data-sources-sftp-credential-access-security.md) for complete patterns including:
-- Two credential access methods compared
-- Connection setup in Versori UI
-- Security best practices
-- Complete code examples for both methods
-
-#### Step 3: SFTP Data Source Configuration
-
-```typescript
-const sftpSource = new SftpDataSource(
-  {
-    type: 'SFTP_CSV',
-    connectionId: 'inventory-sftp',
-    name: 'Inventory SFTP',
-    settings: {
-      ...config.sftp,
-      csvDelimiter: ',',
-      csvHeaders: ['sku', 'location', 'quantity', 'status'],
-      csvSkipEmptyLines: true,
-      csvTrimValues: true,
-    },
-  },
-  logger
-);
-```
-
-**Key options:**
-
-- `csvHeaders`: Column names (auto-detected if CSV has header row)
-- `csvSkipEmptyLines`: Ignore blank lines
-- `csvTrimValues`: Remove whitespace from values
-
-#### Step 4: Field Mapping
-
-```typescript
-const fieldMapper = new UniversalMapper({
-  fields: {
-    ref: { source: 'sku', resolver: 'sdk.uppercase' },
-    locationRef: { source: 'location', resolver: v => `LOC:${v}` },
-    qty: { source: 'quantity', resolver: 'sdk.parseInt' },
-    status: { source: 'status', resolver: statusMapFunction },
-  },
-});
-```
-
-**Mapping strategies:**
-
-- **Built-in resolvers**: `sdk.uppercase`, `sdk.parseInt`, `sdk.formatDate`
-- **Custom functions**: `(value) => transformedValue`
-- **Prefixing**: Add prefixes to IDs (`LOC:WAREHOUSE-A`)
-- **Mapping tables**: Convert codes to values
-
-#### Step 11: Validation
-
-```typescript
-function validateRecords(records: any[]) {
-  const validRecords = records.filter(record => {
-    // Check required fields
-    if (!record.sku || !record.location) return false;
-
-    // Validate data types
-    if (isNaN(Number(record.quantity))) return false;
-
-    // Validate formats
-    if (!/^[A-Z0-9-]+$/i.test(record.sku)) return false;
-
-    return true;
-  });
-
-  return { validRecords, validationErrors };
-}
-```
-
-**Validation levels:**
-
-1. **Required fields**: Must be present
-2. **Data types**: Must be correct type (number, string, etc.)
-3. **Formats**: Must match expected patterns (SKU format, email, etc.)
-4. **Business rules**: Must satisfy domain logic (quantity > 0, etc.)
-
-#### Step 14: Batch API Call
-
-```typescript
-await fluentClient.sendBatch(job.id, {
-  action: 'UPSERT', // Create or update
-  entityType: 'INVENTORY', // Entity type
-  entities: transformedRecords.map(record => ({
-    ref: record.ref,
-    productRef: record.ref,
-    locationRef: record.locationRef,
-    qty: record.qty,
-    status: record.status,
-    type: 'ADJUSTMENT',
-  })),
-});
-```
-
-**Batch API notes:**
-
-- `action: 'UPSERT'` - Creates new or updates existing
-- `entityType: 'INVENTORY'` - Only entity type supported
-- Required fields: `ref`, `productRef`, `locationRef`, `qty`
-- Optional fields: `status`, `type`
-
-#### Step 15: Status Monitoring
-
-```typescript
-async function waitForBatchCompletion(client, jobId, batchId) {
-  let batchStatus = 'PENDING';
-  let attempts = 0;
-  const maxAttempts = 60; // 5 minutes max
-
-  while (batchStatus !== 'COMPLETED' && batchStatus !== 'FAILED' && attempts < maxAttempts) {
-    await new Promise(resolve => setTimeout(resolve, 5000));
-    const statusResult = await client.getBatchStatus(jobId, batchId);
-    batchStatus = statusResult.status;
-    attempts++;
-  }
-
-  if (batchStatus !== 'COMPLETED') {
-    throw new Error('Batch failed or timed out');
-  }
-}
-```
-
-**Why poll for status:**
-
-- Batch API is asynchronous (returns immediately)
-- Need to wait for processing to complete
-- Detect failures early
-- Implement timeout to prevent infinite loops
-
-### Error Handling Strategy
-
-```typescript
-try {
-  // Process file
-  await processFile(file);
-
-  // Archive on success
-  await sftpSource.moveFile(file.name, 'archive/');
-} catch (error) {
-  logger.error(`Failed to process ${file.name}:`, error);
-
-  // Move to error folder for review
-  await sftpSource.moveFile(file.name, 'errors/');
-
-  // Log error in state for audit trail (use KV directly)
-  // const kv = openKv(); // Initialize KV store
-  // await kv.set(['errors', 'inventory', file.name], {
-  //   error: error.message,
-  //   timestamp: new Date().toISOString()
-  // });
-
-  // Continue processing next file (don't throw)
-}
-```
-
-**Error handling principles:**
-
-1. **Log detailed errors** - Include file name, error message, stack trace
-2. **Move failed files** - Quarantine for manual review
-3. **Track failures** - Record in state management
-4. **Continue processing** - One failure shouldn't stop entire batch
-5. **Notify on critical errors** - Alert administrators when needed
-
-### File Archival Pattern
-
-```typescript
-// Archive processed files by date
-await sftpSource.moveFile(
-  '/inventory/updates/INV_2025-01-15.csv',
-  '/inventory/archive/2025-01-15/INV_2025-01-15.csv',
-  false // Don't overwrite if exists
-);
-
-// Move errors to dedicated folder
-await sftpSource.moveFile(
-  '/inventory/updates/BAD_FILE.csv',
-  '/inventory/errors/BAD_FILE.csv',
-  true // Overwrite if exists (append timestamp to avoid loss)
-);
-```
-
-**Archival best practices:**
-
-- Organize by date: `archive/YYYY-MM-DD/filename.csv`
-- Keep processed files for audit trail (at least 30 days)
-- Separate error files for investigation
-- Use retention policies (auto-delete after N days)
-
-## Practice Exercise
-
-**Challenge:** Modify the connector to handle a CSV with additional fields
-
-**New CSV format:**
-
-```csv
-sku,location,on_hand,allocated,damaged,status,last_updated
-SKU-001,WH-A,100,20,5,active,2025-01-15T10:00:00Z
-```
-
-**Requirements:**
-
-1. Calculate available quantity: `on_hand - allocated - damaged`
-2. Map `last_updated` to ISO date format
-3. Add validation: `on_hand >= allocated + damaged`
-4. Include `lastUpdated` attribute in Fluent payload
-
-**Solution:**
-
-```typescript
-const fieldMapper = new UniversalMapper({
-  fields: {
-    ref: { source: 'sku', resolver: 'sdk.uppercase' },
-    locationRef: { source: 'location' },
-    qty: {
-      source: ['on_hand', 'allocated', 'damaged'],
-      resolver: values => {
-        const [onHand, allocated, damaged] = values.map(Number);
-        return onHand - allocated - damaged;
-      },
-    },
-    status: { source: 'status', resolver: 'sdk.uppercase' },
-    lastUpdated: { source: 'last_updated', resolver: 'sdk.formatDate' },
-  },
-});
-
-// Add validation
-function validateRecords(records: any[]) {
-  return records.filter(record => {
-    const onHand = Number(record.on_hand);
-    const allocated = Number(record.allocated);
-    const damaged = Number(record.damaged);
-
-    return onHand >= allocated + damaged;
-  });
-}
-```
-
-## Key Takeaways
-
-- 🎯 Simple connectors follow a clear pattern: List → Download → Parse → Validate → Transform → Send → Archive
-- 🎯 SftpDataSource handles all SFTP operations (list, download, move)
-- 🎯 UniversalMapper provides flexible field transformation with built-in resolvers
-- 🎯 Validation should happen before transformation to catch bad data early
-- 🎯 Batch API requires job creation + batch send + status monitoring
-- 🎯 Error handling should be graceful (log, quarantine, continue)
-- 🎯 State management prevents duplicate processing
-- 🎯 **Security:** Use secure credential access methods on Versori platform (see [SFTP Credential Access & Security](../../../02-CORE-GUIDES/data-sources/data-sources-sftp-credential-access-security.md))
-
-## Next Steps
-
-Continue to [Module 3: Intermediate Scenarios](./connector-scenarios-03-intermediate-scenarios.md) to learn S3 Parquet processing, JSON order creation, and advanced validation patterns.
+# Module 2: Simple Scenarios
+
+**Level:** Beginner
+**Estimated Time:** 30 minutes
+
+## Overview
+
+This module covers simple connector scenarios focusing on SFTP CSV ingestion. You'll build a complete inventory update connector from scratch, learning the fundamental patterns that apply to all connector types.
+
+## Learning Objectives
+
+By the end of this module, you will:
+
+- ✅ Build a complete SFTP CSV to Fluent inventory connector
+- ✅ Configure data sources and parsers
+- ✅ Implement field mapping and validation
+- ✅ Send data via Fluent Batch API
+- ✅ Handle file archival and error management
+- ✅ Understand the complete end-to-end flow
+
+## Scenario 1: SFTP CSV to Fluent Inventory Update
+
+**Goal:** Read inventory CSV files from SFTP, validate, transform, and update inventory in Fluent Commerce.
+
+**Complexity:** Low
+**Components:** SftpDataSource, CSVParserService, UniversalMapper, FluentClient
+**Estimated Lines:** ~250 lines
+
+### Problem Statement
+
+Your warehouse partner sends daily inventory updates via SFTP as CSV files. Each file contains:
+
+- SKU identifier
+- Warehouse location code
+- Available quantity
+- Status (available, unavailable, reserved)
+
+You need to:
+
+1. Connect to SFTP daily
+2. Process all new CSV files
+3. Validate data quality
+4. Transform to Fluent schema
+5. Update inventory positions
+6. Archive processed files
+7. Move failed files to error folder
+
+### CSV File Format
+
+```csv
+sku,location,quantity,status
+SKU-001,WAREHOUSE-A,100,available
+SKU-002,WAREHOUSE-A,50,available
+SKU-003,WAREHOUSE-B,25,reserved
+SKU-001,WAREHOUSE-B,75,available
+```
+
+### Complete Implementation
+
+```typescript
+import {
+  createClient,
+  SftpDataSource,
+  UniversalMapper,
+  StateService,
+  CSVParserService,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+
+// ====================
+// 1. CONFIGURATION
+// ====================
+
+/**
+ * IMPORTANT: Credential Access Methods
+ *
+ * This example shows standalone Node.js configuration using environment variables.
+ *
+ * For Versori Platform deployments, use secure credential access methods:
+ *
+ * ✅ RECOMMENDED: activation.connections (already decoded, works everywhere)
+ * ⚠️ ALTERNATIVE: credentials().get() (requires base64 decoding)
+ * ❌ NEVER: Hardcoded credentials or unsecured environment variables
+ *
+ * See: docs/02-CORE-GUIDES/data-sources/sftp-credential-access-security.md
+ * For complete credential access patterns including:
+ * - Method comparison (credentials().get vs activation.connections)
+ * - Connection configuration in Versori UI
+ * - Security best practices
+ * - Error handling examples
+ * - Debug utilities
+ */
+const config = {
+  sftp: {
+    host: process.env.SFTP_HOST!,
+    username: process.env.SFTP_USERNAME!,
+    password: process.env.SFTP_PASSWORD!,
+    remotePath: '/inventory/updates',
+    filePattern: 'INV_*.csv',
+  },
+  fluent: {
+    baseUrl: process.env.FLUENT_BASE_URL!,
+    clientId: process.env.FLUENT_CLIENT_ID!,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
+    username: process.env.FLUENT_USERNAME!,
+    password: process.env.FLUENT_PASSWORD!,
+    retailerId: process.env.FLUENT_RETAILER_ID!,
+  },
+};
+
+// ====================
+// 2. INITIALIZE SERVICES
+// ====================
+
+const logger = toStructuredLogger(createConsoleLogger(), {
+  logLevel: 'info'
+});
+
+const stateService = new StateService(logger);
+
+// ====================
+// 3. SETUP SFTP DATA SOURCE
+// ====================
+
+const sftpSource = new SftpDataSource(
+  {
+    type: 'SFTP_CSV',
+    connectionId: 'inventory-sftp',
+    name: 'Inventory SFTP',
+    settings: {
+      ...config.sftp,
+      csvDelimiter: ',',
+      csvHeaders: ['sku', 'location', 'quantity', 'status'],
+      csvSkipEmptyLines: true,
+      csvTrimValues: true,
+    },
+  },
+  logger
+);
+
+// ====================
+// 4. SETUP FIELD MAPPING
+// ====================
+
+const fieldMapper = new UniversalMapper({
+  fields: {
+    // Map 'sku' to 'ref', convert to uppercase
+    ref: {
+      source: 'sku',
+      resolver: 'sdk.uppercase',
+    },
+
+    // Map 'location' to 'locationRef' with prefix
+    locationRef: {
+      source: 'location',
+      resolver: value => `LOC:${value}`,
+    },
+
+    // Parse quantity as integer
+    qty: {
+      source: 'quantity',
+      resolver: 'sdk.parseInt',
+    },
+
+    // Transform status values
+    status: {
+      source: 'status',
+      resolver: value => {
+        const statusMap: Record<string, string> = {
+          available: 'ACTIVE',
+          unavailable: 'INACTIVE',
+          reserved: 'RESERVED',
+        };
+        return statusMap[String(value).toLowerCase()] || 'INACTIVE';
+      },
+    },
+  },
+});
+
+// ====================
+// 5. CREATE FLUENT CLIENT
+// ====================
+
+const fluentClient = await createClient({ config: config.fluent });
+
+// ====================
+// 6. MAIN PROCESSING FUNCTION
+// ====================
+
+async function processInventoryUpdates() {
+  logger.info('Starting inventory update process');
+
+  try {
+    // 7. LIST FILES FROM SFTP
+    // Get last processed timestamp from sync state
+    // Initialize KV store (implementation depends on runtime)
+    // const kv = openKv(); // For Deno or Versori: ctx.openKv(':project:')
+    // const syncState = await stateService.getSyncState(kv, 'inventory-sync');
+    // const lastProcessedTimestamp = syncState.lastProcessedTimestamp;
+
+    const files = await sftpSource.listFiles({
+      filePattern: 'INV_*.csv'
+    });
+
+    logger.info(`Found ${files.length} new files to process`);
+
+    // 8. PROCESS EACH FILE
+    for (const file of files) {
+      logger.info(`Processing file: ${file.name}`);
+
+      try {
+        // 9. DOWNLOAD FILE
+        const content = await sftpSource.downloadFile(file.name);
+
+        // 10. PARSE CSV
+        const parser = new CSVParserService();
+        const records = await parser.parse(content as string);
+        logger.info(`Parsed ${records.length} records`);
+
+        // 11. VALIDATION
+        const { validRecords, validationErrors } = validateRecords(records);
+
+        logger.info(
+          `Validation complete: ${validRecords.length} valid, ${validationErrors.length} errors`
+        );
+
+        if (validationErrors.length > 0) {
+          logger.warn('Validation errors found:', validationErrors.slice(0, 5));
+        }
+
+        // 12. TRANSFORM DATA
+        const transformedRecords: any[] = [];
+        for (const record of validRecords) {
+          const result = await fieldMapper.map(record);
+          if (!result.success) {
+            logger.warn('Mapping errors', result.errors);
+            continue;
+          }
+          transformedRecords.push(result.data);
+        }
+
+        // 13. CREATE FLUENT JOB
+        const job = await fluentClient.createJob({
+          name: `Inventory Update - ${file.name}`,
+          retailerId: config.fluent.retailerId,
+        });
+
+        // 14. SEND TO FLUENT VIA BATCH API
+        const batchResult = await fluentClient.sendBatch(job.id, {
+          action: 'UPSERT',
+          entityType: 'INVENTORY',
+          entities: transformedRecords.map(record => ({
+            ref: record.ref,
+            productRef: record.ref,
+            locationRef: record.locationRef,
+            qty: record.qty,
+            status: record.status,
+            type: 'ADJUSTMENT',
+          })),
+        });
+
+        logger.info('Batch sent to Fluent', {
+          batchId: batchResult.id,
+          recordCount: transformedRecords.length,
+          jobId: job.id,
+        });
+
+        // 15. MONITOR BATCH STATUS
+        await waitForBatchCompletion(fluentClient, job.id, batchResult.id);
+
+        // 16. ARCHIVE PROCESSED FILE
+        await sftpSource.moveFile(
+          `/inventory/updates/${file.name}`,
+          `/inventory/archive/${new Date().toISOString().split('T')[0]}/${file.name}`,
+          false
+        );
+
+        // 17. UPDATE STATE
+        // Update sync state using KV adapter (tracks file automatically)
+        // Initialize KV store (implementation depends on runtime)
+        // const kv = openKv(); // For Deno or Versori: ctx.openKv(':project:')
+        // await stateService.updateSyncState(kv, [{
+        //   fileName: file.name,
+        //   lastModified: new Date().toISOString(),
+        //   recordCount: transformedRecords.length
+        // }], 'inventory-sync');
+
+        logger.info(`Successfully processed ${file.name}`);
+      } catch (error) {
+        logger.error(`Failed to process file ${file.name}:`, error);
+
+        // Move to error folder
+        await sftpSource.moveFile(
+          `/inventory/updates/${file.name}`,
+          `/inventory/errors/${file.name}`,
+          true
+        );
+
+        // Log error details (use KV directly for error tracking)
+        // const kv = openKv(); // Initialize KV store
+        // await kv.set(['error-log', 'inventory', file.name], {
+        //   error: (error as Error).message,
+        //   timestamp: new Date().toISOString(),
+        //   stack: (error as Error).stack
+        // });
+      }
+    }
+
+    logger.info('Inventory update process completed');
+  } catch (error) {
+    logger.error('Process failed:', error);
+    throw error;
+  }
+}
+
+// ====================
+// HELPER FUNCTIONS
+// ====================
+
+/**
+ * Validate CSV records
+ */
+function validateRecords(records: any[]) {
+  const validationErrors: any[] = [];
+  const validRecords = records.filter(record => {
+    // Required field validation
+    if (!record.sku || !record.location || record.quantity === undefined) {
+      validationErrors.push({
+        record,
+        error: 'Missing required fields',
+      });
+      return false;
+    }
+
+    // Numeric validation
+    if (isNaN(Number(record.quantity))) {
+      validationErrors.push({
+        record,
+        error: 'Invalid quantity',
+      });
+      return false;
+    }
+
+    // SKU format validation
+    if (!/^[A-Z0-9-]+$/i.test(record.sku)) {
+      validationErrors.push({
+        record,
+        error: 'Invalid SKU format',
+      });
+      return false;
+    }
+
+    return true;
+  });
+
+  return { validRecords, validationErrors };
+}
+
+/**
+ * Wait for batch processing to complete
+ */
+async function waitForBatchCompletion(client: any, jobId: string, batchId: string) {
+  let batchStatus = 'PENDING';
+  let attempts = 0;
+  const maxAttempts = 60; // 5 minutes max
+
+  while (batchStatus !== 'COMPLETED' && batchStatus !== 'FAILED' && attempts < maxAttempts) {
+    await new Promise(resolve => setTimeout(resolve, 5000)); // Wait 5 seconds
+
+    const statusResult = await client.getBatchStatus(jobId, batchId);
+    batchStatus = statusResult.status;
+
+    logger.debug(`Batch status: ${batchStatus}`);
+    attempts++;
+  }
+
+  if (batchStatus === 'COMPLETED') {
+    logger.info('Batch processing completed successfully');
+  } else if (batchStatus === 'FAILED') {
+    throw new Error('Batch processing failed');
+  } else {
+    throw new Error('Batch processing timeout');
+  }
+}
+
+// ====================
+// EXECUTE
+// ====================
+
+processInventoryUpdates()
+  .then(() => console.log('✅ Inventory sync complete'))
+  .catch(err => console.error('❌ Sync failed:', err));
+```
+
+### Code Walkthrough
+
+#### Step 1-2: Configuration and Service Setup
+
+```typescript
+// Configuration separates environment-specific values
+const config = {
+  sftp: { host, username, password, remotePath },
+  fluent: { baseUrl, clientId, clientSecret, retailerId },
+};
+
+// Initialize core services
+const logger = toStructuredLogger(createConsoleLogger(), {
+  logLevel: 'info'
+});
+const stateService = new StateService(logger);
+```
+
+**Why this matters:**
+
+- Configuration centralization makes deployment easier
+- Logger provides structured output for debugging
+- StateService prevents re-processing the same file
+
+**IMPORTANT - Credential Access:**
+
+This example shows **standalone Node.js** configuration using environment variables. For **Versori Platform** deployments:
+
+- ✅ **Use `activation.connections`** for secure credential access (recommended)
+- ⚠️ **Or `credentials().get()`** if needed (requires base64 decoding)
+- ❌ **Never hardcode** credentials in code
+
+**See:** [SFTP Credential Access & Security](../../../02-CORE-GUIDES/data-sources/data-sources-sftp-credential-access-security.md) for complete patterns including:
+- Two credential access methods compared
+- Connection setup in Versori UI
+- Security best practices
+- Complete code examples for both methods
+
+#### Step 3: SFTP Data Source Configuration
+
+```typescript
+const sftpSource = new SftpDataSource(
+  {
+    type: 'SFTP_CSV',
+    connectionId: 'inventory-sftp',
+    name: 'Inventory SFTP',
+    settings: {
+      ...config.sftp,
+      csvDelimiter: ',',
+      csvHeaders: ['sku', 'location', 'quantity', 'status'],
+      csvSkipEmptyLines: true,
+      csvTrimValues: true,
+    },
+  },
+  logger
+);
+```
+
+**Key options:**
+
+- `csvHeaders`: Column names (auto-detected if CSV has header row)
+- `csvSkipEmptyLines`: Ignore blank lines
+- `csvTrimValues`: Remove whitespace from values
+
+#### Step 4: Field Mapping
+
+```typescript
+const fieldMapper = new UniversalMapper({
+  fields: {
+    ref: { source: 'sku', resolver: 'sdk.uppercase' },
+    locationRef: { source: 'location', resolver: v => `LOC:${v}` },
+    qty: { source: 'quantity', resolver: 'sdk.parseInt' },
+    status: { source: 'status', resolver: statusMapFunction },
+  },
+});
+```
+
+**Mapping strategies:**
+
+- **Built-in resolvers**: `sdk.uppercase`, `sdk.parseInt`, `sdk.formatDate`
+- **Custom functions**: `(value) => transformedValue`
+- **Prefixing**: Add prefixes to IDs (`LOC:WAREHOUSE-A`)
+- **Mapping tables**: Convert codes to values
+
+#### Step 11: Validation
+
+```typescript
+function validateRecords(records: any[]) {
+  const validRecords = records.filter(record => {
+    // Check required fields
+    if (!record.sku || !record.location) return false;
+
+    // Validate data types
+    if (isNaN(Number(record.quantity))) return false;
+
+    // Validate formats
+    if (!/^[A-Z0-9-]+$/i.test(record.sku)) return false;
+
+    return true;
+  });
+
+  return { validRecords, validationErrors };
+}
+```
+
+**Validation levels:**
+
+1. **Required fields**: Must be present
+2. **Data types**: Must be correct type (number, string, etc.)
+3. **Formats**: Must match expected patterns (SKU format, email, etc.)
+4. **Business rules**: Must satisfy domain logic (quantity > 0, etc.)
+
+#### Step 14: Batch API Call
+
+```typescript
+await fluentClient.sendBatch(job.id, {
+  action: 'UPSERT', // Create or update
+  entityType: 'INVENTORY', // Entity type
+  entities: transformedRecords.map(record => ({
+    ref: record.ref,
+    productRef: record.ref,
+    locationRef: record.locationRef,
+    qty: record.qty,
+    status: record.status,
+    type: 'ADJUSTMENT',
+  })),
+});
+```
+
+**Batch API notes:**
+
+- `action: 'UPSERT'` - Creates new or updates existing
+- `entityType: 'INVENTORY'` - Only entity type supported
+- Required fields: `ref`, `productRef`, `locationRef`, `qty`
+- Optional fields: `status`, `type`
+
+#### Step 15: Status Monitoring
+
+```typescript
+async function waitForBatchCompletion(client, jobId, batchId) {
+  let batchStatus = 'PENDING';
+  let attempts = 0;
+  const maxAttempts = 60; // 5 minutes max
+
+  while (batchStatus !== 'COMPLETED' && batchStatus !== 'FAILED' && attempts < maxAttempts) {
+    await new Promise(resolve => setTimeout(resolve, 5000));
+    const statusResult = await client.getBatchStatus(jobId, batchId);
+    batchStatus = statusResult.status;
+    attempts++;
+  }
+
+  if (batchStatus !== 'COMPLETED') {
+    throw new Error('Batch failed or timed out');
+  }
+}
+```
+
+**Why poll for status:**
+
+- Batch API is asynchronous (returns immediately)
+- Need to wait for processing to complete
+- Detect failures early
+- Implement timeout to prevent infinite loops
+
+### Error Handling Strategy
+
+```typescript
+try {
+  // Process file
+  await processFile(file);
+
+  // Archive on success
+  await sftpSource.moveFile(file.name, 'archive/');
+} catch (error) {
+  logger.error(`Failed to process ${file.name}:`, error);
+
+  // Move to error folder for review
+  await sftpSource.moveFile(file.name, 'errors/');
+
+  // Log error in state for audit trail (use KV directly)
+  // const kv = openKv(); // Initialize KV store
+  // await kv.set(['errors', 'inventory', file.name], {
+  //   error: error.message,
+  //   timestamp: new Date().toISOString()
+  // });
+
+  // Continue processing next file (don't throw)
+}
+```
+
+**Error handling principles:**
+
+1. **Log detailed errors** - Include file name, error message, stack trace
+2. **Move failed files** - Quarantine for manual review
+3. **Track failures** - Record in state management
+4. **Continue processing** - One failure shouldn't stop entire batch
+5. **Notify on critical errors** - Alert administrators when needed
+
+### File Archival Pattern
+
+```typescript
+// Archive processed files by date
+await sftpSource.moveFile(
+  '/inventory/updates/INV_2025-01-15.csv',
+  '/inventory/archive/2025-01-15/INV_2025-01-15.csv',
+  false // Don't overwrite if exists
+);
+
+// Move errors to dedicated folder
+await sftpSource.moveFile(
+  '/inventory/updates/BAD_FILE.csv',
+  '/inventory/errors/BAD_FILE.csv',
+  true // Overwrite if exists (append timestamp to avoid loss)
+);
+```
+
+**Archival best practices:**
+
+- Organize by date: `archive/YYYY-MM-DD/filename.csv`
+- Keep processed files for audit trail (at least 30 days)
+- Separate error files for investigation
+- Use retention policies (auto-delete after N days)
+
+## Practice Exercise
+
+**Challenge:** Modify the connector to handle a CSV with additional fields
+
+**New CSV format:**
+
+```csv
+sku,location,on_hand,allocated,damaged,status,last_updated
+SKU-001,WH-A,100,20,5,active,2025-01-15T10:00:00Z
+```
+
+**Requirements:**
+
+1. Calculate available quantity: `on_hand - allocated - damaged`
+2. Map `last_updated` to ISO date format
+3. Add validation: `on_hand >= allocated + damaged`
+4. Include `lastUpdated` attribute in Fluent payload
+
+**Solution:**
+
+```typescript
+const fieldMapper = new UniversalMapper({
+  fields: {
+    ref: { source: 'sku', resolver: 'sdk.uppercase' },
+    locationRef: { source: 'location' },
+    qty: {
+      source: ['on_hand', 'allocated', 'damaged'],
+      resolver: values => {
+        const [onHand, allocated, damaged] = values.map(Number);
+        return onHand - allocated - damaged;
+      },
+    },
+    status: { source: 'status', resolver: 'sdk.uppercase' },
+    lastUpdated: { source: 'last_updated', resolver: 'sdk.formatDate' },
+  },
+});
+
+// Add validation
+function validateRecords(records: any[]) {
+  return records.filter(record => {
+    const onHand = Number(record.on_hand);
+    const allocated = Number(record.allocated);
+    const damaged = Number(record.damaged);
+
+    return onHand >= allocated + damaged;
+  });
+}
+```
+
+## Key Takeaways
+
+- 🎯 Simple connectors follow a clear pattern: List → Download → Parse → Validate → Transform → Send → Archive
+- 🎯 SftpDataSource handles all SFTP operations (list, download, move)
+- 🎯 UniversalMapper provides flexible field transformation with built-in resolvers
+- 🎯 Validation should happen before transformation to catch bad data early
+- 🎯 Batch API requires job creation + batch send + status monitoring
+- 🎯 Error handling should be graceful (log, quarantine, continue)
+- 🎯 State management prevents duplicate processing
+- 🎯 **Security:** Use secure credential access methods on Versori platform (see [SFTP Credential Access & Security](../../../02-CORE-GUIDES/data-sources/data-sources-sftp-credential-access-security.md))
+
+## Next Steps
+
+Continue to [Module 3: Intermediate Scenarios](./connector-scenarios-03-intermediate-scenarios.md) to learn S3 Parquet processing, JSON order creation, and advanced validation patterns.