@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

package/docs/02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-02-quick-start.md

@@ -1,658 +1,658 @@
-# Module 2: Quick Start - Your First Ingestion
-
-[← Back to Ingestion Guide](../ingestion-readme.md)
-
-**Module 2 of 9** | **Level**: Beginner | **Time**: 15 minutes
-
----
-
-## Overview
-
-This module walks you through creating your first complete ingestion workflow: reading a CSV file from S3, transforming it with UniversalMapper, and sending it to Fluent Commerce using the Batch API.
-
-## Learning Objectives
-
-By the end of this module, you will:
-
-- ✅ Install and configure the FC Connect SDK
-- ✅ Create a complete S3 → CSV → Batch API workflow
-- ✅ Understand the basic ingestion pattern
-- ✅ Validate field mappings against the schema
-- ✅ Send your first batch to Fluent Commerce
-
----
-
-## Prerequisites
-
-Before starting, ensure you have:
-
-1. **Node.js** >= 18.0.0 installed
-2. **Fluent Commerce credentials**:
-   - API Base URL (e.g., `https://yourcompany.api.fluentcommerce.com`)
-   - OAuth2 Client ID
-   - OAuth2 Client Secret
-   - Retailer ID
-3. **AWS S3 access** (or local CSV file for testing):
-   - Access Key ID
-   - Secret Access Key
-   - Bucket name
-
----
-
-## Step 1: Installation
-
-Install the FC Connect SDK:
-
-```bash
-npm install @fluentcommerce/fc-connect-sdk
-```
-
-**Verify installation:**
-
-```bash
-npx @fluentcommerce/fc-connect-sdk --version
-```
-
----
-
-## Step 2: Environment Configuration
-
-Create a `.env` file with your credentials:
-
-```bash
-# Fluent Commerce OAuth2 (Password Grant)
-FLUENT_BASE_URL=https://yourcompany.api.fluentcommerce.com
-FLUENT_CLIENT_ID=your-oauth-client-id
-FLUENT_CLIENT_SECRET=your-oauth-client-secret
-FLUENT_USERNAME=your-username          # ✅ REQUIRED for password grant
-FLUENT_PASSWORD=your-password          # ✅ REQUIRED for password grant
-FLUENT_RETAILER_ID=1
-
-# AWS S3 (source)
-AWS_ACCESS_KEY_ID=your-aws-access-key
-AWS_SECRET_ACCESS_KEY=your-aws-secret-key
-AWS_REGION=us-east-1
-S3_BUCKET=your-inventory-bucket
-S3_PREFIX=data/
-```
-
-**Load environment variables:**
-
-```bash
-npm install dotenv
-```
-
----
-
-## Step 3: Prepare Sample Data
-
-Create a sample CSV file (`sample-inventory.csv`):
-
-```csv
-sku,warehouse,quantity,status
-SKU-WM-001,WH-001,500,in_stock
-SKU-WM-002,WH-001,150,in_stock
-SKU-KB-001,WH-001,75,reserved
-SKU-WM-003,WH-002,300,discontinued
-SKU-WM-004,WH-002,200,in_stock
-```
-
-**Upload to S3** (or keep local for testing):
-
-```bash
-aws s3 cp sample-inventory.csv s3://your-bucket/data/inventory-2025-01-19.csv
-```
-
----
-
-## Step 4: Create Field Mapping Configuration
-
-Create `field-mappings.json`:
-
-```json
-{
-  "fields": {
-    "ref": {
-      "source": "sku",
-      "required": true,
-      "comment": "✅ InventoryPositionInput.ref (String! required) - Position reference"
-    },
-    "productRef": {
-      "source": "sku",
-      "required": true,
-      "comment": "✅ InventoryPositionInput.productRef (String! required) - Product SKU"
-    },
-    "locationRef": {
-      "source": "warehouse",
-      "required": true,
-      "comment": "✅ InventoryPositionInput.locationRef (String! required) - Warehouse code"
-    },
-    "qty": {
-      "source": "quantity",
-      "resolver": "sdk.parseInt",
-      "required": true,
-      "comment": "✅ InventoryPositionInput.qty (Int! required) - Available quantity"
-    },
-    "status": {
-      "source": "status",
-      "resolver": "custom.mapStatus",
-      "defaultValue": "AVAILABLE",
-      "comment": "✅ InventoryPositionInput.status (String optional) - Position status"
-    }
-  }
-}
-```
-
----
-
-## Step 5: Validate Field Mapping Against Schema
-
-**CRITICAL**: Before running ingestion, validate your mapping:
-
-```bash
-# Step 1: Introspect Fluent Commerce schema
-npx @fluentcommerce/fc-connect-sdk introspect-schema \
-  --url https://yourcompany.api.fluentcommerce.com/graphql \
-  --client-id $FLUENT_CLIENT_ID \
-  --client-secret $FLUENT_CLIENT_SECRET \
-  --output schema.json
-
-# Step 2: Validate field-mappings.json against schema
-npx @fluentcommerce/fc-connect-sdk validate-schema \
-  --mapping field-mappings.json \
-  --schema schema.json
-```
-
-**Expected output:**
-
-```
-✅ Mapping validation successful
-✅ All required fields present: ref, productRef, locationRef, qty
-✅ All field types compatible with InventoryPositionInput schema
-✅ No unknown fields detected
-```
-
-### Preflight Validation (NEW)
-
-Run a preflight validation to catch misconfigurations before execution.
-
-```typescript
-import { PreflightValidator } from '@fluentcommerce/fc-connect-sdk';
-
-const validator = new PreflightValidator(logger);
-await validator.validate({
-  fluent: { client },
-  sftp: { host: process.env.SFTP_HOST, username: process.env.SFTP_USERNAME },
-  config: { processedPath: process.env.PROCESSED_PATH, errorsPath: process.env.ERRORS_PATH },
-});
-```
-
-Checks performed:
-
-- Fluent client reachability/auth
-- SFTP connectivity and directory writability (create directories if missing)
-- Required activation/env variables present
-
----
-
-## Step 6: Complete Ingestion Script
-
-Create `ingest-inventory.ts`:
-
-```typescript
-import 'dotenv/config';
-import {
-  createClient,
-  S3DataSource,
-  CSVParserService,
-  UniversalMapper,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-
-// Custom resolver for status mapping
-const customResolvers = {
-  'custom.mapStatus': (value: string) => {
-    const statusMap: Record<string, string> = {
-      in_stock: 'AVAILABLE',
-      out_of_stock: 'ON_HOLD',
-      discontinued: 'DISCONTINUED',
-      reserved: 'RESERVED',
-    };
-
-    const mapped = statusMap[String(value).toLowerCase()];
-    if (!mapped) {
-      throw new Error(`Unknown status: ${value}`);
-    }
-    return mapped;
-  },
-};
-
-async function ingestInventory() {
-  const logger = toStructuredLogger(createConsoleLogger(), {
-    service: 'inventory-ingestion',
-    correlationId: generateCorrelationId()
-  });
-
-  try {
-    // Step 1: Initialize Fluent client with OAuth2 password grant
-    logger.info('Initializing Fluent client...');
-    const client = await createClient({
-      config: {
-        baseUrl: process.env.FLUENT_BASE_URL!,
-        clientId: process.env.FLUENT_CLIENT_ID!,
-        clientSecret: process.env.FLUENT_CLIENT_SECRET!,
-        username: process.env.FLUENT_USERNAME!,       // ✅ REQUIRED for password grant
-        password: process.env.FLUENT_PASSWORD!,       // ✅ REQUIRED for password grant
-        retailerId: process.env.FLUENT_RETAILER_ID!,
-      }
-    });
-
-    // Step 2: Initialize S3 data source with retry logic
-    logger.info('Initializing S3 data source...');
-    let s3: S3DataSource;
-    try {
-      s3 = new S3DataSource(
-        {
-          type: 'S3_CSV',
-          connectionId: 's3-inventory',
-          name: 'S3 Inventory Source',
-          s3Config: {
-            accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
-            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
-            region: process.env.AWS_REGION!,
-            bucket: process.env.S3_BUCKET!,
-          },
-        },
-        logger
-      );
-    } catch (error) {
-      logger.error('Failed to initialize S3 data source', {
-        error: error.message,
-        bucket: process.env.S3_BUCKET,
-        region: process.env.AWS_REGION
-      });
-      throw new Error(`S3 initialization failed: ${error.message}`);
-    }
-
-    // Step 3: Initialize CSV parser
-    const parser = new CSVParserService(logger);
-    // Options are passed to parse() method, not constructor
-
-    // Step 4: Load field mapping configuration
-    const mappingConfig = require('./field-mappings.json');
-    const mapper = new UniversalMapper(mappingConfig, { customResolvers });
-
-    // Step 5: List files in S3 with error handling
-    logger.info(`Listing files in s3://${process.env.S3_BUCKET}/${process.env.S3_PREFIX}`);
-    let files: any[];
-    try {
-      files = await s3.listFiles({ prefix: process.env.S3_PREFIX });
-      logger.info(`Found ${files.length} files`);
-    } catch (error) {
-      logger.error('Failed to list S3 files', {
-        error: error.message,
-        bucket: process.env.S3_BUCKET,
-        prefix: process.env.S3_PREFIX
-      });
-      throw new Error(`S3 file listing failed: ${error.message}`);
-    }
-
-    // Step 6: Process each file with comprehensive error handling
-    for (const file of files) {
-      logger.info(`Processing file: ${file.path}`);
-
-      try {
-        // Step 6a: Download and parse CSV with error recovery
-        let csvData: string;
-        let records: any[];
-
-        try {
-          csvData = await s3.downloadFile(file.path);
-          logger.info(`Downloaded file: ${file.path} (${csvData.length} bytes)`);
-        } catch (error) {
-          logger.error('Failed to download file from S3', {
-            file: file.path,
-            error: error.message
-          });
-          // Skip to next file on download failure
-          continue;
-        }
-
-        try {
-          records = await parser.parse(csvData);
-          logger.info(`Parsed ${records.length} records from CSV`);
-
-          if (records.length === 0) {
-            logger.warn('No records found in CSV file', { file: file.path });
-            continue;
-          }
-        } catch (error) {
-          logger.error('CSV parsing failed', {
-            file: file.path,
-            error: error.message,
-            errorType: error.constructor.name
-          });
-          // Skip to next file on parse failure
-          continue;
-        }
-
-        // Step 6b: Transform data with UniversalMapper
-        const mappingResult = await mapper.map(records);
-
-        if (!mappingResult.success) {
-          logger.error('Mapping errors detected', {
-            file: file.path,
-            errorCount: mappingResult.errors.length,
-            errors: mappingResult.errors.slice(0, 5) // First 5 errors
-          });
-          // Skip to next file on mapping failure
-          continue;
-        }
-
-        logger.info(`Transformed ${mappingResult.data.length} records`);
-
-        // Step 6c: Create Fluent Commerce job with retry
-        let job: any;
-        try {
-          job = await client.createJob({
-            name: `Inventory Import - ${file.path}`,
-            retailerId: process.env.FLUENT_RETAILER_ID!,
-          });
-          logger.info(`Created job: ${job.id}`);
-        } catch (error) {
-          logger.error('Failed to create Fluent job', {
-            file: file.path,
-            error: error.message
-          });
-          continue;
-        }
-
-        // Step 6d: Send batch with rate limiting and validation error handling
-        let batch: any;
-        try {
-          batch = await client.sendBatch(job.id, {
-            action: 'UPSERT',           // String, not enum
-            entityType: 'INVENTORY',    // String, not enum
-            source: 'S3_CSV_IMPORT',    // REQUIRED - Source system identifier
-            event: 'INVENTORY_UPDATE',  // REQUIRED - Event type identifier
-            entities: mappingResult.data, // ✅ Validated InventoryPositionInput[]
-          });
-          logger.info(`Batch sent: ${batch.id}`);
-        } catch (error) {
-          if (error.code === 'RATE_LIMIT_ERROR' || error.message.includes('rate limit')) {
-            logger.warn('Rate limit hit, waiting before retry', {
-              file: file.path,
-              jobId: job.id
-            });
-            // Wait 5 seconds before retry
-            await new Promise(resolve => setTimeout(resolve, 5000));
-            try {
-              batch = await client.sendBatch(job.id, {
-                action: 'UPSERT',
-                entityType: 'INVENTORY',
-                source: 'S3_CSV_IMPORT',
-                event: 'INVENTORY_UPDATE',
-                entities: mappingResult.data,
-              });
-              logger.info(`Batch sent after retry: ${batch.id}`);
-            } catch (retryError) {
-              logger.error('Batch failed after retry', {
-                file: file.path,
-                jobId: job.id,
-                error: retryError.message
-              });
-              continue;
-            }
-          } else if (error.code === 'VALIDATION_ERROR') {
-            logger.error('Batch validation failed - data does not match schema', {
-              file: file.path,
-              jobId: job.id,
-              error: error.message
-            });
-            continue;
-          } else {
-            logger.error('Batch submission failed', {
-              file: file.path,
-              jobId: job.id,
-              error: error.message,
-              errorCode: error.code
-            });
-            continue;
-          }
-        }
-
-        // Step 6e: Monitor batch status with timeout and error handling
-        let status: any;
-        try {
-          status = await client.getBatchStatus(job.id, batch.id);
-          logger.info(`Initial batch status: ${status.status}`);
-        } catch (error) {
-          logger.error('Failed to get batch status', {
-            file: file.path,
-            jobId: job.id,
-            batchId: batch.id,
-            error: error.message
-          });
-          continue;
-        }
-
-        // Poll for completion with timeout (simplified - see Module 6 for production pattern)
-        const startTime = Date.now();
-        const timeout = 300000; // 5 minutes
-
-        while (status.status === 'PENDING' || status.status === 'PROCESSING') {
-          if (Date.now() - startTime > timeout) {
-            logger.error('Batch processing timeout', {
-              file: file.path,
-              jobId: job.id,
-              batchId: batch.id,
-              timeoutMs: timeout
-            });
-            break;
-          }
-
-          await new Promise(resolve => setTimeout(resolve, 5000)); // Wait 5s
-
-          try {
-            status = await client.getBatchStatus(job.id, batch.id);
-            logger.info(`Batch status: ${status.status}`);
-          } catch (error) {
-            logger.error('Failed to poll batch status', {
-              file: file.path,
-              jobId: job.id,
-              batchId: batch.id,
-              error: error.message
-            });
-            break;
-          }
-        }
-
-        if (status.status === 'COMPLETED') {
-          logger.info(
-            `✅ Successfully ingested ${mappingResult.data.length} records from ${file.path}`
-          );
-        } else if (status.status === 'FAILED') {
-          logger.error(`❌ Batch failed with errors`, {
-            file: file.path,
-            jobId: job.id,
-            batchId: batch.id,
-            errors: status.errors || []
-          });
-        } else {
-          logger.warn(`⚠️ Batch ended with unexpected status: ${status.status}`, {
-            file: file.path,
-            jobId: job.id,
-            batchId: batch.id
-          });
-        }
-      } catch (fileError) {
-        logger.error('File processing failed', {
-          file: file.path,
-          error: fileError.message,
-          stack: fileError.stack
-        });
-        // Continue to next file
-      }
-    }
-
-    logger.info('Ingestion complete!');
-  } catch (error) {
-    logger.error('Critical ingestion failure', {
-      error: error.message,
-      errorType: error.constructor.name,
-      stack: error.stack
-    });
-    throw error;
-  }
-}
-
-// Run ingestion
-ingestInventory().catch(error => {
-  console.error('Fatal error:', error);
-  process.exit(1);
-});
-```
-
----
-
-## Step 7: Run Your First Ingestion
-
-Execute the script:
-
-```bash
-npx ts-node ingest-inventory.ts
-```
-
-**Expected output:**
-
-```
-[INFO] Initializing Fluent client...
-[INFO] Initializing S3 data source...
-[INFO] Listing files in s3://your-bucket/data/
-[INFO] Found 1 files
-[INFO] Processing file: data/inventory-2025-01-19.csv
-[INFO] Parsed 5 records from CSV
-[INFO] Transformed 5 records
-[INFO] Created job: JOB-12345
-[INFO] Batch sent: BATCH-67890
-[INFO] Initial batch status: PENDING
-[INFO] Batch status: PROCESSING
-[INFO] Batch status: COMPLETED
-[INFO] ✅ Successfully ingested 5 records from data/inventory-2025-01-19.csv
-[INFO] Ingestion complete!
-```
-
----
-
-## What Just Happened?
-
-Your script performed the following steps:
-
-1. **Connected to Fluent Commerce** - OAuth2 authentication with client credentials
-2. **Connected to S3** - Listed files in the specified bucket/prefix
-3. **Downloaded CSV** - Retrieved file content from S3
-4. **Parsed CSV** - Converted CSV rows to JavaScript objects
-5. **Transformed Data** - Mapped source fields to Fluent schema using UniversalMapper
-6. **Validated Schema** - Ensured all required fields present and correct types
-7. **Created Job** - Prepared a Batch API job for processing
-8. **Sent Batch** - Submitted inventory positions to Fluent Commerce
-9. **Monitored Status** - Polled until batch completed successfully
-
-**Key Concepts Introduced:**
-
-- ✅ `createClient()` - Universal client factory (auto-detects Versori vs standalone)
-- ✅ `S3DataSource` - Read files from S3 buckets
-- ✅ `CSVParserService` - Parse CSV with headers and validation
-- ✅ `UniversalMapper` - Transform source → Fluent schema
-- ✅ Custom resolvers - Business logic transformations (`custom.mapStatus`)
-- ✅ Batch API - UPSERT inventory positions
-- ✅ Job lifecycle - Create → Send Batch → Monitor Status
-
----
-
-## Troubleshooting
-
-### Issue: "Invalid credentials"
-
-**Problem**: OAuth2 authentication failed
-
-**Solution**:
-
-```bash
-# Verify credentials are correct
-echo $FLUENT_CLIENT_ID
-echo $FLUENT_BASE_URL
-
-# Test authentication manually
-npx @fluentcommerce/fc-connect-sdk introspect-schema \
-  --url $FLUENT_BASE_URL/graphql \
-  --client-id $FLUENT_CLIENT_ID \
-  --client-secret $FLUENT_CLIENT_SECRET
-```
-
-### Issue: "Mapping validation failed"
-
-**Problem**: Field mappings don't match schema
-
-**Solution**:
-
-```bash
-# Re-run schema validation
-npx @fluentcommerce/fc-connect-sdk validate-schema \
-  --mapping field-mappings.json \
-  --schema schema.json
-
-# Check error output for specific field mismatches
-```
-
-### Issue: "Missing required field: locationRef"
-
-**Problem**: CSV missing required column
-
-**Solution**:
-
-```typescript
-// Add validation before mapping
-if (!records.every(r => r.warehouse)) {
-  throw new Error('CSV missing required column: warehouse');
-}
-```
-
-### Issue: "Batch status: FAILED"
-
-**Problem**: Batch processing failed in Fluent Commerce
-
-**Solution**:
-
-```typescript
-// Get detailed error information
-const jobStatus = await client.getJobStatus(job.id);
-console.log('Job errors:', jobStatus.errors);
-
-// Check batch-specific errors
-const batchDetails = await client.getBatchStatus(job.id, batch.id);
-console.log('Batch errors:', batchDetails.errors);
-```
-
----
-
-## Next Steps
-
-Now that you have a working ingestion workflow, explore:
-
-- **[Module 3: Data Sources](./02-core-guides-ingestion-03-data-sources.md)** - SFTP, local files, and custom sources
-- **[Module 4: Field Mapping](./02-core-guides-ingestion-04-field-mapping.md)** - Advanced mapping patterns and resolvers
-- **[Module 6: Batch API](./02-core-guides-ingestion-06-batch-api.md)** - Complete job lifecycle and optimization
-- **[Module 7: State Management](./02-core-guides-ingestion-07-state-management.md)** - Prevent duplicate processing
-
----
-
-[← Previous: Introduction](./02-core-guides-ingestion-01-introduction.md) | [Back to Guide](../ingestion-readme.md) | [Next: Data Sources →](./02-core-guides-ingestion-03-data-sources.md)
-
-## Related Documentation
-
-- [S3 Operations Guide](../../data-sources/modules/data-sources-02-s3-operations.md) - Advanced S3 patterns
-- [Universal Mapping Guide](../../mapping/mapping-readme.md) - Complete mapping reference
-- [CSV Parser Guide](../../parsers/modules/02-core-guides-parsers-02-csv-parser.md) - CSV parsing options
-- [Batch API Reference](../../api-reference/modules/api-reference-01-client-api.md#job-batch-operations) - Complete API documentation
+# Module 2: Quick Start - Your First Ingestion
+
+[← Back to Ingestion Guide](../ingestion-readme.md)
+
+**Module 2 of 9** | **Level**: Beginner | **Time**: 15 minutes
+
+---
+
+## Overview
+
+This module walks you through creating your first complete ingestion workflow: reading a CSV file from S3, transforming it with UniversalMapper, and sending it to Fluent Commerce using the Batch API.
+
+## Learning Objectives
+
+By the end of this module, you will:
+
+- ✅ Install and configure the FC Connect SDK
+- ✅ Create a complete S3 → CSV → Batch API workflow
+- ✅ Understand the basic ingestion pattern
+- ✅ Validate field mappings against the schema
+- ✅ Send your first batch to Fluent Commerce
+
+---
+
+## Prerequisites
+
+Before starting, ensure you have:
+
+1. **Node.js** >= 18.0.0 installed
+2. **Fluent Commerce credentials**:
+   - API Base URL (e.g., `https://yourcompany.api.fluentcommerce.com`)
+   - OAuth2 Client ID
+   - OAuth2 Client Secret
+   - Retailer ID
+3. **AWS S3 access** (or local CSV file for testing):
+   - Access Key ID
+   - Secret Access Key
+   - Bucket name
+
+---
+
+## Step 1: Installation
+
+Install the FC Connect SDK:
+
+```bash
+npm install @fluentcommerce/fc-connect-sdk
+```
+
+**Verify installation:**
+
+```bash
+npx @fluentcommerce/fc-connect-sdk --version
+```
+
+---
+
+## Step 2: Environment Configuration
+
+Create a `.env` file with your credentials:
+
+```bash
+# Fluent Commerce OAuth2 (Password Grant)
+FLUENT_BASE_URL=https://yourcompany.api.fluentcommerce.com
+FLUENT_CLIENT_ID=your-oauth-client-id
+FLUENT_CLIENT_SECRET=your-oauth-client-secret
+FLUENT_USERNAME=your-username          # ✅ REQUIRED for password grant
+FLUENT_PASSWORD=your-password          # ✅ REQUIRED for password grant
+FLUENT_RETAILER_ID=1
+
+# AWS S3 (source)
+AWS_ACCESS_KEY_ID=your-aws-access-key
+AWS_SECRET_ACCESS_KEY=your-aws-secret-key
+AWS_REGION=us-east-1
+S3_BUCKET=your-inventory-bucket
+S3_PREFIX=data/
+```
+
+**Load environment variables:**
+
+```bash
+npm install dotenv
+```
+
+---
+
+## Step 3: Prepare Sample Data
+
+Create a sample CSV file (`sample-inventory.csv`):
+
+```csv
+sku,warehouse,quantity,status
+SKU-WM-001,WH-001,500,in_stock
+SKU-WM-002,WH-001,150,in_stock
+SKU-KB-001,WH-001,75,reserved
+SKU-WM-003,WH-002,300,discontinued
+SKU-WM-004,WH-002,200,in_stock
+```
+
+**Upload to S3** (or keep local for testing):
+
+```bash
+aws s3 cp sample-inventory.csv s3://your-bucket/data/inventory-2025-01-19.csv
+```
+
+---
+
+## Step 4: Create Field Mapping Configuration
+
+Create `field-mappings.json`:
+
+```json
+{
+  "fields": {
+    "ref": {
+      "source": "sku",
+      "required": true,
+      "comment": "✅ InventoryPositionInput.ref (String! required) - Position reference"
+    },
+    "productRef": {
+      "source": "sku",
+      "required": true,
+      "comment": "✅ InventoryPositionInput.productRef (String! required) - Product SKU"
+    },
+    "locationRef": {
+      "source": "warehouse",
+      "required": true,
+      "comment": "✅ InventoryPositionInput.locationRef (String! required) - Warehouse code"
+    },
+    "qty": {
+      "source": "quantity",
+      "resolver": "sdk.parseInt",
+      "required": true,
+      "comment": "✅ InventoryPositionInput.qty (Int! required) - Available quantity"
+    },
+    "status": {
+      "source": "status",
+      "resolver": "custom.mapStatus",
+      "defaultValue": "AVAILABLE",
+      "comment": "✅ InventoryPositionInput.status (String optional) - Position status"
+    }
+  }
+}
+```
+
+---
+
+## Step 5: Validate Field Mapping Against Schema
+
+**CRITICAL**: Before running ingestion, validate your mapping:
+
+```bash
+# Step 1: Introspect Fluent Commerce schema
+npx @fluentcommerce/fc-connect-sdk introspect-schema \
+  --url https://yourcompany.api.fluentcommerce.com/graphql \
+  --client-id $FLUENT_CLIENT_ID \
+  --client-secret $FLUENT_CLIENT_SECRET \
+  --output schema.json
+
+# Step 2: Validate field-mappings.json against schema
+npx @fluentcommerce/fc-connect-sdk validate-schema \
+  --mapping field-mappings.json \
+  --schema schema.json
+```
+
+**Expected output:**
+
+```
+✅ Mapping validation successful
+✅ All required fields present: ref, productRef, locationRef, qty
+✅ All field types compatible with InventoryPositionInput schema
+✅ No unknown fields detected
+```
+
+### Preflight Validation (NEW)
+
+Run a preflight validation to catch misconfigurations before execution.
+
+```typescript
+import { PreflightValidator } from '@fluentcommerce/fc-connect-sdk';
+
+const validator = new PreflightValidator(logger);
+await validator.validate({
+  fluent: { client },
+  sftp: { host: process.env.SFTP_HOST, username: process.env.SFTP_USERNAME },
+  config: { processedPath: process.env.PROCESSED_PATH, errorsPath: process.env.ERRORS_PATH },
+});
+```
+
+Checks performed:
+
+- Fluent client reachability/auth
+- SFTP connectivity and directory writability (create directories if missing)
+- Required activation/env variables present
+
+---
+
+## Step 6: Complete Ingestion Script
+
+Create `ingest-inventory.ts`:
+
+```typescript
+import 'dotenv/config';
+import {
+  createClient,
+  S3DataSource,
+  CSVParserService,
+  UniversalMapper,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+
+// Custom resolver for status mapping
+const customResolvers = {
+  'custom.mapStatus': (value: string) => {
+    const statusMap: Record<string, string> = {
+      in_stock: 'AVAILABLE',
+      out_of_stock: 'ON_HOLD',
+      discontinued: 'DISCONTINUED',
+      reserved: 'RESERVED',
+    };
+
+    const mapped = statusMap[String(value).toLowerCase()];
+    if (!mapped) {
+      throw new Error(`Unknown status: ${value}`);
+    }
+    return mapped;
+  },
+};
+
+async function ingestInventory() {
+  const logger = toStructuredLogger(createConsoleLogger(), {
+    service: 'inventory-ingestion',
+    correlationId: generateCorrelationId()
+  });
+
+  try {
+    // Step 1: Initialize Fluent client with OAuth2 password grant
+    logger.info('Initializing Fluent client...');
+    const client = await createClient({
+      config: {
+        baseUrl: process.env.FLUENT_BASE_URL!,
+        clientId: process.env.FLUENT_CLIENT_ID!,
+        clientSecret: process.env.FLUENT_CLIENT_SECRET!,
+        username: process.env.FLUENT_USERNAME!,       // ✅ REQUIRED for password grant
+        password: process.env.FLUENT_PASSWORD!,       // ✅ REQUIRED for password grant
+        retailerId: process.env.FLUENT_RETAILER_ID!,
+      }
+    });
+
+    // Step 2: Initialize S3 data source with retry logic
+    logger.info('Initializing S3 data source...');
+    let s3: S3DataSource;
+    try {
+      s3 = new S3DataSource(
+        {
+          type: 'S3_CSV',
+          connectionId: 's3-inventory',
+          name: 'S3 Inventory Source',
+          s3Config: {
+            accessKeyId: process.env.AWS_ACCESS_KEY_ID!,
+            secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!,
+            region: process.env.AWS_REGION!,
+            bucket: process.env.S3_BUCKET!,
+          },
+        },
+        logger
+      );
+    } catch (error) {
+      logger.error('Failed to initialize S3 data source', {
+        error: error.message,
+        bucket: process.env.S3_BUCKET,
+        region: process.env.AWS_REGION
+      });
+      throw new Error(`S3 initialization failed: ${error.message}`);
+    }
+
+    // Step 3: Initialize CSV parser
+    const parser = new CSVParserService(logger);
+    // Options are passed to parse() method, not constructor
+
+    // Step 4: Load field mapping configuration
+    const mappingConfig = require('./field-mappings.json');
+    const mapper = new UniversalMapper(mappingConfig, { customResolvers });
+
+    // Step 5: List files in S3 with error handling
+    logger.info(`Listing files in s3://${process.env.S3_BUCKET}/${process.env.S3_PREFIX}`);
+    let files: any[];
+    try {
+      files = await s3.listFiles({ prefix: process.env.S3_PREFIX });
+      logger.info(`Found ${files.length} files`);
+    } catch (error) {
+      logger.error('Failed to list S3 files', {
+        error: error.message,
+        bucket: process.env.S3_BUCKET,
+        prefix: process.env.S3_PREFIX
+      });
+      throw new Error(`S3 file listing failed: ${error.message}`);
+    }
+
+    // Step 6: Process each file with comprehensive error handling
+    for (const file of files) {
+      logger.info(`Processing file: ${file.path}`);
+
+      try {
+        // Step 6a: Download and parse CSV with error recovery
+        let csvData: string;
+        let records: any[];
+
+        try {
+          csvData = await s3.downloadFile(file.path);
+          logger.info(`Downloaded file: ${file.path} (${csvData.length} bytes)`);
+        } catch (error) {
+          logger.error('Failed to download file from S3', {
+            file: file.path,
+            error: error.message
+          });
+          // Skip to next file on download failure
+          continue;
+        }
+
+        try {
+          records = await parser.parse(csvData);
+          logger.info(`Parsed ${records.length} records from CSV`);
+
+          if (records.length === 0) {
+            logger.warn('No records found in CSV file', { file: file.path });
+            continue;
+          }
+        } catch (error) {
+          logger.error('CSV parsing failed', {
+            file: file.path,
+            error: error.message,
+            errorType: error.constructor.name
+          });
+          // Skip to next file on parse failure
+          continue;
+        }
+
+        // Step 6b: Transform data with UniversalMapper
+        const mappingResult = await mapper.map(records);
+
+        if (!mappingResult.success) {
+          logger.error('Mapping errors detected', {
+            file: file.path,
+            errorCount: mappingResult.errors.length,
+            errors: mappingResult.errors.slice(0, 5) // First 5 errors
+          });
+          // Skip to next file on mapping failure
+          continue;
+        }
+
+        logger.info(`Transformed ${mappingResult.data.length} records`);
+
+        // Step 6c: Create Fluent Commerce job with retry
+        let job: any;
+        try {
+          job = await client.createJob({
+            name: `Inventory Import - ${file.path}`,
+            retailerId: process.env.FLUENT_RETAILER_ID!,
+          });
+          logger.info(`Created job: ${job.id}`);
+        } catch (error) {
+          logger.error('Failed to create Fluent job', {
+            file: file.path,
+            error: error.message
+          });
+          continue;
+        }
+
+        // Step 6d: Send batch with rate limiting and validation error handling
+        let batch: any;
+        try {
+          batch = await client.sendBatch(job.id, {
+            action: 'UPSERT',           // String, not enum
+            entityType: 'INVENTORY',    // String, not enum
+            source: 'S3_CSV_IMPORT',    // REQUIRED - Source system identifier
+            event: 'INVENTORY_UPDATE',  // REQUIRED - Event type identifier
+            entities: mappingResult.data, // ✅ Validated InventoryPositionInput[]
+          });
+          logger.info(`Batch sent: ${batch.id}`);
+        } catch (error) {
+          if (error.code === 'RATE_LIMIT_ERROR' || error.message.includes('rate limit')) {
+            logger.warn('Rate limit hit, waiting before retry', {
+              file: file.path,
+              jobId: job.id
+            });
+            // Wait 5 seconds before retry
+            await new Promise(resolve => setTimeout(resolve, 5000));
+            try {
+              batch = await client.sendBatch(job.id, {
+                action: 'UPSERT',
+                entityType: 'INVENTORY',
+                source: 'S3_CSV_IMPORT',
+                event: 'INVENTORY_UPDATE',
+                entities: mappingResult.data,
+              });
+              logger.info(`Batch sent after retry: ${batch.id}`);
+            } catch (retryError) {
+              logger.error('Batch failed after retry', {
+                file: file.path,
+                jobId: job.id,
+                error: retryError.message
+              });
+              continue;
+            }
+          } else if (error.code === 'VALIDATION_ERROR') {
+            logger.error('Batch validation failed - data does not match schema', {
+              file: file.path,
+              jobId: job.id,
+              error: error.message
+            });
+            continue;
+          } else {
+            logger.error('Batch submission failed', {
+              file: file.path,
+              jobId: job.id,
+              error: error.message,
+              errorCode: error.code
+            });
+            continue;
+          }
+        }
+
+        // Step 6e: Monitor batch status with timeout and error handling
+        let status: any;
+        try {
+          status = await client.getBatchStatus(job.id, batch.id);
+          logger.info(`Initial batch status: ${status.status}`);
+        } catch (error) {
+          logger.error('Failed to get batch status', {
+            file: file.path,
+            jobId: job.id,
+            batchId: batch.id,
+            error: error.message
+          });
+          continue;
+        }
+
+        // Poll for completion with timeout (simplified - see Module 6 for production pattern)
+        const startTime = Date.now();
+        const timeout = 300000; // 5 minutes
+
+        while (status.status === 'PENDING' || status.status === 'PROCESSING') {
+          if (Date.now() - startTime > timeout) {
+            logger.error('Batch processing timeout', {
+              file: file.path,
+              jobId: job.id,
+              batchId: batch.id,
+              timeoutMs: timeout
+            });
+            break;
+          }
+
+          await new Promise(resolve => setTimeout(resolve, 5000)); // Wait 5s
+
+          try {
+            status = await client.getBatchStatus(job.id, batch.id);
+            logger.info(`Batch status: ${status.status}`);
+          } catch (error) {
+            logger.error('Failed to poll batch status', {
+              file: file.path,
+              jobId: job.id,
+              batchId: batch.id,
+              error: error.message
+            });
+            break;
+          }
+        }
+
+        if (status.status === 'COMPLETED') {
+          logger.info(
+            `✅ Successfully ingested ${mappingResult.data.length} records from ${file.path}`
+          );
+        } else if (status.status === 'FAILED') {
+          logger.error(`❌ Batch failed with errors`, {
+            file: file.path,
+            jobId: job.id,
+            batchId: batch.id,
+            errors: status.errors || []
+          });
+        } else {
+          logger.warn(`⚠️ Batch ended with unexpected status: ${status.status}`, {
+            file: file.path,
+            jobId: job.id,
+            batchId: batch.id
+          });
+        }
+      } catch (fileError) {
+        logger.error('File processing failed', {
+          file: file.path,
+          error: fileError.message,
+          stack: fileError.stack
+        });
+        // Continue to next file
+      }
+    }
+
+    logger.info('Ingestion complete!');
+  } catch (error) {
+    logger.error('Critical ingestion failure', {
+      error: error.message,
+      errorType: error.constructor.name,
+      stack: error.stack
+    });
+    throw error;
+  }
+}
+
+// Run ingestion
+ingestInventory().catch(error => {
+  console.error('Fatal error:', error);
+  process.exit(1);
+});
+```
+
+---
+
+## Step 7: Run Your First Ingestion
+
+Execute the script:
+
+```bash
+npx ts-node ingest-inventory.ts
+```
+
+**Expected output:**
+
+```
+[INFO] Initializing Fluent client...
+[INFO] Initializing S3 data source...
+[INFO] Listing files in s3://your-bucket/data/
+[INFO] Found 1 files
+[INFO] Processing file: data/inventory-2025-01-19.csv
+[INFO] Parsed 5 records from CSV
+[INFO] Transformed 5 records
+[INFO] Created job: JOB-12345
+[INFO] Batch sent: BATCH-67890
+[INFO] Initial batch status: PENDING
+[INFO] Batch status: PROCESSING
+[INFO] Batch status: COMPLETED
+[INFO] ✅ Successfully ingested 5 records from data/inventory-2025-01-19.csv
+[INFO] Ingestion complete!
+```
+
+---
+
+## What Just Happened?
+
+Your script performed the following steps:
+
+1. **Connected to Fluent Commerce** - OAuth2 authentication with client credentials
+2. **Connected to S3** - Listed files in the specified bucket/prefix
+3. **Downloaded CSV** - Retrieved file content from S3
+4. **Parsed CSV** - Converted CSV rows to JavaScript objects
+5. **Transformed Data** - Mapped source fields to Fluent schema using UniversalMapper
+6. **Validated Schema** - Ensured all required fields present and correct types
+7. **Created Job** - Prepared a Batch API job for processing
+8. **Sent Batch** - Submitted inventory positions to Fluent Commerce
+9. **Monitored Status** - Polled until batch completed successfully
+
+**Key Concepts Introduced:**
+
+- ✅ `createClient()` - Universal client factory (auto-detects Versori vs standalone)
+- ✅ `S3DataSource` - Read files from S3 buckets
+- ✅ `CSVParserService` - Parse CSV with headers and validation
+- ✅ `UniversalMapper` - Transform source → Fluent schema
+- ✅ Custom resolvers - Business logic transformations (`custom.mapStatus`)
+- ✅ Batch API - UPSERT inventory positions
+- ✅ Job lifecycle - Create → Send Batch → Monitor Status
+
+---
+
+## Troubleshooting
+
+### Issue: "Invalid credentials"
+
+**Problem**: OAuth2 authentication failed
+
+**Solution**:
+
+```bash
+# Verify credentials are correct
+echo $FLUENT_CLIENT_ID
+echo $FLUENT_BASE_URL
+
+# Test authentication manually
+npx @fluentcommerce/fc-connect-sdk introspect-schema \
+  --url $FLUENT_BASE_URL/graphql \
+  --client-id $FLUENT_CLIENT_ID \
+  --client-secret $FLUENT_CLIENT_SECRET
+```
+
+### Issue: "Mapping validation failed"
+
+**Problem**: Field mappings don't match schema
+
+**Solution**:
+
+```bash
+# Re-run schema validation
+npx @fluentcommerce/fc-connect-sdk validate-schema \
+  --mapping field-mappings.json \
+  --schema schema.json
+
+# Check error output for specific field mismatches
+```
+
+### Issue: "Missing required field: locationRef"
+
+**Problem**: CSV missing required column
+
+**Solution**:
+
+```typescript
+// Add validation before mapping
+if (!records.every(r => r.warehouse)) {
+  throw new Error('CSV missing required column: warehouse');
+}
+```
+
+### Issue: "Batch status: FAILED"
+
+**Problem**: Batch processing failed in Fluent Commerce
+
+**Solution**:
+
+```typescript
+// Get detailed error information
+const jobStatus = await client.getJobStatus(job.id);
+console.log('Job errors:', jobStatus.errors);
+
+// Check batch-specific errors
+const batchDetails = await client.getBatchStatus(job.id, batch.id);
+console.log('Batch errors:', batchDetails.errors);
+```
+
+---
+
+## Next Steps
+
+Now that you have a working ingestion workflow, explore:
+
+- **[Module 3: Data Sources](./02-core-guides-ingestion-03-data-sources.md)** - SFTP, local files, and custom sources
+- **[Module 4: Field Mapping](./02-core-guides-ingestion-04-field-mapping.md)** - Advanced mapping patterns and resolvers
+- **[Module 6: Batch API](./02-core-guides-ingestion-06-batch-api.md)** - Complete job lifecycle and optimization
+- **[Module 7: State Management](./02-core-guides-ingestion-07-state-management.md)** - Prevent duplicate processing
+
+---
+
+[← Previous: Introduction](./02-core-guides-ingestion-01-introduction.md) | [Back to Guide](../ingestion-readme.md) | [Next: Data Sources →](./02-core-guides-ingestion-03-data-sources.md)
+
+## Related Documentation
+
+- [S3 Operations Guide](../../data-sources/modules/data-sources-02-s3-operations.md) - Advanced S3 patterns
+- [Universal Mapping Guide](../../mapping/mapping-readme.md) - Complete mapping reference
+- [CSV Parser Guide](../../parsers/modules/02-core-guides-parsers-02-csv-parser.md) - CSV parsing options
+- [Batch API Reference](../../api-reference/modules/api-reference-01-client-api.md#job-batch-operations) - Complete API documentation