@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.55

package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-01-foundations.md

@@ -1,452 +1,452 @@
-# Module 1: Connector Foundations
-
-**Level:** Beginner
-**Estimated Time:** 15-20 minutes
-
-## Overview
-
-This module introduces the fundamental concepts and architecture patterns for building connectors with the Fluent Connect SDK. You'll learn what makes a complete connector, how components work together, and the common data flow patterns.
-
-## Learning Objectives
-
-By the end of this module, you will:
-
-- ✅ Understand the complete connector architecture
-- ✅ Identify the key SDK components used in connectors
-- ✅ Understand data flow patterns from source to Fluent Commerce
-- ✅ Recognize when to use different connector patterns
-- ✅ Understand state management and error handling concepts
-
-## What is a Connector?
-
-A **connector** is a complete data integration workflow that:
-
-1. **Reads** data from a source (SFTP, S3, API, etc.)
-2. **Parses** the data format (CSV, JSON, Parquet, XML)
-3. **Validates** data quality and completeness
-4. **Transforms** fields to match Fluent Commerce schema
-5. **Sends** data to Fluent Commerce via Batch API or GraphQL
-6. **Tracks** processing state to prevent duplicates
-7. **Handles** errors and provides monitoring
-
-### Example: Inventory Sync Connector
-
-```typescript
-// A connector orchestrates multiple SDK components:
-
-┌─────────────────────────────────────────────────────────────────┐
-│                    INVENTORY SYNC CONNECTOR                     │
-├─────────────────────────────────────────────────────────────────┤
-│                                                                 │
-│  1. SftpDataSource ──> Downloads CSV files from remote server  │
-│  2. CSVParserService      ──> Parses CSV into JavaScript objects      │
-│  3. Validator      ──> Checks required fields and data types   │
-│  4. UniversalMapper──> Transforms fields to Fluent schema      │
-│  5. FluentClient   ──> Sends batch updates to Fluent API       │
-│  6. StateService   ──> Tracks processed files (no duplicates)  │
-│  7. Logging Utilities ──> Records progress, errors, metrics    │
-│                                                                 │
-└─────────────────────────────────────────────────────────────────┘
-```
-
-## Core SDK Components
-
-### 1. Data Sources
-
-Data sources handle reading from and writing to external systems:
-
-| Component             | Purpose              | Example Use                                     |
-| --------------------- | -------------------- | ----------------------------------------------- |
-| `SftpDataSource`      | SFTP file operations | Read inventory CSVs from supplier SFTP          |
-| `S3DataSource`        | AWS S3 operations    | Read product Parquet files from data lake       |
-| `InventoryDataSource` | Abstract base        | Build custom sources (HTTP API, database, etc.) |
-
-**Key capabilities:**
-
-- List files with pattern matching
-- Download file contents (streaming or buffered)
-- Upload/archive files
-- Move files between folders
-
-### 2. Parsers
-
-Parsers convert file formats into JavaScript objects:
-
-| Component              | Format  | Use Case                                |
-| ---------------------- | ------- | --------------------------------------- |
-| `CSVParserService`     | CSV     | Inventory feeds, simple data exports    |
-| `ParquetParserService` | Parquet | Large datasets, data warehouse exports  |
-| `XMLParserService`     | XML     | SFCC orders, EDI/B2B system integrations |
-| `JSONParserService`    | JSON    | Modern APIs, webhook payloads           |
-
-**Key capabilities:**
-
-- Stream large files (memory efficient)
-- Validate structure and types
-- Handle encoding and special characters
-- Skip malformed records with errors
-
-### 3. Field Mapper
-
-`UniversalMapper` transforms source data to Fluent Commerce schema:
-
-```typescript
-const mapper = new UniversalMapper({
-  fields: {
-    // Target field: { source, transformation }
-    ref: { source: 'sku', resolver: 'sdk.uppercase' },
-    locationRef: { source: 'warehouse_code' },
-    qty: { source: 'quantity', resolver: 'sdk.parseInt' },
-    status: { source: 'status', resolver: v => (v === '1' ? 'ACTIVE' : 'INACTIVE') },
-  },
-});
-```
-
-**Key capabilities:**
-
-- Built-in resolvers (parse numbers, format dates, etc.)
-- Custom transformation functions
-- Nested field mapping
-- Conditional logic
-- Error collection (doesn't fail on single record)
-
-### 4. Fluent Client
-
-`FluentClient` communicates with Fluent Commerce APIs:
-
-```typescript
-const client = await createClient({
-  config: {
-    baseUrl: process.env.FLUENT_BASE_URL,
-    clientId: process.env.FLUENT_CLIENT_ID,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET,
-    retailerId: process.env.FLUENT_RETAILER_ID,
-  },
-});
-```
-
-**Key capabilities:**
-
-- OAuth2 authentication (automatic token refresh)
-- Batch API operations (inventory, positions)
-- GraphQL mutations (orders, products, etc.)
-- GraphQL queries with auto-pagination
-- Error handling and retries
-
-### 5. State Management
-
-`StateService` prevents duplicate processing:
-
-```typescript
-const stateService = new StateService(logger);
-const kv = openKv(); // Or VersoriKVAdapter for Versori platform
-
-// Check if already processed
-if (await stateService.isFileProcessed(kv, file.name, 'workflow-id')) {
-  logger.info('Already processed, skipping');
-  return;
-}
-
-// Process file...
-
-// Mark as complete (updateSyncState tracks file automatically)
-await stateService.updateSyncState(kv, [{
-  fileName: file.name,
-  lastModified: new Date().toISOString(),
-  recordCount: 1000
-}], 'workflow-id');
-```
-
-**Key capabilities:**
-
-- File-level tracking
-- Record-level checkpoints
-- Metadata storage
-- Timestamp tracking
-- Multiple storage backends (memory, S3, Versori KV)
-
-### 6. Logging & Monitoring
-
-The SDK provides functional logging utilities:
-
-**Versori Platform:** Use native `log` from context
-```typescript
-const { log } = ctx;
-log.info('Processing started');
-```
-
-**Standalone:** Use function-based utilities
-```typescript
-import { createConsoleLogger, toStructuredLogger } from '@fluentcommerce/fc-connect-sdk';
-const logger = toStructuredLogger(createConsoleLogger(), { service: 'MyService' });
-
-logger.info('Starting sync', { source: 'SFTP', files: 5 });
-logger.warn('Validation errors', { count: 3 });
-logger.error('Upload failed', error);
-```
-
-## Data Flow Patterns
-
-### Pattern 1: Simple Ingestion (One-Way)
-
-```
-External System → Fluent Commerce
-```
-
-**Steps:**
-
-1. List files from source
-2. Download and parse
-3. Validate and transform
-4. Send to Fluent
-5. Archive processed files
-
-**Example use cases:**
-
-- SFTP CSV inventory updates
-- S3 Parquet product catalog sync
-- JSON order imports
-
-### Pattern 2: Bidirectional Sync
-
-```
-External System ←→ Fluent Commerce
-```
-
-**Steps:**
-
-1. **Inbound:** External → Fluent (ingestion)
-2. **Outbound:** Fluent → External (extraction)
-
-**Example use cases:**
-
-- Inventory sync (external updates Fluent, Fluent exports for reporting)
-- Order fulfillment (orders in, status updates out)
-- Product catalog (master data in, availability out)
-
-### Pattern 3: Multi-Source Aggregation
-
-```
-Source A ─┐
-Source B ─┼─→ Aggregation → Fluent Commerce
-Source C ─┘
-```
-
-**Steps:**
-
-1. Fetch data from multiple sources
-2. Merge and reconcile differences
-3. Apply business rules
-4. Send consolidated data to Fluent
-
-**Example use cases:**
-
-- Multi-warehouse inventory (aggregate quantities)
-- Multi-channel orders (combine platforms)
-- Distributed product data (merge attributes)
-
-### Pattern 4: Event-Driven Processing
-
-```
-Webhook/Event → Connector → Fluent Commerce
-```
-
-**Steps:**
-
-1. Receive webhook payload
-2. Validate and parse
-3. Transform to Fluent schema
-4. Send mutation or event
-
-**Example use cases:**
-
-- Real-time order creation
-- Inventory adjustments on sales
-- Product updates on changes
-
-## When to Use Each Pattern
-
-| Pattern                | Best For              | Complexity | State Needed                  |
-| ---------------------- | --------------------- | ---------- | ----------------------------- |
-| **Simple Ingestion**   | Regular batch updates | Low        | File-level                    |
-| **Bidirectional Sync** | Two-way data exchange | Medium     | File + record-level           |
-| **Multi-Source**       | Data consolidation    | High       | Record-level + reconciliation |
-| **Event-Driven**       | Real-time updates     | Medium     | Deduplication only            |
-
-## Connector Lifecycle
-
-### Phase 1: Initialization
-
-```typescript
-// Setup services
-const logger = toStructuredLogger(createConsoleLogger(), {
-  logLevel: 'info'
-});
-const sftpSource = new SftpDataSource({...}, logger);
-const mapper = new UniversalMapper({...});
-const client = await createClient({...});
-const stateService = new StateService(logger);
-```
-
-### Phase 2: Discovery
-
-```typescript
-// Find new files to process
-const files = await sftpSource.listFiles({ filePattern: '*.csv' });
-logger.info(`Found ${files.length} files`);
-```
-
-### Phase 3: Processing
-
-```typescript
-for (const file of files) {
-  // Download → Parse → Transform → Send
-  const content = await sftpSource.downloadFile(file.name);
-  const records = await parser.parse(content);
-  const transformed = await mapper.mapBatch(records);
-  await client.sendBatch(jobId, { entities: transformed });
-}
-```
-
-### Phase 4: Finalization
-
-```typescript
-// Archive, track state, log metrics
-await sftpSource.moveFile(file.name, 'archive/');
-await stateService.updateSyncState(kv, [{
-  fileName: file.name,
-  lastModified: new Date().toISOString(),
-  recordCount: 1000
-}], 'workflow-id');
-logger.info('Sync complete', { filesProcessed: files.length });
-```
-
-## Error Handling Strategy
-
-### Levels of Error Handling
-
-**1. Record-Level Errors (Continue Processing)**
-
-```typescript
-const transformed = [];
-const errors = [];
-
-for (const record of records) {
-  const result = await mapper.map(record);
-  if (result.success) {
-    transformed.push(result.data);
-  } else {
-    errors.push({ record, errors: result.errors });
-  }
-}
-
-logger.warn(`Skipped ${errors.length} invalid records`);
-```
-
-**2. File-Level Errors (Skip File, Continue Batch)**
-
-```typescript
-try {
-  await processFile(file);
-} catch (error) {
-  logger.error(`File failed: ${file.name}`, error);
-  await moveToErrorFolder(file);
-  // Continue with next file
-}
-```
-
-**3. Job-Level Errors (Fail Fast)**
-
-```typescript
-try {
-  await syncInventory();
-} catch (error) {
-  logger.error('Critical failure', error);
-  await notifyAdministrators(error);
-  throw error; // Stop execution
-}
-```
-
-## State Management Strategies
-
-### Strategy 1: File-Level Tracking (Simple)
-
-**When to use:** Small files, infrequent processing
-**Storage:** Lightweight (just file names + timestamps)
-
-```typescript
-// Check before processing
-if (await stateService.isFileProcessed(kv, 'inventory_2025-01-15.csv', 'workflow-id')) {
-  return; // Skip
-}
-
-// Mark after success
-await stateService.updateSyncState(kv, [{
-  fileName: 'inventory_2025-01-15.csv',
-  lastModified: new Date().toISOString(),
-  recordCount: 1000
-}], 'workflow-id');
-```
-
-### Strategy 2: Record-Level Checkpoints (Advanced)
-
-**When to use:** Large files, streaming, can resume mid-file
-**Storage:** More complex (record IDs, batch numbers)
-
-```typescript
-// Save checkpoint every 1000 records using KV directly
-const kv = openKv();
-if (processedCount % 1000 === 0) {
-  await kv.set(['checkpoint', 'job-123'], {
-    file: currentFile,
-    lastRecordId: record.id,
-    recordsProcessed: processedCount,
-    timestamp: new Date().toISOString(),
-  });
-}
-
-// Resume from checkpoint on failure
-const checkpointData = await kv.get(['checkpoint', 'job-123']);
-const checkpoint = checkpointData?.value;
-if (checkpoint) {
-  startFrom = checkpoint.lastRecordId;
-}
-```
-
-### Strategy 3: Hybrid (File + Record Metadata)
-
-**When to use:** Production systems needing audit trails
-**Storage:** File status + processing metadata
-
-```typescript
-// Use updateSyncState with full metadata
-const kv = openKv();
-await stateService.updateSyncState(kv, [{
-  fileName: file.name,
-  lastModified: new Date().toISOString(),
-  recordCount: 5000
-}], 'workflow-id');
-
-// Store additional custom metadata via KV directly (optional)
-await kv.set(['custom-state', file.name], {
-  status: 'completed',
-  recordsProcessed: 5000,
-  recordsFailed: 12,
-  startTime: startTime.toISOString(),
-  endTime: new Date().toISOString(),
-  processingDuration: Date.now() - startTime,
-});
-```
-
-## Key Takeaways
-
-- 🎯 Connectors orchestrate multiple SDK components into complete workflows
-- 🎯 Data flows through: Source → Parser → Validator → Mapper → Fluent API
-- 🎯 State management prevents duplicate processing (critical for production)
-- 🎯 Error handling should be layered: record-level, file-level, job-level
-- 🎯 Choose the right pattern based on use case (simple, bidirectional, multi-source, event)
-- 🎯 Always include logging and monitoring for production visibility
-
-## Next Steps
-
-Continue to [Module 2: Simple Scenarios](./connector-scenarios-02-simple-scenarios.md) to build your first connector with SFTP CSV ingestion.
+# Module 1: Connector Foundations
+
+**Level:** Beginner
+**Estimated Time:** 15-20 minutes
+
+## Overview
+
+This module introduces the fundamental concepts and architecture patterns for building connectors with the Fluent Connect SDK. You'll learn what makes a complete connector, how components work together, and the common data flow patterns.
+
+## Learning Objectives
+
+By the end of this module, you will:
+
+- ✅ Understand the complete connector architecture
+- ✅ Identify the key SDK components used in connectors
+- ✅ Understand data flow patterns from source to Fluent Commerce
+- ✅ Recognize when to use different connector patterns
+- ✅ Understand state management and error handling concepts
+
+## What is a Connector?
+
+A **connector** is a complete data integration workflow that:
+
+1. **Reads** data from a source (SFTP, S3, API, etc.)
+2. **Parses** the data format (CSV, JSON, Parquet, XML)
+3. **Validates** data quality and completeness
+4. **Transforms** fields to match Fluent Commerce schema
+5. **Sends** data to Fluent Commerce via Batch API or GraphQL
+6. **Tracks** processing state to prevent duplicates
+7. **Handles** errors and provides monitoring
+
+### Example: Inventory Sync Connector
+
+```typescript
+// A connector orchestrates multiple SDK components:
+
+┌─────────────────────────────────────────────────────────────────┐
+│                    INVENTORY SYNC CONNECTOR                     │
+├─────────────────────────────────────────────────────────────────┤
+│                                                                 │
+│  1. SftpDataSource ──> Downloads CSV files from remote server  │
+│  2. CSVParserService      ──> Parses CSV into JavaScript objects      │
+│  3. Validator      ──> Checks required fields and data types   │
+│  4. UniversalMapper──> Transforms fields to Fluent schema      │
+│  5. FluentClient   ──> Sends batch updates to Fluent API       │
+│  6. StateService   ──> Tracks processed files (no duplicates)  │
+│  7. Logging Utilities ──> Records progress, errors, metrics    │
+│                                                                 │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+## Core SDK Components
+
+### 1. Data Sources
+
+Data sources handle reading from and writing to external systems:
+
+| Component             | Purpose              | Example Use                                     |
+| --------------------- | -------------------- | ----------------------------------------------- |
+| `SftpDataSource`      | SFTP file operations | Read inventory CSVs from supplier SFTP          |
+| `S3DataSource`        | AWS S3 operations    | Read product Parquet files from data lake       |
+| `InventoryDataSource` | Abstract base        | Build custom sources (HTTP API, database, etc.) |
+
+**Key capabilities:**
+
+- List files with pattern matching
+- Download file contents (streaming or buffered)
+- Upload/archive files
+- Move files between folders
+
+### 2. Parsers
+
+Parsers convert file formats into JavaScript objects:
+
+| Component              | Format  | Use Case                                |
+| ---------------------- | ------- | --------------------------------------- |
+| `CSVParserService`     | CSV     | Inventory feeds, simple data exports    |
+| `ParquetParserService` | Parquet | Large datasets, data warehouse exports  |
+| `XMLParserService`     | XML     | SFCC orders, EDI/B2B system integrations |
+| `JSONParserService`    | JSON    | Modern APIs, webhook payloads           |
+
+**Key capabilities:**
+
+- Stream large files (memory efficient)
+- Validate structure and types
+- Handle encoding and special characters
+- Skip malformed records with errors
+
+### 3. Field Mapper
+
+`UniversalMapper` transforms source data to Fluent Commerce schema:
+
+```typescript
+const mapper = new UniversalMapper({
+  fields: {
+    // Target field: { source, transformation }
+    ref: { source: 'sku', resolver: 'sdk.uppercase' },
+    locationRef: { source: 'warehouse_code' },
+    qty: { source: 'quantity', resolver: 'sdk.parseInt' },
+    status: { source: 'status', resolver: v => (v === '1' ? 'ACTIVE' : 'INACTIVE') },
+  },
+});
+```
+
+**Key capabilities:**
+
+- Built-in resolvers (parse numbers, format dates, etc.)
+- Custom transformation functions
+- Nested field mapping
+- Conditional logic
+- Error collection (doesn't fail on single record)
+
+### 4. Fluent Client
+
+`FluentClient` communicates with Fluent Commerce APIs:
+
+```typescript
+const client = await createClient({
+  config: {
+    baseUrl: process.env.FLUENT_BASE_URL,
+    clientId: process.env.FLUENT_CLIENT_ID,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET,
+    retailerId: process.env.FLUENT_RETAILER_ID,
+  },
+});
+```
+
+**Key capabilities:**
+
+- OAuth2 authentication (automatic token refresh)
+- Batch API operations (inventory, positions)
+- GraphQL mutations (orders, products, etc.)
+- GraphQL queries with auto-pagination
+- Error handling and retries
+
+### 5. State Management
+
+`StateService` prevents duplicate processing:
+
+```typescript
+const stateService = new StateService(logger);
+const kv = openKv(); // Or VersoriKVAdapter for Versori platform
+
+// Check if already processed
+if (await stateService.isFileProcessed(kv, file.name, 'workflow-id')) {
+  logger.info('Already processed, skipping');
+  return;
+}
+
+// Process file...
+
+// Mark as complete (updateSyncState tracks file automatically)
+await stateService.updateSyncState(kv, [{
+  fileName: file.name,
+  lastModified: new Date().toISOString(),
+  recordCount: 1000
+}], 'workflow-id');
+```
+
+**Key capabilities:**
+
+- File-level tracking
+- Record-level checkpoints
+- Metadata storage
+- Timestamp tracking
+- Multiple storage backends (memory, S3, Versori KV)
+
+### 6. Logging & Monitoring
+
+The SDK provides functional logging utilities:
+
+**Versori Platform:** Use native `log` from context
+```typescript
+const { log } = ctx;
+log.info('Processing started');
+```
+
+**Standalone:** Use function-based utilities
+```typescript
+import { createConsoleLogger, toStructuredLogger } from '@fluentcommerce/fc-connect-sdk';
+const logger = toStructuredLogger(createConsoleLogger(), { service: 'MyService' });
+
+logger.info('Starting sync', { source: 'SFTP', files: 5 });
+logger.warn('Validation errors', { count: 3 });
+logger.error('Upload failed', error);
+```
+
+## Data Flow Patterns
+
+### Pattern 1: Simple Ingestion (One-Way)
+
+```
+External System → Fluent Commerce
+```
+
+**Steps:**
+
+1. List files from source
+2. Download and parse
+3. Validate and transform
+4. Send to Fluent
+5. Archive processed files
+
+**Example use cases:**
+
+- SFTP CSV inventory updates
+- S3 Parquet product catalog sync
+- JSON order imports
+
+### Pattern 2: Bidirectional Sync
+
+```
+External System ←→ Fluent Commerce
+```
+
+**Steps:**
+
+1. **Inbound:** External → Fluent (ingestion)
+2. **Outbound:** Fluent → External (extraction)
+
+**Example use cases:**
+
+- Inventory sync (external updates Fluent, Fluent exports for reporting)
+- Order fulfillment (orders in, status updates out)
+- Product catalog (master data in, availability out)
+
+### Pattern 3: Multi-Source Aggregation
+
+```
+Source A ─┐
+Source B ─┼─→ Aggregation → Fluent Commerce
+Source C ─┘
+```
+
+**Steps:**
+
+1. Fetch data from multiple sources
+2. Merge and reconcile differences
+3. Apply business rules
+4. Send consolidated data to Fluent
+
+**Example use cases:**
+
+- Multi-warehouse inventory (aggregate quantities)
+- Multi-channel orders (combine platforms)
+- Distributed product data (merge attributes)
+
+### Pattern 4: Event-Driven Processing
+
+```
+Webhook/Event → Connector → Fluent Commerce
+```
+
+**Steps:**
+
+1. Receive webhook payload
+2. Validate and parse
+3. Transform to Fluent schema
+4. Send mutation or event
+
+**Example use cases:**
+
+- Real-time order creation
+- Inventory adjustments on sales
+- Product updates on changes
+
+## When to Use Each Pattern
+
+| Pattern                | Best For              | Complexity | State Needed                  |
+| ---------------------- | --------------------- | ---------- | ----------------------------- |
+| **Simple Ingestion**   | Regular batch updates | Low        | File-level                    |
+| **Bidirectional Sync** | Two-way data exchange | Medium     | File + record-level           |
+| **Multi-Source**       | Data consolidation    | High       | Record-level + reconciliation |
+| **Event-Driven**       | Real-time updates     | Medium     | Deduplication only            |
+
+## Connector Lifecycle
+
+### Phase 1: Initialization
+
+```typescript
+// Setup services
+const logger = toStructuredLogger(createConsoleLogger(), {
+  logLevel: 'info'
+});
+const sftpSource = new SftpDataSource({...}, logger);
+const mapper = new UniversalMapper({...});
+const client = await createClient({...});
+const stateService = new StateService(logger);
+```
+
+### Phase 2: Discovery
+
+```typescript
+// Find new files to process
+const files = await sftpSource.listFiles({ filePattern: '*.csv' });
+logger.info(`Found ${files.length} files`);
+```
+
+### Phase 3: Processing
+
+```typescript
+for (const file of files) {
+  // Download → Parse → Transform → Send
+  const content = await sftpSource.downloadFile(file.name);
+  const records = await parser.parse(content);
+  const transformed = await mapper.mapBatch(records);
+  await client.sendBatch(jobId, { entities: transformed });
+}
+```
+
+### Phase 4: Finalization
+
+```typescript
+// Archive, track state, log metrics
+await sftpSource.moveFile(file.name, 'archive/');
+await stateService.updateSyncState(kv, [{
+  fileName: file.name,
+  lastModified: new Date().toISOString(),
+  recordCount: 1000
+}], 'workflow-id');
+logger.info('Sync complete', { filesProcessed: files.length });
+```
+
+## Error Handling Strategy
+
+### Levels of Error Handling
+
+**1. Record-Level Errors (Continue Processing)**
+
+```typescript
+const transformed = [];
+const errors = [];
+
+for (const record of records) {
+  const result = await mapper.map(record);
+  if (result.success) {
+    transformed.push(result.data);
+  } else {
+    errors.push({ record, errors: result.errors });
+  }
+}
+
+logger.warn(`Skipped ${errors.length} invalid records`);
+```
+
+**2. File-Level Errors (Skip File, Continue Batch)**
+
+```typescript
+try {
+  await processFile(file);
+} catch (error) {
+  logger.error(`File failed: ${file.name}`, error);
+  await moveToErrorFolder(file);
+  // Continue with next file
+}
+```
+
+**3. Job-Level Errors (Fail Fast)**
+
+```typescript
+try {
+  await syncInventory();
+} catch (error) {
+  logger.error('Critical failure', error);
+  await notifyAdministrators(error);
+  throw error; // Stop execution
+}
+```
+
+## State Management Strategies
+
+### Strategy 1: File-Level Tracking (Simple)
+
+**When to use:** Small files, infrequent processing
+**Storage:** Lightweight (just file names + timestamps)
+
+```typescript
+// Check before processing
+if (await stateService.isFileProcessed(kv, 'inventory_2025-01-15.csv', 'workflow-id')) {
+  return; // Skip
+}
+
+// Mark after success
+await stateService.updateSyncState(kv, [{
+  fileName: 'inventory_2025-01-15.csv',
+  lastModified: new Date().toISOString(),
+  recordCount: 1000
+}], 'workflow-id');
+```
+
+### Strategy 2: Record-Level Checkpoints (Advanced)
+
+**When to use:** Large files, streaming, can resume mid-file
+**Storage:** More complex (record IDs, batch numbers)
+
+```typescript
+// Save checkpoint every 1000 records using KV directly
+const kv = openKv();
+if (processedCount % 1000 === 0) {
+  await kv.set(['checkpoint', 'job-123'], {
+    file: currentFile,
+    lastRecordId: record.id,
+    recordsProcessed: processedCount,
+    timestamp: new Date().toISOString(),
+  });
+}
+
+// Resume from checkpoint on failure
+const checkpointData = await kv.get(['checkpoint', 'job-123']);
+const checkpoint = checkpointData?.value;
+if (checkpoint) {
+  startFrom = checkpoint.lastRecordId;
+}
+```
+
+### Strategy 3: Hybrid (File + Record Metadata)
+
+**When to use:** Production systems needing audit trails
+**Storage:** File status + processing metadata
+
+```typescript
+// Use updateSyncState with full metadata
+const kv = openKv();
+await stateService.updateSyncState(kv, [{
+  fileName: file.name,
+  lastModified: new Date().toISOString(),
+  recordCount: 5000
+}], 'workflow-id');
+
+// Store additional custom metadata via KV directly (optional)
+await kv.set(['custom-state', file.name], {
+  status: 'completed',
+  recordsProcessed: 5000,
+  recordsFailed: 12,
+  startTime: startTime.toISOString(),
+  endTime: new Date().toISOString(),
+  processingDuration: Date.now() - startTime,
+});
+```
+
+## Key Takeaways
+
+- 🎯 Connectors orchestrate multiple SDK components into complete workflows
+- 🎯 Data flows through: Source → Parser → Validator → Mapper → Fluent API
+- 🎯 State management prevents duplicate processing (critical for production)
+- 🎯 Error handling should be layered: record-level, file-level, job-level
+- 🎯 Choose the right pattern based on use case (simple, bidirectional, multi-source, event)
+- 🎯 Always include logging and monitoring for production visibility
+
+## Next Steps
+
+Continue to [Module 2: Simple Scenarios](./connector-scenarios-02-simple-scenarios.md) to build your first connector with SFTP CSV ingestion.