@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-07-reference.md

@@ -1,445 +1,445 @@
-# Module 7: Connector Patterns Reference
-
-**Level:** Reference
-**Estimated Time:** As needed
-
-## Overview
-
-This module provides a quick reference catalog of all connector patterns, components, and common code snippets.
-
-## Connector Patterns Catalog
-
-### Pattern 1: SFTP CSV to Fluent Inventory
-
-- **Complexity:** Low
-- **Components:** SftpDataSource, CSVParserService, UniversalMapper, FluentClient
-- **Use Case:** Daily inventory updates from warehouse
-- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
-
-### Pattern 2: S3 Parquet to Fluent Products
-
-- **Complexity:** Medium
-- **Components:** S3DataSource, ParquetParserService, UniversalMapper, FluentClient
-- **Use Case:** Product catalog sync from data lake
-- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
-
-### Pattern 3: SFTP JSON to Fluent Orders
-
-- **Complexity:** Medium
-- **Components:** SftpDataSource, JSONParserService, SchemaValidationService
-- **Use Case:** Order creation from e-commerce exports
-- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
-
-### Pattern 4: S3 CSV with State Management
-
-- **Complexity:** High
-- **Components:** S3DataSource, CSVParserService, StateService, FluentClient
-- **Use Case:** Large file processing with recovery
-- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
-
-### Pattern 5: Multi-Source Aggregation
-
-- **Complexity:** High
-- **Components:** SftpDataSource, S3DataSource, Multiple parsers, Reconciliation
-- **Use Case:** Aggregate inventory from multiple warehouses
-- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
-
-### Pattern 6: Bidirectional Sync
-
-- **Complexity:** Medium
-- **Components:** FluentClient, ExtractionOrchestrator, SftpDataSource
-- **Use Case:** Export Fluent data to external systems
-- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
-
-## Component Reference
-
-### Data Sources
-
-| Component             | Protocol | Formats            | Key Methods                                                        |
-| --------------------- | -------- | ------------------ | ------------------------------------------------------------------ |
-| `SftpDataSource`      | SFTP     | CSV, JSON, XML     | `listFiles()`, `downloadFile()`, `uploadFile()`, `moveFile()`   |
-| `S3DataSource`        | AWS S3   | CSV, JSON, Parquet | `listFiles()`, `downloadFile()`, `uploadFile()`, `getFileStream()` |
-| `InventoryDataSource` | Custom   | Any                | Extend for custom sources                                          |
-
-### Parsers
-
-| Component              | Format  | Streaming | Key Options                                         |
-| ---------------------- | ------- | --------- | --------------------------------------------------- |
-| `CSVParserService`     | CSV     | Yes       | `delimiter`, `headers`, `skipEmptyLines`, `maxRows` |
-| `ParquetParserService` | Parquet | No        | `columns`, `maxRows`                                |
-| `XMLParserService`     | XML     | Yes       | `pathMapping`, `rootElement`                        |
-| `JSONParserService`    | JSON    | Yes       | `jsonFormat`, `validate`                            |
-
-### Transformation
-
-| Component           | Purpose              | Key Methods                |
-| ------------------- | -------------------- | -------------------------- |
-| `UniversalMapper`   | Field transformation | `map()`, `mapBatch()`      |
-| `SchemaValidationService` | GraphQL schema validation | `validateMutation()`, `validatePayload()` |
-| `ExtractionOrchestrator` | High-level data extraction | `extract()`, with auto-pagination |
-
-### State Management
-
-⚠️ **IMPORTANT:** StateService requires KV adapter passed to EVERY method call. The constructor takes only logger (optional).
-
-| Component                | Storage        | Use Case                     |
-| ------------------------ | -------------- | ---------------------------- |
-| `StateService`           | KV adapter-based | File tracking, deduplication, checkpointing |
-| `VersoriKVAdapter`       | Versori KV     | Versori platform state       |
-| `VersoriFileTracker`     | Versori KV     | Simple file tracking         |
-
-#### VersoriFileTracker Usage
-
-**Simple file tracking without StateService complexity:**
-
-```typescript
-import { schedule, fn } from '@versori/run';
-import { VersoriFileTracker, SftpDataSource } from '@fluentcommerce/fc-connect-sdk';
-
-export const inventorySync = schedule("inventory-sync", "0 * * * *")
-  .then(fn("process-files", async (ctx) => {
-    const { log } = ctx;
-
-    // Create file tracker (simpler than StateService)
-    const fileTracker = new VersoriFileTracker(openKv(), 'inventory-sync');
-
-    // List files from SFTP
-    const sftp = new SftpDataSource(sftpConfig, log);
-    const files = await sftp.listFiles();
-
-    for (const file of files) {
-      // Skip already processed files
-      if (await fileTracker.wasFileProcessed(file.name)) {
-        log.info(`Skipping already processed file: ${file.name}`);
-        continue;
-      }
-
-      try {
-        // Download and process file
-        const content = await sftp.downloadFile(file.path);
-        const records = await processFile(content);
-
-        // Mark as processed with metadata
-        await fileTracker.markFileProcessed(file.name, {
-          recordCount: records.length
-        });
-
-        log.info(`Processed ${file.name}: ${records.length} records`);
-
-        // Optional: Track last processed file
-        await fileTracker.setLastProcessedFile(file.name);
-
-      } catch (error) {
-        log.error(`Failed to process ${file.name}:`, error);
-        // Don't mark as processed - will retry next run
-      }
-    }
-
-    // Get last processed file for reporting
-    const lastFile = await fileTracker.getLastProcessedFile();
-    return { success: true, lastProcessedFile: lastFile };
-  }));
-```
-
-**VersoriFileTracker API:**
-
-| Method | Parameters | Returns | Description |
-|--------|-----------|---------|-------------|
-| `wasFileProcessed(fileName)` | `fileName: string` | `Promise<boolean>` | Check if file was already processed |
-| `markFileProcessed(fileName, metadata?)` | `fileName: string`, `metadata?: { recordCount?: number }` | `Promise<void>` | Mark file as processed with optional metadata |
-| `getLastProcessedFile()` | - | `Promise<string \| null>` | Get the last processed file name |
-| `setLastProcessedFile(fileName)` | `fileName: string` | `Promise<void>` | Set the last processed file |
-
-**When to use VersoriFileTracker vs StateService:**
-
-| Feature | VersoriFileTracker | StateService |
-|---------|-------------------|--------------|
-| **Simplicity** | ✅ Simple 4-method API | Complex with adapter pattern |
-| **File Tracking** | ✅ Built-in | Requires adapter + workflow ID |
-| **Metadata** | ✅ Basic (recordCount) | ✅ Rich metadata support |
-| **Checkpointing** | ❌ Not supported | ✅ Record-level checkpointing |
-| **Multi-workflow** | ✅ Via prefix | ✅ Via workflow ID |
-| **Best For** | Simple file-based ingestion | Complex stateful workflows |
-
-**Recommendation:** Use `VersoriFileTracker` for simple file processing workflows on Versori. Use `StateService` for complex stateful operations or cross-runtime support.
-
-## Common Code Patterns
-
-### Initialize Connector
-
-```typescript
-import {
-  createClient,
-  SftpDataSource,
-  UniversalMapper,
-  StateService,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-
-const logger = toStructuredLogger(createConsoleLogger(), {
-  service: 'connector-name',
-  module: 'workflow-name'
-});
-
-// StateService constructor takes only logger (optional)
-const stateService = new StateService(logger);
-
-const dataSource = new SftpDataSource(
-  {
-    type: 'SFTP_CSV',
-    connectionId: 'my-sftp',
-    name: 'My SFTP',
-    settings: {
-      /* ... */
-    },
-  },
-  logger
-);
-
-const mapper = new UniversalMapper({
-  fields: {
-    /* ... */
-  },
-});
-const client = await createClient({
-  /* ... */
-});
-```
-
-### List and Process Files
-
-```typescript
-// ⚠️ IMPORTANT: StateService requires KV adapter passed to EVERY method call
-// Constructor takes only logger (optional)
-const stateService = new StateService(logger);
-const kvAdapter = new VersoriKVAdapter(openKv());
-const workflowId = 'my-workflow';
-
-const files = await dataSource.listFiles({
-  filePattern: '*.csv'
-});
-
-for (const file of files) {
-  // KV adapter MUST be passed to every method
-  if (await stateService.isFileProcessed(kvAdapter, file.name, workflowId)) continue;
-
-  const content = await dataSource.downloadFile(file.name);
-  const parser = new CSVParserService();
-  const records = await parser.parse(content as string);
-
-  // Process records...
-
-  // KV adapter MUST be passed to every method
-  await stateService.updateSyncState(kvAdapter, [{
-    fileName: file.name,
-    lastModified: new Date().toISOString(),
-    recordCount: records.length
-  }], workflowId);
-}
-```
-
-### Transform and Validate
-
-```typescript
-const validRecords = [];
-const errors = [];
-
-for (const record of records) {
-  // Validate
-  if (!record.sku || !record.quantity) {
-    errors.push({ record, error: 'Missing required fields' });
-    continue;
-  }
-
-  // Transform
-  const result = await mapper.map(record);
-  if (result.success) {
-    validRecords.push(result.data);
-  } else {
-    errors.push({ record, errors: result.errors });
-  }
-}
-```
-
-### Send to Fluent Batch API
-
-```typescript
-const job = await client.createJob({
-  name: 'Inventory Update',
-  retailerId: process.env.FLUENT_RETAILER_ID!,
-});
-
-await client.sendBatch(job.id, {
-  action: 'UPSERT',
-  entityType: 'INVENTORY',
-  entities: transformedRecords,
-});
-
-// Monitor status
-const status = await client.getBatchStatus(job.id, batchId);
-```
-
-### Send via GraphQL Mutation
-
-```typescript
-const mutation = `
-  mutation CreateOrder($input: CreateOrderInput!) {
-    createOrder(input: $input) {
-      id
-      ref
-      status
-    }
-  }
-`;
-
-const result = await client.graphql({ query: mutation, variables: { input: orderData } });
-```
-
-### Extract from Fluent
-
-```typescript
-const query = `
-  query GetOrders($retailerId: ID!, $first: Int!, $after: String) {
-    orders(retailerId: $retailerId, first: $first, after: $after) {
-      edges {
-        node { id ref status customer { email } }
-        cursor
-      }
-      pageInfo { hasNextPage }
-    }
-  }
-`;
-
-const result = await client.graphql({
-  query,
-  variables: { retailerId, first: 100 },
-  pagination: { maxPages: 50 },
-});
-
-const orders = result.data.orders.edges.map(e => e.node);
-```
-
-## Error Handling Patterns
-
-### Record-Level (Skip Invalid)
-
-```typescript
-const validRecords = records.filter(record => {
-  try {
-    validateRecord(record);
-    return true;
-  } catch (error) {
-    logger.warn('Invalid record', { record, error });
-    return false;
-  }
-});
-```
-
-### File-Level (Quarantine)
-
-```typescript
-try {
-  await processFile(file);
-} catch (error) {
-  logger.error(`File failed: ${file.name}`, error);
-  await dataSource.moveFile(file.name, 'errors/');
-}
-```
-
-### Job-Level (Fail Fast)
-
-```typescript
-try {
-  await runConnector();
-} catch (error) {
-  logger.error('Critical failure', error);
-  await notifyAdmins(error);
-  throw error;
-}
-```
-
-## Environment Configuration
-
-```bash
-# Fluent Commerce
-FLUENT_BASE_URL=https://api.fluentcommerce.com
-FLUENT_CLIENT_ID=your-client-id
-FLUENT_CLIENT_SECRET=***SECRET***
-FLUENT_RETAILER_ID=1
-
-# SFTP Source
-SFTP_HOST=sftp.partner.com
-SFTP_USERNAME=user
-SFTP_PASSWORD=***SECRET***
-
-# S3 Source
-S3_BUCKET=data-bucket
-AWS_REGION=us-east-1
-AWS_ACCESS_KEY_ID=***SECRET***
-AWS_SECRET_ACCESS_KEY=***SECRET***
-
-# Connector Settings
-LOG_LEVEL=info
-BATCH_SIZE=100
-STATE_STORAGE=s3
-```
-
-## Performance Tuning
-
-| Scenario                      | Recommended Setting                 | Notes                   |
-| ----------------------------- | ----------------------------------- | ----------------------- |
-| Small files (< 1000 records)  | Batch size: 500                     | Process in single batch |
-| Medium files (1K-10K records) | Batch size: 200-500                 | Balance speed vs memory |
-| Large files (> 10K records)   | Batch size: 100-200, Streaming: Yes | Use checkpoints         |
-| High-frequency updates        | Batch size: 50-100                  | Optimize for latency    |
-| Low-frequency bulk            | Batch size: 500-1000                | Optimize for throughput |
-
-## Testing Checklist
-
-- [ ] Unit tests for parsers
-- [ ] Unit tests for mappers
-- [ ] Unit tests for validators
-- [ ] Integration tests for data sources
-- [ ] Integration tests for Fluent API
-- [ ] End-to-end tests with test data
-- [ ] Performance tests with large files
-- [ ] Error handling tests
-- [ ] State management tests
-- [ ] Recovery from failure tests
-
-## Deployment Checklist
-
-- [ ] Environment variables configured
-- [ ] Secrets stored securely
-- [ ] Logging configured
-- [ ] Monitoring/alerts set up
-- [ ] Health checks implemented
-- [ ] State storage configured
-- [ ] Error notification configured
-- [ ] Backup/recovery plan documented
-- [ ] Performance baselines established
-- [ ] Documentation updated
-
-## Quick Links
-
-- [Full Connector Examples](../examples/)
-- [Common Patterns](../examples/common-patterns.ts)
-- [Universal Mapping Guide](../../../02-CORE-GUIDES/mapping/mapping-readme.md)
-- [Auto-Pagination Guide](../../../02-CORE-GUIDES/auto-pagination/auto-pagination-readme.md)
-- [API Reference](../../../02-CORE-GUIDES/api-reference/api-reference-readme.md)
-- [Getting Started](../../../00-START-HERE/getting-started.md)
-
-## Summary
-
-This reference module provides:
-
-- ✅ Catalog of all connector patterns
-- ✅ Component reference tables
-- ✅ Common code patterns (copy-paste ready)
-- ✅ Error handling strategies
-- ✅ Configuration templates
-- ✅ Performance tuning guidelines
-- ✅ Testing and deployment checklists
-
-Use this module as a quick reference when building connectors.
+# Module 7: Connector Patterns Reference
+
+**Level:** Reference
+**Estimated Time:** As needed
+
+## Overview
+
+This module provides a quick reference catalog of all connector patterns, components, and common code snippets.
+
+## Connector Patterns Catalog
+
+### Pattern 1: SFTP CSV to Fluent Inventory
+
+- **Complexity:** Low
+- **Components:** SftpDataSource, CSVParserService, UniversalMapper, FluentClient
+- **Use Case:** Daily inventory updates from warehouse
+- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
+
+### Pattern 2: S3 Parquet to Fluent Products
+
+- **Complexity:** Medium
+- **Components:** S3DataSource, ParquetParserService, UniversalMapper, FluentClient
+- **Use Case:** Product catalog sync from data lake
+- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
+
+### Pattern 3: SFTP JSON to Fluent Orders
+
+- **Complexity:** Medium
+- **Components:** SftpDataSource, JSONParserService, SchemaValidationService
+- **Use Case:** Order creation from e-commerce exports
+- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
+
+### Pattern 4: S3 CSV with State Management
+
+- **Complexity:** High
+- **Components:** S3DataSource, CSVParserService, StateService, FluentClient
+- **Use Case:** Large file processing with recovery
+- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
+
+### Pattern 5: Multi-Source Aggregation
+
+- **Complexity:** High
+- **Components:** SftpDataSource, S3DataSource, Multiple parsers, Reconciliation
+- **Use Case:** Aggregate inventory from multiple warehouses
+- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
+
+### Pattern 6: Bidirectional Sync
+
+- **Complexity:** Medium
+- **Components:** FluentClient, ExtractionOrchestrator, SftpDataSource
+- **Use Case:** Export Fluent data to external systems
+- **Full Example:** [See Common Patterns](../examples/common-patterns.ts)
+
+## Component Reference
+
+### Data Sources
+
+| Component             | Protocol | Formats            | Key Methods                                                        |
+| --------------------- | -------- | ------------------ | ------------------------------------------------------------------ |
+| `SftpDataSource`      | SFTP     | CSV, JSON, XML     | `listFiles()`, `downloadFile()`, `uploadFile()`, `moveFile()`   |
+| `S3DataSource`        | AWS S3   | CSV, JSON, Parquet | `listFiles()`, `downloadFile()`, `uploadFile()`, `getFileStream()` |
+| `InventoryDataSource` | Custom   | Any                | Extend for custom sources                                          |
+
+### Parsers
+
+| Component              | Format  | Streaming | Key Options                                         |
+| ---------------------- | ------- | --------- | --------------------------------------------------- |
+| `CSVParserService`     | CSV     | Yes       | `delimiter`, `headers`, `skipEmptyLines`, `maxRows` |
+| `ParquetParserService` | Parquet | No        | `columns`, `maxRows`                                |
+| `XMLParserService`     | XML     | Yes       | `pathMapping`, `rootElement`                        |
+| `JSONParserService`    | JSON    | Yes       | `jsonFormat`, `validate`                            |
+
+### Transformation
+
+| Component           | Purpose              | Key Methods                |
+| ------------------- | -------------------- | -------------------------- |
+| `UniversalMapper`   | Field transformation | `map()`, `mapBatch()`      |
+| `SchemaValidationService` | GraphQL schema validation | `validateMutation()`, `validatePayload()` |
+| `ExtractionOrchestrator` | High-level data extraction | `extract()`, with auto-pagination |
+
+### State Management
+
+⚠️ **IMPORTANT:** StateService requires KV adapter passed to EVERY method call. The constructor takes only logger (optional).
+
+| Component                | Storage        | Use Case                     |
+| ------------------------ | -------------- | ---------------------------- |
+| `StateService`           | KV adapter-based | File tracking, deduplication, checkpointing |
+| `VersoriKVAdapter`       | Versori KV     | Versori platform state       |
+| `VersoriFileTracker`     | Versori KV     | Simple file tracking         |
+
+#### VersoriFileTracker Usage
+
+**Simple file tracking without StateService complexity:**
+
+```typescript
+import { schedule, fn } from '@versori/run';
+import { VersoriFileTracker, SftpDataSource } from '@fluentcommerce/fc-connect-sdk';
+
+export const inventorySync = schedule("inventory-sync", "0 * * * *")
+  .then(fn("process-files", async (ctx) => {
+    const { log } = ctx;
+
+    // Create file tracker (simpler than StateService)
+    const fileTracker = new VersoriFileTracker(openKv(), 'inventory-sync');
+
+    // List files from SFTP
+    const sftp = new SftpDataSource(sftpConfig, log);
+    const files = await sftp.listFiles();
+
+    for (const file of files) {
+      // Skip already processed files
+      if (await fileTracker.wasFileProcessed(file.name)) {
+        log.info(`Skipping already processed file: ${file.name}`);
+        continue;
+      }
+
+      try {
+        // Download and process file
+        const content = await sftp.downloadFile(file.path);
+        const records = await processFile(content);
+
+        // Mark as processed with metadata
+        await fileTracker.markFileProcessed(file.name, {
+          recordCount: records.length
+        });
+
+        log.info(`Processed ${file.name}: ${records.length} records`);
+
+        // Optional: Track last processed file
+        await fileTracker.setLastProcessedFile(file.name);
+
+      } catch (error) {
+        log.error(`Failed to process ${file.name}:`, error);
+        // Don't mark as processed - will retry next run
+      }
+    }
+
+    // Get last processed file for reporting
+    const lastFile = await fileTracker.getLastProcessedFile();
+    return { success: true, lastProcessedFile: lastFile };
+  }));
+```
+
+**VersoriFileTracker API:**
+
+| Method | Parameters | Returns | Description |
+|--------|-----------|---------|-------------|
+| `wasFileProcessed(fileName)` | `fileName: string` | `Promise<boolean>` | Check if file was already processed |
+| `markFileProcessed(fileName, metadata?)` | `fileName: string`, `metadata?: { recordCount?: number }` | `Promise<void>` | Mark file as processed with optional metadata |
+| `getLastProcessedFile()` | - | `Promise<string \| null>` | Get the last processed file name |
+| `setLastProcessedFile(fileName)` | `fileName: string` | `Promise<void>` | Set the last processed file |
+
+**When to use VersoriFileTracker vs StateService:**
+
+| Feature | VersoriFileTracker | StateService |
+|---------|-------------------|--------------|
+| **Simplicity** | ✅ Simple 4-method API | Complex with adapter pattern |
+| **File Tracking** | ✅ Built-in | Requires adapter + workflow ID |
+| **Metadata** | ✅ Basic (recordCount) | ✅ Rich metadata support |
+| **Checkpointing** | ❌ Not supported | ✅ Record-level checkpointing |
+| **Multi-workflow** | ✅ Via prefix | ✅ Via workflow ID |
+| **Best For** | Simple file-based ingestion | Complex stateful workflows |
+
+**Recommendation:** Use `VersoriFileTracker` for simple file processing workflows on Versori. Use `StateService` for complex stateful operations or cross-runtime support.
+
+## Common Code Patterns
+
+### Initialize Connector
+
+```typescript
+import {
+  createClient,
+  SftpDataSource,
+  UniversalMapper,
+  StateService,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+
+const logger = toStructuredLogger(createConsoleLogger(), {
+  service: 'connector-name',
+  module: 'workflow-name'
+});
+
+// StateService constructor takes only logger (optional)
+const stateService = new StateService(logger);
+
+const dataSource = new SftpDataSource(
+  {
+    type: 'SFTP_CSV',
+    connectionId: 'my-sftp',
+    name: 'My SFTP',
+    settings: {
+      /* ... */
+    },
+  },
+  logger
+);
+
+const mapper = new UniversalMapper({
+  fields: {
+    /* ... */
+  },
+});
+const client = await createClient({
+  /* ... */
+});
+```
+
+### List and Process Files
+
+```typescript
+// ⚠️ IMPORTANT: StateService requires KV adapter passed to EVERY method call
+// Constructor takes only logger (optional)
+const stateService = new StateService(logger);
+const kvAdapter = new VersoriKVAdapter(openKv());
+const workflowId = 'my-workflow';
+
+const files = await dataSource.listFiles({
+  filePattern: '*.csv'
+});
+
+for (const file of files) {
+  // KV adapter MUST be passed to every method
+  if (await stateService.isFileProcessed(kvAdapter, file.name, workflowId)) continue;
+
+  const content = await dataSource.downloadFile(file.name);
+  const parser = new CSVParserService();
+  const records = await parser.parse(content as string);
+
+  // Process records...
+
+  // KV adapter MUST be passed to every method
+  await stateService.updateSyncState(kvAdapter, [{
+    fileName: file.name,
+    lastModified: new Date().toISOString(),
+    recordCount: records.length
+  }], workflowId);
+}
+```
+
+### Transform and Validate
+
+```typescript
+const validRecords = [];
+const errors = [];
+
+for (const record of records) {
+  // Validate
+  if (!record.sku || !record.quantity) {
+    errors.push({ record, error: 'Missing required fields' });
+    continue;
+  }
+
+  // Transform
+  const result = await mapper.map(record);
+  if (result.success) {
+    validRecords.push(result.data);
+  } else {
+    errors.push({ record, errors: result.errors });
+  }
+}
+```
+
+### Send to Fluent Batch API
+
+```typescript
+const job = await client.createJob({
+  name: 'Inventory Update',
+  retailerId: process.env.FLUENT_RETAILER_ID!,
+});
+
+await client.sendBatch(job.id, {
+  action: 'UPSERT',
+  entityType: 'INVENTORY',
+  entities: transformedRecords,
+});
+
+// Monitor status
+const status = await client.getBatchStatus(job.id, batchId);
+```
+
+### Send via GraphQL Mutation
+
+```typescript
+const mutation = `
+  mutation CreateOrder($input: CreateOrderInput!) {
+    createOrder(input: $input) {
+      id
+      ref
+      status
+    }
+  }
+`;
+
+const result = await client.graphql({ query: mutation, variables: { input: orderData } });
+```
+
+### Extract from Fluent
+
+```typescript
+const query = `
+  query GetOrders($retailerId: ID!, $first: Int!, $after: String) {
+    orders(retailerId: $retailerId, first: $first, after: $after) {
+      edges {
+        node { id ref status customer { email } }
+        cursor
+      }
+      pageInfo { hasNextPage }
+    }
+  }
+`;
+
+const result = await client.graphql({
+  query,
+  variables: { retailerId, first: 100 },
+  pagination: { maxPages: 50 },
+});
+
+const orders = result.data.orders.edges.map(e => e.node);
+```
+
+## Error Handling Patterns
+
+### Record-Level (Skip Invalid)
+
+```typescript
+const validRecords = records.filter(record => {
+  try {
+    validateRecord(record);
+    return true;
+  } catch (error) {
+    logger.warn('Invalid record', { record, error });
+    return false;
+  }
+});
+```
+
+### File-Level (Quarantine)
+
+```typescript
+try {
+  await processFile(file);
+} catch (error) {
+  logger.error(`File failed: ${file.name}`, error);
+  await dataSource.moveFile(file.name, 'errors/');
+}
+```
+
+### Job-Level (Fail Fast)
+
+```typescript
+try {
+  await runConnector();
+} catch (error) {
+  logger.error('Critical failure', error);
+  await notifyAdmins(error);
+  throw error;
+}
+```
+
+## Environment Configuration
+
+```bash
+# Fluent Commerce
+FLUENT_BASE_URL=https://api.fluentcommerce.com
+FLUENT_CLIENT_ID=your-client-id
+FLUENT_CLIENT_SECRET=***SECRET***
+FLUENT_RETAILER_ID=1
+
+# SFTP Source
+SFTP_HOST=sftp.partner.com
+SFTP_USERNAME=user
+SFTP_PASSWORD=***SECRET***
+
+# S3 Source
+S3_BUCKET=data-bucket
+AWS_REGION=us-east-1
+AWS_ACCESS_KEY_ID=***SECRET***
+AWS_SECRET_ACCESS_KEY=***SECRET***
+
+# Connector Settings
+LOG_LEVEL=info
+BATCH_SIZE=100
+STATE_STORAGE=s3
+```
+
+## Performance Tuning
+
+| Scenario                      | Recommended Setting                 | Notes                   |
+| ----------------------------- | ----------------------------------- | ----------------------- |
+| Small files (< 1000 records)  | Batch size: 500                     | Process in single batch |
+| Medium files (1K-10K records) | Batch size: 200-500                 | Balance speed vs memory |
+| Large files (> 10K records)   | Batch size: 100-200, Streaming: Yes | Use checkpoints         |
+| High-frequency updates        | Batch size: 50-100                  | Optimize for latency    |
+| Low-frequency bulk            | Batch size: 500-1000                | Optimize for throughput |
+
+## Testing Checklist
+
+- [ ] Unit tests for parsers
+- [ ] Unit tests for mappers
+- [ ] Unit tests for validators
+- [ ] Integration tests for data sources
+- [ ] Integration tests for Fluent API
+- [ ] End-to-end tests with test data
+- [ ] Performance tests with large files
+- [ ] Error handling tests
+- [ ] State management tests
+- [ ] Recovery from failure tests
+
+## Deployment Checklist
+
+- [ ] Environment variables configured
+- [ ] Secrets stored securely
+- [ ] Logging configured
+- [ ] Monitoring/alerts set up
+- [ ] Health checks implemented
+- [ ] State storage configured
+- [ ] Error notification configured
+- [ ] Backup/recovery plan documented
+- [ ] Performance baselines established
+- [ ] Documentation updated
+
+## Quick Links
+
+- [Full Connector Examples](../examples/)
+- [Common Patterns](../examples/common-patterns.ts)
+- [Universal Mapping Guide](../../../02-CORE-GUIDES/mapping/mapping-readme.md)
+- [Auto-Pagination Guide](../../../02-CORE-GUIDES/auto-pagination/auto-pagination-readme.md)
+- [API Reference](../../../02-CORE-GUIDES/api-reference/api-reference-readme.md)
+- [Getting Started](../../../00-START-HERE/getting-started.md)
+
+## Summary
+
+This reference module provides:
+
+- ✅ Catalog of all connector patterns
+- ✅ Component reference tables
+- ✅ Common code patterns (copy-paste ready)
+- ✅ Error handling strategies
+- ✅ Configuration templates
+- ✅ Performance tuning guidelines
+- ✅ Testing and deployment checklists
+
+Use this module as a quick reference when building connectors.