@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

package/docs/03-PATTERN-GUIDES/connector-scenarios/modules/connector-scenarios-05-bidirectional-sync.md

@@ -1,233 +1,233 @@
-# Module 5: Bidirectional Sync
-
-**Level:** Advanced
-**Estimated Time:** 45 minutes
-
-## Overview
-
-This module covers bidirectional sync patterns - extracting data from Fluent Commerce and exporting to external systems. You'll learn how to build reverse data flows for reporting, analytics, and system integration.
-
-## Learning Objectives
-
-By the end of this module, you will:
-
-- ✅ Extract data from Fluent Commerce using GraphQL queries
-- ✅ Transform Fluent data for external systems
-- ✅ Export to multiple formats (JSON, CSV, JSONL)
-- ✅ Upload to SFTP and S3
-- ✅ Implement extract-transform-load (ETL) patterns
-- ✅ Generate export summaries and manifests
-
-## Scenario 6: Fluent to SFTP Export
-
-**Goal:** Extract orders, inventory, and products from Fluent Commerce and export to SFTP for external systems.
-
-**Complexity:** Medium
-**Components:** FluentClient, ExtractionOrchestrator, SftpDataSource
-**Estimated Lines:** ~300 lines
-
-### Complete Implementation
-
-See full implementation in [connector-scenarios.md lines 1244-1563] - extract orders, inventory positions, and products from Fluent, transform for external systems, and write to SFTP as JSON/CSV/JSONL.
-
-### Key Patterns
-
-**1. Pagination for Large Datasets**
-
-```typescript
-const orderQuery = `
-  query GetOrders($retailerId: ID!, $first: Int!, $after: String, $from: DateTime) {
-    orders(retailerId: $retailerId, first: $first, after: $after, createdFrom: $from) {
-      edges {
-        node { id ref status customer { email firstName lastName } }
-        cursor
-      }
-      pageInfo { hasNextPage }
-    }
-  }
-`;
-
-// Use auto-pagination
-const ordersResult = await fluentClient.graphql({
-  query: orderQuery,
-  variables: { retailerId, first: 100, from: startDate },
-  pagination: { maxPages: 100 },
-});
-```
-
-**2. Data Transformation for Export**
-
-```typescript
-// Transform Fluent order to external format
-const exportOrders = orders.map(order => ({
-  orderId: order.ref,
-  fluentId: order.id,
-  status: order.status,
-  customer: {
-    email: order.customer.email,
-    name: `${order.customer.firstName} ${order.customer.lastName}`,
-  },
-  lineItems: order.items.map(item => ({
-    sku: item.productRef,
-    quantity: item.quantity,
-    unitPrice: item.price,
-  })),
-}));
-```
-
-**3. Multiple Export Formats**
-
-```typescript
-// JSON export
-await sftpExport.uploadFile(
-  `/exports/orders/orders_${date}.json`,
-  JSON.stringify(exportOrders)
-);
-
-// CSV export
-const csvContent = [
-  csvHeaders.join(','),
-  ...records.map(row => csvHeaders.map(h => row[h]).join(',')),
-].join('\n');
-
-await sftpExport.uploadFile(`/exports/inventory/inventory_${date}.csv`, csvContent);
-
-// JSONL (JSON Lines) export
-const jsonlContent = products.map(product => JSON.stringify(product)).join('\n');
-
-await sftpExport.uploadFile(`/exports/products/products_${date}.jsonl`, jsonlContent);
-```
-
-**4. Export Summary**
-
-```typescript
-const summary = {
-  exportDate: new Date().toISOString().split('T')[0],
-  exportedAt: new Date().toISOString(),
-  statistics: {
-    orders: orders.length,
-    inventoryPositions: inventory.length,
-    products: products.length,
-  },
-  files: [
-    `/exports/orders/orders_${date}.json`,
-    `/exports/inventory/inventory_${date}.csv`,
-    `/exports/products/products_${date}.jsonl`,
-  ],
-};
-
-await sftpExport.uploadFile(`/exports/summary_${date}.json`, JSON.stringify(summary));
-```
-
-## Extraction Best Practices
-
-### 1. Use Incremental Exports
-
-**Don't export everything every time:**
-
-```typescript
-// Filter by date range
-const from = lastExportDate || getYesterday();
-const to = new Date().toISOString();
-
-const query = `
-  query GetRecentOrders($from: DateTime!, $to: DateTime!) {
-    orders(createdFrom: $from, createdTo: $to) {
-      edges { node { ... } }
-    }
-  }
-`;
-```
-
-### 2. Handle Large Datasets with Pagination
-
-**Use auto-pagination with safety limits:**
-
-```typescript
-const result = await fluentClient.graphql({
-  query,
-  variables: { first: 500 },
-  pagination: {
-    maxPages: 200,
-    maxRecords: 100000,
-    onProgress: (page, total) => {
-      logger.info(`Extracted page ${page}: ${total} records`);
-    },
-  },
-});
-```
-
-### 3. Transform for Target System
-
-**Map Fluent schema to external schema:**
-
-```typescript
-// Fluent inventory → External inventory feed
-const externalFormat = inventoryPositions.map(pos => ({
-  sku: pos.productRef,
-  location: pos.locationRef,
-  totalQty: pos.qty,
-  availableQty: pos.availableQty,
-  reservedQty: pos.reservedQty,
-  status: pos.status,
-  lastUpdated: pos.updatedOn,
-}));
-```
-
-### 4. Generate Manifests
-
-**Create manifest files for consumers:**
-
-```typescript
-const manifest = {
-  exportId: generateUUID(),
-  exportDate: new Date().toISOString(),
-  dataType: 'orders',
-  recordCount: orders.length,
-  fileFormat: 'json',
-  filePath: `/exports/orders/orders_${date}.json`,
-  checksum: calculateChecksum(content),
-  schema: {
-    version: '1.0',
-    fields: ['orderId', 'status', 'customer', 'lineItems'],
-  },
-};
-```
-
-## Scheduling Exports
-
-### Cron-Based Scheduling
-
-```typescript
-import * as cron from 'node-cron';
-
-// Daily export at 2 AM
-cron.schedule('0 2 * * *', async () => {
-  logger.info('Starting scheduled export');
-  await exportFluentDataToSFTP();
-});
-
-// Hourly inventory export
-cron.schedule('0 * * * *', async () => {
-  await exportInventorySnapshot();
-});
-```
-
-### Event-Driven Exports
-
-```typescript
-// Example event-driven hooks can be added in your runtime if available
-```
-
-## Key Takeaways
-
-- 🎯 Extraction uses GraphQL queries with auto-pagination
-- 🎯 Transform Fluent schema to match external system requirements
-- 🎯 Support multiple export formats (JSON, CSV, JSONL)
-- 🎯 Use incremental exports (date filters) for efficiency
-- 🎯 Generate manifests for data consumers
-- 🎯 Schedule exports based on business requirements
-
-## Next Steps
-
-Continue to [Module 6: Production Patterns](./connector-scenarios-06-production-patterns.md) to learn error handling, monitoring, and deployment best practices.
+# Module 5: Bidirectional Sync
+
+**Level:** Advanced
+**Estimated Time:** 45 minutes
+
+## Overview
+
+This module covers bidirectional sync patterns - extracting data from Fluent Commerce and exporting to external systems. You'll learn how to build reverse data flows for reporting, analytics, and system integration.
+
+## Learning Objectives
+
+By the end of this module, you will:
+
+- ✅ Extract data from Fluent Commerce using GraphQL queries
+- ✅ Transform Fluent data for external systems
+- ✅ Export to multiple formats (JSON, CSV, JSONL)
+- ✅ Upload to SFTP and S3
+- ✅ Implement extract-transform-load (ETL) patterns
+- ✅ Generate export summaries and manifests
+
+## Scenario 6: Fluent to SFTP Export
+
+**Goal:** Extract orders, inventory, and products from Fluent Commerce and export to SFTP for external systems.
+
+**Complexity:** Medium
+**Components:** FluentClient, ExtractionOrchestrator, SftpDataSource
+**Estimated Lines:** ~300 lines
+
+### Complete Implementation
+
+See full implementation in [connector-scenarios.md lines 1244-1563] - extract orders, inventory positions, and products from Fluent, transform for external systems, and write to SFTP as JSON/CSV/JSONL.
+
+### Key Patterns
+
+**1. Pagination for Large Datasets**
+
+```typescript
+const orderQuery = `
+  query GetOrders($retailerId: ID!, $first: Int!, $after: String, $from: DateTime) {
+    orders(retailerId: $retailerId, first: $first, after: $after, createdFrom: $from) {
+      edges {
+        node { id ref status customer { email firstName lastName } }
+        cursor
+      }
+      pageInfo { hasNextPage }
+    }
+  }
+`;
+
+// Use auto-pagination
+const ordersResult = await fluentClient.graphql({
+  query: orderQuery,
+  variables: { retailerId, first: 100, from: startDate },
+  pagination: { maxPages: 100 },
+});
+```
+
+**2. Data Transformation for Export**
+
+```typescript
+// Transform Fluent order to external format
+const exportOrders = orders.map(order => ({
+  orderId: order.ref,
+  fluentId: order.id,
+  status: order.status,
+  customer: {
+    email: order.customer.email,
+    name: `${order.customer.firstName} ${order.customer.lastName}`,
+  },
+  lineItems: order.items.map(item => ({
+    sku: item.productRef,
+    quantity: item.quantity,
+    unitPrice: item.price,
+  })),
+}));
+```
+
+**3. Multiple Export Formats**
+
+```typescript
+// JSON export
+await sftpExport.uploadFile(
+  `/exports/orders/orders_${date}.json`,
+  JSON.stringify(exportOrders)
+);
+
+// CSV export
+const csvContent = [
+  csvHeaders.join(','),
+  ...records.map(row => csvHeaders.map(h => row[h]).join(',')),
+].join('\n');
+
+await sftpExport.uploadFile(`/exports/inventory/inventory_${date}.csv`, csvContent);
+
+// JSONL (JSON Lines) export
+const jsonlContent = products.map(product => JSON.stringify(product)).join('\n');
+
+await sftpExport.uploadFile(`/exports/products/products_${date}.jsonl`, jsonlContent);
+```
+
+**4. Export Summary**
+
+```typescript
+const summary = {
+  exportDate: new Date().toISOString().split('T')[0],
+  exportedAt: new Date().toISOString(),
+  statistics: {
+    orders: orders.length,
+    inventoryPositions: inventory.length,
+    products: products.length,
+  },
+  files: [
+    `/exports/orders/orders_${date}.json`,
+    `/exports/inventory/inventory_${date}.csv`,
+    `/exports/products/products_${date}.jsonl`,
+  ],
+};
+
+await sftpExport.uploadFile(`/exports/summary_${date}.json`, JSON.stringify(summary));
+```
+
+## Extraction Best Practices
+
+### 1. Use Incremental Exports
+
+**Don't export everything every time:**
+
+```typescript
+// Filter by date range
+const from = lastExportDate || getYesterday();
+const to = new Date().toISOString();
+
+const query = `
+  query GetRecentOrders($from: DateTime!, $to: DateTime!) {
+    orders(createdFrom: $from, createdTo: $to) {
+      edges { node { ... } }
+    }
+  }
+`;
+```
+
+### 2. Handle Large Datasets with Pagination
+
+**Use auto-pagination with safety limits:**
+
+```typescript
+const result = await fluentClient.graphql({
+  query,
+  variables: { first: 500 },
+  pagination: {
+    maxPages: 200,
+    maxRecords: 100000,
+    onProgress: (page, total) => {
+      logger.info(`Extracted page ${page}: ${total} records`);
+    },
+  },
+});
+```
+
+### 3. Transform for Target System
+
+**Map Fluent schema to external schema:**
+
+```typescript
+// Fluent inventory → External inventory feed
+const externalFormat = inventoryPositions.map(pos => ({
+  sku: pos.productRef,
+  location: pos.locationRef,
+  totalQty: pos.qty,
+  availableQty: pos.availableQty,
+  reservedQty: pos.reservedQty,
+  status: pos.status,
+  lastUpdated: pos.updatedOn,
+}));
+```
+
+### 4. Generate Manifests
+
+**Create manifest files for consumers:**
+
+```typescript
+const manifest = {
+  exportId: generateUUID(),
+  exportDate: new Date().toISOString(),
+  dataType: 'orders',
+  recordCount: orders.length,
+  fileFormat: 'json',
+  filePath: `/exports/orders/orders_${date}.json`,
+  checksum: calculateChecksum(content),
+  schema: {
+    version: '1.0',
+    fields: ['orderId', 'status', 'customer', 'lineItems'],
+  },
+};
+```
+
+## Scheduling Exports
+
+### Cron-Based Scheduling
+
+```typescript
+import * as cron from 'node-cron';
+
+// Daily export at 2 AM
+cron.schedule('0 2 * * *', async () => {
+  logger.info('Starting scheduled export');
+  await exportFluentDataToSFTP();
+});
+
+// Hourly inventory export
+cron.schedule('0 * * * *', async () => {
+  await exportInventorySnapshot();
+});
+```
+
+### Event-Driven Exports
+
+```typescript
+// Example event-driven hooks can be added in your runtime if available
+```
+
+## Key Takeaways
+
+- 🎯 Extraction uses GraphQL queries with auto-pagination
+- 🎯 Transform Fluent schema to match external system requirements
+- 🎯 Support multiple export formats (JSON, CSV, JSONL)
+- 🎯 Use incremental exports (date filters) for efficiency
+- 🎯 Generate manifests for data consumers
+- 🎯 Schedule exports based on business requirements
+
+## Next Steps
+
+Continue to [Module 6: Production Patterns](./connector-scenarios-06-production-patterns.md) to learn error handling, monitoring, and deployment best practices.