@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

package/docs/02-CORE-GUIDES/api-reference/modules/api-reference-10-extraction.md

@@ -1,557 +1,557 @@
-# Module 10: Extraction
-
-**Level:** Intermediate to Advanced
-**Category:** Data Export & Integration
-
-## Overview
-
-Extraction involves querying data from Fluent Commerce GraphQL API and exporting it to external systems in various formats (CSV, JSON, XML, Parquet). The SDK provides `ExtractionOrchestrator` for high-level extraction workflows with auto-pagination, format conversion, and S3 output.
-
-## Table of Contents
-
-- [ExtractionOrchestrator Service](#extractionorchestrator-service)
-- [Format Conversion](#format-conversion)
-- [Partitioning Strategies](#partitioning-strategies)
-- [S3 Output](#s3-output)
-- [Auto-Pagination](#auto-pagination)
-- [Data Enrichment](#data-enrichment)
-- [Performance Optimization](#performance-optimization)
-- [Common Patterns](#common-patterns)
-- [Error Handling](#error-handling)
-- [Troubleshooting](#troubleshooting)
-- [See Also](#see-also)
-
----
-
-## ExtractionOrchestrator Service
-
-High-level service for end-to-end extraction workflows:
-
-```typescript
-import { createClient, ExtractionOrchestrator } from '@fluentcommerce/fc-connect-sdk';
-
-const client = await createClient({ config });
-const orchestrator = new ExtractionOrchestrator(client, console);
-
-// Extract virtual positions to CSV on S3
-const result = await orchestrator.extractToS3({
-  query: `
-    query GetPositions($first: Int, $after: String) {
-      virtualPositions(first: $first, after: $after) {
-        edges {
-          node {
-            id
-            ref
-            productRef
-            groupRef
-            quantity
-            status
-          }
-        }
-        pageInfo {
-          hasNextPage
-          endCursor
-        }
-      }
-    }
-  `,
-  connectionName: 'virtualPositions',  // GraphQL connection name
-  outputFormat: 'csv',                // csv | json | xml | parquet
-  s3Config: {
-    bucket: 'my-exports',
-    keyPrefix: 'fluent/virtual-positions/',
-    filename: `positions-${Date.now()}.csv`
-  },
-  pagination: {
-    pageSize: 250,
-    maxPages: Infinity
-  }
-});
-
-console.log(`Extracted ${result.recordCount} records to ${result.s3Key}`);
-```
-
-**Key Features:**
-- Automatic pagination through all results
-- Format conversion (JSON → CSV/XML/Parquet)
-- Direct S3 upload (no local disk)
-- Progress tracking and logging
-- Error recovery and retry
-
----
-
-## Format Conversion
-
-The SDK supports multiple output formats:
-
-### CSV Export
-
-```typescript
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'inventoryPositions',
-  outputFormat: 'csv',
-  s3Config: {
-    bucket: 'exports',
-    keyPrefix: 'inventory/',
-    filename: 'positions.csv'
-  },
-  csvOptions: {
-    headers: true,            // Include header row
-    delimiter: ',',           // Column separator
-    quote: '"',               // Quote character
-    includeNull: false       // Skip null values
-  }
-});
-```
-
-**Output:**
-```csv
-id,ref,productRef,locationRef,onHand,status
-123,POS-001,SKU-100,LOC-A,50,ACTIVE
-124,POS-002,SKU-101,LOC-B,25,ACTIVE
-```
-
-### JSON Export
-
-```typescript
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'orders',
-  outputFormat: 'json',
-  s3Config: {
-    bucket: 'exports',
-    keyPrefix: 'orders/',
-    filename: 'orders.json'
-  },
-  jsonOptions: {
-    pretty: true,           // Pretty-print JSON
-    arrayFormat: true       // Output as JSON array
-  }
-});
-```
-
-**Output:**
-```json
-[
-  {
-    "id": "123",
-    "ref": "ORD-001",
-    "status": "BOOKED",
-    "totalPrice": 99.99
-  },
-  {
-    "id": "124",
-    "ref": "ORD-002",
-    "status": "BOOKED",
-    "totalPrice": 149.99
-  }
-]
-```
-
-### XML Export
-
-```typescript
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'products',
-  outputFormat: 'xml',
-  s3Config: {
-    bucket: 'exports',
-    keyPrefix: 'products/',
-    filename: 'products.xml'
-  },
-  xmlOptions: {
-    rootElement: 'Products',    // Root element name
-    itemElement: 'Product',     // Individual record element
-    declaration: true          // Include <?xml declaration
-  }
-});
-```
-
-**Output:**
-```xml
-<?xml version="1.0" encoding="UTF-8"?>
-<Products>
-  <Product>
-    <id>123</id>
-    <ref>PROD-001</ref>
-    <name>Widget</name>
-    <status>ACTIVE</status>
-  </Product>
-  <Product>
-    <id>124</id>
-    <ref>PROD-002</ref>
-    <name>Gadget</name>
-    <status>ACTIVE</status>
-  </Product>
-</Products>
-```
-
-### Parquet Export
-
-```typescript
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'virtualPositions',
-  outputFormat: 'parquet',
-  s3Config: {
-    bucket: 'datalake',
-    keyPrefix: 'fluent/positions/date=2025-01-15/',
-    filename: 'positions.parquet'
-  },
-  parquetOptions: {
-    compression: 'SNAPPY',    // SNAPPY | GZIP | NONE
-    rowGroupSize: 5000       // Records per row group
-  }
-});
-```
-
----
-
-## Partitioning Strategies
-
-For large datasets, partition exports by date, location, or other dimensions:
-
-### Date Partitioning
-
-```typescript
-// Partition by extraction date (Hive-style)
-const today = new Date().toISOString().split('T')[0];
-
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'orders',
-  outputFormat: 'parquet',
-  s3Config: {
-    bucket: 'datalake',
-    keyPrefix: `fluent/orders/date=${today}/`,  // date=2025-01-15/
-    filename: `orders-${Date.now()}.parquet`
-  }
-});
-```
-
-### Multi-Dimensional Partitioning
-
-```typescript
-// Partition by retailer + date
-const retailerId = process.env.FLUENT_RETAILER_ID;
-const today = new Date().toISOString().split('T')[0];
-
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'virtualPositions',
-  outputFormat: 'parquet',
-  s3Config: {
-    bucket: 'datalake',
-    keyPrefix: `fluent/positions/retailer=${retailerId}/date=${today}/`,
-    filename: `positions-${Date.now()}.parquet`
-  }
-});
-```
-
----
-
-## S3 Output
-
-### Direct S3 Upload
-
-The ExtractionOrchestrator streams directly to S3 without local disk:
-
-```typescript
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'inventoryPositions',
-  outputFormat: 'csv',
-  s3Config: {
-    bucket: 'my-bucket',
-    keyPrefix: 'exports/inventory/',
-    filename: 'positions.csv',
-    region: 'us-east-1',        // AWS region
-    acl: 'private'              // S3 ACL
-  },
-  awsCredentials: {
-    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
-  }
-});
-```
-
-### S3 Path Construction
-
-```typescript
-// Generates: s3://my-bucket/exports/inventory/positions.csv
-const s3Path = `s3://${s3Config.bucket}/${s3Config.keyPrefix}${s3Config.filename}`;
-```
-
-### Custom S3 Configuration
-
-```typescript
-import { S3DataSource } from '@fluentcommerce/fc-connect-sdk';
-
-const s3 = new S3DataSource({
-  type: 'S3_CSV',
-  settings: {
-    bucket: 'my-bucket',
-    region: 'us-west-2',
-    credentials: {
-      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
-      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
-    }
-  }
-});
-
-// Manual upload
-await s3.writeFile('exports/data.csv', csvContent);
-```
-
----
-
-## Auto-Pagination
-
-ExtractionOrchestrator handles pagination automatically:
-
-```typescript
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'virtualPositions',
-  outputFormat: 'csv',
-  s3Config: { /* ... */ },
-  pagination: {
-    pageSize: 250,          // Records per page
-    maxPages: Infinity,     // Extract all pages
-    delayMs: 500           // Delay between pages (rate limiting)
-  }
-});
-```
-
-**Under the hood:**
-1. Fetches first page (250 records)
-2. Checks `pageInfo.hasNextPage`
-3. If true, fetches next page with `endCursor`
-4. Repeats until `hasNextPage: false`
-5. Streams all records to S3
-
----
-
-## Data Enrichment
-
-Transform records during extraction:
-
-```typescript
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'orders',
-  outputFormat: 'csv',
-  s3Config: { /* ... */ },
-  transformFn: (record) => ({
-    ...record,
-    extractedAt: new Date().toISOString(),
-    totalPriceUSD: record.totalPrice,
-    currency: 'USD'
-  })
-});
-```
-
-**Use Cases:**
-- Add extraction timestamp
-- Compute derived fields
-- Normalize data formats
-- Filter sensitive fields
-
----
-
-## Performance Optimization
-
-### Parallel Extractions
-
-Extract multiple entities in parallel:
-
-```typescript
-await Promise.all([
-  orchestrator.extractToS3({ query: positionsQuery, connectionName: 'inventoryPositions', /* ... */ }),
-  orchestrator.extractToS3({ query: ordersQuery, connectionName: 'orders', /* ... */ }),
-  orchestrator.extractToS3({ query: productsQuery, connectionName: 'products', /* ... */ })
-]);
-```
-
-### Streaming Large Datasets
-
-For 100K+ records, use streaming:
-
-```typescript
-// ExtractionOrchestrator streams by default (no memory accumulation)
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'virtualPositions',
-  outputFormat: 'parquet',
-  s3Config: { /* ... */ },
-  pagination: {
-    pageSize: 500,  // Larger pages for throughput
-    delayMs: 0     // No delay (maximize speed)
-  }
-});
-```
-
-### Compression
-
-Reduce S3 storage costs with compression:
-
-```typescript
-// Parquet with SNAPPY compression
-await orchestrator.extractToS3({
-  query,
-  connectionName: 'orders',
-  outputFormat: 'parquet',
-  s3Config: { /* ... */ },
-  parquetOptions: {
-    compression: 'SNAPPY'  // 10-20x size reduction
-  }
-});
-```
-
----
-
-## Common Patterns
-
-### Pattern 1: Daily Extract Job
-
-```typescript
-import { schedule } from '@versori/run';
-import { createClient, ExtractionOrchestrator } from '@fluentcommerce/fc-connect-sdk';
-
-export const dailyExtract = schedule('daily-extract', '0 2 * * *', async (ctx) => {
-  const { log } = ctx;
-  const client = await createClient({ ...ctx, log });
-  const orchestrator = new ExtractionOrchestrator(client, log);
-
-  const today = new Date().toISOString().split('T')[0];
-
-  await orchestrator.extractToS3({
-    query: virtualPositionsQuery,
-    connectionName: 'virtualPositions',
-    outputFormat: 'parquet',
-    s3Config: {
-      bucket: 'datalake',
-      keyPrefix: `fluent/positions/date=${today}/`,
-      filename: `positions-${Date.now()}.parquet`
-    }
-  });
-
-  log.info('Daily extract complete');
-});
-```
-
-### Pattern 2: Multi-Format Export
-
-```typescript
-async function exportAllFormats(client, query) {
-  const orchestrator = new ExtractionOrchestrator(client, console);
-
-  await Promise.all([
-    // CSV for BI tools
-    orchestrator.extractToS3({
-      query,
-      connectionName: 'orders',
-      outputFormat: 'csv',
-      s3Config: { bucket: 'exports', keyPrefix: 'csv/', filename: 'orders.csv' }
-    }),
-
-    // JSON for APIs
-    orchestrator.extractToS3({
-      query,
-      connectionName: 'orders',
-      outputFormat: 'json',
-      s3Config: { bucket: 'exports', keyPrefix: 'json/', filename: 'orders.json' }
-    }),
-
-    // Parquet for data lake
-    orchestrator.extractToS3({
-      query,
-      connectionName: 'orders',
-      outputFormat: 'parquet',
-      s3Config: { bucket: 'datalake', keyPrefix: 'orders/', filename: 'orders.parquet' }
-    })
-  ]);
-}
-```
-
----
-
-## Error Handling
-
-```typescript
-try {
-  const result = await orchestrator.extractToS3({
-    query,
-    connectionName: 'orders',
-    outputFormat: 'csv',
-    s3Config: { /* ... */ }
-  });
-
-  console.log(`Success: ${result.recordCount} records extracted`);
-} catch (error) {
-  if (error.code === 'ECONNREFUSED') {
-    console.error('Cannot connect to Fluent API');
-  } else if (error.code === 'NoSuchBucket') {
-    console.error('S3 bucket does not exist');
-  } else if (error.message.includes('GraphQL')) {
-    console.error('GraphQL query error:', error.message);
-  } else {
-    console.error('Extraction failed:', error);
-  }
-}
-```
-
----
-
-## Troubleshooting
-
-### Issue 1: Out of Memory
-
-**Symptom:** Process crashes with OOM error
-
-**Cause:** Accumulating too many records in memory
-
-**Solution:** ExtractionOrchestrator streams by default, but ensure pagination is enabled
-
-### Issue 2: Slow Extraction
-
-**Symptom:** Takes hours to extract large dataset
-
-**Solution:** Increase page size and reduce delay
-```typescript
-pagination: {
-  pageSize: 500,   // Larger pages
-  delayMs: 0      // No delay
-}
-```
-
-### Issue 3: S3 Upload Fails
-
-**Symptom:** "Access Denied" or "NoSuchBucket" errors
-
-**Solution:** Verify AWS credentials and bucket permissions
-
----
-
-## See Also
-
-**Related Modules:**
-- [Module 05: Services](./api-reference-05-services.md) - ExtractionOrchestrator details
-- [Module 06: Data Sources](./api-reference-06-data-sources.md) - S3DataSource
-- [Module 08: Pagination](./api-reference-08-pagination.md) - Auto-pagination
-
-**Complete Guides:**
-- [Extraction Guide](../../extraction/extraction-readme.md) - Complete extraction series
-- [Parquet Guide](../../../03-PATTERN-GUIDES/parquet/parquet-readme.md) - Parquet export patterns
-
-**Templates:**
-- [Extraction Templates](../../../01-TEMPLATES/versori/workflows/extraction/graphql-queries/) - Production examples
-
----
-
-**Navigation:**
-- [Back to Module List](./readme.md)
-- [Previous: Module 09 - Webhook Validation](./api-reference-09-webhook-validation.md)
-- [Next: Module 11 - Error Handling](./api-reference-11-error-handling.md)
+# Module 10: Extraction
+
+**Level:** Intermediate to Advanced
+**Category:** Data Export & Integration
+
+## Overview
+
+Extraction involves querying data from Fluent Commerce GraphQL API and exporting it to external systems in various formats (CSV, JSON, XML, Parquet). The SDK provides `ExtractionOrchestrator` for high-level extraction workflows with auto-pagination, format conversion, and S3 output.
+
+## Table of Contents
+
+- [ExtractionOrchestrator Service](#extractionorchestrator-service)
+- [Format Conversion](#format-conversion)
+- [Partitioning Strategies](#partitioning-strategies)
+- [S3 Output](#s3-output)
+- [Auto-Pagination](#auto-pagination)
+- [Data Enrichment](#data-enrichment)
+- [Performance Optimization](#performance-optimization)
+- [Common Patterns](#common-patterns)
+- [Error Handling](#error-handling)
+- [Troubleshooting](#troubleshooting)
+- [See Also](#see-also)
+
+---
+
+## ExtractionOrchestrator Service
+
+High-level service for end-to-end extraction workflows:
+
+```typescript
+import { createClient, ExtractionOrchestrator } from '@fluentcommerce/fc-connect-sdk';
+
+const client = await createClient({ config });
+const orchestrator = new ExtractionOrchestrator(client, console);
+
+// Extract virtual positions to CSV on S3
+const result = await orchestrator.extractToS3({
+  query: `
+    query GetPositions($first: Int, $after: String) {
+      virtualPositions(first: $first, after: $after) {
+        edges {
+          node {
+            id
+            ref
+            productRef
+            groupRef
+            quantity
+            status
+          }
+        }
+        pageInfo {
+          hasNextPage
+          endCursor
+        }
+      }
+    }
+  `,
+  connectionName: 'virtualPositions',  // GraphQL connection name
+  outputFormat: 'csv',                // csv | json | xml | parquet
+  s3Config: {
+    bucket: 'my-exports',
+    keyPrefix: 'fluent/virtual-positions/',
+    filename: `positions-${Date.now()}.csv`
+  },
+  pagination: {
+    pageSize: 250,
+    maxPages: Infinity
+  }
+});
+
+console.log(`Extracted ${result.recordCount} records to ${result.s3Key}`);
+```
+
+**Key Features:**
+- Automatic pagination through all results
+- Format conversion (JSON → CSV/XML/Parquet)
+- Direct S3 upload (no local disk)
+- Progress tracking and logging
+- Error recovery and retry
+
+---
+
+## Format Conversion
+
+The SDK supports multiple output formats:
+
+### CSV Export
+
+```typescript
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'inventoryPositions',
+  outputFormat: 'csv',
+  s3Config: {
+    bucket: 'exports',
+    keyPrefix: 'inventory/',
+    filename: 'positions.csv'
+  },
+  csvOptions: {
+    headers: true,            // Include header row
+    delimiter: ',',           // Column separator
+    quote: '"',               // Quote character
+    includeNull: false       // Skip null values
+  }
+});
+```
+
+**Output:**
+```csv
+id,ref,productRef,locationRef,onHand,status
+123,POS-001,SKU-100,LOC-A,50,ACTIVE
+124,POS-002,SKU-101,LOC-B,25,ACTIVE
+```
+
+### JSON Export
+
+```typescript
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'orders',
+  outputFormat: 'json',
+  s3Config: {
+    bucket: 'exports',
+    keyPrefix: 'orders/',
+    filename: 'orders.json'
+  },
+  jsonOptions: {
+    pretty: true,           // Pretty-print JSON
+    arrayFormat: true       // Output as JSON array
+  }
+});
+```
+
+**Output:**
+```json
+[
+  {
+    "id": "123",
+    "ref": "ORD-001",
+    "status": "BOOKED",
+    "totalPrice": 99.99
+  },
+  {
+    "id": "124",
+    "ref": "ORD-002",
+    "status": "BOOKED",
+    "totalPrice": 149.99
+  }
+]
+```
+
+### XML Export
+
+```typescript
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'products',
+  outputFormat: 'xml',
+  s3Config: {
+    bucket: 'exports',
+    keyPrefix: 'products/',
+    filename: 'products.xml'
+  },
+  xmlOptions: {
+    rootElement: 'Products',    // Root element name
+    itemElement: 'Product',     // Individual record element
+    declaration: true          // Include <?xml declaration
+  }
+});
+```
+
+**Output:**
+```xml
+<?xml version="1.0" encoding="UTF-8"?>
+<Products>
+  <Product>
+    <id>123</id>
+    <ref>PROD-001</ref>
+    <name>Widget</name>
+    <status>ACTIVE</status>
+  </Product>
+  <Product>
+    <id>124</id>
+    <ref>PROD-002</ref>
+    <name>Gadget</name>
+    <status>ACTIVE</status>
+  </Product>
+</Products>
+```
+
+### Parquet Export
+
+```typescript
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'virtualPositions',
+  outputFormat: 'parquet',
+  s3Config: {
+    bucket: 'datalake',
+    keyPrefix: 'fluent/positions/date=2025-01-15/',
+    filename: 'positions.parquet'
+  },
+  parquetOptions: {
+    compression: 'SNAPPY',    // SNAPPY | GZIP | NONE
+    rowGroupSize: 5000       // Records per row group
+  }
+});
+```
+
+---
+
+## Partitioning Strategies
+
+For large datasets, partition exports by date, location, or other dimensions:
+
+### Date Partitioning
+
+```typescript
+// Partition by extraction date (Hive-style)
+const today = new Date().toISOString().split('T')[0];
+
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'orders',
+  outputFormat: 'parquet',
+  s3Config: {
+    bucket: 'datalake',
+    keyPrefix: `fluent/orders/date=${today}/`,  // date=2025-01-15/
+    filename: `orders-${Date.now()}.parquet`
+  }
+});
+```
+
+### Multi-Dimensional Partitioning
+
+```typescript
+// Partition by retailer + date
+const retailerId = process.env.FLUENT_RETAILER_ID;
+const today = new Date().toISOString().split('T')[0];
+
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'virtualPositions',
+  outputFormat: 'parquet',
+  s3Config: {
+    bucket: 'datalake',
+    keyPrefix: `fluent/positions/retailer=${retailerId}/date=${today}/`,
+    filename: `positions-${Date.now()}.parquet`
+  }
+});
+```
+
+---
+
+## S3 Output
+
+### Direct S3 Upload
+
+The ExtractionOrchestrator streams directly to S3 without local disk:
+
+```typescript
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'inventoryPositions',
+  outputFormat: 'csv',
+  s3Config: {
+    bucket: 'my-bucket',
+    keyPrefix: 'exports/inventory/',
+    filename: 'positions.csv',
+    region: 'us-east-1',        // AWS region
+    acl: 'private'              // S3 ACL
+  },
+  awsCredentials: {
+    accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+    secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
+  }
+});
+```
+
+### S3 Path Construction
+
+```typescript
+// Generates: s3://my-bucket/exports/inventory/positions.csv
+const s3Path = `s3://${s3Config.bucket}/${s3Config.keyPrefix}${s3Config.filename}`;
+```
+
+### Custom S3 Configuration
+
+```typescript
+import { S3DataSource } from '@fluentcommerce/fc-connect-sdk';
+
+const s3 = new S3DataSource({
+  type: 'S3_CSV',
+  settings: {
+    bucket: 'my-bucket',
+    region: 'us-west-2',
+    credentials: {
+      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
+      secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY
+    }
+  }
+});
+
+// Manual upload
+await s3.writeFile('exports/data.csv', csvContent);
+```
+
+---
+
+## Auto-Pagination
+
+ExtractionOrchestrator handles pagination automatically:
+
+```typescript
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'virtualPositions',
+  outputFormat: 'csv',
+  s3Config: { /* ... */ },
+  pagination: {
+    pageSize: 250,          // Records per page
+    maxPages: Infinity,     // Extract all pages
+    delayMs: 500           // Delay between pages (rate limiting)
+  }
+});
+```
+
+**Under the hood:**
+1. Fetches first page (250 records)
+2. Checks `pageInfo.hasNextPage`
+3. If true, fetches next page with `endCursor`
+4. Repeats until `hasNextPage: false`
+5. Streams all records to S3
+
+---
+
+## Data Enrichment
+
+Transform records during extraction:
+
+```typescript
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'orders',
+  outputFormat: 'csv',
+  s3Config: { /* ... */ },
+  transformFn: (record) => ({
+    ...record,
+    extractedAt: new Date().toISOString(),
+    totalPriceUSD: record.totalPrice,
+    currency: 'USD'
+  })
+});
+```
+
+**Use Cases:**
+- Add extraction timestamp
+- Compute derived fields
+- Normalize data formats
+- Filter sensitive fields
+
+---
+
+## Performance Optimization
+
+### Parallel Extractions
+
+Extract multiple entities in parallel:
+
+```typescript
+await Promise.all([
+  orchestrator.extractToS3({ query: positionsQuery, connectionName: 'inventoryPositions', /* ... */ }),
+  orchestrator.extractToS3({ query: ordersQuery, connectionName: 'orders', /* ... */ }),
+  orchestrator.extractToS3({ query: productsQuery, connectionName: 'products', /* ... */ })
+]);
+```
+
+### Streaming Large Datasets
+
+For 100K+ records, use streaming:
+
+```typescript
+// ExtractionOrchestrator streams by default (no memory accumulation)
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'virtualPositions',
+  outputFormat: 'parquet',
+  s3Config: { /* ... */ },
+  pagination: {
+    pageSize: 500,  // Larger pages for throughput
+    delayMs: 0     // No delay (maximize speed)
+  }
+});
+```
+
+### Compression
+
+Reduce S3 storage costs with compression:
+
+```typescript
+// Parquet with SNAPPY compression
+await orchestrator.extractToS3({
+  query,
+  connectionName: 'orders',
+  outputFormat: 'parquet',
+  s3Config: { /* ... */ },
+  parquetOptions: {
+    compression: 'SNAPPY'  // 10-20x size reduction
+  }
+});
+```
+
+---
+
+## Common Patterns
+
+### Pattern 1: Daily Extract Job
+
+```typescript
+import { schedule } from '@versori/run';
+import { createClient, ExtractionOrchestrator } from '@fluentcommerce/fc-connect-sdk';
+
+export const dailyExtract = schedule('daily-extract', '0 2 * * *', async (ctx) => {
+  const { log } = ctx;
+  const client = await createClient({ ...ctx, log });
+  const orchestrator = new ExtractionOrchestrator(client, log);
+
+  const today = new Date().toISOString().split('T')[0];
+
+  await orchestrator.extractToS3({
+    query: virtualPositionsQuery,
+    connectionName: 'virtualPositions',
+    outputFormat: 'parquet',
+    s3Config: {
+      bucket: 'datalake',
+      keyPrefix: `fluent/positions/date=${today}/`,
+      filename: `positions-${Date.now()}.parquet`
+    }
+  });
+
+  log.info('Daily extract complete');
+});
+```
+
+### Pattern 2: Multi-Format Export
+
+```typescript
+async function exportAllFormats(client, query) {
+  const orchestrator = new ExtractionOrchestrator(client, console);
+
+  await Promise.all([
+    // CSV for BI tools
+    orchestrator.extractToS3({
+      query,
+      connectionName: 'orders',
+      outputFormat: 'csv',
+      s3Config: { bucket: 'exports', keyPrefix: 'csv/', filename: 'orders.csv' }
+    }),
+
+    // JSON for APIs
+    orchestrator.extractToS3({
+      query,
+      connectionName: 'orders',
+      outputFormat: 'json',
+      s3Config: { bucket: 'exports', keyPrefix: 'json/', filename: 'orders.json' }
+    }),
+
+    // Parquet for data lake
+    orchestrator.extractToS3({
+      query,
+      connectionName: 'orders',
+      outputFormat: 'parquet',
+      s3Config: { bucket: 'datalake', keyPrefix: 'orders/', filename: 'orders.parquet' }
+    })
+  ]);
+}
+```
+
+---
+
+## Error Handling
+
+```typescript
+try {
+  const result = await orchestrator.extractToS3({
+    query,
+    connectionName: 'orders',
+    outputFormat: 'csv',
+    s3Config: { /* ... */ }
+  });
+
+  console.log(`Success: ${result.recordCount} records extracted`);
+} catch (error) {
+  if (error.code === 'ECONNREFUSED') {
+    console.error('Cannot connect to Fluent API');
+  } else if (error.code === 'NoSuchBucket') {
+    console.error('S3 bucket does not exist');
+  } else if (error.message.includes('GraphQL')) {
+    console.error('GraphQL query error:', error.message);
+  } else {
+    console.error('Extraction failed:', error);
+  }
+}
+```
+
+---
+
+## Troubleshooting
+
+### Issue 1: Out of Memory
+
+**Symptom:** Process crashes with OOM error
+
+**Cause:** Accumulating too many records in memory
+
+**Solution:** ExtractionOrchestrator streams by default, but ensure pagination is enabled
+
+### Issue 2: Slow Extraction
+
+**Symptom:** Takes hours to extract large dataset
+
+**Solution:** Increase page size and reduce delay
+```typescript
+pagination: {
+  pageSize: 500,   // Larger pages
+  delayMs: 0      // No delay
+}
+```
+
+### Issue 3: S3 Upload Fails
+
+**Symptom:** "Access Denied" or "NoSuchBucket" errors
+
+**Solution:** Verify AWS credentials and bucket permissions
+
+---
+
+## See Also
+
+**Related Modules:**
+- [Module 05: Services](./api-reference-05-services.md) - ExtractionOrchestrator details
+- [Module 06: Data Sources](./api-reference-06-data-sources.md) - S3DataSource
+- [Module 08: Pagination](./api-reference-08-pagination.md) - Auto-pagination
+
+**Complete Guides:**
+- [Extraction Guide](../../extraction/extraction-readme.md) - Complete extraction series
+- [Parquet Guide](../../../03-PATTERN-GUIDES/parquet/parquet-readme.md) - Parquet export patterns
+
+**Templates:**
+- [Extraction Templates](../../../01-TEMPLATES/versori/workflows/extraction/graphql-queries/) - Production examples
+
+---
+
+**Navigation:**
+- [Back to Module List](./readme.md)
+- [Previous: Module 09 - Webhook Validation](./api-reference-09-webhook-validation.md)
+- [Next: Module 11 - Error Handling](./api-reference-11-error-handling.md)