@fluentcommerce/fc-connect-sdk 0.1.54 → 0.1.56

package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-08-extraction-orchestrator.md

@@ -1,1008 +1,1008 @@
-# Module 8: ExtractionOrchestrator - Simplified Data Extraction
-
-> **Reusable orchestrator for extracting and transforming GraphQL data with auto-pagination**
-
-## 📚 What You'll Learn
-
-- ✅ How to use ExtractionOrchestrator for simple data extraction workflows
-- ✅ Path-based result extraction from nested GraphQL responses
-- ✅ Automatic handling of GraphQL connection patterns (edges.node)
-- ✅ Extraction limits and validation
-- ✅ Statistics tracking and error handling
-- ✅ When to use ExtractionOrchestrator vs manual extraction
-
-## 🎯 Overview
-
-`ExtractionOrchestrator` is a high-level service that simplifies the common pattern of:
-
-1. **Execute** GraphQL query with auto-pagination
-2. **Extract** data from nested response paths
-3. **Validate** extracted records (optional)
-4. **Track** statistics and errors
-
-### When to Use
-
-✅ **Use ExtractionOrchestrator when:**
-
-- Extracting data from a single GraphQL query
-- Need auto-pagination handling
-- Want path-based extraction from nested responses
-- Need extraction limits (maxRecords, maxPages, timeout)
-- Want built-in statistics and error tracking
-
-❌ **Don't use when:**
-
-- Need to combine multiple queries (do separate calls)
-- Complex business logic transformations (use UniversalMapper)
-- Real-time streaming (use direct GraphQL calls)
-
-## 🚀 Quick Start
-
-### Basic Extraction
-
-```typescript
-import { ExtractionOrchestrator, createClient } from '@fluentcommerce/fc-connect-sdk';
-
-const client = await createClient({
-  config: {
-    baseUrl: process.env.FLUENT_BASE_URL!,
-    clientId: process.env.FLUENT_CLIENT_ID!,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET!
-    // retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
-  },
-});
-
-const orchestrator = new ExtractionOrchestrator(client, console);
-
-// Extract virtual positions
-const result = await orchestrator.extract({
-  query: `
-    query GetVirtualPositions($first: Int!, $after: String) {
-      virtualPositions(first: $first, after: $after) {
-        edges {
-          node {
-            id
-            ref
-            productRef
-            qty
-            groupRef
-          }
-          cursor
-        }
-        pageInfo {
-          hasNextPage
-          # Note: Fluent doesn't return endCursor - use edge cursors instead
-        }
-      }
-    }
-  `,
-  resultPath: 'virtualPositions.edges.node',
-  pageSize: 200,
-});
-
-console.log(`Extracted ${result.data.length} records`);
-console.log(`Pages fetched: ${result.stats.totalPages}`);
-```
-
-## 📖 Core Concepts
-
-### 1. Result Path Extraction
-
-The `resultPath` parameter uses dot notation to navigate nested GraphQL responses:
-
-```typescript
-// Simple path using dot notation
-resultPath: 'virtualPositions.edges.node';
-
-// The orchestrator automatically:
-// 1. Navigates to virtualPositions
-// 2. Extracts edges array
-// 3. Extracts node from each edge
-```
-
-### 2. Auto-Detection of Edges.Node Pattern
-
-The orchestrator automatically detects and extracts from GraphQL connection patterns:
-
-```typescript
-// If response has structure:
-{
-  virtualPositions: {
-    edges: [{ node: { id: '1', ref: 'VP1' } }, { node: { id: '2', ref: 'VP2' } }];
-  }
-}
-
-// Path: 'virtualPositions.edges'
-// Result: [{ id: '1', ref: 'VP1' }, { id: '2', ref: 'VP2' }]
-// Automatically extracts 'node' from each edge!
-
-// Path: 'virtualPositions.edges.node'
-// Result: Same as above (explicitly extracting node)
-```
-
-### 3. Extraction Limits and Pagination Notes
-
-Control extraction scope with multiple limit options:
-
-```typescript
-const result = await orchestrator.extract({
-  query: myQuery,
-  resultPath: 'data.edges.node',
-
-  // Limit options (all optional)
-  pageSize: 200, // Records per page (default: 200)
-  maxRecords: 10000, // Total records to extract
-  maxPages: 50, // Maximum pages to fetch
-  timeout: 120000, // Query timeout in ms (default: 60000)
-});
-
-// Note:
-// - 'first' and 'after' are managed by the orchestrator; do not include them in variables
-// - Pagination is handled automatically using the SDK's built-in auto-pagination
-```
-
-**Priority when multiple limits are set:**
-
-1. `maxRecords` - Stops when total records reach limit
-2. `maxPages` - Stops after fetching specified pages
-3. API returns `hasNextPage: false`
-
-### 4. Pagination Direction (Forward vs Backward)
-
-**NEW:** ExtractionOrchestrator now supports both forward and backward pagination.
-
-#### Forward Pagination (Default)
-
-```typescript
-// Forward pagination uses first/after (default behavior)
-const result = await orchestrator.extract({
-  query: `
-    query GetItems($first: Int!, $after: String) {
-      items(first: $first, after: $after) {
-        edges {
-          node { id name }
-          cursor
-        }
-        pageInfo { hasNextPage }
-      }
-    }
-  `,
-  resultPath: 'items.edges.node',
-  direction: 'forward', // Optional - this is the default
-});
-```
-
-#### Backward Pagination (Reverse)
-
-```typescript
-// Backward pagination uses last/before
-const result = await orchestrator.extract({
-  query: `
-    query GetItems($last: Int!, $before: String) {
-      items(last: $last, before: $before) {
-        edges {
-          node { id name }
-          cursor
-        }
-        pageInfo { hasPreviousPage }
-      }
-    }
-  `,
-  resultPath: 'items.edges.node',
-  direction: 'backward', // ✅ Enable reverse pagination
-});
-```
-
-#### Query Requirements by Direction
-
-| Direction | Parameters Required | PageInfo Required | Cursor Selection |
-|-----------|-------------------|-------------------|------------------|
-| `forward` (default) | `$first`, `$after` | `hasNextPage` | Last cursor in edges |
-| `backward` | `$last`, `$before` | `hasPreviousPage` | First cursor in edges |
-
-#### Important Rules
-
-**✅ DO:**
-- Declare `$first/$after` (forward) OR `$last/$before` (backward) in your query signature
-- Include the corresponding parameters in your field arguments: `items(first: $first, after: $after)`
-- Always include `cursor` in edges
-- Include correct `pageInfo` field: `hasNextPage` for forward, `hasPreviousPage` for backward
-
-**❌ DON'T:**
-- Mix `first/after` with `direction: 'backward'` - query must match direction
-- Include pagination parameters in `variables` - orchestrator injects them from `pageSize`/cursor
-- Omit `cursor` from edges - orchestrator needs cursors for pagination
-- Forget `hasPreviousPage` when using `direction: 'backward'` - validation will fail
-
-#### Use Cases for Backward Pagination
-
-**When to use backward:**
-- 📅 Fetching most recent records first (e.g., latest orders, recent activity)
-- 🔄 Reverse-chronological feeds
-- 🎯 "Load more above" UI patterns
-- 📊 Time-series data from newest to oldest
-
-```typescript
-// Example: Get latest 1000 orders (most recent first)
-const recentOrders = await orchestrator.extract({
-  query: `
-    query GetRecentOrders($last: Int!, $before: String) {
-      orders(last: $last, before: $before, orderBy: { field: CREATED_ON, direction: DESC }) {
-        edges {
-          node {
-            id
-            ref
-            createdOn
-            status
-          }
-          cursor
-        }
-        pageInfo { hasPreviousPage }
-      }
-    }
-  `,
-  resultPath: 'orders.edges.node',
-  direction: 'backward',
-  maxRecords: 1000,
-  pageSize: 200,
-});
-
-console.log(`Fetched ${recentOrders.data.length} orders (newest first)`);
-```
-
-## 🛠️ Complete API Reference
-
-### Constructor
-
-```typescript
-constructor(
-  client: FluentClient | FluentVersoriClient,
-  logger: StructuredLogger
-)
-```
-
-**Parameters:**
-
-- `client` - Fluent API client instance
-- `logger` - Structured logger (can use `console`)
-
-### extract() Method
-
-```typescript
-async extract<T = any>(
-  options: ExtractionOptions<T>
-): Promise<ExtractionResult<T>>
-```
-
-### ExtractionOptions
-
-```typescript
-interface ExtractionOptions<T = any> {
-  // Required
-  query: string; // GraphQL query
-  resultPath: string; // Path to extract (e.g., 'virtualPositions.edges.node')
-
-  // Optional
-  variables?: Record<string, any>; // Query variables (default: {})
-  pageSize?: number; // Records per page (default: 200)
-  maxRecords?: number; // Max total records (default: unlimited)
-  maxPages?: number; // Max pages to fetch (default: unlimited)
-  timeout?: number; // Query timeout ms (default: 60000)
-  direction?: 'forward' | 'backward'; // Pagination direction (default: 'forward')
-  validateItem?: (item: T) => boolean; // Optional per-item validation
-  errorHandling?: 'throw' | 'partial'; // How to handle GraphQL errors (default: 'throw')
-  operationName?: string; // Optional operation name for logging
-}
-```
-
-**Notes:**
-- `direction`: `'forward'` (default) uses `first/after` params, requires `pageInfo.hasNextPage`. `'backward'` uses `last/before` params, requires `pageInfo.hasPreviousPage`. Your GraphQL query must match the direction (don't mix `first/after` with `direction: 'backward'`)
-- `errorHandling`: `'throw'` (default) throws `GraphQLExecutionError` on any errors. `'partial'` continues extraction and accumulates errors in `stats.partialErrors`. See [Partial Response Support](../../api-reference/modules/api-reference-12-partial-responses.md) for details.
-
-### ExtractionResult
-
-```typescript
-interface ExtractionResult<T = any> {
-  data: T[]; // Extracted records
-  stats: ExtractionStats; // Statistics
-  errors?: ExtractionError[]; // Non-fatal errors (validation failures)
-}
-
-interface ExtractionStats {
-  totalRecords: number; // Total records extracted
-  totalPages: number; // Number of pages fetched
-  duration: number; // Extraction duration in milliseconds
-  truncated: boolean; // Whether extraction was limited
-  truncationReason?: 'maxPages' | 'maxRecords' | 'timeout'; // Reason for truncation
-  validRecords?: number; // Valid records (if validation used)
-  invalidRecords?: number; // Invalid records (if validation used)
-  direction?: 'forward' | 'backward'; // Pagination direction used
-  partialErrors?: GraphQLError[]; // GraphQL errors (only when errorHandling: 'partial')
-}
-
-interface ExtractionError {
-  type: 'graphql' | 'network' | 'validation' | 'parsing'; // Error type
-  message: string; // Error message
-  recordIndex?: number; // Record index (for validation errors)
-  details?: any; // Additional error details
-}
-```
-
-## 📋 Usage Patterns
-
-### Pattern 1: Simple Extraction
-
-```typescript
-const orchestrator = new ExtractionOrchestrator(client, console);
-
-const result = await orchestrator.extract({
-  query: `
-    query GetProducts($first: Int!, $after: String) {
-      products(first: $first, after: $after) {
-        edges {
-          node {
-            id
-            ref
-            name
-            type
-            status
-          }
-          cursor
-        }
-        pageInfo { hasNextPage }
-      }
-    }
-  `,
-  resultPath: 'products.edges.node',
-  pageSize: 100,
-});
-
-console.log(`Extracted ${result.data.length} products`);
-```
-
-### Pattern 2: Extraction with Limits
-
-```typescript
-// Extract first 5000 records only
-const result = await orchestrator.extract({
-  query: inventoryQuery,
-  resultPath: 'inventoryPositions.edges.node',
-  maxRecords: 5000,
-  pageSize: 500, // Use larger pages for efficiency
-});
-
-if (result.stats.truncated) {
-  console.log('Extraction was limited to 5000 records');
-}
-```
-
-### Pattern 3: Extraction with Validation
-
-```typescript
-const result = await orchestrator.extract({
-  query: virtualPositionsQuery,
-  resultPath: 'virtualPositions.edges.node',
-
-  // Validate each item during extraction
-  validateItem: item => {
-    return (
-      item.ref && // ref is required
-      item.productRef && // productRef is required
-      typeof item.qty === 'number' && // qty must be a number
-      item.qty >= 0 // qty must be non-negative
-    );
-  },
-});
-
-console.log(`Valid records: ${result.stats.validRecords}`);
-console.log(`Invalid records: ${result.stats.invalidRecords}`);
-
-// Review validation errors (recordIndex, type, details)
-if (result.errors && result.errors.length > 0) {
-  console.log(
-    'Validation errors:',
-    result.errors.map(e => ({
-      type: e.type,
-      recordIndex: e.recordIndex,
-      details: e.details,
-    }))
-  );
-}
-```
-
-### Pattern 4: Extraction with Variables
-
-```typescript
-// Extract orders for specific retailer and date range
-const result = await orchestrator.extract({
-  query: `
-    query GetOrders(
-      $first: Int!,
-      $after: String,
-      $retailerId: ID!,
-      $startDate: DateTime!
-    ) {
-      orders(
-        first: $first,
-        after: $after,
-        retailerId: $retailerId,
-        createdOn: { from: $startDate }
-      ) {
-        edges {
-          node {
-            id
-            ref
-            status
-            createdOn
-            totalPrice
-          }
-          cursor
-        }
-        pageInfo { hasNextPage }
-      }
-    }
-  `,
-  resultPath: 'orders.edges.node',
-  variables: {
-    startDate: '2025-01-01T00:00:00Z',
-  },
-  pageSize: 100,
-});
-```
-
-### Pattern 5: Nested Path Extraction
-
-```typescript
-// Extract from deeply nested response
-const result = await orchestrator.extract({
-  query: `
-    query GetCatalogues($first: Int!, $after: String) {
-      catalogues(first: $first, after: $after) {
-        edges {
-          node {
-            id
-            ref
-            name
-            products {
-              edges {
-                node {
-                  id
-                  ref
-                  name
-                }
-              }
-            }
-          }
-          cursor
-        }
-        pageInfo { hasNextPage }
-      }
-    }
-  `,
-  // Extract products from catalogues
-  resultPath: 'catalogues.edges.node.products.edges.node',
-  pageSize: 50,
-});
-```
-
-## 🎯 Real-World Examples
-
-### Example 1: Virtual Positions Extraction for SFTP Export
-
-```typescript
-import {
-  ExtractionOrchestrator,
-  createClient,
-  SftpDataSource,
-  createConsoleLogger,
-  toStructuredLogger
-} from '@fluentcommerce/fc-connect-sdk';
-
-async function exportVirtualPositionsToSftp() {
-  // Initialize services
-  const logger = toStructuredLogger(createConsoleLogger(), {
-    logLevel: 'info'
-  });
-  const client = await createClient({
-    config: {
-      baseUrl: process.env.FLUENT_BASE_URL!,
-      clientId: process.env.FLUENT_CLIENT_ID!,
-      clientSecret: process.env.FLUENT_CLIENT_SECRET!
-      // retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
-    }
-  });
-
-  const sftp = new SftpDataSource(
-    {
-      type: 'SFTP_CSV',
-      sftpConfig: {
-        settings: {
-          host: process.env.SFTP_HOST!,
-          port: 22,
-          username: process.env.SFTP_USERNAME!,
-          password: process.env.SFTP_PASSWORD!,
-        },
-      },
-      connectionId: 'sftp-export',
-    },
-    logger
-  );
-
-  // Extract virtual positions
-  const orchestrator = new ExtractionOrchestrator(client, logger);
-
-  const result = await orchestrator.extract({
-    query: `
-      query GetVirtualPositions($first: Int!, $after: String) {
-        virtualPositions(first: $first, after: $after) {
-          edges {
-            node {
-              id
-              ref
-              productRef
-              qty
-              groupRef
-              createdOn
-              updatedOn
-            }
-            cursor
-          }
-          pageInfo {
-            hasNextPage
-            # Note: Fluent doesn't return endCursor - use edge cursors instead
-          }
-        }
-      }
-    `,
-    resultPath: 'virtualPositions.edges.node',
-    pageSize: 500,
-    maxRecords: 50000,
-
-    // Validate data quality
-    validateItem: item => {
-      return item.ref && item.productRef && typeof item.qty === 'number';
-    },
-  });
-
-  logger.info('Extraction complete', {
-    totalRecords: result.stats.totalRecords,
-    totalPages: result.stats.totalPages,
-    duration: result.stats.duration,
-    validRecords: result.stats.validRecords,
-    invalidRecords: result.stats.invalidRecords,
-  });
-
-  // Convert to CSV format
-  const csv = [
-    // Header
-    'ref,productRef,qty,groupRef,createdOn,updatedOn',
-    // Data rows
-    ...result.data.map(
-      item =>
-        `${item.ref},${item.productRef},${item.qty},${item.groupRef},${item.createdOn},${item.updatedOn}`
-    ),
-  ].join('\n');
-
-  // Upload to SFTP
-  try {
-    const filename = `/exports/virtual-positions-${new Date().toISOString()}.csv`;
-    await sftp.uploadFile(filename, csv);
-
-    logger.info('Export complete', { filename, recordCount: result.data.length });
-  } finally {
-    // Always dispose SFTP connection to prevent connection pool leaks
-    await sftp.dispose();
-  }
-}
-```
-
-### Example 2: Multi-Query Extraction Pattern
-
-```typescript
-// For multiple queries, call ExtractionOrchestrator separately for each
-async function extractMultipleEntities() {
-  const orchestrator = new ExtractionOrchestrator(client, logger);
-
-  // Execute extractions in parallel
-  const [products, locations, inventories] = await Promise.all([
-    // Extract products
-    orchestrator.extract({
-      query: productsQuery,
-      resultPath: 'products.edges.node',
-      pageSize: 200,
-    }),
-
-    // Extract locations
-    orchestrator.extract({
-      query: locationsQuery,
-      resultPath: 'locations.edges.node',
-      pageSize: 100,
-    }),
-
-    // Extract inventory positions
-    orchestrator.extract({
-      query: inventoryQuery,
-      resultPath: 'inventoryPositions.edges.node',
-      maxRecords: 10000,
-      pageSize: 500,
-    }),
-  ]);
-
-  // Combine or process separately in your business logic
-  return {
-    products: products.data,
-    locations: locations.data,
-    inventories: inventories.data,
-    stats: {
-      totalProducts: products.stats.totalRecords,
-      totalLocations: locations.stats.totalRecords,
-      totalInventories: inventories.stats.totalRecords,
-      totalDuration:
-        products.stats.duration + locations.stats.duration + inventories.stats.duration,
-    },
-  };
-}
-```
-
-### Example 3: Extraction with Transformation
-
-```typescript
-import { ExtractionOrchestrator, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
-
-async function extractAndTransform() {
-  const orchestrator = new ExtractionOrchestrator(client, logger);
-
-  // Step 1: Extract
-  const extraction = await orchestrator.extract({
-    query: virtualPositionsQuery,
-    resultPath: 'virtualPositions.edges.node',
-    pageSize: 500,
-  });
-
-  // Step 2: Transform with UniversalMapper
-  const mapper = new UniversalMapper({
-    fields: {
-      reference: { source: 'ref', required: true },
-      product: { source: 'productRef', required: true },
-      quantity: { source: 'qty', resolver: 'sdk.parseInt' },
-      group: { source: 'groupRef', resolver: 'sdk.trim' },
-      lastUpdated: { source: 'updatedOn', resolver: 'sdk.formatDate' },
-    },
-  });
-
-  const transformation = await mapper.map(extraction.data);
-
-  if (!transformation.success) {
-    logger.error('Transformation failed', { errors: transformation.errors });
-    throw new Error('Data transformation failed');
-  }
-
-  return transformation.data;
-}
-```
-
-## ⚠️ Error Handling
-
-### Handling Extraction Errors
-
-```typescript
-try {
-  const result = await orchestrator.extract(options);
-
-  // Check for validation errors
-  if (result.errors && result.errors.length > 0) {
-    logger.warn(`${result.errors.length} records failed validation`);
-
-    // Log first few errors for debugging
-    result.errors.slice(0, 5).forEach(err => {
-      logger.debug('Validation error', {
-        type: err.type,
-        message: err.message,
-        recordIndex: err.recordIndex,
-        details: err.details,
-      });
-    });
-  }
-
-  // Check if extraction was truncated
-  if (result.stats.truncated) {
-    logger.info('Extraction was limited by maxRecords or maxPages');
-  }
-
-  return result.data;
-} catch (error) {
-  // Fatal errors (GraphQL errors, network issues, invalid config)
-  logger.error('Extraction failed', error as Error);
-  throw error;
-}
-```
-
-### Partial Response Mode (errorHandling: 'partial')
-
-When GraphQL returns errors but some data is available, you can use partial mode to continue extraction:
-
-```typescript
-const result = await orchestrator.extract({
-  query: ORDERS_QUERY,
-  resultPath: 'orders.edges.node',
-  errorHandling: 'partial', // Continue extraction even with errors
-});
-
-// Check for partial errors in stats
-if (result.stats.partialErrors) {
-  logger.warn('Extraction completed with partial errors', {
-    recordsExtracted: result.data.length,
-    errorCount: result.stats.partialErrors.length,
-  });
-
-  // Process available data
-  console.log(`Extracted ${result.data.length} records despite errors`);
-
-  // Log errors for investigation
-  result.stats.partialErrors.forEach(err => {
-    logger.warn('GraphQL error during extraction', {
-      message: err.message,
-      path: err.path,
-    });
-  });
-}
-```
-
-**Edge Case: Null Data with Errors**
-
-When GraphQL returns errors with `data: null` or `data: undefined`:
-
-```typescript
-// GraphQL response: { data: null, errors: [{ message: 'Order item not found' }] }
-const result = await orchestrator.extract({
-  query: ORDERS_QUERY,
-  resultPath: 'orders.edges.node',
-  errorHandling: 'partial',
-});
-
-// Returns empty array with errors in stats
-expect(result.data).toEqual([]);
-expect(result.stats.partialErrors).toHaveLength(1);
-```
-
-**Behavior:**
-- **Partial mode**: Returns empty array `[]`, accumulates errors in `stats.partialErrors`, stops pagination if no data
-- **Throw mode** (default): Throws `GraphQLExecutionError` immediately (backward compatible)
-
-**When to Use Partial Mode:**
-- ✅ Batch queries where some items may fail independently
-- ✅ Reporting/analytics where partial data is better than none
-- ✅ Non-critical extractions where you want to process available data
-
-**When to Use Throw Mode (default):**
-- ✅ Critical data where consistency matters
-- ✅ Single record operations
-- ✅ When you need to fail fast on any errors
-
-### Common Error Scenarios
-
-| Error                          | Cause                        | Solution                                          |
-| ------------------------------ | ---------------------------- | ------------------------------------------------- |
-| `Query is required`            | Empty query string           | Provide valid GraphQL query                       |
-| `Result path is required`      | Empty resultPath             | Specify path to extract (e.g., 'data.edges.node') |
-| `Page size must be positive`   | pageSize ≤ 0                 | Use pageSize ≥ 1                                  |
-| `Max records must be positive` | maxRecords ≤ 0               | Use maxRecords ≥ 1                                |
-| `Max pages must be positive`   | maxPages ≤ 0                 | Use maxPages ≥ 1                                  |
-| `Timeout must be positive`     | timeout ≤ 0                  | Use timeout ≥ 1000 (1 second minimum)             |
-| `Path extraction failed`       | Invalid resultPath           | Check path matches response structure             |
-| `GraphQL Error: ...`           | Invalid query or permissions | Validate query with schema introspection          |
-
-## 🎓 Best Practices
-
-### 1. Choose Appropriate Page Size
-
-```typescript
-// ❌ Too small - many API calls
-pageSize: 10;
-
-// ✅ Balanced - good for most use cases
-pageSize: 200;
-
-// ✅ Large datasets - fewer API calls
-pageSize: 500;
-
-// ⚠️ Very large - may cause memory issues
-pageSize: 1000;
-```
-
-### 2. Use Extraction Limits for Large Datasets
-
-```typescript
-// ✅ Prevent unbounded extraction
-const result = await orchestrator.extract({
-  query: largeDatasetQuery,
-  resultPath: 'data.edges.node',
-  maxRecords: 100000, // Hard limit
-  maxPages: 200, // Backup limit
-  timeout: 300000, // 5 minute timeout
-});
-```
-
-### 3. Implement Validation for Data Quality
-
-```typescript
-// ✅ Validate critical fields during extraction
-validateItem: item => {
-  // Required fields present
-  if (!item.ref || !item.productRef) return false;
-
-  // Valid data types
-  if (typeof item.qty !== 'number') return false;
-
-  // Business logic validation
-  if (item.qty < 0) return false;
-
-  return true;
-};
-```
-
-### 4. Monitor Statistics
-
-```typescript
-const result = await orchestrator.extract(options);
-
-// Log comprehensive statistics
-logger.info('Extraction complete', {
-  totalRecords: result.stats.totalRecords,
-  totalPages: result.stats.totalPages,
-  duration: result.stats.duration,
-  validRecords: result.stats.validRecords,
-  invalidRecords: result.stats.invalidRecords,
-  truncated: result.stats.truncated,
-  truncationReason: result.stats.truncationReason,
-
-  // Calculate average records per page
-  avgRecordsPerPage: Math.round(result.stats.totalRecords / result.stats.totalPages),
-});
-```
-
-### 5. Use with JobTracker for Async Workflows
-
-```typescript
-import { ExtractionOrchestrator, JobTracker } from '@fluentcommerce/fc-connect-sdk';
-
-async function scheduledExtraction() {
-  const tracker = new JobTracker(kvAdapter, logger);
-
-  try {
-    // Start job
-    const job = await tracker.startJob({
-      triggeredBy: 'schedule',
-      details: { type: 'virtual-positions-extract' },
-    });
-
-    // Start extraction
-    await tracker.update(job.id, {
-      status: 'PROCESSING',
-      stage: 'extracting',
-    });
-
-    const orchestrator = new ExtractionOrchestrator(client, logger);
-    const result = await orchestrator.extract({
-      query: virtualPositionsQuery,
-      resultPath: 'virtualPositions.edges.node',
-      maxRecords: 50000,
-    });
-
-    // Complete
-    await tracker.complete(job.id, {
-      recordsExtracted: result.stats.totalRecords,
-      totalPages: result.stats.totalPages,
-      duration: result.stats.duration,
-    });
-
-    return result;
-  } catch (error) {
-    // Mark failed via update to avoid relying on specific error helpers
-    await tracker.update(job.id, { status: 'FAILED', error: (error as Error).message });
-    throw error;
-  }
-}
-```
-
-## 🔗 Related Documentation
-
-- [Module 2: Basic Extraction](./02-core-guides-extraction-02-basic-extraction.md) - Manual GraphQL extraction
-- [Module 4: Data Enrichment](./02-core-guides-extraction-04-data-enrichment.md) - Multi-query patterns
-- [Module 5: Transformation](./02-core-guides-extraction-05-transformation.md) - UniversalMapper guide
-- [Job Tracker](../../advanced-services/advanced-services-job-tracker.md) - Async job management
-- [Auto-Pagination Guide](../../auto-pagination/) - Pagination internals
-
-## 📊 Performance Considerations
-
-### Memory Usage
-
-```typescript
-// ⚠️ High memory - stores all records in memory
-const result = await orchestrator.extract({
-  query: myQuery,
-  resultPath: 'data.edges.node',
-  maxRecords: 1000000, // 1 million records!
-});
-
-// ✅ Better - use maxRecords to limit extraction
-// Note: ExtractionOrchestrator handles pagination internally
-// For processing in chunks, use multiple extraction calls with date ranges
-async function extractByDateRange() {
-  const dateRanges = [
-    { from: '2025-01-01', to: '2025-01-31' },
-    { from: '2025-02-01', to: '2025-02-28' },
-    // ... more ranges
-  ];
-
-  for (const range of dateRanges) {
-    const result = await orchestrator.extract({
-      query: myQuery,
-      resultPath: 'data.edges.node',
-      variables: {
-        dateRangeFilter: range,
-      },
-      maxRecords: 100000, // Limit per date range
-    });
-
-    // Process chunk
-    await processChunk(result.data);
-  }
-}
-```
-
-### API Rate Limiting
-
-```typescript
-// Add delay between large extractions
-async function extractMultipleWithDelay() {
-  const queries = [query1, query2, query3];
-  const results = [];
-
-  for (const query of queries) {
-    const result = await orchestrator.extract({
-      query,
-      resultPath: 'data.edges.node',
-      maxRecords: 50000,
-    });
-
-    results.push(result);
-
-    // Delay 1 second between extractions
-    await new Promise(resolve => setTimeout(resolve, 1000));
-  }
-
-  return results;
-}
-```
-
-## ✅ Summary
-
-**ExtractionOrchestrator simplifies:**
-
-- ✅ GraphQL query execution with auto-pagination
-- ✅ Path-based extraction from nested responses
-- ✅ Automatic edges.node pattern handling
-- ✅ Extraction limits and validation
-- ✅ Statistics tracking and error handling
-
-**Use it when you need:**
-
-- Single-query data extraction
-- Automatic pagination handling
-- Path-based result extraction
-- Built-in validation and statistics
-
-**For complex scenarios:**
-
-- Multiple queries → Make separate calls, use `Promise.all()`
-- Business logic → Use `UniversalMapper` for transformation
-- Real-time streaming → Use direct `client.graphql()` calls
-
----
-
-**Next Steps:**
-
-- [Job Tracker](../../advanced-services/advanced-services-job-tracker.md) - Async job status tracking
-- [Module 5: Transformation](./02-core-guides-extraction-05-transformation.md) - Transform extracted data
-- [Examples Directory](../examples/) - Working code examples
+# Module 8: ExtractionOrchestrator - Simplified Data Extraction
+
+> **Reusable orchestrator for extracting and transforming GraphQL data with auto-pagination**
+
+## 📚 What You'll Learn
+
+- ✅ How to use ExtractionOrchestrator for simple data extraction workflows
+- ✅ Path-based result extraction from nested GraphQL responses
+- ✅ Automatic handling of GraphQL connection patterns (edges.node)
+- ✅ Extraction limits and validation
+- ✅ Statistics tracking and error handling
+- ✅ When to use ExtractionOrchestrator vs manual extraction
+
+## 🎯 Overview
+
+`ExtractionOrchestrator` is a high-level service that simplifies the common pattern of:
+
+1. **Execute** GraphQL query with auto-pagination
+2. **Extract** data from nested response paths
+3. **Validate** extracted records (optional)
+4. **Track** statistics and errors
+
+### When to Use
+
+✅ **Use ExtractionOrchestrator when:**
+
+- Extracting data from a single GraphQL query
+- Need auto-pagination handling
+- Want path-based extraction from nested responses
+- Need extraction limits (maxRecords, maxPages, timeout)
+- Want built-in statistics and error tracking
+
+❌ **Don't use when:**
+
+- Need to combine multiple queries (do separate calls)
+- Complex business logic transformations (use UniversalMapper)
+- Real-time streaming (use direct GraphQL calls)
+
+## 🚀 Quick Start
+
+### Basic Extraction
+
+```typescript
+import { ExtractionOrchestrator, createClient } from '@fluentcommerce/fc-connect-sdk';
+
+const client = await createClient({
+  config: {
+    baseUrl: process.env.FLUENT_BASE_URL!,
+    clientId: process.env.FLUENT_CLIENT_ID!,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET!
+    // retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
+  },
+});
+
+const orchestrator = new ExtractionOrchestrator(client, console);
+
+// Extract virtual positions
+const result = await orchestrator.extract({
+  query: `
+    query GetVirtualPositions($first: Int!, $after: String) {
+      virtualPositions(first: $first, after: $after) {
+        edges {
+          node {
+            id
+            ref
+            productRef
+            qty
+            groupRef
+          }
+          cursor
+        }
+        pageInfo {
+          hasNextPage
+          # Note: Fluent doesn't return endCursor - use edge cursors instead
+        }
+      }
+    }
+  `,
+  resultPath: 'virtualPositions.edges.node',
+  pageSize: 200,
+});
+
+console.log(`Extracted ${result.data.length} records`);
+console.log(`Pages fetched: ${result.stats.totalPages}`);
+```
+
+## 📖 Core Concepts
+
+### 1. Result Path Extraction
+
+The `resultPath` parameter uses dot notation to navigate nested GraphQL responses:
+
+```typescript
+// Simple path using dot notation
+resultPath: 'virtualPositions.edges.node';
+
+// The orchestrator automatically:
+// 1. Navigates to virtualPositions
+// 2. Extracts edges array
+// 3. Extracts node from each edge
+```
+
+### 2. Auto-Detection of Edges.Node Pattern
+
+The orchestrator automatically detects and extracts from GraphQL connection patterns:
+
+```typescript
+// If response has structure:
+{
+  virtualPositions: {
+    edges: [{ node: { id: '1', ref: 'VP1' } }, { node: { id: '2', ref: 'VP2' } }];
+  }
+}
+
+// Path: 'virtualPositions.edges'
+// Result: [{ id: '1', ref: 'VP1' }, { id: '2', ref: 'VP2' }]
+// Automatically extracts 'node' from each edge!
+
+// Path: 'virtualPositions.edges.node'
+// Result: Same as above (explicitly extracting node)
+```
+
+### 3. Extraction Limits and Pagination Notes
+
+Control extraction scope with multiple limit options:
+
+```typescript
+const result = await orchestrator.extract({
+  query: myQuery,
+  resultPath: 'data.edges.node',
+
+  // Limit options (all optional)
+  pageSize: 200, // Records per page (default: 200)
+  maxRecords: 10000, // Total records to extract
+  maxPages: 50, // Maximum pages to fetch
+  timeout: 120000, // Query timeout in ms (default: 60000)
+});
+
+// Note:
+// - 'first' and 'after' are managed by the orchestrator; do not include them in variables
+// - Pagination is handled automatically using the SDK's built-in auto-pagination
+```
+
+**Priority when multiple limits are set:**
+
+1. `maxRecords` - Stops when total records reach limit
+2. `maxPages` - Stops after fetching specified pages
+3. API returns `hasNextPage: false`
+
+### 4. Pagination Direction (Forward vs Backward)
+
+**NEW:** ExtractionOrchestrator now supports both forward and backward pagination.
+
+#### Forward Pagination (Default)
+
+```typescript
+// Forward pagination uses first/after (default behavior)
+const result = await orchestrator.extract({
+  query: `
+    query GetItems($first: Int!, $after: String) {
+      items(first: $first, after: $after) {
+        edges {
+          node { id name }
+          cursor
+        }
+        pageInfo { hasNextPage }
+      }
+    }
+  `,
+  resultPath: 'items.edges.node',
+  direction: 'forward', // Optional - this is the default
+});
+```
+
+#### Backward Pagination (Reverse)
+
+```typescript
+// Backward pagination uses last/before
+const result = await orchestrator.extract({
+  query: `
+    query GetItems($last: Int!, $before: String) {
+      items(last: $last, before: $before) {
+        edges {
+          node { id name }
+          cursor
+        }
+        pageInfo { hasPreviousPage }
+      }
+    }
+  `,
+  resultPath: 'items.edges.node',
+  direction: 'backward', // ✅ Enable reverse pagination
+});
+```
+
+#### Query Requirements by Direction
+
+| Direction | Parameters Required | PageInfo Required | Cursor Selection |
+|-----------|-------------------|-------------------|------------------|
+| `forward` (default) | `$first`, `$after` | `hasNextPage` | Last cursor in edges |
+| `backward` | `$last`, `$before` | `hasPreviousPage` | First cursor in edges |
+
+#### Important Rules
+
+**✅ DO:**
+- Declare `$first/$after` (forward) OR `$last/$before` (backward) in your query signature
+- Include the corresponding parameters in your field arguments: `items(first: $first, after: $after)`
+- Always include `cursor` in edges
+- Include correct `pageInfo` field: `hasNextPage` for forward, `hasPreviousPage` for backward
+
+**❌ DON'T:**
+- Mix `first/after` with `direction: 'backward'` - query must match direction
+- Include pagination parameters in `variables` - orchestrator injects them from `pageSize`/cursor
+- Omit `cursor` from edges - orchestrator needs cursors for pagination
+- Forget `hasPreviousPage` when using `direction: 'backward'` - validation will fail
+
+#### Use Cases for Backward Pagination
+
+**When to use backward:**
+- 📅 Fetching most recent records first (e.g., latest orders, recent activity)
+- 🔄 Reverse-chronological feeds
+- 🎯 "Load more above" UI patterns
+- 📊 Time-series data from newest to oldest
+
+```typescript
+// Example: Get latest 1000 orders (most recent first)
+const recentOrders = await orchestrator.extract({
+  query: `
+    query GetRecentOrders($last: Int!, $before: String) {
+      orders(last: $last, before: $before, orderBy: { field: CREATED_ON, direction: DESC }) {
+        edges {
+          node {
+            id
+            ref
+            createdOn
+            status
+          }
+          cursor
+        }
+        pageInfo { hasPreviousPage }
+      }
+    }
+  `,
+  resultPath: 'orders.edges.node',
+  direction: 'backward',
+  maxRecords: 1000,
+  pageSize: 200,
+});
+
+console.log(`Fetched ${recentOrders.data.length} orders (newest first)`);
+```
+
+## 🛠️ Complete API Reference
+
+### Constructor
+
+```typescript
+constructor(
+  client: FluentClient | FluentVersoriClient,
+  logger: StructuredLogger
+)
+```
+
+**Parameters:**
+
+- `client` - Fluent API client instance
+- `logger` - Structured logger (can use `console`)
+
+### extract() Method
+
+```typescript
+async extract<T = any>(
+  options: ExtractionOptions<T>
+): Promise<ExtractionResult<T>>
+```
+
+### ExtractionOptions
+
+```typescript
+interface ExtractionOptions<T = any> {
+  // Required
+  query: string; // GraphQL query
+  resultPath: string; // Path to extract (e.g., 'virtualPositions.edges.node')
+
+  // Optional
+  variables?: Record<string, any>; // Query variables (default: {})
+  pageSize?: number; // Records per page (default: 200)
+  maxRecords?: number; // Max total records (default: unlimited)
+  maxPages?: number; // Max pages to fetch (default: unlimited)
+  timeout?: number; // Query timeout ms (default: 60000)
+  direction?: 'forward' | 'backward'; // Pagination direction (default: 'forward')
+  validateItem?: (item: T) => boolean; // Optional per-item validation
+  errorHandling?: 'throw' | 'partial'; // How to handle GraphQL errors (default: 'throw')
+  operationName?: string; // Optional operation name for logging
+}
+```
+
+**Notes:**
+- `direction`: `'forward'` (default) uses `first/after` params, requires `pageInfo.hasNextPage`. `'backward'` uses `last/before` params, requires `pageInfo.hasPreviousPage`. Your GraphQL query must match the direction (don't mix `first/after` with `direction: 'backward'`)
+- `errorHandling`: `'throw'` (default) throws `GraphQLExecutionError` on any errors. `'partial'` continues extraction and accumulates errors in `stats.partialErrors`. See [Partial Response Support](../../api-reference/modules/api-reference-12-partial-responses.md) for details.
+
+### ExtractionResult
+
+```typescript
+interface ExtractionResult<T = any> {
+  data: T[]; // Extracted records
+  stats: ExtractionStats; // Statistics
+  errors?: ExtractionError[]; // Non-fatal errors (validation failures)
+}
+
+interface ExtractionStats {
+  totalRecords: number; // Total records extracted
+  totalPages: number; // Number of pages fetched
+  duration: number; // Extraction duration in milliseconds
+  truncated: boolean; // Whether extraction was limited
+  truncationReason?: 'maxPages' | 'maxRecords' | 'timeout'; // Reason for truncation
+  validRecords?: number; // Valid records (if validation used)
+  invalidRecords?: number; // Invalid records (if validation used)
+  direction?: 'forward' | 'backward'; // Pagination direction used
+  partialErrors?: GraphQLError[]; // GraphQL errors (only when errorHandling: 'partial')
+}
+
+interface ExtractionError {
+  type: 'graphql' | 'network' | 'validation' | 'parsing'; // Error type
+  message: string; // Error message
+  recordIndex?: number; // Record index (for validation errors)
+  details?: any; // Additional error details
+}
+```
+
+## 📋 Usage Patterns
+
+### Pattern 1: Simple Extraction
+
+```typescript
+const orchestrator = new ExtractionOrchestrator(client, console);
+
+const result = await orchestrator.extract({
+  query: `
+    query GetProducts($first: Int!, $after: String) {
+      products(first: $first, after: $after) {
+        edges {
+          node {
+            id
+            ref
+            name
+            type
+            status
+          }
+          cursor
+        }
+        pageInfo { hasNextPage }
+      }
+    }
+  `,
+  resultPath: 'products.edges.node',
+  pageSize: 100,
+});
+
+console.log(`Extracted ${result.data.length} products`);
+```
+
+### Pattern 2: Extraction with Limits
+
+```typescript
+// Extract first 5000 records only
+const result = await orchestrator.extract({
+  query: inventoryQuery,
+  resultPath: 'inventoryPositions.edges.node',
+  maxRecords: 5000,
+  pageSize: 500, // Use larger pages for efficiency
+});
+
+if (result.stats.truncated) {
+  console.log('Extraction was limited to 5000 records');
+}
+```
+
+### Pattern 3: Extraction with Validation
+
+```typescript
+const result = await orchestrator.extract({
+  query: virtualPositionsQuery,
+  resultPath: 'virtualPositions.edges.node',
+
+  // Validate each item during extraction
+  validateItem: item => {
+    return (
+      item.ref && // ref is required
+      item.productRef && // productRef is required
+      typeof item.qty === 'number' && // qty must be a number
+      item.qty >= 0 // qty must be non-negative
+    );
+  },
+});
+
+console.log(`Valid records: ${result.stats.validRecords}`);
+console.log(`Invalid records: ${result.stats.invalidRecords}`);
+
+// Review validation errors (recordIndex, type, details)
+if (result.errors && result.errors.length > 0) {
+  console.log(
+    'Validation errors:',
+    result.errors.map(e => ({
+      type: e.type,
+      recordIndex: e.recordIndex,
+      details: e.details,
+    }))
+  );
+}
+```
+
+### Pattern 4: Extraction with Variables
+
+```typescript
+// Extract orders for specific retailer and date range
+const result = await orchestrator.extract({
+  query: `
+    query GetOrders(
+      $first: Int!,
+      $after: String,
+      $retailerId: ID!,
+      $startDate: DateTime!
+    ) {
+      orders(
+        first: $first,
+        after: $after,
+        retailerId: $retailerId,
+        createdOn: { from: $startDate }
+      ) {
+        edges {
+          node {
+            id
+            ref
+            status
+            createdOn
+            totalPrice
+          }
+          cursor
+        }
+        pageInfo { hasNextPage }
+      }
+    }
+  `,
+  resultPath: 'orders.edges.node',
+  variables: {
+    startDate: '2025-01-01T00:00:00Z',
+  },
+  pageSize: 100,
+});
+```
+
+### Pattern 5: Nested Path Extraction
+
+```typescript
+// Extract from deeply nested response
+const result = await orchestrator.extract({
+  query: `
+    query GetCatalogues($first: Int!, $after: String) {
+      catalogues(first: $first, after: $after) {
+        edges {
+          node {
+            id
+            ref
+            name
+            products {
+              edges {
+                node {
+                  id
+                  ref
+                  name
+                }
+              }
+            }
+          }
+          cursor
+        }
+        pageInfo { hasNextPage }
+      }
+    }
+  `,
+  // Extract products from catalogues
+  resultPath: 'catalogues.edges.node.products.edges.node',
+  pageSize: 50,
+});
+```
+
+## 🎯 Real-World Examples
+
+### Example 1: Virtual Positions Extraction for SFTP Export
+
+```typescript
+import {
+  ExtractionOrchestrator,
+  createClient,
+  SftpDataSource,
+  createConsoleLogger,
+  toStructuredLogger
+} from '@fluentcommerce/fc-connect-sdk';
+
+async function exportVirtualPositionsToSftp() {
+  // Initialize services
+  const logger = toStructuredLogger(createConsoleLogger(), {
+    logLevel: 'info'
+  });
+  const client = await createClient({
+    config: {
+      baseUrl: process.env.FLUENT_BASE_URL!,
+      clientId: process.env.FLUENT_CLIENT_ID!,
+      clientSecret: process.env.FLUENT_CLIENT_SECRET!
+      // retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
+    }
+  });
+
+  const sftp = new SftpDataSource(
+    {
+      type: 'SFTP_CSV',
+      sftpConfig: {
+        settings: {
+          host: process.env.SFTP_HOST!,
+          port: 22,
+          username: process.env.SFTP_USERNAME!,
+          password: process.env.SFTP_PASSWORD!,
+        },
+      },
+      connectionId: 'sftp-export',
+    },
+    logger
+  );
+
+  // Extract virtual positions
+  const orchestrator = new ExtractionOrchestrator(client, logger);
+
+  const result = await orchestrator.extract({
+    query: `
+      query GetVirtualPositions($first: Int!, $after: String) {
+        virtualPositions(first: $first, after: $after) {
+          edges {
+            node {
+              id
+              ref
+              productRef
+              qty
+              groupRef
+              createdOn
+              updatedOn
+            }
+            cursor
+          }
+          pageInfo {
+            hasNextPage
+            # Note: Fluent doesn't return endCursor - use edge cursors instead
+          }
+        }
+      }
+    `,
+    resultPath: 'virtualPositions.edges.node',
+    pageSize: 500,
+    maxRecords: 50000,
+
+    // Validate data quality
+    validateItem: item => {
+      return item.ref && item.productRef && typeof item.qty === 'number';
+    },
+  });
+
+  logger.info('Extraction complete', {
+    totalRecords: result.stats.totalRecords,
+    totalPages: result.stats.totalPages,
+    duration: result.stats.duration,
+    validRecords: result.stats.validRecords,
+    invalidRecords: result.stats.invalidRecords,
+  });
+
+  // Convert to CSV format
+  const csv = [
+    // Header
+    'ref,productRef,qty,groupRef,createdOn,updatedOn',
+    // Data rows
+    ...result.data.map(
+      item =>
+        `${item.ref},${item.productRef},${item.qty},${item.groupRef},${item.createdOn},${item.updatedOn}`
+    ),
+  ].join('\n');
+
+  // Upload to SFTP
+  try {
+    const filename = `/exports/virtual-positions-${new Date().toISOString()}.csv`;
+    await sftp.uploadFile(filename, csv);
+
+    logger.info('Export complete', { filename, recordCount: result.data.length });
+  } finally {
+    // Always dispose SFTP connection to prevent connection pool leaks
+    await sftp.dispose();
+  }
+}
+```
+
+### Example 2: Multi-Query Extraction Pattern
+
+```typescript
+// For multiple queries, call ExtractionOrchestrator separately for each
+async function extractMultipleEntities() {
+  const orchestrator = new ExtractionOrchestrator(client, logger);
+
+  // Execute extractions in parallel
+  const [products, locations, inventories] = await Promise.all([
+    // Extract products
+    orchestrator.extract({
+      query: productsQuery,
+      resultPath: 'products.edges.node',
+      pageSize: 200,
+    }),
+
+    // Extract locations
+    orchestrator.extract({
+      query: locationsQuery,
+      resultPath: 'locations.edges.node',
+      pageSize: 100,
+    }),
+
+    // Extract inventory positions
+    orchestrator.extract({
+      query: inventoryQuery,
+      resultPath: 'inventoryPositions.edges.node',
+      maxRecords: 10000,
+      pageSize: 500,
+    }),
+  ]);
+
+  // Combine or process separately in your business logic
+  return {
+    products: products.data,
+    locations: locations.data,
+    inventories: inventories.data,
+    stats: {
+      totalProducts: products.stats.totalRecords,
+      totalLocations: locations.stats.totalRecords,
+      totalInventories: inventories.stats.totalRecords,
+      totalDuration:
+        products.stats.duration + locations.stats.duration + inventories.stats.duration,
+    },
+  };
+}
+```
+
+### Example 3: Extraction with Transformation
+
+```typescript
+import { ExtractionOrchestrator, UniversalMapper } from '@fluentcommerce/fc-connect-sdk';
+
+async function extractAndTransform() {
+  const orchestrator = new ExtractionOrchestrator(client, logger);
+
+  // Step 1: Extract
+  const extraction = await orchestrator.extract({
+    query: virtualPositionsQuery,
+    resultPath: 'virtualPositions.edges.node',
+    pageSize: 500,
+  });
+
+  // Step 2: Transform with UniversalMapper
+  const mapper = new UniversalMapper({
+    fields: {
+      reference: { source: 'ref', required: true },
+      product: { source: 'productRef', required: true },
+      quantity: { source: 'qty', resolver: 'sdk.parseInt' },
+      group: { source: 'groupRef', resolver: 'sdk.trim' },
+      lastUpdated: { source: 'updatedOn', resolver: 'sdk.formatDate' },
+    },
+  });
+
+  const transformation = await mapper.map(extraction.data);
+
+  if (!transformation.success) {
+    logger.error('Transformation failed', { errors: transformation.errors });
+    throw new Error('Data transformation failed');
+  }
+
+  return transformation.data;
+}
+```
+
+## ⚠️ Error Handling
+
+### Handling Extraction Errors
+
+```typescript
+try {
+  const result = await orchestrator.extract(options);
+
+  // Check for validation errors
+  if (result.errors && result.errors.length > 0) {
+    logger.warn(`${result.errors.length} records failed validation`);
+
+    // Log first few errors for debugging
+    result.errors.slice(0, 5).forEach(err => {
+      logger.debug('Validation error', {
+        type: err.type,
+        message: err.message,
+        recordIndex: err.recordIndex,
+        details: err.details,
+      });
+    });
+  }
+
+  // Check if extraction was truncated
+  if (result.stats.truncated) {
+    logger.info('Extraction was limited by maxRecords or maxPages');
+  }
+
+  return result.data;
+} catch (error) {
+  // Fatal errors (GraphQL errors, network issues, invalid config)
+  logger.error('Extraction failed', error as Error);
+  throw error;
+}
+```
+
+### Partial Response Mode (errorHandling: 'partial')
+
+When GraphQL returns errors but some data is available, you can use partial mode to continue extraction:
+
+```typescript
+const result = await orchestrator.extract({
+  query: ORDERS_QUERY,
+  resultPath: 'orders.edges.node',
+  errorHandling: 'partial', // Continue extraction even with errors
+});
+
+// Check for partial errors in stats
+if (result.stats.partialErrors) {
+  logger.warn('Extraction completed with partial errors', {
+    recordsExtracted: result.data.length,
+    errorCount: result.stats.partialErrors.length,
+  });
+
+  // Process available data
+  console.log(`Extracted ${result.data.length} records despite errors`);
+
+  // Log errors for investigation
+  result.stats.partialErrors.forEach(err => {
+    logger.warn('GraphQL error during extraction', {
+      message: err.message,
+      path: err.path,
+    });
+  });
+}
+```
+
+**Edge Case: Null Data with Errors**
+
+When GraphQL returns errors with `data: null` or `data: undefined`:
+
+```typescript
+// GraphQL response: { data: null, errors: [{ message: 'Order item not found' }] }
+const result = await orchestrator.extract({
+  query: ORDERS_QUERY,
+  resultPath: 'orders.edges.node',
+  errorHandling: 'partial',
+});
+
+// Returns empty array with errors in stats
+expect(result.data).toEqual([]);
+expect(result.stats.partialErrors).toHaveLength(1);
+```
+
+**Behavior:**
+- **Partial mode**: Returns empty array `[]`, accumulates errors in `stats.partialErrors`, stops pagination if no data
+- **Throw mode** (default): Throws `GraphQLExecutionError` immediately (backward compatible)
+
+**When to Use Partial Mode:**
+- ✅ Batch queries where some items may fail independently
+- ✅ Reporting/analytics where partial data is better than none
+- ✅ Non-critical extractions where you want to process available data
+
+**When to Use Throw Mode (default):**
+- ✅ Critical data where consistency matters
+- ✅ Single record operations
+- ✅ When you need to fail fast on any errors
+
+### Common Error Scenarios
+
+| Error                          | Cause                        | Solution                                          |
+| ------------------------------ | ---------------------------- | ------------------------------------------------- |
+| `Query is required`            | Empty query string           | Provide valid GraphQL query                       |
+| `Result path is required`      | Empty resultPath             | Specify path to extract (e.g., 'data.edges.node') |
+| `Page size must be positive`   | pageSize ≤ 0                 | Use pageSize ≥ 1                                  |
+| `Max records must be positive` | maxRecords ≤ 0               | Use maxRecords ≥ 1                                |
+| `Max pages must be positive`   | maxPages ≤ 0                 | Use maxPages ≥ 1                                  |
+| `Timeout must be positive`     | timeout ≤ 0                  | Use timeout ≥ 1000 (1 second minimum)             |
+| `Path extraction failed`       | Invalid resultPath           | Check path matches response structure             |
+| `GraphQL Error: ...`           | Invalid query or permissions | Validate query with schema introspection          |
+
+## 🎓 Best Practices
+
+### 1. Choose Appropriate Page Size
+
+```typescript
+// ❌ Too small - many API calls
+pageSize: 10;
+
+// ✅ Balanced - good for most use cases
+pageSize: 200;
+
+// ✅ Large datasets - fewer API calls
+pageSize: 500;
+
+// ⚠️ Very large - may cause memory issues
+pageSize: 1000;
+```
+
+### 2. Use Extraction Limits for Large Datasets
+
+```typescript
+// ✅ Prevent unbounded extraction
+const result = await orchestrator.extract({
+  query: largeDatasetQuery,
+  resultPath: 'data.edges.node',
+  maxRecords: 100000, // Hard limit
+  maxPages: 200, // Backup limit
+  timeout: 300000, // 5 minute timeout
+});
+```
+
+### 3. Implement Validation for Data Quality
+
+```typescript
+// ✅ Validate critical fields during extraction
+validateItem: item => {
+  // Required fields present
+  if (!item.ref || !item.productRef) return false;
+
+  // Valid data types
+  if (typeof item.qty !== 'number') return false;
+
+  // Business logic validation
+  if (item.qty < 0) return false;
+
+  return true;
+};
+```
+
+### 4. Monitor Statistics
+
+```typescript
+const result = await orchestrator.extract(options);
+
+// Log comprehensive statistics
+logger.info('Extraction complete', {
+  totalRecords: result.stats.totalRecords,
+  totalPages: result.stats.totalPages,
+  duration: result.stats.duration,
+  validRecords: result.stats.validRecords,
+  invalidRecords: result.stats.invalidRecords,
+  truncated: result.stats.truncated,
+  truncationReason: result.stats.truncationReason,
+
+  // Calculate average records per page
+  avgRecordsPerPage: Math.round(result.stats.totalRecords / result.stats.totalPages),
+});
+```
+
+### 5. Use with JobTracker for Async Workflows
+
+```typescript
+import { ExtractionOrchestrator, JobTracker } from '@fluentcommerce/fc-connect-sdk';
+
+async function scheduledExtraction() {
+  const tracker = new JobTracker(kvAdapter, logger);
+
+  try {
+    // Start job
+    const job = await tracker.startJob({
+      triggeredBy: 'schedule',
+      details: { type: 'virtual-positions-extract' },
+    });
+
+    // Start extraction
+    await tracker.update(job.id, {
+      status: 'PROCESSING',
+      stage: 'extracting',
+    });
+
+    const orchestrator = new ExtractionOrchestrator(client, logger);
+    const result = await orchestrator.extract({
+      query: virtualPositionsQuery,
+      resultPath: 'virtualPositions.edges.node',
+      maxRecords: 50000,
+    });
+
+    // Complete
+    await tracker.complete(job.id, {
+      recordsExtracted: result.stats.totalRecords,
+      totalPages: result.stats.totalPages,
+      duration: result.stats.duration,
+    });
+
+    return result;
+  } catch (error) {
+    // Mark failed via update to avoid relying on specific error helpers
+    await tracker.update(job.id, { status: 'FAILED', error: (error as Error).message });
+    throw error;
+  }
+}
+```
+
+## 🔗 Related Documentation
+
+- [Module 2: Basic Extraction](./02-core-guides-extraction-02-basic-extraction.md) - Manual GraphQL extraction
+- [Module 4: Data Enrichment](./02-core-guides-extraction-04-data-enrichment.md) - Multi-query patterns
+- [Module 5: Transformation](./02-core-guides-extraction-05-transformation.md) - UniversalMapper guide
+- [Job Tracker](../../advanced-services/advanced-services-job-tracker.md) - Async job management
+- [Auto-Pagination Guide](../../auto-pagination/) - Pagination internals
+
+## 📊 Performance Considerations
+
+### Memory Usage
+
+```typescript
+// ⚠️ High memory - stores all records in memory
+const result = await orchestrator.extract({
+  query: myQuery,
+  resultPath: 'data.edges.node',
+  maxRecords: 1000000, // 1 million records!
+});
+
+// ✅ Better - use maxRecords to limit extraction
+// Note: ExtractionOrchestrator handles pagination internally
+// For processing in chunks, use multiple extraction calls with date ranges
+async function extractByDateRange() {
+  const dateRanges = [
+    { from: '2025-01-01', to: '2025-01-31' },
+    { from: '2025-02-01', to: '2025-02-28' },
+    // ... more ranges
+  ];
+
+  for (const range of dateRanges) {
+    const result = await orchestrator.extract({
+      query: myQuery,
+      resultPath: 'data.edges.node',
+      variables: {
+        dateRangeFilter: range,
+      },
+      maxRecords: 100000, // Limit per date range
+    });
+
+    // Process chunk
+    await processChunk(result.data);
+  }
+}
+```
+
+### API Rate Limiting
+
+```typescript
+// Add delay between large extractions
+async function extractMultipleWithDelay() {
+  const queries = [query1, query2, query3];
+  const results = [];
+
+  for (const query of queries) {
+    const result = await orchestrator.extract({
+      query,
+      resultPath: 'data.edges.node',
+      maxRecords: 50000,
+    });
+
+    results.push(result);
+
+    // Delay 1 second between extractions
+    await new Promise(resolve => setTimeout(resolve, 1000));
+  }
+
+  return results;
+}
+```
+
+## ✅ Summary
+
+**ExtractionOrchestrator simplifies:**
+
+- ✅ GraphQL query execution with auto-pagination
+- ✅ Path-based extraction from nested responses
+- ✅ Automatic edges.node pattern handling
+- ✅ Extraction limits and validation
+- ✅ Statistics tracking and error handling
+
+**Use it when you need:**
+
+- Single-query data extraction
+- Automatic pagination handling
+- Path-based result extraction
+- Built-in validation and statistics
+
+**For complex scenarios:**
+
+- Multiple queries → Make separate calls, use `Promise.all()`
+- Business logic → Use `UniversalMapper` for transformation
+- Real-time streaming → Use direct `client.graphql()` calls
+
+---
+
+**Next Steps:**
+
+- [Job Tracker](../../advanced-services/advanced-services-job-tracker.md) - Async job status tracking
+- [Module 5: Transformation](./02-core-guides-extraction-05-transformation.md) - Transform extracted data
+- [Examples Directory](../examples/) - Working code examples