@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

package/docs/02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-02-basic-extraction.md

@@ -1,823 +1,823 @@
-# Module 2: Basic Extraction
-
-**Level:** Beginner
-**Estimated Time:** 20 minutes
-
-## Overview
-
-This module teaches you how to perform basic data extraction from Fluent Commerce using validated GraphQL queries and automatic pagination.
-
-**Note:** For most extraction use cases, the **ExtractionOrchestrator** (covered in [Module 8](./02-core-guides-extraction-08-extraction-orchestrator.md)) provides a simpler, higher-level API. This module shows the manual approach for cases where you need fine-grained control.
-
-## Learning Objectives
-
-By the end of this module, you will:
-
-- ✅ Write validated GraphQL extraction queries
-- ✅ Use auto-pagination to fetch large datasets
-- ✅ Extract common Fluent entities (inventory, products, orders)
-- ✅ Handle pagination responses correctly
-- ✅ Process extracted data
-- ✅ Understand when to use manual extraction vs ExtractionOrchestrator
-
-## Your First Extraction Query
-
-### Step 1: Initialize the Client
-
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-
-// Create client (auto-detects Node.js, Deno, or Versori context)
-const client = await createClient({
-  config: {
-    baseUrl: process.env.FLUENT_BASE_URL!,
-    clientId: process.env.FLUENT_CLIENT_ID!,
-    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
-    username: process.env.FLUENT_USERNAME!,
-    password: process.env.FLUENT_PASSWORD!
-    // Note: retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
-  }
-});
-```
-
-### Step 2: Write a Simple Query
-
-```typescript
-const query = `
-  query ExtractVirtualPositions($first: Int!, $after: String) {
-    virtualPositions(first: $first, after: $after) {
-      edges {
-        node {
-          id
-          ref
-          productRef
-          groupRef
-          quantity
-          status
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-      }
-    }
-  }
-`;
-```
-
-**Field Validation:**
-
-- ✅ `id` - VirtualPosition.id (ID!)
-- ✅ `ref` - VirtualPosition.ref (String!)
-- ✅ `productRef` - VirtualPosition.productRef (String!)
-- ✅ `groupRef` - VirtualPosition.groupRef (String)
-- ✅ `quantity` - VirtualPosition.quantity (Int!)
-- ✅ `status` - VirtualPosition.status (String)
-
-> **⚠️ IMPORTANT: VirtualPosition vs InventoryPosition Field Differences**
->
-> These two types have **different field names** for quantity:
-> - **VirtualPosition**: Uses `quantity` (Int!)
-> - **InventoryPosition**: Uses `qty` (Int!) and `onHand` (Int!)
->
-> Always check the GraphQL schema when switching between entity types!
-
-### Step 3: Execute with Auto-Pagination
-
-```typescript
-const result = await client.graphql({
-  query,
-  variables: { first: 100 },
-  pagination: {
-    maxRecords: 10000, // Stop after 10K records
-    maxPages: 100, // OR stop after 100 pages
-    onProgress: (page, totalRecords) => {
-      console.log(`Fetched page ${page}: ${totalRecords} records so far`);
-    },
-  },
-});
-```
-
-### Step 4: Process the Results
-
-```typescript
-// Auto-pagination merges all pages into single response
-const allRecords = result.data.virtualPositions.edges.map(edge => edge.node);
-
-console.log(`Total records: ${allRecords.length}`);
-console.log(`Pages fetched: ${result.extensions.pagination.totalPages}`);
-console.log(`Time elapsed: ${result.extensions.pagination.timeElapsed}ms`);
-
-// Process each record
-for (const record of allRecords) {
-  console.log(`Product ${record.productRef}: ${record.quantity} units`);
-}
-```
-
-## Complete Example
-
-### Approach 1: Using ExtractionOrchestrator (Recommended)
-
-For most use cases, use the high-level ExtractionOrchestrator:
-
-```typescript
-import { createClient, ExtractionOrchestrator } from '@fluentcommerce/fc-connect-sdk';
-
-async function extractVirtualPositions() {
-  const client = await createClient({
-    config: {
-      baseUrl: process.env.FLUENT_BASE_URL!,
-      clientId: process.env.FLUENT_CLIENT_ID!,
-      clientSecret: process.env.FLUENT_CLIENT_SECRET!
-      // retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
-    },
-  });
-
-  const orchestrator = new ExtractionOrchestrator(client, console);
-
-  try {
-    const result = await orchestrator.extract({
-      query: `
-        query ExtractVirtualPositions($first: Int!, $after: String) {
-          virtualPositions(first: $first, after: $after) {
-            edges {
-              node {
-                id ref productRef groupRef quantity status
-                createdOn updatedOn
-              }
-              cursor
-            }
-            pageInfo { hasNextPage }
-          }
-        }
-      `,
-      resultPath: 'virtualPositions.edges.node',
-      pageSize: 100,
-      maxRecords: 10000
-    });
-
-    console.log('\n=== Extraction Summary ===');
-    console.log(`Total records: ${result.data.length}`);
-    console.log(`Pages: ${result.stats.totalPages}`);
-    console.log(`Duration: ${result.stats.duration}ms`);
-
-    return result.data;
-  } catch (error) {
-    console.error('Extraction failed:', error);
-    throw error;
-  }
-}
-
-extractVirtualPositions()
-  .then(() => console.log('Extraction complete'))
-  .catch(error => console.error('Error:', error));
-```
-
-### Approach 2: Manual Extraction (For Advanced Control)
-
-For fine-grained control over pagination and response handling:
-
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-
-async function extractVirtualPositions() {
-  // Initialize client
-  const client = await createClient({
-    config: {
-      baseUrl: process.env.FLUENT_BASE_URL!,
-      clientId: process.env.FLUENT_CLIENT_ID!,
-      clientSecret: process.env.FLUENT_CLIENT_SECRET!
-      // retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
-    },
-  });
-
-  // Define query
-  const query = `
-    query ExtractVirtualPositions($first: Int!, $after: String) {
-      virtualPositions(first: $first, after: $after) {
-        edges {
-          node {
-            id
-            ref
-            productRef
-            groupRef
-            quantity
-            status
-            createdOn
-            updatedOn
-          }
-          cursor
-        }
-        pageInfo {
-          hasNextPage
-        }
-      }
-    }
-  `;
-
-  try {
-    // Execute with auto-pagination
-    const result = await client.graphql({
-      query,
-      variables: { first: 100 },
-      pagination: {
-        maxRecords: 10000,
-        onProgress: (page, totalRecords) => {
-          console.log(`Page ${page}: ${totalRecords} records`);
-        },
-      },
-    });
-
-    // Extract records
-    const records = result.data.virtualPositions.edges.map(e => e.node);
-
-    console.log('\n=== Extraction Summary ===');
-    console.log(`Total records: ${records.length}`);
-    console.log(`Pages: ${result.extensions.pagination.totalPages}`);
-    console.log(`Time: ${result.extensions.pagination.timeElapsed}ms`);
-
-    // Show sample records
-    console.log('\n=== Sample Records ===');
-    records.slice(0, 5).forEach(record => {
-      console.log(`- ${record.ref}: ${record.quantity} units of ${record.productRef}`);
-    });
-
-    return records;
-  } catch (error) {
-    console.error('Extraction failed:', error);
-    throw error;
-  }
-}
-
-// Run extraction
-extractVirtualPositions()
-  .then(() => console.log('Extraction complete'))
-  .catch(error => console.error('Error:', error));
-```
-
-## Common Entity Extractions
-
-### 1. Inventory Positions
-
-```typescript
-const query = `
-  query ExtractInventory($first: Int!, $after: String) {
-    inventoryPositions(first: $first, after: $after) {
-      edges {
-        node {
-          id
-          ref
-          productRef
-          locationRef
-          qty              # ✅ Note: 'qty' not 'quantity'
-          status
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-      }
-    }
-  }
-`;
-
-const result = await client.graphql({
-  query,
-  variables: { first: 100 },
-  pagination: { maxRecords: 50000 },
-});
-
-const inventory = result.data.inventoryPositions.edges.map(e => e.node);
-```
-
-### 2. Products
-
-```typescript
-const query = `
-  query ExtractProducts($first: Int!, $after: String) {
-    products(first: $first, after: $after) {
-      edges {
-        node {
-          id
-          ref
-          name
-          gtin
-          status
-          type
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-      }
-    }
-  }
-`;
-
-const result = await client.graphql({
-  query,
-  variables: { first: 50 }, // Smaller page size for products
-  pagination: { maxRecords: 10000 },
-});
-
-const products = result.data.products.edges.map(e => e.node);
-```
-
-### 3. Orders
-
-```typescript
-const query = `
-  query ExtractOrders($first: Int!, $after: String) {
-    orders(first: $first, after: $after) {
-      edges {
-        node {
-          id
-          ref
-          type
-          status
-          totalPrice
-          totalTaxPrice
-          currency
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-      }
-    }
-  }
-`;
-
-const result = await client.graphql({
-  query,
-  variables: { first: 50 },
-  pagination: { maxRecords: 20000 },
-});
-
-const orders = result.data.orders.edges.map(e => e.node);
-```
-
-### 4. Locations
-
-```typescript
-const query = `
-  query ExtractLocations($first: Int!, $after: String) {
-    locations(first: $first, after: $after) {
-      edges {
-        node {
-          id
-          ref
-          name
-          type
-          status
-          createdOn
-          updatedOn
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-      }
-    }
-  }
-`;
-
-const result = await client.graphql({
-  query,
-  variables: { first: 100 },
-  pagination: { maxRecords: 5000 },
-});
-
-const locations = result.data.locations.edges.map(e => e.node);
-```
-
-## Understanding Pagination Responses
-
-### Response Structure
-
-```typescript
-{
-  data: {
-    virtualPositions: {
-      edges: [
-        {
-          node: { id: "1", ref: "VP-001", ... },
-          cursor: "abc123"
-        },
-        {
-          node: { id: "2", ref: "VP-002", ... },
-          cursor: "def456"
-        }
-      ],
-      pageInfo: {
-        hasNextPage: true
-      }
-    }
-  },
-  extensions: {
-    autoPagination: {
-      totalPages: 15,
-      totalRecords: 1487,
-      truncated: false,
-      timeElapsed: 8324
-    }
-  }
-}
-```
-
-### Auto-Pagination Extensions
-
-The SDK adds `extensions.autoPagination` metadata:
-
-| Field          | Type    | Description                     |
-| -------------- | ------- | ------------------------------- |
-| `totalPages`   | number  | Number of pages fetched         |
-| `totalRecords` | number  | Total records returned          |
-| `truncated`    | boolean | True if stopped by safety limit |
-| `duration`     | number  | Total time in milliseconds      |
-
-### Accessing Data
-
-```typescript
-// Get all records
-const allRecords = result.data.virtualPositions.edges.map(e => e.node);
-
-// Check if truncated
-if (result.extensions.pagination.truncated) {
-  console.warn('Results truncated by safety limit');
-  console.log(`Fetched ${result.extensions.pagination.totalRecords} of total`);
-}
-
-// Performance metrics
-console.log(
-  `Fetched ${result.extensions.pagination.totalRecords} records in ${result.extensions.pagination.timeElapsed}ms`
-);
-console.log(
-  `Rate: ${((result.extensions.pagination.totalRecords / result.extensions.pagination.timeElapsed) * 1000).toFixed(0)} records/sec`
-);
-```
-
-## Filtering Results
-
-### GraphQL Filters
-
-```typescript
-// Filter by status
-const query = `
-  query ExtractActiveInventory($first: Int!, $after: String, $filter: JSON) {
-    inventoryPositions(
-      first: $first
-      after: $after
-      filter: $filter
-    ) {
-      edges {
-        node { id ref qty status }
-        cursor
-      }
-      pageInfo { hasNextPage }
-    }
-  }
-`;
-
-const result = await client.graphql({
-  query,
-  variables: {
-    first: 100,
-    filter: {
-      status: { equalTo: 'AVAILABLE' },
-    },
-  },
-  pagination: { maxRecords: 50000 },
-});
-```
-
-### Date Range Filters
-
-```typescript
-// Filter by date range
-const fromDate = new Date('2024-01-01').toISOString();
-const toDate = new Date('2024-12-31').toISOString();
-
-const query = `
-  query ExtractRecentOrders($first: Int!, $after: String, $from: DateTime!, $to: DateTime!) {
-    orders(
-      first: $first
-      after: $after
-      createdOn: { from: $from, to: $to }
-    ) {
-      edges {
-        node { id ref createdOn totalPrice }
-        cursor
-      }
-      pageInfo { hasNextPage }
-    }
-  }
-`;
-
-const result = await client.graphql({
-  query,
-  variables: {
-    first: 100,
-    from: fromDate,
-    to: toDate,
-  },
-  pagination: { maxRecords: 20000 },
-});
-```
-
-### Complex Filters
-
-```typescript
-// Multiple filter conditions
-const query = `
-  query ExtractFilteredProducts($first: Int!, $after: String, $filter: JSON) {
-    products(
-      first: $first
-      after: $after
-      filter: $filter
-    ) {
-      edges {
-        node { id ref name status type }
-        cursor
-      }
-      pageInfo { hasNextPage }
-    }
-  }
-`;
-
-const result = await client.graphql({
-  query,
-  variables: {
-    first: 50,
-    filter: {
-      AND: [{ status: { equalTo: 'ACTIVE' } }, { type: { equalTo: 'STANDARD' } }],
-    },
-  },
-  pagination: { maxRecords: 10000 },
-});
-```
-
-## Error Handling
-
-### Comprehensive Error Handling with Retry Logic
-
-```typescript
-import { createClient } from '@fluentcommerce/fc-connect-sdk';
-
-async function extractWithErrorHandling() {
-  const client = await createClient({
-    config: {
-      baseUrl: process.env.FLUENT_BASE_URL!,
-      clientId: process.env.FLUENT_CLIENT_ID!,
-      clientSecret: process.env.FLUENT_CLIENT_SECRET!,
-      username: process.env.FLUENT_USERNAME!,
-      password: process.env.FLUENT_PASSWORD!
-    }
-  });
-
-  const query = `
-    query ExtractVirtualPositions($first: Int!, $after: String) {
-      virtualPositions(first: $first, after: $after) {
-        edges {
-          node {
-            id ref productRef groupRef quantity status
-            createdOn updatedOn
-          }
-          cursor
-        }
-        pageInfo { hasNextPage }
-      }
-    }
-  `;
-
-  try {
-    const result = await client.graphql({
-      query,
-      variables: { first: 100 },
-      pagination: { maxRecords: 10000 },
-    });
-
-    // Check for GraphQL errors
-    if (result.errors && result.errors.length > 0) {
-      console.error('GraphQL errors detected:', {
-        errorCount: result.errors.length,
-        errors: result.errors.map(e => ({
-          message: e.message,
-          path: e.path,
-          extensions: e.extensions
-        }))
-      });
-      throw new Error(`GraphQL query failed: ${result.errors[0].message}`);
-    }
-
-    // Validate data structure
-    if (!result.data?.virtualPositions?.edges) {
-      throw new Error('Unexpected response structure - missing virtualPositions.edges');
-    }
-
-    const records = result.data.virtualPositions.edges.map(e => e.node);
-    console.log(`✅ Extracted ${records.length} records successfully`);
-
-    return records;
-  } catch (error) {
-    console.error('Extraction failed:', {
-      error: error.message,
-      errorType: error.constructor.name
-    });
-
-    // Check for specific error types with appropriate handling
-    if (error.message.includes('authentication') || error.message.includes('401')) {
-      console.error('Authentication failed - check credentials');
-      throw new Error('Authentication failure - verify OAuth2 credentials');
-    } else if (error.message.includes('timeout') || error.code === 'TIMEOUT_ERROR') {
-      console.error('Request timeout - try smaller page size');
-      throw new Error('Timeout error - reduce pagination page size');
-    } else if (error.message.includes('rate limit') || error.code === 'RATE_LIMIT_ERROR') {
-      console.error('Rate limit exceeded - implement exponential backoff');
-      throw new Error('Rate limit error - wait before retrying');
-    } else if (error.message.includes('network') || error.code === 'ECONNREFUSED') {
-      console.error('Network error - check connectivity');
-      throw new Error('Network error - verify API endpoint');
-    } else {
-      throw error;
-    }
-  }
-}
-
-// Retry logic for transient errors
-async function extractWithRetry(maxRetries: number = 3) {
-  let lastError: Error;
-
-  for (let attempt = 0; attempt < maxRetries; attempt++) {
-    try {
-      return await extractWithErrorHandling();
-    } catch (error) {
-      lastError = error;
-
-      // Only retry for transient errors
-      if (error.message.includes('timeout') ||
-          error.message.includes('rate limit') ||
-          error.message.includes('network')) {
-
-        const delay = Math.pow(2, attempt) * 1000; // Exponential backoff
-        console.log(`Retry attempt ${attempt + 1}/${maxRetries} after ${delay}ms`);
-        await new Promise(resolve => setTimeout(resolve, delay));
-      } else {
-        // Don't retry for authentication or validation errors
-        throw error;
-      }
-    }
-  }
-
-  throw lastError!;
-}
-```
-
-### Validation Errors with Detailed Logging
-
-```typescript
-async function extractWithValidation() {
-  try {
-    const result = await client.graphql({ query, variables });
-
-    // Check for GraphQL errors
-    if (result.errors && result.errors.length > 0) {
-      console.error('GraphQL errors:', {
-        query: query.substring(0, 100) + '...',
-        variables,
-        errors: result.errors.map(e => ({
-          message: e.message,
-          path: e.path
-        }))
-      });
-      throw new Error('Query returned GraphQL errors');
-    }
-
-    // Validate data structure
-    if (!result.data?.virtualPositions?.edges) {
-      console.error('Invalid response structure:', {
-        hasData: !!result.data,
-        hasVirtualPositions: !!result.data?.virtualPositions,
-        actualStructure: Object.keys(result.data || {})
-      });
-      throw new Error('Unexpected response structure');
-    }
-
-    const records = result.data.virtualPositions.edges.map(e => e.node);
-
-    if (records.length === 0) {
-      console.warn('No records found - verify query filters');
-    }
-
-    return records;
-  } catch (error) {
-    console.error('Validation error:', error.message);
-    throw error;
-  }
-}
-```
-
-## Practice Exercise
-
-**Goal**: Extract all products created in the last 30 days
-
-**Requirements:**
-
-1. Query the `products` connection
-2. Filter by `createdOn` date range
-3. Include fields: `id`, `ref`, `name`, `status`, `createdOn`
-4. Use auto-pagination with max 5000 records
-5. Log progress every page
-
-**Solution:**
-
-```typescript
-async function extractRecentProducts() {
-  const client = await createClient({ config });
-
-  const thirtyDaysAgo = new Date();
-  thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30);
-
-  const query = `
-    query ExtractRecentProducts($first: Int!, $after: String, $from: DateTime!) {
-      products(
-        first: $first
-        after: $after
-        createdOn: { from: $from }
-      ) {
-        edges {
-          node {
-            id
-            ref
-            name
-            status
-            createdOn
-          }
-          cursor
-        }
-        pageInfo {
-          hasNextPage
-        }
-      }
-    }
-  `;
-
-  const result = await client.graphql({
-    query,
-    variables: {
-      first: 50,
-      from: thirtyDaysAgo.toISOString(),
-    },
-    pagination: {
-      maxRecords: 5000,
-      onProgress: (page, totalRecords) => {
-        console.log(`Page ${page}: ${totalRecords} products fetched`);
-      },
-    },
-  });
-
-  const products = result.data.products.edges.map(e => e.node);
-
-  console.log(`\nTotal products created in last 30 days: ${products.length}`);
-
-  return products;
-}
-```
-
-## When to Use Each Approach
-
-| Feature | ExtractionOrchestrator | Manual Extraction |
-|---------|------------------------|-------------------|
-| **Simplicity** | ✅ High - minimal code | ⚠️ Medium - more code |
-| **Path Extraction** | ✅ Automatic with `resultPath` | ❌ Manual with `.map()` |
-| **Validation** | ✅ Built-in per-item validation | ❌ Manual validation |
-| **Statistics** | ✅ Comprehensive stats object | ⚠️ Via `extensions.autoPagination` |
-| **Error Handling** | ✅ Structured error tracking | ⚠️ Try/catch only |
-| **Fine Control** | ⚠️ Limited customization | ✅ Full control |
-| **Best For** | Most extraction tasks | Custom pagination logic |
-
-**Recommendation:** Start with `ExtractionOrchestrator` for 90% of use cases. Switch to manual extraction only when you need custom pagination behavior or response processing.
-
-## Key Takeaways
-
-- 🎯 **Use ExtractionOrchestrator for simplicity** - handles path extraction and validation
-- 🎯 **Manual extraction for control** - when you need custom pagination logic
-- 🎯 Always include pagination variables (`$first`, `$after`) in queries
-- 🎯 Use auto-pagination to avoid manual cursor management
-- 🎯 Configure safety limits (`maxRecords`, `maxPages`, `timeout`)
-- 🎯 Use `onProgress` callbacks for long-running extractions
-- 🎯 Check `extensions.autoPagination` for metadata (manual approach)
-- 🎯 Handle errors gracefully with try/catch
-- 🎯 Validate field names against schema before use
-
-## Next Steps
-
-Continue to [Module 3: Parquet Processing](./02-core-guides-extraction-03-parquet-processing.md) to learn how to extract data from Parquet files and combine it with GraphQL enrichment.
+# Module 2: Basic Extraction
+
+**Level:** Beginner
+**Estimated Time:** 20 minutes
+
+## Overview
+
+This module teaches you how to perform basic data extraction from Fluent Commerce using validated GraphQL queries and automatic pagination.
+
+**Note:** For most extraction use cases, the **ExtractionOrchestrator** (covered in [Module 8](./02-core-guides-extraction-08-extraction-orchestrator.md)) provides a simpler, higher-level API. This module shows the manual approach for cases where you need fine-grained control.
+
+## Learning Objectives
+
+By the end of this module, you will:
+
+- ✅ Write validated GraphQL extraction queries
+- ✅ Use auto-pagination to fetch large datasets
+- ✅ Extract common Fluent entities (inventory, products, orders)
+- ✅ Handle pagination responses correctly
+- ✅ Process extracted data
+- ✅ Understand when to use manual extraction vs ExtractionOrchestrator
+
+## Your First Extraction Query
+
+### Step 1: Initialize the Client
+
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+
+// Create client (auto-detects Node.js, Deno, or Versori context)
+const client = await createClient({
+  config: {
+    baseUrl: process.env.FLUENT_BASE_URL!,
+    clientId: process.env.FLUENT_CLIENT_ID!,
+    clientSecret: process.env.FLUENT_CLIENT_SECRET!,
+    username: process.env.FLUENT_USERNAME!,
+    password: process.env.FLUENT_PASSWORD!
+    // Note: retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
+  }
+});
+```
+
+### Step 2: Write a Simple Query
+
+```typescript
+const query = `
+  query ExtractVirtualPositions($first: Int!, $after: String) {
+    virtualPositions(first: $first, after: $after) {
+      edges {
+        node {
+          id
+          ref
+          productRef
+          groupRef
+          quantity
+          status
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+      }
+    }
+  }
+`;
+```
+
+**Field Validation:**
+
+- ✅ `id` - VirtualPosition.id (ID!)
+- ✅ `ref` - VirtualPosition.ref (String!)
+- ✅ `productRef` - VirtualPosition.productRef (String!)
+- ✅ `groupRef` - VirtualPosition.groupRef (String)
+- ✅ `quantity` - VirtualPosition.quantity (Int!)
+- ✅ `status` - VirtualPosition.status (String)
+
+> **⚠️ IMPORTANT: VirtualPosition vs InventoryPosition Field Differences**
+>
+> These two types have **different field names** for quantity:
+> - **VirtualPosition**: Uses `quantity` (Int!)
+> - **InventoryPosition**: Uses `qty` (Int!) and `onHand` (Int!)
+>
+> Always check the GraphQL schema when switching between entity types!
+
+### Step 3: Execute with Auto-Pagination
+
+```typescript
+const result = await client.graphql({
+  query,
+  variables: { first: 100 },
+  pagination: {
+    maxRecords: 10000, // Stop after 10K records
+    maxPages: 100, // OR stop after 100 pages
+    onProgress: (page, totalRecords) => {
+      console.log(`Fetched page ${page}: ${totalRecords} records so far`);
+    },
+  },
+});
+```
+
+### Step 4: Process the Results
+
+```typescript
+// Auto-pagination merges all pages into single response
+const allRecords = result.data.virtualPositions.edges.map(edge => edge.node);
+
+console.log(`Total records: ${allRecords.length}`);
+console.log(`Pages fetched: ${result.extensions.pagination.totalPages}`);
+console.log(`Time elapsed: ${result.extensions.pagination.timeElapsed}ms`);
+
+// Process each record
+for (const record of allRecords) {
+  console.log(`Product ${record.productRef}: ${record.quantity} units`);
+}
+```
+
+## Complete Example
+
+### Approach 1: Using ExtractionOrchestrator (Recommended)
+
+For most use cases, use the high-level ExtractionOrchestrator:
+
+```typescript
+import { createClient, ExtractionOrchestrator } from '@fluentcommerce/fc-connect-sdk';
+
+async function extractVirtualPositions() {
+  const client = await createClient({
+    config: {
+      baseUrl: process.env.FLUENT_BASE_URL!,
+      clientId: process.env.FLUENT_CLIENT_ID!,
+      clientSecret: process.env.FLUENT_CLIENT_SECRET!
+      // retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
+    },
+  });
+
+  const orchestrator = new ExtractionOrchestrator(client, console);
+
+  try {
+    const result = await orchestrator.extract({
+      query: `
+        query ExtractVirtualPositions($first: Int!, $after: String) {
+          virtualPositions(first: $first, after: $after) {
+            edges {
+              node {
+                id ref productRef groupRef quantity status
+                createdOn updatedOn
+              }
+              cursor
+            }
+            pageInfo { hasNextPage }
+          }
+        }
+      `,
+      resultPath: 'virtualPositions.edges.node',
+      pageSize: 100,
+      maxRecords: 10000
+    });
+
+    console.log('\n=== Extraction Summary ===');
+    console.log(`Total records: ${result.data.length}`);
+    console.log(`Pages: ${result.stats.totalPages}`);
+    console.log(`Duration: ${result.stats.duration}ms`);
+
+    return result.data;
+  } catch (error) {
+    console.error('Extraction failed:', error);
+    throw error;
+  }
+}
+
+extractVirtualPositions()
+  .then(() => console.log('Extraction complete'))
+  .catch(error => console.error('Error:', error));
+```
+
+### Approach 2: Manual Extraction (For Advanced Control)
+
+For fine-grained control over pagination and response handling:
+
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+
+async function extractVirtualPositions() {
+  // Initialize client
+  const client = await createClient({
+    config: {
+      baseUrl: process.env.FLUENT_BASE_URL!,
+      clientId: process.env.FLUENT_CLIENT_ID!,
+      clientSecret: process.env.FLUENT_CLIENT_SECRET!
+      // retailerId is OPTIONAL for GraphQL queries (only required for Job/Event API)
+    },
+  });
+
+  // Define query
+  const query = `
+    query ExtractVirtualPositions($first: Int!, $after: String) {
+      virtualPositions(first: $first, after: $after) {
+        edges {
+          node {
+            id
+            ref
+            productRef
+            groupRef
+            quantity
+            status
+            createdOn
+            updatedOn
+          }
+          cursor
+        }
+        pageInfo {
+          hasNextPage
+        }
+      }
+    }
+  `;
+
+  try {
+    // Execute with auto-pagination
+    const result = await client.graphql({
+      query,
+      variables: { first: 100 },
+      pagination: {
+        maxRecords: 10000,
+        onProgress: (page, totalRecords) => {
+          console.log(`Page ${page}: ${totalRecords} records`);
+        },
+      },
+    });
+
+    // Extract records
+    const records = result.data.virtualPositions.edges.map(e => e.node);
+
+    console.log('\n=== Extraction Summary ===');
+    console.log(`Total records: ${records.length}`);
+    console.log(`Pages: ${result.extensions.pagination.totalPages}`);
+    console.log(`Time: ${result.extensions.pagination.timeElapsed}ms`);
+
+    // Show sample records
+    console.log('\n=== Sample Records ===');
+    records.slice(0, 5).forEach(record => {
+      console.log(`- ${record.ref}: ${record.quantity} units of ${record.productRef}`);
+    });
+
+    return records;
+  } catch (error) {
+    console.error('Extraction failed:', error);
+    throw error;
+  }
+}
+
+// Run extraction
+extractVirtualPositions()
+  .then(() => console.log('Extraction complete'))
+  .catch(error => console.error('Error:', error));
+```
+
+## Common Entity Extractions
+
+### 1. Inventory Positions
+
+```typescript
+const query = `
+  query ExtractInventory($first: Int!, $after: String) {
+    inventoryPositions(first: $first, after: $after) {
+      edges {
+        node {
+          id
+          ref
+          productRef
+          locationRef
+          qty              # ✅ Note: 'qty' not 'quantity'
+          status
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+      }
+    }
+  }
+`;
+
+const result = await client.graphql({
+  query,
+  variables: { first: 100 },
+  pagination: { maxRecords: 50000 },
+});
+
+const inventory = result.data.inventoryPositions.edges.map(e => e.node);
+```
+
+### 2. Products
+
+```typescript
+const query = `
+  query ExtractProducts($first: Int!, $after: String) {
+    products(first: $first, after: $after) {
+      edges {
+        node {
+          id
+          ref
+          name
+          gtin
+          status
+          type
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+      }
+    }
+  }
+`;
+
+const result = await client.graphql({
+  query,
+  variables: { first: 50 }, // Smaller page size for products
+  pagination: { maxRecords: 10000 },
+});
+
+const products = result.data.products.edges.map(e => e.node);
+```
+
+### 3. Orders
+
+```typescript
+const query = `
+  query ExtractOrders($first: Int!, $after: String) {
+    orders(first: $first, after: $after) {
+      edges {
+        node {
+          id
+          ref
+          type
+          status
+          totalPrice
+          totalTaxPrice
+          currency
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+      }
+    }
+  }
+`;
+
+const result = await client.graphql({
+  query,
+  variables: { first: 50 },
+  pagination: { maxRecords: 20000 },
+});
+
+const orders = result.data.orders.edges.map(e => e.node);
+```
+
+### 4. Locations
+
+```typescript
+const query = `
+  query ExtractLocations($first: Int!, $after: String) {
+    locations(first: $first, after: $after) {
+      edges {
+        node {
+          id
+          ref
+          name
+          type
+          status
+          createdOn
+          updatedOn
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+      }
+    }
+  }
+`;
+
+const result = await client.graphql({
+  query,
+  variables: { first: 100 },
+  pagination: { maxRecords: 5000 },
+});
+
+const locations = result.data.locations.edges.map(e => e.node);
+```
+
+## Understanding Pagination Responses
+
+### Response Structure
+
+```typescript
+{
+  data: {
+    virtualPositions: {
+      edges: [
+        {
+          node: { id: "1", ref: "VP-001", ... },
+          cursor: "abc123"
+        },
+        {
+          node: { id: "2", ref: "VP-002", ... },
+          cursor: "def456"
+        }
+      ],
+      pageInfo: {
+        hasNextPage: true
+      }
+    }
+  },
+  extensions: {
+    autoPagination: {
+      totalPages: 15,
+      totalRecords: 1487,
+      truncated: false,
+      timeElapsed: 8324
+    }
+  }
+}
+```
+
+### Auto-Pagination Extensions
+
+The SDK adds `extensions.autoPagination` metadata:
+
+| Field          | Type    | Description                     |
+| -------------- | ------- | ------------------------------- |
+| `totalPages`   | number  | Number of pages fetched         |
+| `totalRecords` | number  | Total records returned          |
+| `truncated`    | boolean | True if stopped by safety limit |
+| `duration`     | number  | Total time in milliseconds      |
+
+### Accessing Data
+
+```typescript
+// Get all records
+const allRecords = result.data.virtualPositions.edges.map(e => e.node);
+
+// Check if truncated
+if (result.extensions.pagination.truncated) {
+  console.warn('Results truncated by safety limit');
+  console.log(`Fetched ${result.extensions.pagination.totalRecords} of total`);
+}
+
+// Performance metrics
+console.log(
+  `Fetched ${result.extensions.pagination.totalRecords} records in ${result.extensions.pagination.timeElapsed}ms`
+);
+console.log(
+  `Rate: ${((result.extensions.pagination.totalRecords / result.extensions.pagination.timeElapsed) * 1000).toFixed(0)} records/sec`
+);
+```
+
+## Filtering Results
+
+### GraphQL Filters
+
+```typescript
+// Filter by status
+const query = `
+  query ExtractActiveInventory($first: Int!, $after: String, $filter: JSON) {
+    inventoryPositions(
+      first: $first
+      after: $after
+      filter: $filter
+    ) {
+      edges {
+        node { id ref qty status }
+        cursor
+      }
+      pageInfo { hasNextPage }
+    }
+  }
+`;
+
+const result = await client.graphql({
+  query,
+  variables: {
+    first: 100,
+    filter: {
+      status: { equalTo: 'AVAILABLE' },
+    },
+  },
+  pagination: { maxRecords: 50000 },
+});
+```
+
+### Date Range Filters
+
+```typescript
+// Filter by date range
+const fromDate = new Date('2024-01-01').toISOString();
+const toDate = new Date('2024-12-31').toISOString();
+
+const query = `
+  query ExtractRecentOrders($first: Int!, $after: String, $from: DateTime!, $to: DateTime!) {
+    orders(
+      first: $first
+      after: $after
+      createdOn: { from: $from, to: $to }
+    ) {
+      edges {
+        node { id ref createdOn totalPrice }
+        cursor
+      }
+      pageInfo { hasNextPage }
+    }
+  }
+`;
+
+const result = await client.graphql({
+  query,
+  variables: {
+    first: 100,
+    from: fromDate,
+    to: toDate,
+  },
+  pagination: { maxRecords: 20000 },
+});
+```
+
+### Complex Filters
+
+```typescript
+// Multiple filter conditions
+const query = `
+  query ExtractFilteredProducts($first: Int!, $after: String, $filter: JSON) {
+    products(
+      first: $first
+      after: $after
+      filter: $filter
+    ) {
+      edges {
+        node { id ref name status type }
+        cursor
+      }
+      pageInfo { hasNextPage }
+    }
+  }
+`;
+
+const result = await client.graphql({
+  query,
+  variables: {
+    first: 50,
+    filter: {
+      AND: [{ status: { equalTo: 'ACTIVE' } }, { type: { equalTo: 'STANDARD' } }],
+    },
+  },
+  pagination: { maxRecords: 10000 },
+});
+```
+
+## Error Handling
+
+### Comprehensive Error Handling with Retry Logic
+
+```typescript
+import { createClient } from '@fluentcommerce/fc-connect-sdk';
+
+async function extractWithErrorHandling() {
+  const client = await createClient({
+    config: {
+      baseUrl: process.env.FLUENT_BASE_URL!,
+      clientId: process.env.FLUENT_CLIENT_ID!,
+      clientSecret: process.env.FLUENT_CLIENT_SECRET!,
+      username: process.env.FLUENT_USERNAME!,
+      password: process.env.FLUENT_PASSWORD!
+    }
+  });
+
+  const query = `
+    query ExtractVirtualPositions($first: Int!, $after: String) {
+      virtualPositions(first: $first, after: $after) {
+        edges {
+          node {
+            id ref productRef groupRef quantity status
+            createdOn updatedOn
+          }
+          cursor
+        }
+        pageInfo { hasNextPage }
+      }
+    }
+  `;
+
+  try {
+    const result = await client.graphql({
+      query,
+      variables: { first: 100 },
+      pagination: { maxRecords: 10000 },
+    });
+
+    // Check for GraphQL errors
+    if (result.errors && result.errors.length > 0) {
+      console.error('GraphQL errors detected:', {
+        errorCount: result.errors.length,
+        errors: result.errors.map(e => ({
+          message: e.message,
+          path: e.path,
+          extensions: e.extensions
+        }))
+      });
+      throw new Error(`GraphQL query failed: ${result.errors[0].message}`);
+    }
+
+    // Validate data structure
+    if (!result.data?.virtualPositions?.edges) {
+      throw new Error('Unexpected response structure - missing virtualPositions.edges');
+    }
+
+    const records = result.data.virtualPositions.edges.map(e => e.node);
+    console.log(`✅ Extracted ${records.length} records successfully`);
+
+    return records;
+  } catch (error) {
+    console.error('Extraction failed:', {
+      error: error.message,
+      errorType: error.constructor.name
+    });
+
+    // Check for specific error types with appropriate handling
+    if (error.message.includes('authentication') || error.message.includes('401')) {
+      console.error('Authentication failed - check credentials');
+      throw new Error('Authentication failure - verify OAuth2 credentials');
+    } else if (error.message.includes('timeout') || error.code === 'TIMEOUT_ERROR') {
+      console.error('Request timeout - try smaller page size');
+      throw new Error('Timeout error - reduce pagination page size');
+    } else if (error.message.includes('rate limit') || error.code === 'RATE_LIMIT_ERROR') {
+      console.error('Rate limit exceeded - implement exponential backoff');
+      throw new Error('Rate limit error - wait before retrying');
+    } else if (error.message.includes('network') || error.code === 'ECONNREFUSED') {
+      console.error('Network error - check connectivity');
+      throw new Error('Network error - verify API endpoint');
+    } else {
+      throw error;
+    }
+  }
+}
+
+// Retry logic for transient errors
+async function extractWithRetry(maxRetries: number = 3) {
+  let lastError: Error;
+
+  for (let attempt = 0; attempt < maxRetries; attempt++) {
+    try {
+      return await extractWithErrorHandling();
+    } catch (error) {
+      lastError = error;
+
+      // Only retry for transient errors
+      if (error.message.includes('timeout') ||
+          error.message.includes('rate limit') ||
+          error.message.includes('network')) {
+
+        const delay = Math.pow(2, attempt) * 1000; // Exponential backoff
+        console.log(`Retry attempt ${attempt + 1}/${maxRetries} after ${delay}ms`);
+        await new Promise(resolve => setTimeout(resolve, delay));
+      } else {
+        // Don't retry for authentication or validation errors
+        throw error;
+      }
+    }
+  }
+
+  throw lastError!;
+}
+```
+
+### Validation Errors with Detailed Logging
+
+```typescript
+async function extractWithValidation() {
+  try {
+    const result = await client.graphql({ query, variables });
+
+    // Check for GraphQL errors
+    if (result.errors && result.errors.length > 0) {
+      console.error('GraphQL errors:', {
+        query: query.substring(0, 100) + '...',
+        variables,
+        errors: result.errors.map(e => ({
+          message: e.message,
+          path: e.path
+        }))
+      });
+      throw new Error('Query returned GraphQL errors');
+    }
+
+    // Validate data structure
+    if (!result.data?.virtualPositions?.edges) {
+      console.error('Invalid response structure:', {
+        hasData: !!result.data,
+        hasVirtualPositions: !!result.data?.virtualPositions,
+        actualStructure: Object.keys(result.data || {})
+      });
+      throw new Error('Unexpected response structure');
+    }
+
+    const records = result.data.virtualPositions.edges.map(e => e.node);
+
+    if (records.length === 0) {
+      console.warn('No records found - verify query filters');
+    }
+
+    return records;
+  } catch (error) {
+    console.error('Validation error:', error.message);
+    throw error;
+  }
+}
+```
+
+## Practice Exercise
+
+**Goal**: Extract all products created in the last 30 days
+
+**Requirements:**
+
+1. Query the `products` connection
+2. Filter by `createdOn` date range
+3. Include fields: `id`, `ref`, `name`, `status`, `createdOn`
+4. Use auto-pagination with max 5000 records
+5. Log progress every page
+
+**Solution:**
+
+```typescript
+async function extractRecentProducts() {
+  const client = await createClient({ config });
+
+  const thirtyDaysAgo = new Date();
+  thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30);
+
+  const query = `
+    query ExtractRecentProducts($first: Int!, $after: String, $from: DateTime!) {
+      products(
+        first: $first
+        after: $after
+        createdOn: { from: $from }
+      ) {
+        edges {
+          node {
+            id
+            ref
+            name
+            status
+            createdOn
+          }
+          cursor
+        }
+        pageInfo {
+          hasNextPage
+        }
+      }
+    }
+  `;
+
+  const result = await client.graphql({
+    query,
+    variables: {
+      first: 50,
+      from: thirtyDaysAgo.toISOString(),
+    },
+    pagination: {
+      maxRecords: 5000,
+      onProgress: (page, totalRecords) => {
+        console.log(`Page ${page}: ${totalRecords} products fetched`);
+      },
+    },
+  });
+
+  const products = result.data.products.edges.map(e => e.node);
+
+  console.log(`\nTotal products created in last 30 days: ${products.length}`);
+
+  return products;
+}
+```
+
+## When to Use Each Approach
+
+| Feature | ExtractionOrchestrator | Manual Extraction |
+|---------|------------------------|-------------------|
+| **Simplicity** | ✅ High - minimal code | ⚠️ Medium - more code |
+| **Path Extraction** | ✅ Automatic with `resultPath` | ❌ Manual with `.map()` |
+| **Validation** | ✅ Built-in per-item validation | ❌ Manual validation |
+| **Statistics** | ✅ Comprehensive stats object | ⚠️ Via `extensions.autoPagination` |
+| **Error Handling** | ✅ Structured error tracking | ⚠️ Try/catch only |
+| **Fine Control** | ⚠️ Limited customization | ✅ Full control |
+| **Best For** | Most extraction tasks | Custom pagination logic |
+
+**Recommendation:** Start with `ExtractionOrchestrator` for 90% of use cases. Switch to manual extraction only when you need custom pagination behavior or response processing.
+
+## Key Takeaways
+
+- 🎯 **Use ExtractionOrchestrator for simplicity** - handles path extraction and validation
+- 🎯 **Manual extraction for control** - when you need custom pagination logic
+- 🎯 Always include pagination variables (`$first`, `$after`) in queries
+- 🎯 Use auto-pagination to avoid manual cursor management
+- 🎯 Configure safety limits (`maxRecords`, `maxPages`, `timeout`)
+- 🎯 Use `onProgress` callbacks for long-running extractions
+- 🎯 Check `extensions.autoPagination` for metadata (manual approach)
+- 🎯 Handle errors gracefully with try/catch
+- 🎯 Validate field names against schema before use
+
+## Next Steps
+
+Continue to [Module 3: Parquet Processing](./02-core-guides-extraction-03-parquet-processing.md) to learn how to extract data from Parquet files and combine it with GraphQL enrichment.