@fluentcommerce/fc-connect-sdk 0.1.53 → 0.1.55

package/docs/01-TEMPLATES/versori/workflows/extraction/graphql-queries/template-extraction-orders-to-s3-csv.md

@@ -1,1953 +1,1953 @@
----
-template_id: tpl-extract-orders-to-s3-csv
-canonical_filename: template-extraction-orders-to-s3-csv.md
-version: 2.0.0
-sdk_version: ^0.1.39
-runtime: versori
-direction: extraction
-source: fluent-graphql
-destination: s3-csv
-entity: orders
-format: csv
-logging: versori
-status: stable
-features:
-  - memory-management
-  - enhanced-logging
-  - pagination-progress
----
-
-# Template: Extraction - Orders to S3 CSV
-
-**Template Version:** 2.0.0
-**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
-**Last Updated:** 2025-01-24
-**Deployment Target:** Versori Platform
-
-**🆕 Version 2.0.0 Enhancements:**
-- ✅ **Memory Management** - Clear large result sets after processing batches
-- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
-- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
-
----
-
-## 📚 STEP 1: Load These Docs (Human Checklist)
-
-1. REQUIRED (load all)
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
-
-Copy-paste list (open these):
-fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
-fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
-fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
-fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
-
----
-
-## 📋 STEP 2: Tell Your AI (Prompt)
-
-Copy/paste this prompt after loading the documentation above:
-
-```
-Create a Versori scheduled extractor for orders that uses ExtractionOrchestrator + JobTracker, incremental updatedOn with a 60s overlap buffer, transforms via UniversalMapper, generates CSV with CSVParserService.stringify(), uploads to S3 using S3DataSource. Include 3 workflows: scheduled, ad-hoc webhook, and job-status query with native Versori logging.
-```
-
----
-
-## 📦 SDK Imports (Verified - Versori Optimized)
-
-```typescript
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  ExtractionOrchestrator,
-  JobTracker,
-  UniversalMapper,
-  CSVParserService,
-  S3DataSource,
-  VersoriKVAdapter,
-} from '@fluentcommerce/fc-connect-sdk';
-
-import { schedule, webhook, http, fn } from '@versori/run';
-```
-
----
-
-# Versori Scheduled: Orders Extraction to S3 CSV (Incremental)
-
-**FC Connect SDK Use Case Guide**
-
-> **SDK**: [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
-> **Installation**: `npm install @fluentcommerce/fc-connect-sdk`
-
-Context: Scheduled Versori workflow that extracts new/updated orders from Fluent Commerce via GraphQL query with **ExtractionOrchestrator**, **JobTracker**, and **incremental timestamp tracking**, transforms with `UniversalMapper`, and writes **CSV files** to S3 for 3PL/fulfillment systems.
-
-**Pattern**: EXTRACTION (Fluent → S3 CSV)
-**Complexity**: High | Runtime: Versori Platform (Scheduled)
-
----
-
-## ⚠️ IMPORTANT: Production-Ready Base Template
-
-> **📋 BASE TEMPLATE - Ready for Production (Customize for Your Needs)**
->
-> This is a **production-ready base template** demonstrating FC Connect SDK best practices for order extraction workflows with CSV output to S3.
->
-> **✅ INCLUDED FEATURES:**
->
-> - ✅ Comprehensive error handling with retry logic
-> - ✅ S3 upload with proper error handling
-> - ✅ State management with overlap buffer (prevents missed records)
-> - ✅ Job tracking with lifecycle management
-> - ✅ Security (credential masking in logs)
-> - ✅ UTC time enforcement (prevents timezone bugs)
-> - ✅ Incremental extraction (safe, efficient, production-ready)
-> - ✅ Natural rate limiting via timestamps
->
-> **📝 BEFORE DEPLOYING:**
->
-> 1. Review and customize activation variables for your environment
-> 2. Test with sample data in your Versori workspace
-> 3. Adjust safety limits (pageSize, maxRecords) if needed
-> 4. Configure monitoring alerts for extraction failures
-> 5. Verify S3 bucket credentials and paths
->
-> **This base template follows SDK best practices - tweak specific to your needs.**
-
----
-
-## What You'll Build
-
-- **Incremental extraction** using `updatedOn >= (lastRunTime - buffer)` with **overlap buffer**
-- **ExtractionOrchestrator** for auto-pagination and path-based extraction
-- **JobTracker** for lifecycle management
-- **State management** with VersoriKV to track last successful run
-- **Safety buffer** (60 seconds) to handle clock skew and race conditions
-- GraphQL query with nested order lines
-- UniversalMapper transformation with line item flattening
-- **CSV file generation** with CSVParserService
-- **S3 upload** to 3PL/fulfillment system
-- **3 workflow patterns**: scheduled, ad-hoc webhook, job status query
-- **Failure recovery** with timestamp tracking
-
-## Business Use Case
-
-**Hourly order feed to 3PL/fulfillment center:**
-
-- Extract new and updated orders since last run
-- Flatten line items (one CSV row per item)
-- Generate CSV file with order header + line items
-- Upload to S3 bucket for 3PL integration
-- Run every hour to enable real-time fulfillment
-- Support order updates (address changes, item modifications)
-- Standard CSV format for WMS/ERP imports
-
-## SDK Methods Used
-
-```typescript
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  ExtractionOrchestrator,
-  JobTracker,
-  UniversalMapper,
-  CSVParserService,
-  S3DataSource,
-  VersoriKVAdapter,
-} from '@fluentcommerce/fc-connect-sdk';
-
-await createClient(ctx); // Versori-aware client
-const orchestrator = new ExtractionOrchestrator(client, log); // Auto-pagination
-const tracker = new JobTracker(kv, log); // Job lifecycle tracking
-await orchestrator.extract({ query, resultPath, variables, pageSize, maxRecords }); // Extract
-new VersoriKVAdapter(ctx.openKv(':project:')); // State management
-new UniversalMapper(exportMapping); // Field transformation
-const csvParser = new CSVParserService(); // CSV generation
-csvParser.stringify(records, { headers: true }); // Generate CSV content (synchronous)
-await s3.upload(key, buffer, contentType, metadata); // S3 upload
-```
-
-## Activation Variables
-
-```json
-{
-  "retailerId": "your-retailer-id",
-  "s3BucketName": "3pl-orders-export",
-  "awsAccessKeyId": "AKIAXXXXXXXXXXXX",
-  "awsSecretAccessKey": "********",
-  "awsRegion": "us-east-1",
-  "s3Prefix": "orders/new/",
-  "pageSize": 200,
-  "maxRecords": 10000,
-  "fallbackStartDate": "2024-01-01T00:00:00Z",
-  "overlapBufferSeconds": "60",
-  "validateConnection": "true"
-}
-```
-
-## Export Mapping Configuration
-
-Create file: `./config/orders.export.csv.json`
-
-```json
-{
-  "name": "orders.export.csv",
-  "version": "1.0.0",
-  "description": "Fluent Orders → 3PL CSV Export",
-  "fields": {
-    "order_id": { "source": "ref", "required": true, "resolver": "sdk.trim" },
-    "order_date": { "source": "createdOn", "required": true, "resolver": "sdk.formatDateShort" },
-    "order_status": { "source": "status", "required": true, "resolver": "sdk.uppercase" },
-    "customer_name": { "source": "customer.firstName", "required": false, "resolver": "sdk.trim" },
-    "customer_email": { "source": "customer.email", "required": false, "resolver": "sdk.trim" },
-    "ship_to_name": { "source": "deliveryAddress.name", "required": true, "resolver": "sdk.trim" },
-    "ship_to_address1": {
-      "source": "deliveryAddress.street1",
-      "required": true,
-      "resolver": "sdk.trim"
-    },
-    "ship_to_city": { "source": "deliveryAddress.city", "required": true, "resolver": "sdk.trim" },
-    "ship_to_state": {
-      "source": "deliveryAddress.state",
-      "required": true,
-      "resolver": "sdk.uppercase"
-    },
-    "ship_to_zip": {
-      "source": "deliveryAddress.postcode",
-      "required": true,
-      "resolver": "sdk.trim"
-    },
-    "ship_to_country": {
-      "source": "deliveryAddress.country",
-      "required": true,
-      "resolver": "sdk.uppercase"
-    },
-    "line_item_sku": { "source": "product.ref", "required": true, "resolver": "sdk.trim" },
-    "line_item_qty": { "source": "quantity", "required": true, "resolver": "sdk.parseInt" },
-    "line_item_price": { "source": "price", "required": true, "resolver": "sdk.parseFloat" }
-  }
-}
-```
-
-## GraphQL Query
-
-```graphql
-query GetOrders($retailerId: ID!, $updatedAfter: DateTime!, $first: Int!, $after: String) {
-  orders(
-    retailerId: $retailerId
-    updatedOn: { after: $updatedAfter }
-    first: $first
-    after: $after
-  ) {
-    edges {
-      node {
-        id
-        ref
-        status
-        createdOn
-        updatedOn
-        customer {
-          firstName
-          lastName
-          email
-        }
-        deliveryAddress {
-          name
-          street1
-          street2
-          city
-          state
-          postcode
-          country
-        }
-        items {
-          id
-          quantity
-          price
-          product {
-            ref
-            name
-          }
-        }
-      }
-      cursor
-    }
-    pageInfo {
-      hasNextPage
-    }
-  }
-}
-```
-
----
-
-## Versori Workflows Structure
-
-**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
-
-**Trigger Types:**
-- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
-- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
-- **`workflow()`** → Durable workflows (advanced, rarely used)
-
-**Execution Steps (chained to triggers):**
-- **`http()`** → External API calls (chained from schedule/webhook)
-- **`fn()`** → Internal processing (chained from schedule/webhook)
-
-### Recommended Project Structure (MemoryInterpreter Pattern)
-
-```
-orders-extraction/
-├── index.ts                              # Entry point - exports all workflows (MemoryInterpreter pattern)
-└── src/
-    ├── workflows/
-    │   ├── scheduled/
-    │   │   └── daily-orders-extraction.ts  # Scheduled: Daily orders extraction
-    │   │
-    │   └── webhook/
-    │       ├── adhoc-orders-extraction.ts  # Webhook: Manual trigger
-    │       └── job-status-check.ts         # Webhook: Status query
-    │
-    ├── services/
-    │   └── orders-extraction.service.ts   # Shared orchestration logic (reusable)
-    │
-    └── config/
-        └── orders.export.csv.json         # Mapping configuration
-```
-
-**MemoryInterpreter Pattern:**
-- `index.ts` exports workflow definitions (lightweight registration)
-- Workflow files delegate to service functions for business logic
-- Service functions contain the actual implementation
-- This pattern enables better memory management and code organization
-
-**Note:** The code examples below demonstrate what goes in each workflow file. You would split them into separate files following the structure above.
-
----
-
-```csv
-order_id,order_date,order_status,customer_name,customer_email,ship_to_name,ship_to_address1,ship_to_city,ship_to_state,ship_to_zip,ship_to_country,line_item_sku,line_item_qty,line_item_price
-ORD-001,2025-01-22,CREATED,John,john@example.com,John Smith,123 Main St,New York,NY,10001,US,SKU-001,2,29.99
-ORD-001,2025-01-22,CREATED,John,john@example.com,John Smith,123 Main St,New York,NY,10001,US,SKU-002,1,49.99
-ORD-002,2025-01-22,PAID,Jane,jane@example.com,Jane Doe,456 Oak Ave,Los Angeles,CA,90001,US,SKU-003,1,19.99
-```
-
-**Note**: Each line item becomes a separate row with duplicated order header data (standard 3PL format).
-
----
-
-## Complete Implementation
-
-### Workflow 1: Scheduled Extraction (Main Workflow)
-
-**File: `src/workflows/scheduled/daily-orders-extraction.ts`**
-
-```typescript
-import { schedule, http } from '@versori/run';
-import { JobTracker } from '@fluentcommerce/fc-connect-sdk';
-import { processOrdersExtraction } from '../../services/orders-extraction.service';
-
-/**
- * Scheduled workflow: Hourly orders extraction to S3 CSV
- * Runs every hour
- *
- * DELEGATION PATTERN (MemoryInterpreter):
- * - Workflow receives ctx from Versori
- * - Passes entire ctx to service function
- * - Service handles all business logic
- */
-export const ordersExtractionScheduled = schedule('orders-extract-hourly', '0 * * * *').then(
-  http('extract-orders', { connection: 'fluent_commerce' }, async (ctx: any) => {
-    const { log, openKv } = ctx;
-    const executionStartTime = Date.now();
-    const jobId = `SCHEDULED_ORD_${new Date().toISOString().replace(/[:.]/g, '-')}_${Date.now()}`;
-    const tracker = new JobTracker(openKv(':project:'), log);
-
-    log.info('🚀 [WORKFLOW] Starting scheduled orders extraction', { jobId });
-
-    await tracker.createJob(jobId, {
-      triggeredBy: 'schedule',
-      stage: 'initialization',
-      startTime: executionStartTime,
-    });
-
-    await tracker.updateJob(jobId, { status: 'processing' });
-
-    try {
-      // DELEGATION: Pass entire ctx to service function
-      const result = await processOrdersExtraction(ctx, jobId, tracker);
-
-      const duration = Date.now() - executionStartTime;
-
-      if (result.success) {
-        await tracker.markCompleted(jobId, { ...result, duration });
-        log.info('✅ [WORKFLOW] Orders extraction completed successfully', {
-          jobId,
-          ordersExtracted: result.ordersExtracted,
-          fileName: result.fileName,
-          duration: `${duration}ms`,
-        });
-      } else {
-        await tracker.markFailed(jobId, result.error || 'Unknown error');
-        log.error('❌ [WORKFLOW] Orders extraction failed', {
-          jobId,
-          error: result.error,
-          duration: `${duration}ms`,
-        });
-      }
-
-      return { success: true, jobId, ...result, duration };
-    } catch (e: any) {
-      const errorMessage = e instanceof Error ? e.message : String(e);
-      const duration = Date.now() - executionStartTime;
-
-      await tracker.markFailed(jobId, errorMessage);
-
-      log.error('❌ [WORKFLOW] Orders extraction failed with exception', {
-        jobId,
-        error: errorMessage,
-        stack: e instanceof Error ? e.stack : undefined,
-        duration: `${duration}ms`,
-      });
-
-      return {
-        success: false,
-        jobId,
-        error: errorMessage,
-        duration,
-        recommendations: [
-          'Check SFTP/S3 connection credentials',
-          'Verify GraphQL query permissions',
-          'Review error stack trace for details'
-        ]
-      };
-    }
-  })
-);
-```
-
-**File: `src/services/orders-extraction.service.ts`**
-
-```typescript
-import { Buffer } from 'node:buffer';
-import {
-  createClient,
-  ExtractionOrchestrator,
-  JobTracker,
-  UniversalMapper,
-  CSVParserService,
-  S3DataSource,
-  VersoriKVAdapter,
-} from '@fluentcommerce/fc-connect-sdk';
-import ordersExportMapping from '../config/orders.export.csv.json' with { type: 'json' };
-
-const ORDERS_QUERY = `
-  query GetOrders($retailerId: ID!, $updatedAfter: DateTime!, $first: Int!, $after: String) {
-    orders(
-      retailerId: $retailerId
-      updatedOn: { after: $updatedAfter }
-      first: $first
-      after: $after
-    ) {
-      edges {
-        node {
-          id
-          ref
-          status
-          createdOn
-          updatedOn
-          customer {
-            firstName
-            lastName
-            email
-          }
-          deliveryAddress {
-            name
-            street1
-            street2
-            city
-            state
-            postcode
-            country
-          }
-          items {
-            id
-            quantity
-            price
-            product {
-              ref
-              name
-            }
-          }
-        }
-        cursor
-      }
-      pageInfo {
-        hasNextPage
-      }
-    }
-  }
-`;
-
-/**
- * Main orders extraction service function
- *
- * This function orchestrates the complete extraction process:
- * 1. Client initialization
- * 2. State management and incremental sync
- * 3. GraphQL extraction with pagination
- * 4. Data transformation and CSV generation
- * 5. S3 upload and state tracking
- *
- * @param ctx - Versori context object containing fetch, connections, log, activation, openKv
- * @param jobId - Job ID for tracking
- * @param tracker - JobTracker instance for job lifecycle management
- */
-export async function processOrdersExtraction(ctx: any, jobId: string, tracker: JobTracker) {
-  const { log, openKv, activation } = ctx;
-  const startTime = Date.now();
-
-  log.info('🔍 [OrdersExtraction] ==================== EXECUTION START ====================');
-  log.info('🔍 [ExtractionOrchestrator] Starting extraction');
-
-  try {
-    // STEP 1/8: Extract context and configuration
-    log.info('📄 [OrdersExtraction] STEP 1/8: Loading configuration');
-    const retailerId = activation?.getVariable('retailerId');
-    const pageSize = parseInt(activation?.getVariable('pageSize') || '200', 10);
-    const maxRecords = parseInt(activation?.getVariable('maxRecords') || '10000', 10);
-    const fallbackStartDate =
-      activation?.getVariable('fallbackStartDate') || '2024-01-01T00:00:00Z';
-    const overlapBufferSeconds = parseInt(
-      activation?.getVariable('overlapBufferSeconds') || '60',
-      10
-    );
-    const OVERLAP_BUFFER_MS = overlapBufferSeconds * 1000;
-    const validateConnection = activation?.getVariable('validateConnection') === 'true';
-
-    const s3Config = {
-      bucket: activation?.getVariable('s3BucketName'),
-      region: activation?.getVariable('awsRegion') || 'us-east-1',
-      accessKeyId: activation?.getVariable('awsAccessKeyId'),
-      secretAccessKey: activation?.getVariable('awsSecretAccessKey'),
-    };
-    const s3Prefix = activation?.getVariable('s3Prefix') || 'orders/new/';
-
-    const missing: string[] = [];
-    if (!retailerId) missing.push('retailerId');
-    if (!s3Config.bucket) missing.push('s3BucketName');
-    if (!s3Config.accessKeyId) missing.push('awsAccessKeyId');
-    if (!s3Config.secretAccessKey) missing.push('awsSecretAccessKey');
-    if (missing.length) {
-      log.error('❌ [OrdersExtraction] Missing required activation variables', { missing });
-      return {
-        success: false,
-        error: `Missing required variables: ${missing.join(', ')}`,
-        recommendations: [
-          'Check Activation Variables in Versori dashboard',
-          'Ensure all required credentials are configured',
-          'Verify connection settings for fluent_commerce and S3'
-        ]
-      };
-    }
-
-    // STEP 2/8: Initialize SDK services
-    log.info('📄 [OrdersExtraction] STEP 2/8: Initializing SDK services');
-    const client = await createClient(ctx);
-    const orchestrator = new ExtractionOrchestrator(client, log);
-    const kv = new VersoriKVAdapter(openKv(':project:'));
-    const mapper = new UniversalMapper(ordersExportMapping);
-    const csvParser = new CSVParserService();
-    const s3 = new S3DataSource(
-      {
-        type: 'S3_CSV',
-        connectionId: 's3-orders-export',
-        name: 'S3 Orders Export',
-        s3Config,
-      },
-      log
-    );
-
-    // Validate S3 connection if enabled
-    if (validateConnection) {
-      log.info('🔍 [OrdersExtraction] Validating S3 connection');
-      try {
-        await s3.validateConnection();
-        log.info('✅ [OrdersExtraction] S3 connection validated successfully');
-      } catch (validationError: any) {
-        log.error('❌ [OrdersExtraction] S3 connection validation failed', {
-          error: validationError.message,
-        });
-        return {
-          success: false,
-          error: `S3 connection validation failed: ${validationError.message}`,
-          recommendations: [
-            'Verify S3 bucket exists and is accessible',
-            'Check AWS credentials (accessKeyId, secretAccessKey)',
-            'Ensure correct AWS region is specified',
-            'Verify IAM permissions for S3 operations'
-          ]
-        };
-      }
-    }
-
-    // STEP 3/8: Create job and load state
-    log.info('📄 [OrdersExtraction] STEP 3/8: Loading state for incremental sync');
-
-    const stateKey = ['extraction', 'orders', 'lastRunTime'];
-    const lastRunState = await kv.get(stateKey);
-    const rawLastRunTime = lastRunState?.value?.timestamp || fallbackStartDate;
-
-    // Apply overlap buffer for query (WITH buffer)
-    const bufferedLastRunTime = new Date(
-      new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_MS
-    ).toISOString();
-
-    log.info('🔍 [OrdersExtraction] Starting incremental extraction with overlap buffer', {
-      rawLastRunTime,
-      bufferedLastRunTime,
-      overlapBufferSeconds,
-      retailerId,
-    });
-
-    // STEP 4/8: Execute extraction with ExtractionOrchestrator
-    log.info('📄 [OrdersExtraction] STEP 4/8: Executing GraphQL extraction');
-    const extractionStartTime = Date.now();
-
-    const result = await orchestrator.extract({
-      query: ORDERS_QUERY,
-      resultPath: 'orders.edges.node',
-      variables: {
-        retailerId,
-        updatedAfter: bufferedLastRunTime, // ← WITH overlap buffer
-      },
-      pageSize,
-      maxRecords,
-    });
-
-    const orders = result.data || [];
-    const extractionDuration = Date.now() - extractionStartTime;
-
-    log.info('✅ [ExtractionOrchestrator] Extraction completed', {
-      totalRecords: result.stats.totalRecords,
-      totalPages: result.stats.totalPages,
-      validRecords: result.stats.validRecords ?? orders.length,
-      errors: result.errors ? result.errors.length : 0,
-      duration: `${extractionDuration}ms`,
-    });
-
-    if (result.errors && result.errors.length > 0) {
-      log.warn('⚠️ [OrdersExtraction] Non-fatal extraction errors encountered', {
-        errorCount: result.errors.length,
-        sampleErrors: result.errors.slice(0, 3),
-      });
-    }
-
-    if (orders.length === 0) {
-      const duration = Date.now() - startTime;
-      log.info('✅ [OrdersExtraction] No new orders to extract');
-      await kv.set(stateKey, {
-        timestamp: new Date().toISOString(),
-        orderCount: 0,
-        extractedAt: new Date().toISOString(),
-      });
-      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
-      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
-      return {
-        success: true,
-        message: 'No new orders to extract',
-        lastRunTime: rawLastRunTime,
-        duration,
-      };
-    }
-
-    log.info('📄 [OrdersExtraction] Orders retrieved', { count: orders.length });
-
-    // STEP 5/8: Transform and flatten line items (bulk mapping)
-    log.info('📄 [OrdersExtraction] STEP 5/8: Transforming and flattening line items');
-    const transformStartTime = Date.now();
-
-    // Collect all flattened records first (order + item combinations)
-    const flattenedInputs: any[] = [];
-
-    for (const order of orders) {
-      const lineItems = order.items || [];
-
-      if (lineItems.length === 0) {
-        log.warn('⚠️ [OrdersExtraction] Order has no line items, skipping', {
-          orderId: order.id,
-          ref: order.ref,
-        });
-        continue;
-      }
-
-      // Create flattened records (one per line item)
-      for (const item of lineItems) {
-        flattenedInputs.push({
-          ...order, // Order header data
-          ...item, // Line item data (overwrites items array with individual item)
-        });
-      }
-    }
-
-    if (flattenedInputs.length === 0) {
-      const duration = Date.now() - startTime;
-      log.error('❌ [OrdersExtraction] All records skipped (no line items)');
-      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
-      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
-      return {
-        success: false,
-        error: 'All orders skipped because they contain no line items',
-        duration,
-        recommendations: [
-          'Check order data quality in Fluent Commerce',
-          'Verify line items are being populated correctly',
-          'Review GraphQL query to ensure items are included'
-        ]
-      };
-    }
-
-    // Bulk mapping with UniversalMapper
-    const mappingResult = await mapper.map(flattenedInputs);
-    const transformDuration = Date.now() - transformStartTime;
-
-    if (!mappingResult.success) {
-      const mappingErrors = mappingResult.errors || ['Unknown mapping failure'];
-      const duration = Date.now() - startTime;
-      log.error('❌ [OrdersExtraction] Mapping failed - terminating job', {
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-        duration: `${duration}ms`,
-      });
-
-      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
-      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
-
-      return {
-        success: false,
-        error: `Transformation failed: ${mappingErrors[0] || 'Unknown error'}`,
-        errors: mappingErrors,
-        duration,
-        recommendations: [
-          'Check mapping configuration in orders.export.csv.json',
-          'Verify source field paths match GraphQL response',
-          'Review sample errors for specific field issues',
-          'Validate resolver functions are working correctly'
-        ]
-      };
-    }
-
-    const flattenedRecords = mappingResult.data || [];
-    const mappingErrors = mappingResult.errors || [];
-
-    if (mappingErrors.length > 0) {
-      log.warn('⚠️ [OrdersExtraction] Some records failed transformation', {
-        errorCount: mappingErrors.length,
-        sampleErrors: mappingErrors.slice(0, 3),
-      });
-    }
-
-    if (mappingResult.skippedFields && mappingResult.skippedFields.length > 0) {
-      log.info('ℹ️ [OrdersExtraction] Optional fields skipped (undefined values)', {
-        skippedFields: mappingResult.skippedFields,
-        note: 'These fields were not present in source data. Add defaultValue to mapping config if they should always appear.',
-      });
-    }
-
-    if (flattenedRecords.length === 0) {
-      const duration = Date.now() - startTime;
-      log.error('❌ [OrdersExtraction] All records failed mapping');
-      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
-      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
-      return {
-        success: false,
-        error: 'All records failed mapping',
-        errors: mappingErrors,
-        duration,
-        recommendations: [
-          'Review mapping configuration thoroughly',
-          'Check for schema changes in Fluent Commerce',
-          'Validate all required fields are present',
-          'Test with sample data first'
-        ]
-      };
-    }
-
-    log.info('✅ [OrdersExtraction] Records transformed', {
-      orders: orders.length,
-      lineItems: flattenedRecords.length,
-      failed: mappingErrors.length,
-      duration: `${transformDuration}ms`,
-    });
-
-    // STEP 6/8: Generate CSV content (synchronous, not async)
-    log.info('📄 [OrdersExtraction] STEP 6/8: Generating CSV content');
-    const csvStartTime = Date.now();
-    const csvContent = csvParser.stringify(flattenedRecords, { headers: true });
-    const csvDuration = Date.now() - csvStartTime;
-
-    const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
-    const fileName = `orders-${timestamp}.csv`;
-    const s3Key = `${s3Prefix}${fileName}`;
-
-    log.info('✅ [OrdersExtraction] Generated CSV file', {
-      fileName,
-      size: csvContent.length,
-      orderCount: orders.length,
-      lineItemCount: flattenedRecords.length,
-      duration: `${csvDuration}ms`,
-    });
-
-    // STEP 7/8: Upload to S3
-    log.info('📄 [OrdersExtraction] STEP 7/8: Uploading to S3');
-    const uploadStartTime = Date.now();
-
-    try {
-      await s3.upload(
-        s3Key,
-        Buffer.from(csvContent, 'utf8'),
-        'text/csv',
-        {
-          orderCount: String(orders.length),
-          lineItemCount: String(flattenedRecords.length),
-          extractedAt: new Date().toISOString(),
-          lastRunTime: rawLastRunTime,
-          jobId,
-        }
-      );
-
-      const uploadDuration = Date.now() - uploadStartTime;
-      log.info('✅ [OrdersExtraction] CSV file uploaded to S3', {
-        s3Key,
-        duration: `${uploadDuration}ms`,
-      });
-    } catch (uploadError: any) {
-      const duration = Date.now() - startTime;
-      log.error('❌ [OrdersExtraction] S3 upload failed', {
-        error: uploadError.message,
-        s3Key,
-        duration: `${duration}ms`,
-      });
-
-      // Dispose S3 connection
-      try {
-        await s3.dispose();
-      } catch (disposeError: any) {
-        log.error('⚠️ [OrdersExtraction] Failed to dispose S3 connection', {
-          error: disposeError.message,
-        });
-      }
-
-      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
-      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
-
-      return {
-        success: false,
-        error: `S3 upload failed: ${uploadError.message}`,
-        duration,
-        recommendations: [
-          'Verify S3 bucket exists and is accessible',
-          'Check AWS credentials and permissions',
-          'Ensure network connectivity to S3',
-          'Verify S3 bucket region matches configuration',
-          'Check file size limits and quotas'
-        ]
-      };
-    }
-
-    // STEP 8/8: Update state and complete job
-    log.info('📄 [OrdersExtraction] STEP 8/8: Updating state and completing job');
-
-    // Calculate MAX(updatedOn) from extracted orders (WITHOUT buffer)
-    const maxUpdatedOn = orders.reduce((max, order) => {
-      const orderTime = new Date(order.updatedOn).getTime();
-      return orderTime > max ? orderTime : max;
-    }, new Date(rawLastRunTime).getTime());
-
-    const newTimestamp = new Date(maxUpdatedOn).toISOString();
-
-    // Save state WITHOUT buffer - buffer only applied to query
-    await kv.set(stateKey, {
-      timestamp: newTimestamp, // ← WITHOUT buffer
-      orderCount: orders.length,
-      lineItemCount: flattenedRecords.length,
-      extractedAt: new Date().toISOString(),
-      fileName,
-      s3Key,
-      overlapBufferSeconds,
-      jobId,
-      errors: mappingErrors.length > 0 ? mappingErrors : undefined,
-    });
-
-    log.info('✅ [OrdersExtraction] State updated with new timestamp', {
-      newTimestamp,
-      recordsProcessed: orders.length,
-      nextRunWillQueryFrom: new Date(
-        new Date(newTimestamp).getTime() - OVERLAP_BUFFER_MS
-      ).toISOString(),
-    });
-
-    // Dispose S3 connection
-    try {
-      await s3.dispose();
-      log.info('✅ [OrdersExtraction] S3 connection disposed successfully');
-    } catch (disposeError: any) {
-      log.error('⚠️ [OrdersExtraction] Failed to dispose S3 connection', {
-        error: disposeError.message,
-      });
-    }
-
-    const duration = Date.now() - startTime;
-
-    log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
-    log.info('⏱️ [OrdersExtraction] Total Duration: ' + duration + 'ms');
-
-    return {
-      success: true,
-      ordersExtracted: orders.length,
-      lineItemsExtracted: flattenedRecords.length,
-      recordsFailed: mappingErrors.length,
-      fileName,
-      s3Key,
-      lastRunTime: rawLastRunTime,
-      newTimestamp,
-      duration,
-      errors: mappingErrors.length > 0 ? mappingErrors : undefined,
-    };
-  } catch (error: any) {
-    const duration = Date.now() - startTime;
-    log.error('❌ [OrdersExtraction] Fatal error', {
-      message: error?.message,
-      stack: error?.stack,
-      duration: `${duration}ms`,
-    });
-
-    log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
-    log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
-
-    return {
-      success: false,
-      error: error instanceof Error ? error.message : String(error),
-      stack: error instanceof Error ? error.stack : undefined,
-      errorType: error instanceof Error ? error.constructor.name : 'UnknownError',
-      duration,
-      recommendations: [
-        'Review error stack trace for root cause',
-        'Check all credentials and connection settings',
-        'Verify GraphQL query syntax and permissions',
-        'Ensure all activation variables are set correctly',
-        'Check network connectivity to Fluent Commerce and S3'
-      ]
-    };
-  }
-}
-```
-
-**File: `index.ts`**
-
-```typescript
-/**
- * Entry point - Export all workflows for Versori platform
- *
- * MemoryInterpreter Pattern:
- * This file exports workflow definitions that delegate to service functions.
- * Each workflow is lightweight and imports business logic from services.
- */
-
-// Scheduled workflows
-export { ordersExtractionScheduled } from './src/workflows/scheduled/daily-orders-extraction';
-
-// Webhook workflows
-export { ordersExtractionWebhook } from './src/workflows/webhook/adhoc-orders-extraction';
-export { ordersExtractionStatus } from './src/workflows/webhook/job-status-check';
-```
-
-### Workflow 2: Ad-Hoc Webhook Trigger
-
-**File: `src/workflows/webhook/adhoc-orders-extraction.ts`**
-
-```typescript
-import { webhook, http } from '@versori/run';
-import { JobTracker } from '@fluentcommerce/fc-connect-sdk';
-import { processOrdersExtraction } from '../../services/orders-extraction.service';
-
-/**
- * Webhook workflow: Ad-hoc orders extraction (manual trigger)
- *
- * DELEGATION PATTERN (MemoryInterpreter):
- * - Reuses the same service function as scheduled workflow
- * - Lightweight wrapper that delegates to shared business logic
- */
-export const ordersExtractionWebhook = webhook('orders-extract-webhook', {
-  connection: 'orders-adhoc',
-  response: { mode: 'sync' }, // ✅ Sync mode: response sent when handler returns
-}).then(
-  http('trigger-extraction', { connection: 'fluent_commerce' }, async (ctx: any) => {
-    const { log, openKv } = ctx;
-    const executionStartTime = Date.now();
-    const jobId = `ADHOC_ORD_${new Date().toISOString().replace(/[:.]/g, '-')}_${Date.now()}`;
-    const tracker = new JobTracker(openKv(':project:'), log);
-
-    log.info('🚀 [WEBHOOK] Adhoc orders extraction triggered', { jobId });
-
-    // Create job entry FIRST (awaited to ensure job exists in KV)
-    await tracker.createJob(jobId, {
-      triggeredBy: 'webhook',
-      stage: 'initialization',
-      status: 'queued',
-      startTime: executionStartTime,
-      createdAt: new Date().toISOString(),
-    });
-
-    // ✅ Fire-and-forget: Start background processing WITHOUT await
-    // The promise continues execution after we return the response
-    processOrdersExtraction(ctx, jobId, tracker)
-      .then((result) => {
-      const duration = Date.now() - executionStartTime;
-      if (result.success) {
-          log.info('✅ [BACKGROUND] Orders extraction completed successfully', {
-          jobId,
-          ordersExtracted: result.ordersExtracted,
-          fileName: result.fileName,
-          duration: `${duration}ms`,
-        });
-          return tracker.markCompleted(jobId, { ...result, duration });
-      } else {
-          log.error('❌ [BACKGROUND] Orders extraction failed', {
-          jobId,
-          error: result.error,
-          duration: `${duration}ms`,
-        });
-          return tracker.markFailed(jobId, result.error || 'Unknown error');
-        }
-      })
-      .catch((error: unknown) => {
-        const errorMessage = error instanceof Error ? error.message : String(error);
-        const errorStack = error instanceof Error ? error.stack : undefined;
-      const duration = Date.now() - executionStartTime;
-
-        log.error('❌ [BACKGROUND] Orders extraction failed with exception', {
-        jobId,
-        error: errorMessage,
-          stack: errorStack,
-        duration: `${duration}ms`,
-      });
-
-        return tracker.markFailed(jobId, errorMessage);
-      });
-
-    // Return immediately with jobId (response sent with this return value)
-      return {
-      success: true,
-        jobId,
-      message: 'Orders extraction started in background',
-      statusEndpoint: `https://{workspace}.versori.run/orders-job-status`,
-      note: 'Poll the status endpoint with jobId to check progress',
-    };
-  })
-);
-```
-
-### Workflow 3: Job Status Query
-
-**File: `src/workflows/webhook/job-status-check.ts`**
-
-```typescript
-import { webhook, fn } from '@versori/run';
-import { JobTracker, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
-
-/**
- * Webhook workflow: Job status query (monitor extraction progress)
- *
- * Simple query endpoint - no delegation needed
- */
-export const ordersExtractionStatus = webhook('orders-extract-status', {
-  connection: 'orders-job-status',
-}).then(
-  fn('get-status', async (ctx: any) => {
-    const { log, openKv } = ctx;
-    const kv = new VersoriKVAdapter(openKv(':project:'));
-    const tracker = new JobTracker(kv, log);
-
-    const url = new URL(ctx.request.url);
-    const jobId = url.searchParams.get('jobId');
-
-    if (!jobId) {
-      log.warn('⚠️ [JobStatus] Missing jobId query parameter');
-      return {
-        success: false,
-        error: 'Missing jobId query parameter',
-        usage: 'GET /orders-extract-status?jobId=<job-id>',
-        recommendations: [
-          'Provide jobId as query parameter',
-          'Example: /orders-extract-status?jobId=SCHEDULED_ORD_2025-01-22_12345'
-        ]
-      };
-    }
-
-    log.info('🔍 [JobStatus] Retrieving job status', { jobId });
-
-    const job = await tracker.getJob(jobId);
-
-    if (!job) {
-      log.warn('⚠️ [JobStatus] Job not found', { jobId });
-      return {
-        success: false,
-        error: `Job not found: ${jobId}`,
-        recommendations: [
-          'Verify jobId is correct',
-          'Check if job has expired (default TTL: 7 days)',
-          'Ensure job was created successfully'
-        ]
-      };
-    }
-
-    log.info('✅ [JobStatus] Job status retrieved', { jobId, status: job.status });
-
-    return {
-      success: true,
-      job: {
-        id: job.id,
-        name: job.name,
-        status: job.status,
-        createdAt: job.createdAt,
-        startedAt: job.startedAt,
-        completedAt: job.completedAt,
-        duration: job.duration,
-        result: job.result,
-        error: job.error,
-        metadata: job.metadata,
-      },
-    };
-  })
-);
-```
-
----
-
-## Key Patterns Explained
-
-### Pattern 1: Line Item Flattening
-
-```typescript
-// BEFORE: 1 order with 3 line items
-{
-  ref: "ORD-001",
-  items: [
-    { product: { ref: "SKU-A" }, quantity: 2 },
-    { product: { ref: "SKU-B" }, quantity: 1 },
-    { product: { ref: "SKU-C" }, quantity: 3 }
-  ]
-}
-
-// AFTER: 3 CSV rows (order data repeated)
-[
-  { order_id: "ORD-001", line_item_sku: "SKU-A", line_item_qty: 2 },
-  { order_id: "ORD-001", line_item_sku: "SKU-B", line_item_qty: 1 },
-  { order_id: "ORD-001", line_item_sku: "SKU-C", line_item_qty: 3 }
-]
-```
-
-**Why?** Most 3PL systems expect flattened CSV format, not nested JSON.
-
-### Pattern 2: Overlap Buffer (Query WITH, Save WITHOUT)
-
-```typescript
-// Query uses bufferedLastRunTime (WITH overlap buffer)
-const bufferedLastRunTime = new Date(
-  new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_MS
-).toISOString();
-
-const result = await orchestrator.extract({
-  variables: {
-    updatedAfter: bufferedLastRunTime, // ← WITH buffer
-  },
-});
-
-// Save MAX(updatedOn) WITHOUT buffer
-const maxUpdatedOn = orders.reduce(
-  (max, order) => Math.max(max, new Date(order.updatedOn).getTime()),
-  new Date(rawLastRunTime).getTime()
-);
-
-await kv.set(stateKey, {
-  timestamp: new Date(maxUpdatedOn).toISOString(), // ← WITHOUT buffer
-});
-```
-
-**Why?** Buffer prevents missed records due to clock skew. Next run applies buffer again to saved timestamp.
-
-### Pattern 3: CSVParserService Usage
-
-```typescript
-// ✅ CORRECT - Use CSVParserService.stringify()
-const csvParser = new CSVParserService();
-const csvContent = csvParser.stringify(flattenedRecords, { headers: true });
-
-// ❌ WRONG - Don't use CSVBuilder (doesn't exist)
-const builder = new CSVBuilder();
-builder.addRows(records);
-const csvContent = builder.build();
-```
-
-**Why?** `CSVParserService` is the correct SDK service for CSV generation. It has no constructor parameters.
-
-### Pattern 4: 3-Workflow Pattern
-
-1. **Scheduled**: Hourly automated extraction
-2. **Ad-hoc Webhook**: On-demand trigger with API key auth
-3. **Job Status**: Query job progress via webhook
-
-**Why?** Provides flexibility for monitoring, testing, and emergency extractions.
-
----
-
-## Testing Checklist
-
-**Before deploying to production:**
-
-### 1. Schema Validation
-
-- [ ] Run `npx fc-connect introspect-schema` to download current schema
-- [ ] Run `npx fc-connect validate-schema` to check mapping against schema
-- [ ] Verify all nested `source` paths exist (customer.email, deliveryAddress.city, product.ref)
-
-### 2. Mapping Testing
-
-- [ ] Test with sample data (maxRecords=10, small orders)
-- [ ] Verify all required fields are populated
-- [ ] Verify SDK resolvers work correctly (trim, uppercase, parseInt, parseFloat)
-- [ ] Verify line item flattening creates correct CSV structure
-
-### 3. State Management
-
-- [ ] Verify overlap buffer prevents duplicate misses
-- [ ] Test state recovery after failure (run should retry)
-- [ ] Verify timestamp is saved WITHOUT buffer
-- [ ] Test fallback to `fallbackStartDate` on first run
-
-### 4. S3 Operations
-
-- [ ] Test S3 connection with credentials
-- [ ] Verify file upload to correct prefix
-- [ ] Verify CSV file is valid (headers, row format)
-- [ ] Verify metadata is attached to uploaded file
-
-### 5. Workflow Testing
-
-- [ ] Test scheduled workflow (hourly cron)
-- [ ] Test ad-hoc webhook trigger with API key
-- [ ] Test job status query endpoint
-
----
-
-## Monitoring & Alerting
-
-### Success Response Example
-
-```json
-{
-  "success": true,
-  "jobId": "SCHEDULED_ORD_20251102_140000_abc123",
-  "recordsExtracted": 1523,
-  "fileName": "orders-2025-11-02T14-00-00-000Z.csv",
-  "s3Path": "s3://bucket/orders/orders-2025-11-02T14-00-00-000Z.csv",
-  "metrics": {
-    "extractionDurationMs": 12543,
-    "totalPages": 8,
-    "pageSize": 200,
-    "mappingErrors": 0,
-    "fileSizeBytes": 524288,
-    "uploadDurationMs": 1234
-  },
-  "timestamps": {
-    "extractionStart": "2025-11-02T14:00:00.000Z",
-    "extractionEnd": "2025-11-02T14:00:12.543Z",
-    "uploadComplete": "2025-11-02T14:00:13.777Z"
-  },
-  "state": {
-    "previousTimestamp": "2025-11-02T13:00:00.000Z",
-    "newTimestamp": "2025-11-02T13:59:58.123Z",
-    "stateUpdated": true,
-    "overlapBufferSeconds": 60
-  }
-}
-```
-
-### Error Response Example
-
-```json
-{
-  "success": false,
-  "jobId": "ADHOC_ORD_20251102_140500_xyz789",
-  "error": "S3 upload failed: Connection timeout",
-  "errorCategory": "NETWORK",
-  "recordsExtracted": 0,
-  "stage": "s3_upload",
-  "details": {
-    "message": "Failed to upload file after 3 retry attempts",
-    "retryAttempts": 3,
-    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
-  },
-  "state": {
-    "stateUpdated": false,
-    "willRetryNextRun": true,
-    "note": "State not advanced - next extraction will retry same time window"
-  }
-}
-```
-
-### Key Metrics to Track
-
-```typescript
-const METRICS = {
-  // Extraction Performance
-  extractionDurationMs: Date.now() - extractionStart,
-  recordCount: records.length,
-  pageCount: extractionResult.stats.totalPages,
-  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
-
-  // Transformation Performance
-  transformedCount: transformedRecords.length,
-  failedCount: mappingErrors.length,
-  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
-
-  // File Generation
-  fileSizeMB: (csvContent.length / (1024 * 1024)).toFixed(2),
-
-  // Upload Performance
-  uploadDurationMs: uploadEnd - uploadStart,
-  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
-
-  // State Management
-  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
-  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
-};
-
-log.info('Extraction metrics', metrics);
-```
-
-### Alert Thresholds
-
-```typescript
-const ALERT_THRESHOLDS = {
-  // Duration Alerts
-  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
-  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
-  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
-
-  // Error Rate Alerts
-  MAX_ERROR_RATE: 0.05, // 5% mapping errors
-  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
-
-  // Volume Alerts
-  MAX_RECORDS_PER_RUN: 100000,
-  MIN_RECORDS_WARNING: 0, // Alert if no records found
-  MAX_FILE_SIZE_MB: 150, // 150MB
-
-  // State Alerts
-  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
-  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
-};
-
-// Check thresholds
-if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
-  log.warn('Extraction duration exceeded threshold', {
-    duration: metrics.extractionDurationMs,
-    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
-    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
-  });
-}
-```
-
-### Monitoring Dashboard Queries
-
-**Versori Platform Logs Query:**
-
-```
-# Successful extractions
-log_level:info AND message:"Extraction complete" AND jobId:*
-
-# Failed extractions
-log_level:error AND message:"Extraction workflow failed" AND jobId:*
-
-# Performance issues
-extractionDurationMs:>300000 OR uploadDurationMs:>120000
-
-# High error rates
-errorRate:>5
-
-# State management issues
-stateUpdated:false AND success:true
-```
-
-### Common Issues and Solutions
-
-**Issue**: "Extraction timeout after 10 minutes"
-
-- **Cause**: Too many records in single extraction
-- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
-- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
-
-**Issue**: "Mapping errors for 50% of records"
-
-- **Cause**: Schema mismatch between GraphQL response and mapping config
-- **Fix**: Run schema validation, update mapping config paths
-- **Prevention**: Use `npx fc-connect validate-schema` before deployment
-
-**Issue**: "S3 connection timeout"
-
-- **Cause**: Network issues, firewall, or connection pool exhaustion
-- **Fix**: Check S3 credentials, verify network connectivity
-- **Prevention**: Implement connection health checks, monitor connection status
-
-**Issue**: "State not updating after successful extraction"
-
-- **Cause**: KV write failure or intentional retry logic
-- **Fix**: Check KV logs, verify state update code executed
-- **Prevention**: Add KV write verification, log state updates explicitly
-
-**Issue**: "First run exceeds record limits"
-
-- **Cause**: No previous timestamp, fetches all historical records
-- **Fix**: Set fallbackStartDate close to current date, apply additional filters
-- **Prevention**: Use appropriate fallbackStartDate for initial runs
-
-**Issue**: "Excessive duplicate records in output"
-
-- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
-- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
-- **Prevention**: Monitor duplicate rates, verify state update logic
-
----
-
-## Troubleshooting Quick Reference
-
-| Error Message | Likely Cause | Solution |
-|--------------|--------------|----------|
-| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
-| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
-| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
-| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
-| "S3 authentication failed" | Invalid credentials | Verify S3 credentials in activation variables |
-| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
-| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
-| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
-| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
-| "CSV generation failed" | Format-specific error | Check CSV generation logic, validate output |
-
----
-
----
-
-### Pattern 7: State Management & Date Overrides
-
-**Use Case**: Understand how state management works with scheduled and ad-hoc extractions.
-
-**How it works**:
-
-VersoriKV stores the last successful extraction timestamp to enable incremental sync:
-
-```typescript
-interface ExtractionState {
-  timestamp: string; // Last run timestamp (WITHOUT overlap buffer)
-  recordCount: number; // Number of records extracted
-  extractedAt: string; // When extraction completed
-  fileName?: string; // Generated filename
-  s3Key?: string; // S3 upload path
-  overlapBufferSeconds?: number; // Buffer configuration
-}
-```
-
-**State Priority Chain** (highest to lowest):
-
-1. **`fromDate` override** (manual date in webhook payload) - Highest priority
-2. **Stored state** (`await kv.get(stateKey)`) - Normal incremental mode
-3. **`fallbackStartDate`** (activation variable) - First run fallback
-
-**Three Scenarios**:
-
-#### Scenario 1: Normal Scheduled Runs (Incremental)
-
-```typescript
-// Payload: {} (empty - no overrides)
-
-// Behavior:
-// 1. Load last timestamp from KV: "2025-01-22T10:00:00Z"
-// 2. Apply overlap buffer: "2025-01-22T09:59:00Z" (query WITH buffer)
-// 3. Extract records updated since buffered time
-// 4. Calculate MAX(updatedOn) from results: "2025-01-22T14:30:00Z"
-// 5. Save new timestamp WITHOUT buffer: "2025-01-22T14:30:00Z"
-// 6. Next run starts from "2025-01-22T14:29:00Z" (with buffer)
-```
-
-**Test**:
-
-```bash
-# Trigger scheduled run (no payload needed)
-# State advances automatically
-curl -X POST https://workspace.versori.run/orders-extract-hourly
-```
-
-#### Scenario 2: Ad-hoc Extraction WITH State Update
-
-```typescript
-// Payload: { "fromDate": "2025-01-01T00:00:00Z", "updateState": true }
-
-// Behavior:
-// 1. Ignore stored state
-// 2. Use fromDate: "2025-01-01T00:00:00Z" (no buffer applied to manual dates)
-// 3. Extract all records since 2025-01-01
-// 4. Calculate MAX(updatedOn): "2025-01-22T14:30:00Z"
-// 5. Save new timestamp: "2025-01-22T14:30:00Z" (updates state!)
-// 6. Next scheduled run starts from this new timestamp
-```
-
-**Use Case**: One-time catch-up extraction that advances the state pointer.
-
-**Test**:
-
-```bash
-curl -X POST https://workspace.versori.run/orders-extract-webhook \
-  -H "x-api-key: your-api-key" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "fromDate": "2025-01-01T00:00:00Z",
-    "updateState": true
-  }'
-```
-
-#### Scenario 3: Ad-hoc Extraction WITHOUT State Update
-
-```typescript
-// Payload: { "fromDate": "2025-01-01T00:00:00Z", "updateState": false }
-
-// Behavior:
-// 1. Ignore stored state
-// 2. Use fromDate: "2025-01-01T00:00:00Z"
-// 3. Extract all records since 2025-01-01
-// 4. DO NOT update state
-// 5. Next scheduled run uses previous timestamp (unaffected)
-```
-
-**Use Case**: Historical backfill or testing without affecting incremental sync.
-
-**Test**:
-
-```bash
-curl -X POST https://workspace.versori.run/orders-extract-webhook \
-  -H "x-api-key: your-api-key" \
-  -H "Content-Type: application/json" \
-  -d '{
-    "fromDate": "2025-01-01T00:00:00Z",
-    "toDate": "2025-01-31T23:59:59Z",
-    "updateState": false
-  }'
-```
-
-**Why this matters**:
-
-- **Incremental sync** relies on state continuity
-- **Manual overrides** allow catch-up without breaking incremental flow
-- **Overlap buffer** prevents missed records at time boundaries
-- **State isolation** lets you test/backfill without affecting production sync
-
----
-
-### Pattern 8: Optional GraphQL Query Logging
-
-**Use Case**: Debug extraction issues by logging the exact GraphQL query sent to Fluent Commerce API.
-
-**When to use**:
-
-- ✅ Debugging pagination issues
-- ✅ Verifying query variables (dates, filters, limits)
-- ✅ Development and testing
-- ❌ Production (verbose logs, potential secrets in variables)
-
-**How to enable**:
-
-Set `DEBUG_GRAPHQL=true` environment variable in Versori activation settings.
-
-**Implementation**:
-
-```typescript
-// In your extraction workflow
-const DEBUG_GRAPHQL = activation?.getVariable('DEBUG_GRAPHQL') === 'true';
-
-if (DEBUG_GRAPHQL) {
-  log.info('GraphQL Query Debug', {
-    query: ORDERS_QUERY,
-    variables: {
-      retailerId,
-      updatedAfter: bufferedLastRunTime,
-      first: pageSize,
-      after: null, // First page
-    },
-    pagination: {
-      pageSize,
-      maxRecords,
-      currentPage: 1,
-    },
-  });
-}
-
-const extractionResult = await orchestrator.extract({
-  query: ORDERS_QUERY,
-  resultPath: 'orders.edges.node',
-  variables: {
-    retailerId,
-    updatedAfter: bufferedLastRunTime,
-  },
-  pageSize,
-  maxRecords,
-});
-
-if (DEBUG_GRAPHQL) {
-  log.info('GraphQL Response Debug', {
-    totalRecords: extractionResult.stats.totalRecords,
-    totalPages: extractionResult.stats.totalPages,
-    validRecords: extractionResult.stats.validRecords ?? extractionResult.data.length,
-    firstRecordId: extractionResult.data[0]?.id,
-    lastRecordId: extractionResult.data[extractionResult.data.length - 1]?.id,
-  });
-}
-```
-
-**What gets logged**:
-
-```json
-{
-  "level": "info",
-  "message": "GraphQL Query Debug",
-  "query": "query GetOrders($retailerId: ID!, $updatedAfter: DateTime!, ...)",
-  "variables": {
-    "retailerId": "ACME",
-    "updatedAfter": "2025-01-22T09:59:00Z",
-    "first": 200,
-    "after": null
-  },
-  "pagination": {
-    "pageSize": 200,
-    "maxRecords": 10000,
-    "currentPage": 1
-  }
-}
-```
-
-**Versori Environment Variables**:
-
-Add to activation settings:
-
-```json
-{
-  "DEBUG_GRAPHQL": "true"
-}
-```
-
-**Testing**:
-
-```bash
-# Enable debug logging
-curl -X POST https://workspace.versori.run/orders-extract-hourly
-
-# Check Versori logs for "GraphQL Query Debug" entries
-# Verify query structure and variables are correct
-```
-
-**Sample Debug Output**:
-
-```
-[INFO] GraphQL Query Debug
-  query: "query GetOrders($retailerId: ID!, $updatedAfter: DateTime!, $first: Int!, $after: String) { ... }"
-  variables: { retailerId: "ACME", updatedAfter: "2025-01-22T09:59:00Z", first: 200, after: null }
-  pagination: { pageSize: 200, maxRecords: 10000, currentPage: 1 }
-
-[INFO] Extraction complete
-  totalRecords: 150
-  totalPages: 1
-  validRecords: 150
-  failedValidations: 0
-
-[INFO] GraphQL Response Debug
-  totalRecords: 150
-  totalPages: 1
-  validRecords: 150
-  firstRecordId: "order_123"
-  lastRecordId: "order_272"
-```
-
-**Key Benefits**:
-
-- Quickly identify pagination configuration issues
-- Verify date filters are applied correctly
-- Debug "no records found" scenarios
-- Validate ExtractionOrchestrator variable injection
-
-**Production Best Practice**: Disable `DEBUG_GRAPHQL` in production to reduce log volume and avoid logging sensitive data.
-
----
-
-## Troubleshooting
-
-**Issue**: "All records failed mapping"
-
-**Cause**: Schema mismatch or incorrect field paths
-
-**Solutions**:
-
-1. Run `npx fc-connect introspect-schema` to get current schema
-2. Run `npx fc-connect validate-schema` to check mapping
-3. Review error details: `errors.map(e => e.errors)`
-
----
-
-**Issue**: "State not updating, same orders exported every run"
-
-**Cause**: KV storage write failure or timestamp calculation error
-
-**Solutions**:
-
-1. Check KV logs for write errors
-2. Verify state update code is reached: add logging
-3. Check `newTimestamp` calculation logic
-4. Manually inspect KV value: `await kv.get(['extraction', 'orders', 'lastRunTime'])`
-
----
-
-**Issue**: "Orders with no line items cause mapping errors"
-
-**Cause**: `order.items` is empty array or null
-
-**Solutions**:
-
-```typescript
-if (lineItems.length === 0) {
-  log.warn('Order has no line items, skipping', { orderId: order.id, ref: order.ref });
-  continue;
-}
-```
-
----
-
-## See Also
-
-**Related Templates:**
-
-- [Orders to SFTP XML](./template-extraction-orders-to-sftp-xml.md) - Same entity, different format/destination
-- [Fulfillments to S3 CSV](./template-extraction-fulfillments-to-sftp-csv.md) - Fulfillment lifecycle tracking
-- [Products to S3 JSON](./template-extraction-products-to-s3-json.md) - Product catalog extraction
-
-**SDK Documentation:**
-
-- [ExtractionOrchestrator Guide](../../../../../02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-08-extraction-orchestrator.md) - Auto-pagination and path-based extraction
-- [JobTracker Reference](../../../../../02-CORE-GUIDES/api-reference/modules/api-reference-05-services.md#jobtracker-new) - Job lifecycle management
-- [Universal Mapping Guide](../../../../../02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-01-introduction.md) - Field transformation
-- [CSVParserService API](../../../../../02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-02-csv-parser.md) - CSV generation
-
----
-
-### Pattern 8: Backward Pagination (Optional - Advanced)
-
-**Use Case**: Extract data in reverse chronological order (newest to oldest) instead of oldest to newest.
-
-**When to Use**:
-
-- ✅ Need most recent records first (e.g., latest orders, recent inventory updates)
-- ✅ Time-bounded reverse traversal for auditing
-- ✅ Display newest-first in UI/reports
-- ❌ **Don't use for standard incremental sync** - use forward pagination (default)
-
-**GraphQL Query Requirements**:
-
-Your query must support backward pagination by including `$last` and `$before`:
-
-```graphql
-query GetData(
-  $retailerId: ID!
-  $first: Int # For forward pagination
-  $after: String # For forward pagination
-  $last: Int # For backward pagination
-  $before: String # For backward pagination
-) {
-  data(retailerId: $retailerId, first: $first, after: $after, last: $last, before: $before) {
-    edges {
-      cursor # ✅ REQUIRED
-      node {
-        id
-        createdAt
-        # ... other fields
-      }
-    }
-    pageInfo {
-      hasNextPage # For forward
-      hasPreviousPage # ✅ REQUIRED for backward
-    }
-  }
-}
-```
-
-**Implementation**:
-
-```typescript
-// Backward pagination - newest records first
-const result = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  variables: {
-    retailerId,
-    dateRangeFilter: { from: bufferedLastRunTime, to: effectiveEndTime },
-    // ❌ Don't include last/before - orchestrator injects them
-  },
-  pageSize: 200,
-  direction: 'backward', // ✅ Enable reverse pagination
-  maxRecords: 10000,
-});
-
-// Records are returned in reverse chronological order
-console.log(result.data[0].createdAt); // Newest
-console.log(result.data[result.data.length - 1].createdAt); // Oldest (within range)
-```
-
-**Key Differences from Forward Pagination**:
-
-| Aspect                 | Forward (Default)                | Backward                |
-| ---------------------- | -------------------------------- | ----------------------- |
-| **Direction**          | `direction: 'forward'` (default) | `direction: 'backward'` |
-| **Variables Injected** | `first`, `after`                 | `last`, `before`        |
-| **PageInfo Field**     | `hasNextPage`                    | `hasPreviousPage`       |
-| **Cursor Source**      | Last edge of page                | First edge of page      |
-| **Record Order**       | Oldest → Newest                  | Newest → Oldest         |
-
-**Important Notes**:
-
-1. **Orchestrator injects variables**: Don't pass `last` or `before` in your variables object - the orchestrator injects them based on `pageSize` and cursor tracking.
-
-2. **Query signature**: Your GraphQL query must declare `$last` and `$before` parameters even if you don't pass them explicitly.
-
-3. **PageInfo requirement**: Response must include `pageInfo.hasPreviousPage` or the orchestrator will throw an error.
-
-4. **Cursor requirement**: Each edge must include `cursor` field for pagination to work.
-
-**Example: Extract Latest 1000 Orders**
-
-```typescript
-const latestOrders = await orchestrator.extract({
-  query: ORDERS_QUERY,
-  resultPath: 'orders.edges.node',
-  variables: {
-    retailerId,
-    statuses: ['BOOKED', 'ALLOCATED'],
-  },
-  direction: 'backward', // Start from newest
-  maxRecords: 1000, // Stop after 1000 records
-  pageSize: 100, // 100 per page = 10 pages
-});
-
-// latestOrders.data[0] is the newest order
-// latestOrders.data[999] is the 1000th newest order
-```
-
-**When to Use Forward vs Backward**:
-
-```typescript
-// ✅ Forward (default) - For incremental sync
-const incrementalData = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  variables: {
-    dateRangeFilter: { from: lastSyncTime, to: now },
-  },
-  // direction defaults to 'forward'
-  // Processes oldest → newest for proper sequencing
-});
-
-// ✅ Backward - For "latest N records" use cases
-const latestData = await orchestrator.extract({
-  query: YOUR_QUERY,
-  resultPath: 'data.edges.node',
-  direction: 'backward',
-  maxRecords: 100, // Just get latest 100
-  // Gets newest → oldest
-});
-```
-
-**Pagination Variables Reference**:
-
-| Variable | Forward     | Backward    | Injected By  | Notes                    |
-| -------- | ----------- | ----------- | ------------ | ------------------------ |
-| `first`  | ✅ Used     | ❌ Not used | Orchestrator | From `pageSize`          |
-| `after`  | ✅ Used     | ❌ Not used | Orchestrator | From cursor (last edge)  |
-| `last`   | ❌ Not used | ✅ Used     | Orchestrator | From `pageSize`          |
-| `before` | ❌ Not used | ✅ Used     | Orchestrator | From cursor (first edge) |
-
-**Common Mistakes to Avoid**:
-
-```typescript
-// ❌ WRONG - Don't pass pagination variables
-const result = await orchestrator.extract({
-  variables: {
-    last: 200, // ❌ Orchestrator will override this
-    before: cursor, // ❌ Orchestrator manages cursor
-  },
-  direction: 'backward',
-});
-
-// ✅ CORRECT - Let orchestrator inject pagination
-const result = await orchestrator.extract({
-  variables: {
-    retailerId, // ✅ Your business variables only
-  },
-  pageSize: 200, // ✅ Orchestrator uses this for last/before
-  direction: 'backward',
-});
-```
-
-#### Optional: Reverse Pagination
-
-- Default: forward pagination ($first/$after) with pageInfo.hasNextPage.
-- Reverse: declare $last/$before in the query and include pageInfo.hasPreviousPage; set direction='backward' in the orchestrator call.
-
-GraphQL:
-
-```graphql
-query GetOrdersBackward(
-  $retailerId: ID!
-  $dateRangeFilter: DateRange
-  $last: Int!
-  $before: String
-) {
-  orders(retailerId: $retailerId, updatedOn: $dateRangeFilter, last: $last, before: $before) {
-    edges {
-      cursor
-      node {
-        id
-        ref
-        updatedOn
-      }
-    }
-    pageInfo {
-      hasPreviousPage
-    }
-  }
-}
-```
-
-SDK:
-
-```typescript
-await orchestrator.extract({
-  query: ORDERS_BACKWARD_QUERY,
-  resultPath: 'orders.edges.node',
-  variables: { retailerId, dateRangeFilter },
-  pageSize,
-  direction: 'backward',
-});
-```
+---
+template_id: tpl-extract-orders-to-s3-csv
+canonical_filename: template-extraction-orders-to-s3-csv.md
+version: 2.0.0
+sdk_version: ^0.1.39
+runtime: versori
+direction: extraction
+source: fluent-graphql
+destination: s3-csv
+entity: orders
+format: csv
+logging: versori
+status: stable
+features:
+  - memory-management
+  - enhanced-logging
+  - pagination-progress
+---
+
+# Template: Extraction - Orders to S3 CSV
+
+**Template Version:** 2.0.0
+**SDK Version:** @fluentcommerce/fc-connect-sdk@^0.1.39
+**Last Updated:** 2025-01-24
+**Deployment Target:** Versori Platform
+
+**🆕 Version 2.0.0 Enhancements:**
+- ✅ **Memory Management** - Clear large result sets after processing batches
+- ✅ **Enhanced Logging** - Pagination progress tracking with emoji indicators (📊, 📥, ✅)
+- ✅ **Pagination Progress** - Real-time page-by-page progress logging with metrics
+
+---
+
+## 📚 STEP 1: Load These Docs (Human Checklist)
+
+1. REQUIRED (load all)
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+   - [ ] fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+   - [ ] fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+   - [ ] fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+
+Copy-paste list (open these):
+fc-connect-sdk/docs/02-CORE-GUIDES/api-reference/
+fc-connect-sdk/docs/02-CORE-GUIDES/mapping/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/data-sources/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/parsers/
+fc-connect-sdk/docs/03-PATTERN-GUIDES/extraction/
+fc-connect-sdk/docs/04-REFERENCE/platforms/versori/
+
+---
+
+## 📋 STEP 2: Tell Your AI (Prompt)
+
+Copy/paste this prompt after loading the documentation above:
+
+```
+Create a Versori scheduled extractor for orders that uses ExtractionOrchestrator + JobTracker, incremental updatedOn with a 60s overlap buffer, transforms via UniversalMapper, generates CSV with CSVParserService.stringify(), uploads to S3 using S3DataSource. Include 3 workflows: scheduled, ad-hoc webhook, and job-status query with native Versori logging.
+```
+
+---
+
+## 📦 SDK Imports (Verified - Versori Optimized)
+
+```typescript
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  ExtractionOrchestrator,
+  JobTracker,
+  UniversalMapper,
+  CSVParserService,
+  S3DataSource,
+  VersoriKVAdapter,
+} from '@fluentcommerce/fc-connect-sdk';
+
+import { schedule, webhook, http, fn } from '@versori/run';
+```
+
+---
+
+# Versori Scheduled: Orders Extraction to S3 CSV (Incremental)
+
+**FC Connect SDK Use Case Guide**
+
+> **SDK**: [@fluentcommerce/fc-connect-sdk](https://www.npmjs.com/package/@fluentcommerce/fc-connect-sdk)
+> **Installation**: `npm install @fluentcommerce/fc-connect-sdk`
+
+Context: Scheduled Versori workflow that extracts new/updated orders from Fluent Commerce via GraphQL query with **ExtractionOrchestrator**, **JobTracker**, and **incremental timestamp tracking**, transforms with `UniversalMapper`, and writes **CSV files** to S3 for 3PL/fulfillment systems.
+
+**Pattern**: EXTRACTION (Fluent → S3 CSV)
+**Complexity**: High | Runtime: Versori Platform (Scheduled)
+
+---
+
+## ⚠️ IMPORTANT: Production-Ready Base Template
+
+> **📋 BASE TEMPLATE - Ready for Production (Customize for Your Needs)**
+>
+> This is a **production-ready base template** demonstrating FC Connect SDK best practices for order extraction workflows with CSV output to S3.
+>
+> **✅ INCLUDED FEATURES:**
+>
+> - ✅ Comprehensive error handling with retry logic
+> - ✅ S3 upload with proper error handling
+> - ✅ State management with overlap buffer (prevents missed records)
+> - ✅ Job tracking with lifecycle management
+> - ✅ Security (credential masking in logs)
+> - ✅ UTC time enforcement (prevents timezone bugs)
+> - ✅ Incremental extraction (safe, efficient, production-ready)
+> - ✅ Natural rate limiting via timestamps
+>
+> **📝 BEFORE DEPLOYING:**
+>
+> 1. Review and customize activation variables for your environment
+> 2. Test with sample data in your Versori workspace
+> 3. Adjust safety limits (pageSize, maxRecords) if needed
+> 4. Configure monitoring alerts for extraction failures
+> 5. Verify S3 bucket credentials and paths
+>
+> **This base template follows SDK best practices - tweak specific to your needs.**
+
+---
+
+## What You'll Build
+
+- **Incremental extraction** using `updatedOn >= (lastRunTime - buffer)` with **overlap buffer**
+- **ExtractionOrchestrator** for auto-pagination and path-based extraction
+- **JobTracker** for lifecycle management
+- **State management** with VersoriKV to track last successful run
+- **Safety buffer** (60 seconds) to handle clock skew and race conditions
+- GraphQL query with nested order lines
+- UniversalMapper transformation with line item flattening
+- **CSV file generation** with CSVParserService
+- **S3 upload** to 3PL/fulfillment system
+- **3 workflow patterns**: scheduled, ad-hoc webhook, job status query
+- **Failure recovery** with timestamp tracking
+
+## Business Use Case
+
+**Hourly order feed to 3PL/fulfillment center:**
+
+- Extract new and updated orders since last run
+- Flatten line items (one CSV row per item)
+- Generate CSV file with order header + line items
+- Upload to S3 bucket for 3PL integration
+- Run every hour to enable real-time fulfillment
+- Support order updates (address changes, item modifications)
+- Standard CSV format for WMS/ERP imports
+
+## SDK Methods Used
+
+```typescript
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  ExtractionOrchestrator,
+  JobTracker,
+  UniversalMapper,
+  CSVParserService,
+  S3DataSource,
+  VersoriKVAdapter,
+} from '@fluentcommerce/fc-connect-sdk';
+
+await createClient(ctx); // Versori-aware client
+const orchestrator = new ExtractionOrchestrator(client, log); // Auto-pagination
+const tracker = new JobTracker(kv, log); // Job lifecycle tracking
+await orchestrator.extract({ query, resultPath, variables, pageSize, maxRecords }); // Extract
+new VersoriKVAdapter(ctx.openKv(':project:')); // State management
+new UniversalMapper(exportMapping); // Field transformation
+const csvParser = new CSVParserService(); // CSV generation
+csvParser.stringify(records, { headers: true }); // Generate CSV content (synchronous)
+await s3.upload(key, buffer, contentType, metadata); // S3 upload
+```
+
+## Activation Variables
+
+```json
+{
+  "retailerId": "your-retailer-id",
+  "s3BucketName": "3pl-orders-export",
+  "awsAccessKeyId": "AKIAXXXXXXXXXXXX",
+  "awsSecretAccessKey": "********",
+  "awsRegion": "us-east-1",
+  "s3Prefix": "orders/new/",
+  "pageSize": 200,
+  "maxRecords": 10000,
+  "fallbackStartDate": "2024-01-01T00:00:00Z",
+  "overlapBufferSeconds": "60",
+  "validateConnection": "true"
+}
+```
+
+## Export Mapping Configuration
+
+Create file: `./config/orders.export.csv.json`
+
+```json
+{
+  "name": "orders.export.csv",
+  "version": "1.0.0",
+  "description": "Fluent Orders → 3PL CSV Export",
+  "fields": {
+    "order_id": { "source": "ref", "required": true, "resolver": "sdk.trim" },
+    "order_date": { "source": "createdOn", "required": true, "resolver": "sdk.formatDateShort" },
+    "order_status": { "source": "status", "required": true, "resolver": "sdk.uppercase" },
+    "customer_name": { "source": "customer.firstName", "required": false, "resolver": "sdk.trim" },
+    "customer_email": { "source": "customer.email", "required": false, "resolver": "sdk.trim" },
+    "ship_to_name": { "source": "deliveryAddress.name", "required": true, "resolver": "sdk.trim" },
+    "ship_to_address1": {
+      "source": "deliveryAddress.street1",
+      "required": true,
+      "resolver": "sdk.trim"
+    },
+    "ship_to_city": { "source": "deliveryAddress.city", "required": true, "resolver": "sdk.trim" },
+    "ship_to_state": {
+      "source": "deliveryAddress.state",
+      "required": true,
+      "resolver": "sdk.uppercase"
+    },
+    "ship_to_zip": {
+      "source": "deliveryAddress.postcode",
+      "required": true,
+      "resolver": "sdk.trim"
+    },
+    "ship_to_country": {
+      "source": "deliveryAddress.country",
+      "required": true,
+      "resolver": "sdk.uppercase"
+    },
+    "line_item_sku": { "source": "product.ref", "required": true, "resolver": "sdk.trim" },
+    "line_item_qty": { "source": "quantity", "required": true, "resolver": "sdk.parseInt" },
+    "line_item_price": { "source": "price", "required": true, "resolver": "sdk.parseFloat" }
+  }
+}
+```
+
+## GraphQL Query
+
+```graphql
+query GetOrders($retailerId: ID!, $updatedAfter: DateTime!, $first: Int!, $after: String) {
+  orders(
+    retailerId: $retailerId
+    updatedOn: { after: $updatedAfter }
+    first: $first
+    after: $after
+  ) {
+    edges {
+      node {
+        id
+        ref
+        status
+        createdOn
+        updatedOn
+        customer {
+          firstName
+          lastName
+          email
+        }
+        deliveryAddress {
+          name
+          street1
+          street2
+          city
+          state
+          postcode
+          country
+        }
+        items {
+          id
+          quantity
+          price
+          product {
+            ref
+            name
+          }
+        }
+      }
+      cursor
+    }
+    pageInfo {
+      hasNextPage
+    }
+  }
+}
+```
+
+---
+
+## Versori Workflows Structure
+
+**Key Concept**: Versori workflows are organized by **trigger type** at the first level, then by **specific workflow** with descriptive file names.
+
+**Trigger Types:**
+- **`schedule()`** → Time-based triggers (cron expressions) - NOT exposed as HTTP endpoints
+- **`webhook()`** → HTTP-based triggers (event-driven) - Creates HTTP endpoints
+- **`workflow()`** → Durable workflows (advanced, rarely used)
+
+**Execution Steps (chained to triggers):**
+- **`http()`** → External API calls (chained from schedule/webhook)
+- **`fn()`** → Internal processing (chained from schedule/webhook)
+
+### Recommended Project Structure (MemoryInterpreter Pattern)
+
+```
+orders-extraction/
+├── index.ts                              # Entry point - exports all workflows (MemoryInterpreter pattern)
+└── src/
+    ├── workflows/
+    │   ├── scheduled/
+    │   │   └── daily-orders-extraction.ts  # Scheduled: Daily orders extraction
+    │   │
+    │   └── webhook/
+    │       ├── adhoc-orders-extraction.ts  # Webhook: Manual trigger
+    │       └── job-status-check.ts         # Webhook: Status query
+    │
+    ├── services/
+    │   └── orders-extraction.service.ts   # Shared orchestration logic (reusable)
+    │
+    └── config/
+        └── orders.export.csv.json         # Mapping configuration
+```
+
+**MemoryInterpreter Pattern:**
+- `index.ts` exports workflow definitions (lightweight registration)
+- Workflow files delegate to service functions for business logic
+- Service functions contain the actual implementation
+- This pattern enables better memory management and code organization
+
+**Note:** The code examples below demonstrate what goes in each workflow file. You would split them into separate files following the structure above.
+
+---
+
+```csv
+order_id,order_date,order_status,customer_name,customer_email,ship_to_name,ship_to_address1,ship_to_city,ship_to_state,ship_to_zip,ship_to_country,line_item_sku,line_item_qty,line_item_price
+ORD-001,2025-01-22,CREATED,John,john@example.com,John Smith,123 Main St,New York,NY,10001,US,SKU-001,2,29.99
+ORD-001,2025-01-22,CREATED,John,john@example.com,John Smith,123 Main St,New York,NY,10001,US,SKU-002,1,49.99
+ORD-002,2025-01-22,PAID,Jane,jane@example.com,Jane Doe,456 Oak Ave,Los Angeles,CA,90001,US,SKU-003,1,19.99
+```
+
+**Note**: Each line item becomes a separate row with duplicated order header data (standard 3PL format).
+
+---
+
+## Complete Implementation
+
+### Workflow 1: Scheduled Extraction (Main Workflow)
+
+**File: `src/workflows/scheduled/daily-orders-extraction.ts`**
+
+```typescript
+import { schedule, http } from '@versori/run';
+import { JobTracker } from '@fluentcommerce/fc-connect-sdk';
+import { processOrdersExtraction } from '../../services/orders-extraction.service';
+
+/**
+ * Scheduled workflow: Hourly orders extraction to S3 CSV
+ * Runs every hour
+ *
+ * DELEGATION PATTERN (MemoryInterpreter):
+ * - Workflow receives ctx from Versori
+ * - Passes entire ctx to service function
+ * - Service handles all business logic
+ */
+export const ordersExtractionScheduled = schedule('orders-extract-hourly', '0 * * * *').then(
+  http('extract-orders', { connection: 'fluent_commerce' }, async (ctx: any) => {
+    const { log, openKv } = ctx;
+    const executionStartTime = Date.now();
+    const jobId = `SCHEDULED_ORD_${new Date().toISOString().replace(/[:.]/g, '-')}_${Date.now()}`;
+    const tracker = new JobTracker(openKv(':project:'), log);
+
+    log.info('🚀 [WORKFLOW] Starting scheduled orders extraction', { jobId });
+
+    await tracker.createJob(jobId, {
+      triggeredBy: 'schedule',
+      stage: 'initialization',
+      startTime: executionStartTime,
+    });
+
+    await tracker.updateJob(jobId, { status: 'processing' });
+
+    try {
+      // DELEGATION: Pass entire ctx to service function
+      const result = await processOrdersExtraction(ctx, jobId, tracker);
+
+      const duration = Date.now() - executionStartTime;
+
+      if (result.success) {
+        await tracker.markCompleted(jobId, { ...result, duration });
+        log.info('✅ [WORKFLOW] Orders extraction completed successfully', {
+          jobId,
+          ordersExtracted: result.ordersExtracted,
+          fileName: result.fileName,
+          duration: `${duration}ms`,
+        });
+      } else {
+        await tracker.markFailed(jobId, result.error || 'Unknown error');
+        log.error('❌ [WORKFLOW] Orders extraction failed', {
+          jobId,
+          error: result.error,
+          duration: `${duration}ms`,
+        });
+      }
+
+      return { success: true, jobId, ...result, duration };
+    } catch (e: any) {
+      const errorMessage = e instanceof Error ? e.message : String(e);
+      const duration = Date.now() - executionStartTime;
+
+      await tracker.markFailed(jobId, errorMessage);
+
+      log.error('❌ [WORKFLOW] Orders extraction failed with exception', {
+        jobId,
+        error: errorMessage,
+        stack: e instanceof Error ? e.stack : undefined,
+        duration: `${duration}ms`,
+      });
+
+      return {
+        success: false,
+        jobId,
+        error: errorMessage,
+        duration,
+        recommendations: [
+          'Check SFTP/S3 connection credentials',
+          'Verify GraphQL query permissions',
+          'Review error stack trace for details'
+        ]
+      };
+    }
+  })
+);
+```
+
+**File: `src/services/orders-extraction.service.ts`**
+
+```typescript
+import { Buffer } from 'node:buffer';
+import {
+  createClient,
+  ExtractionOrchestrator,
+  JobTracker,
+  UniversalMapper,
+  CSVParserService,
+  S3DataSource,
+  VersoriKVAdapter,
+} from '@fluentcommerce/fc-connect-sdk';
+import ordersExportMapping from '../config/orders.export.csv.json' with { type: 'json' };
+
+const ORDERS_QUERY = `
+  query GetOrders($retailerId: ID!, $updatedAfter: DateTime!, $first: Int!, $after: String) {
+    orders(
+      retailerId: $retailerId
+      updatedOn: { after: $updatedAfter }
+      first: $first
+      after: $after
+    ) {
+      edges {
+        node {
+          id
+          ref
+          status
+          createdOn
+          updatedOn
+          customer {
+            firstName
+            lastName
+            email
+          }
+          deliveryAddress {
+            name
+            street1
+            street2
+            city
+            state
+            postcode
+            country
+          }
+          items {
+            id
+            quantity
+            price
+            product {
+              ref
+              name
+            }
+          }
+        }
+        cursor
+      }
+      pageInfo {
+        hasNextPage
+      }
+    }
+  }
+`;
+
+/**
+ * Main orders extraction service function
+ *
+ * This function orchestrates the complete extraction process:
+ * 1. Client initialization
+ * 2. State management and incremental sync
+ * 3. GraphQL extraction with pagination
+ * 4. Data transformation and CSV generation
+ * 5. S3 upload and state tracking
+ *
+ * @param ctx - Versori context object containing fetch, connections, log, activation, openKv
+ * @param jobId - Job ID for tracking
+ * @param tracker - JobTracker instance for job lifecycle management
+ */
+export async function processOrdersExtraction(ctx: any, jobId: string, tracker: JobTracker) {
+  const { log, openKv, activation } = ctx;
+  const startTime = Date.now();
+
+  log.info('🔍 [OrdersExtraction] ==================== EXECUTION START ====================');
+  log.info('🔍 [ExtractionOrchestrator] Starting extraction');
+
+  try {
+    // STEP 1/8: Extract context and configuration
+    log.info('📄 [OrdersExtraction] STEP 1/8: Loading configuration');
+    const retailerId = activation?.getVariable('retailerId');
+    const pageSize = parseInt(activation?.getVariable('pageSize') || '200', 10);
+    const maxRecords = parseInt(activation?.getVariable('maxRecords') || '10000', 10);
+    const fallbackStartDate =
+      activation?.getVariable('fallbackStartDate') || '2024-01-01T00:00:00Z';
+    const overlapBufferSeconds = parseInt(
+      activation?.getVariable('overlapBufferSeconds') || '60',
+      10
+    );
+    const OVERLAP_BUFFER_MS = overlapBufferSeconds * 1000;
+    const validateConnection = activation?.getVariable('validateConnection') === 'true';
+
+    const s3Config = {
+      bucket: activation?.getVariable('s3BucketName'),
+      region: activation?.getVariable('awsRegion') || 'us-east-1',
+      accessKeyId: activation?.getVariable('awsAccessKeyId'),
+      secretAccessKey: activation?.getVariable('awsSecretAccessKey'),
+    };
+    const s3Prefix = activation?.getVariable('s3Prefix') || 'orders/new/';
+
+    const missing: string[] = [];
+    if (!retailerId) missing.push('retailerId');
+    if (!s3Config.bucket) missing.push('s3BucketName');
+    if (!s3Config.accessKeyId) missing.push('awsAccessKeyId');
+    if (!s3Config.secretAccessKey) missing.push('awsSecretAccessKey');
+    if (missing.length) {
+      log.error('❌ [OrdersExtraction] Missing required activation variables', { missing });
+      return {
+        success: false,
+        error: `Missing required variables: ${missing.join(', ')}`,
+        recommendations: [
+          'Check Activation Variables in Versori dashboard',
+          'Ensure all required credentials are configured',
+          'Verify connection settings for fluent_commerce and S3'
+        ]
+      };
+    }
+
+    // STEP 2/8: Initialize SDK services
+    log.info('📄 [OrdersExtraction] STEP 2/8: Initializing SDK services');
+    const client = await createClient(ctx);
+    const orchestrator = new ExtractionOrchestrator(client, log);
+    const kv = new VersoriKVAdapter(openKv(':project:'));
+    const mapper = new UniversalMapper(ordersExportMapping);
+    const csvParser = new CSVParserService();
+    const s3 = new S3DataSource(
+      {
+        type: 'S3_CSV',
+        connectionId: 's3-orders-export',
+        name: 'S3 Orders Export',
+        s3Config,
+      },
+      log
+    );
+
+    // Validate S3 connection if enabled
+    if (validateConnection) {
+      log.info('🔍 [OrdersExtraction] Validating S3 connection');
+      try {
+        await s3.validateConnection();
+        log.info('✅ [OrdersExtraction] S3 connection validated successfully');
+      } catch (validationError: any) {
+        log.error('❌ [OrdersExtraction] S3 connection validation failed', {
+          error: validationError.message,
+        });
+        return {
+          success: false,
+          error: `S3 connection validation failed: ${validationError.message}`,
+          recommendations: [
+            'Verify S3 bucket exists and is accessible',
+            'Check AWS credentials (accessKeyId, secretAccessKey)',
+            'Ensure correct AWS region is specified',
+            'Verify IAM permissions for S3 operations'
+          ]
+        };
+      }
+    }
+
+    // STEP 3/8: Create job and load state
+    log.info('📄 [OrdersExtraction] STEP 3/8: Loading state for incremental sync');
+
+    const stateKey = ['extraction', 'orders', 'lastRunTime'];
+    const lastRunState = await kv.get(stateKey);
+    const rawLastRunTime = lastRunState?.value?.timestamp || fallbackStartDate;
+
+    // Apply overlap buffer for query (WITH buffer)
+    const bufferedLastRunTime = new Date(
+      new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_MS
+    ).toISOString();
+
+    log.info('🔍 [OrdersExtraction] Starting incremental extraction with overlap buffer', {
+      rawLastRunTime,
+      bufferedLastRunTime,
+      overlapBufferSeconds,
+      retailerId,
+    });
+
+    // STEP 4/8: Execute extraction with ExtractionOrchestrator
+    log.info('📄 [OrdersExtraction] STEP 4/8: Executing GraphQL extraction');
+    const extractionStartTime = Date.now();
+
+    const result = await orchestrator.extract({
+      query: ORDERS_QUERY,
+      resultPath: 'orders.edges.node',
+      variables: {
+        retailerId,
+        updatedAfter: bufferedLastRunTime, // ← WITH overlap buffer
+      },
+      pageSize,
+      maxRecords,
+    });
+
+    const orders = result.data || [];
+    const extractionDuration = Date.now() - extractionStartTime;
+
+    log.info('✅ [ExtractionOrchestrator] Extraction completed', {
+      totalRecords: result.stats.totalRecords,
+      totalPages: result.stats.totalPages,
+      validRecords: result.stats.validRecords ?? orders.length,
+      errors: result.errors ? result.errors.length : 0,
+      duration: `${extractionDuration}ms`,
+    });
+
+    if (result.errors && result.errors.length > 0) {
+      log.warn('⚠️ [OrdersExtraction] Non-fatal extraction errors encountered', {
+        errorCount: result.errors.length,
+        sampleErrors: result.errors.slice(0, 3),
+      });
+    }
+
+    if (orders.length === 0) {
+      const duration = Date.now() - startTime;
+      log.info('✅ [OrdersExtraction] No new orders to extract');
+      await kv.set(stateKey, {
+        timestamp: new Date().toISOString(),
+        orderCount: 0,
+        extractedAt: new Date().toISOString(),
+      });
+      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
+      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
+      return {
+        success: true,
+        message: 'No new orders to extract',
+        lastRunTime: rawLastRunTime,
+        duration,
+      };
+    }
+
+    log.info('📄 [OrdersExtraction] Orders retrieved', { count: orders.length });
+
+    // STEP 5/8: Transform and flatten line items (bulk mapping)
+    log.info('📄 [OrdersExtraction] STEP 5/8: Transforming and flattening line items');
+    const transformStartTime = Date.now();
+
+    // Collect all flattened records first (order + item combinations)
+    const flattenedInputs: any[] = [];
+
+    for (const order of orders) {
+      const lineItems = order.items || [];
+
+      if (lineItems.length === 0) {
+        log.warn('⚠️ [OrdersExtraction] Order has no line items, skipping', {
+          orderId: order.id,
+          ref: order.ref,
+        });
+        continue;
+      }
+
+      // Create flattened records (one per line item)
+      for (const item of lineItems) {
+        flattenedInputs.push({
+          ...order, // Order header data
+          ...item, // Line item data (overwrites items array with individual item)
+        });
+      }
+    }
+
+    if (flattenedInputs.length === 0) {
+      const duration = Date.now() - startTime;
+      log.error('❌ [OrdersExtraction] All records skipped (no line items)');
+      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
+      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
+      return {
+        success: false,
+        error: 'All orders skipped because they contain no line items',
+        duration,
+        recommendations: [
+          'Check order data quality in Fluent Commerce',
+          'Verify line items are being populated correctly',
+          'Review GraphQL query to ensure items are included'
+        ]
+      };
+    }
+
+    // Bulk mapping with UniversalMapper
+    const mappingResult = await mapper.map(flattenedInputs);
+    const transformDuration = Date.now() - transformStartTime;
+
+    if (!mappingResult.success) {
+      const mappingErrors = mappingResult.errors || ['Unknown mapping failure'];
+      const duration = Date.now() - startTime;
+      log.error('❌ [OrdersExtraction] Mapping failed - terminating job', {
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+        duration: `${duration}ms`,
+      });
+
+      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
+      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
+
+      return {
+        success: false,
+        error: `Transformation failed: ${mappingErrors[0] || 'Unknown error'}`,
+        errors: mappingErrors,
+        duration,
+        recommendations: [
+          'Check mapping configuration in orders.export.csv.json',
+          'Verify source field paths match GraphQL response',
+          'Review sample errors for specific field issues',
+          'Validate resolver functions are working correctly'
+        ]
+      };
+    }
+
+    const flattenedRecords = mappingResult.data || [];
+    const mappingErrors = mappingResult.errors || [];
+
+    if (mappingErrors.length > 0) {
+      log.warn('⚠️ [OrdersExtraction] Some records failed transformation', {
+        errorCount: mappingErrors.length,
+        sampleErrors: mappingErrors.slice(0, 3),
+      });
+    }
+
+    if (mappingResult.skippedFields && mappingResult.skippedFields.length > 0) {
+      log.info('ℹ️ [OrdersExtraction] Optional fields skipped (undefined values)', {
+        skippedFields: mappingResult.skippedFields,
+        note: 'These fields were not present in source data. Add defaultValue to mapping config if they should always appear.',
+      });
+    }
+
+    if (flattenedRecords.length === 0) {
+      const duration = Date.now() - startTime;
+      log.error('❌ [OrdersExtraction] All records failed mapping');
+      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
+      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
+      return {
+        success: false,
+        error: 'All records failed mapping',
+        errors: mappingErrors,
+        duration,
+        recommendations: [
+          'Review mapping configuration thoroughly',
+          'Check for schema changes in Fluent Commerce',
+          'Validate all required fields are present',
+          'Test with sample data first'
+        ]
+      };
+    }
+
+    log.info('✅ [OrdersExtraction] Records transformed', {
+      orders: orders.length,
+      lineItems: flattenedRecords.length,
+      failed: mappingErrors.length,
+      duration: `${transformDuration}ms`,
+    });
+
+    // STEP 6/8: Generate CSV content (synchronous, not async)
+    log.info('📄 [OrdersExtraction] STEP 6/8: Generating CSV content');
+    const csvStartTime = Date.now();
+    const csvContent = csvParser.stringify(flattenedRecords, { headers: true });
+    const csvDuration = Date.now() - csvStartTime;
+
+    const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
+    const fileName = `orders-${timestamp}.csv`;
+    const s3Key = `${s3Prefix}${fileName}`;
+
+    log.info('✅ [OrdersExtraction] Generated CSV file', {
+      fileName,
+      size: csvContent.length,
+      orderCount: orders.length,
+      lineItemCount: flattenedRecords.length,
+      duration: `${csvDuration}ms`,
+    });
+
+    // STEP 7/8: Upload to S3
+    log.info('📄 [OrdersExtraction] STEP 7/8: Uploading to S3');
+    const uploadStartTime = Date.now();
+
+    try {
+      await s3.upload(
+        s3Key,
+        Buffer.from(csvContent, 'utf8'),
+        'text/csv',
+        {
+          orderCount: String(orders.length),
+          lineItemCount: String(flattenedRecords.length),
+          extractedAt: new Date().toISOString(),
+          lastRunTime: rawLastRunTime,
+          jobId,
+        }
+      );
+
+      const uploadDuration = Date.now() - uploadStartTime;
+      log.info('✅ [OrdersExtraction] CSV file uploaded to S3', {
+        s3Key,
+        duration: `${uploadDuration}ms`,
+      });
+    } catch (uploadError: any) {
+      const duration = Date.now() - startTime;
+      log.error('❌ [OrdersExtraction] S3 upload failed', {
+        error: uploadError.message,
+        s3Key,
+        duration: `${duration}ms`,
+      });
+
+      // Dispose S3 connection
+      try {
+        await s3.dispose();
+      } catch (disposeError: any) {
+        log.error('⚠️ [OrdersExtraction] Failed to dispose S3 connection', {
+          error: disposeError.message,
+        });
+      }
+
+      log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
+      log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
+
+      return {
+        success: false,
+        error: `S3 upload failed: ${uploadError.message}`,
+        duration,
+        recommendations: [
+          'Verify S3 bucket exists and is accessible',
+          'Check AWS credentials and permissions',
+          'Ensure network connectivity to S3',
+          'Verify S3 bucket region matches configuration',
+          'Check file size limits and quotas'
+        ]
+      };
+    }
+
+    // STEP 8/8: Update state and complete job
+    log.info('📄 [OrdersExtraction] STEP 8/8: Updating state and completing job');
+
+    // Calculate MAX(updatedOn) from extracted orders (WITHOUT buffer)
+    const maxUpdatedOn = orders.reduce((max, order) => {
+      const orderTime = new Date(order.updatedOn).getTime();
+      return orderTime > max ? orderTime : max;
+    }, new Date(rawLastRunTime).getTime());
+
+    const newTimestamp = new Date(maxUpdatedOn).toISOString();
+
+    // Save state WITHOUT buffer - buffer only applied to query
+    await kv.set(stateKey, {
+      timestamp: newTimestamp, // ← WITHOUT buffer
+      orderCount: orders.length,
+      lineItemCount: flattenedRecords.length,
+      extractedAt: new Date().toISOString(),
+      fileName,
+      s3Key,
+      overlapBufferSeconds,
+      jobId,
+      errors: mappingErrors.length > 0 ? mappingErrors : undefined,
+    });
+
+    log.info('✅ [OrdersExtraction] State updated with new timestamp', {
+      newTimestamp,
+      recordsProcessed: orders.length,
+      nextRunWillQueryFrom: new Date(
+        new Date(newTimestamp).getTime() - OVERLAP_BUFFER_MS
+      ).toISOString(),
+    });
+
+    // Dispose S3 connection
+    try {
+      await s3.dispose();
+      log.info('✅ [OrdersExtraction] S3 connection disposed successfully');
+    } catch (disposeError: any) {
+      log.error('⚠️ [OrdersExtraction] Failed to dispose S3 connection', {
+        error: disposeError.message,
+      });
+    }
+
+    const duration = Date.now() - startTime;
+
+    log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
+    log.info('⏱️ [OrdersExtraction] Total Duration: ' + duration + 'ms');
+
+    return {
+      success: true,
+      ordersExtracted: orders.length,
+      lineItemsExtracted: flattenedRecords.length,
+      recordsFailed: mappingErrors.length,
+      fileName,
+      s3Key,
+      lastRunTime: rawLastRunTime,
+      newTimestamp,
+      duration,
+      errors: mappingErrors.length > 0 ? mappingErrors : undefined,
+    };
+  } catch (error: any) {
+    const duration = Date.now() - startTime;
+    log.error('❌ [OrdersExtraction] Fatal error', {
+      message: error?.message,
+      stack: error?.stack,
+      duration: `${duration}ms`,
+    });
+
+    log.info('🔍 [OrdersExtraction] ==================== EXECUTION END ====================');
+    log.info('⏱️ [OrdersExtraction] Duration: ' + duration + 'ms');
+
+    return {
+      success: false,
+      error: error instanceof Error ? error.message : String(error),
+      stack: error instanceof Error ? error.stack : undefined,
+      errorType: error instanceof Error ? error.constructor.name : 'UnknownError',
+      duration,
+      recommendations: [
+        'Review error stack trace for root cause',
+        'Check all credentials and connection settings',
+        'Verify GraphQL query syntax and permissions',
+        'Ensure all activation variables are set correctly',
+        'Check network connectivity to Fluent Commerce and S3'
+      ]
+    };
+  }
+}
+```
+
+**File: `index.ts`**
+
+```typescript
+/**
+ * Entry point - Export all workflows for Versori platform
+ *
+ * MemoryInterpreter Pattern:
+ * This file exports workflow definitions that delegate to service functions.
+ * Each workflow is lightweight and imports business logic from services.
+ */
+
+// Scheduled workflows
+export { ordersExtractionScheduled } from './src/workflows/scheduled/daily-orders-extraction';
+
+// Webhook workflows
+export { ordersExtractionWebhook } from './src/workflows/webhook/adhoc-orders-extraction';
+export { ordersExtractionStatus } from './src/workflows/webhook/job-status-check';
+```
+
+### Workflow 2: Ad-Hoc Webhook Trigger
+
+**File: `src/workflows/webhook/adhoc-orders-extraction.ts`**
+
+```typescript
+import { webhook, http } from '@versori/run';
+import { JobTracker } from '@fluentcommerce/fc-connect-sdk';
+import { processOrdersExtraction } from '../../services/orders-extraction.service';
+
+/**
+ * Webhook workflow: Ad-hoc orders extraction (manual trigger)
+ *
+ * DELEGATION PATTERN (MemoryInterpreter):
+ * - Reuses the same service function as scheduled workflow
+ * - Lightweight wrapper that delegates to shared business logic
+ */
+export const ordersExtractionWebhook = webhook('orders-extract-webhook', {
+  connection: 'orders-adhoc',
+  response: { mode: 'sync' }, // ✅ Sync mode: response sent when handler returns
+}).then(
+  http('trigger-extraction', { connection: 'fluent_commerce' }, async (ctx: any) => {
+    const { log, openKv } = ctx;
+    const executionStartTime = Date.now();
+    const jobId = `ADHOC_ORD_${new Date().toISOString().replace(/[:.]/g, '-')}_${Date.now()}`;
+    const tracker = new JobTracker(openKv(':project:'), log);
+
+    log.info('🚀 [WEBHOOK] Adhoc orders extraction triggered', { jobId });
+
+    // Create job entry FIRST (awaited to ensure job exists in KV)
+    await tracker.createJob(jobId, {
+      triggeredBy: 'webhook',
+      stage: 'initialization',
+      status: 'queued',
+      startTime: executionStartTime,
+      createdAt: new Date().toISOString(),
+    });
+
+    // ✅ Fire-and-forget: Start background processing WITHOUT await
+    // The promise continues execution after we return the response
+    processOrdersExtraction(ctx, jobId, tracker)
+      .then((result) => {
+      const duration = Date.now() - executionStartTime;
+      if (result.success) {
+          log.info('✅ [BACKGROUND] Orders extraction completed successfully', {
+          jobId,
+          ordersExtracted: result.ordersExtracted,
+          fileName: result.fileName,
+          duration: `${duration}ms`,
+        });
+          return tracker.markCompleted(jobId, { ...result, duration });
+      } else {
+          log.error('❌ [BACKGROUND] Orders extraction failed', {
+          jobId,
+          error: result.error,
+          duration: `${duration}ms`,
+        });
+          return tracker.markFailed(jobId, result.error || 'Unknown error');
+        }
+      })
+      .catch((error: unknown) => {
+        const errorMessage = error instanceof Error ? error.message : String(error);
+        const errorStack = error instanceof Error ? error.stack : undefined;
+      const duration = Date.now() - executionStartTime;
+
+        log.error('❌ [BACKGROUND] Orders extraction failed with exception', {
+        jobId,
+        error: errorMessage,
+          stack: errorStack,
+        duration: `${duration}ms`,
+      });
+
+        return tracker.markFailed(jobId, errorMessage);
+      });
+
+    // Return immediately with jobId (response sent with this return value)
+      return {
+      success: true,
+        jobId,
+      message: 'Orders extraction started in background',
+      statusEndpoint: `https://{workspace}.versori.run/orders-job-status`,
+      note: 'Poll the status endpoint with jobId to check progress',
+    };
+  })
+);
+```
+
+### Workflow 3: Job Status Query
+
+**File: `src/workflows/webhook/job-status-check.ts`**
+
+```typescript
+import { webhook, fn } from '@versori/run';
+import { JobTracker, VersoriKVAdapter } from '@fluentcommerce/fc-connect-sdk';
+
+/**
+ * Webhook workflow: Job status query (monitor extraction progress)
+ *
+ * Simple query endpoint - no delegation needed
+ */
+export const ordersExtractionStatus = webhook('orders-extract-status', {
+  connection: 'orders-job-status',
+}).then(
+  fn('get-status', async (ctx: any) => {
+    const { log, openKv } = ctx;
+    const kv = new VersoriKVAdapter(openKv(':project:'));
+    const tracker = new JobTracker(kv, log);
+
+    const url = new URL(ctx.request.url);
+    const jobId = url.searchParams.get('jobId');
+
+    if (!jobId) {
+      log.warn('⚠️ [JobStatus] Missing jobId query parameter');
+      return {
+        success: false,
+        error: 'Missing jobId query parameter',
+        usage: 'GET /orders-extract-status?jobId=<job-id>',
+        recommendations: [
+          'Provide jobId as query parameter',
+          'Example: /orders-extract-status?jobId=SCHEDULED_ORD_2025-01-22_12345'
+        ]
+      };
+    }
+
+    log.info('🔍 [JobStatus] Retrieving job status', { jobId });
+
+    const job = await tracker.getJob(jobId);
+
+    if (!job) {
+      log.warn('⚠️ [JobStatus] Job not found', { jobId });
+      return {
+        success: false,
+        error: `Job not found: ${jobId}`,
+        recommendations: [
+          'Verify jobId is correct',
+          'Check if job has expired (default TTL: 7 days)',
+          'Ensure job was created successfully'
+        ]
+      };
+    }
+
+    log.info('✅ [JobStatus] Job status retrieved', { jobId, status: job.status });
+
+    return {
+      success: true,
+      job: {
+        id: job.id,
+        name: job.name,
+        status: job.status,
+        createdAt: job.createdAt,
+        startedAt: job.startedAt,
+        completedAt: job.completedAt,
+        duration: job.duration,
+        result: job.result,
+        error: job.error,
+        metadata: job.metadata,
+      },
+    };
+  })
+);
+```
+
+---
+
+## Key Patterns Explained
+
+### Pattern 1: Line Item Flattening
+
+```typescript
+// BEFORE: 1 order with 3 line items
+{
+  ref: "ORD-001",
+  items: [
+    { product: { ref: "SKU-A" }, quantity: 2 },
+    { product: { ref: "SKU-B" }, quantity: 1 },
+    { product: { ref: "SKU-C" }, quantity: 3 }
+  ]
+}
+
+// AFTER: 3 CSV rows (order data repeated)
+[
+  { order_id: "ORD-001", line_item_sku: "SKU-A", line_item_qty: 2 },
+  { order_id: "ORD-001", line_item_sku: "SKU-B", line_item_qty: 1 },
+  { order_id: "ORD-001", line_item_sku: "SKU-C", line_item_qty: 3 }
+]
+```
+
+**Why?** Most 3PL systems expect flattened CSV format, not nested JSON.
+
+### Pattern 2: Overlap Buffer (Query WITH, Save WITHOUT)
+
+```typescript
+// Query uses bufferedLastRunTime (WITH overlap buffer)
+const bufferedLastRunTime = new Date(
+  new Date(rawLastRunTime).getTime() - OVERLAP_BUFFER_MS
+).toISOString();
+
+const result = await orchestrator.extract({
+  variables: {
+    updatedAfter: bufferedLastRunTime, // ← WITH buffer
+  },
+});
+
+// Save MAX(updatedOn) WITHOUT buffer
+const maxUpdatedOn = orders.reduce(
+  (max, order) => Math.max(max, new Date(order.updatedOn).getTime()),
+  new Date(rawLastRunTime).getTime()
+);
+
+await kv.set(stateKey, {
+  timestamp: new Date(maxUpdatedOn).toISOString(), // ← WITHOUT buffer
+});
+```
+
+**Why?** Buffer prevents missed records due to clock skew. Next run applies buffer again to saved timestamp.
+
+### Pattern 3: CSVParserService Usage
+
+```typescript
+// ✅ CORRECT - Use CSVParserService.stringify()
+const csvParser = new CSVParserService();
+const csvContent = csvParser.stringify(flattenedRecords, { headers: true });
+
+// ❌ WRONG - Don't use CSVBuilder (doesn't exist)
+const builder = new CSVBuilder();
+builder.addRows(records);
+const csvContent = builder.build();
+```
+
+**Why?** `CSVParserService` is the correct SDK service for CSV generation. It has no constructor parameters.
+
+### Pattern 4: 3-Workflow Pattern
+
+1. **Scheduled**: Hourly automated extraction
+2. **Ad-hoc Webhook**: On-demand trigger with API key auth
+3. **Job Status**: Query job progress via webhook
+
+**Why?** Provides flexibility for monitoring, testing, and emergency extractions.
+
+---
+
+## Testing Checklist
+
+**Before deploying to production:**
+
+### 1. Schema Validation
+
+- [ ] Run `npx fc-connect introspect-schema` to download current schema
+- [ ] Run `npx fc-connect validate-schema` to check mapping against schema
+- [ ] Verify all nested `source` paths exist (customer.email, deliveryAddress.city, product.ref)
+
+### 2. Mapping Testing
+
+- [ ] Test with sample data (maxRecords=10, small orders)
+- [ ] Verify all required fields are populated
+- [ ] Verify SDK resolvers work correctly (trim, uppercase, parseInt, parseFloat)
+- [ ] Verify line item flattening creates correct CSV structure
+
+### 3. State Management
+
+- [ ] Verify overlap buffer prevents duplicate misses
+- [ ] Test state recovery after failure (run should retry)
+- [ ] Verify timestamp is saved WITHOUT buffer
+- [ ] Test fallback to `fallbackStartDate` on first run
+
+### 4. S3 Operations
+
+- [ ] Test S3 connection with credentials
+- [ ] Verify file upload to correct prefix
+- [ ] Verify CSV file is valid (headers, row format)
+- [ ] Verify metadata is attached to uploaded file
+
+### 5. Workflow Testing
+
+- [ ] Test scheduled workflow (hourly cron)
+- [ ] Test ad-hoc webhook trigger with API key
+- [ ] Test job status query endpoint
+
+---
+
+## Monitoring & Alerting
+
+### Success Response Example
+
+```json
+{
+  "success": true,
+  "jobId": "SCHEDULED_ORD_20251102_140000_abc123",
+  "recordsExtracted": 1523,
+  "fileName": "orders-2025-11-02T14-00-00-000Z.csv",
+  "s3Path": "s3://bucket/orders/orders-2025-11-02T14-00-00-000Z.csv",
+  "metrics": {
+    "extractionDurationMs": 12543,
+    "totalPages": 8,
+    "pageSize": 200,
+    "mappingErrors": 0,
+    "fileSizeBytes": 524288,
+    "uploadDurationMs": 1234
+  },
+  "timestamps": {
+    "extractionStart": "2025-11-02T14:00:00.000Z",
+    "extractionEnd": "2025-11-02T14:00:12.543Z",
+    "uploadComplete": "2025-11-02T14:00:13.777Z"
+  },
+  "state": {
+    "previousTimestamp": "2025-11-02T13:00:00.000Z",
+    "newTimestamp": "2025-11-02T13:59:58.123Z",
+    "stateUpdated": true,
+    "overlapBufferSeconds": 60
+  }
+}
+```
+
+### Error Response Example
+
+```json
+{
+  "success": false,
+  "jobId": "ADHOC_ORD_20251102_140500_xyz789",
+  "error": "S3 upload failed: Connection timeout",
+  "errorCategory": "NETWORK",
+  "recordsExtracted": 0,
+  "stage": "s3_upload",
+  "details": {
+    "message": "Failed to upload file after 3 retry attempts",
+    "retryAttempts": 3,
+    "lastError": "ETIMEDOUT: Connection timed out after 30000ms"
+  },
+  "state": {
+    "stateUpdated": false,
+    "willRetryNextRun": true,
+    "note": "State not advanced - next extraction will retry same time window"
+  }
+}
+```
+
+### Key Metrics to Track
+
+```typescript
+const METRICS = {
+  // Extraction Performance
+  extractionDurationMs: Date.now() - extractionStart,
+  recordCount: records.length,
+  pageCount: extractionResult.stats.totalPages,
+  avgRecordsPerPage: records.length / extractionResult.stats.totalPages,
+
+  // Transformation Performance
+  transformedCount: transformedRecords.length,
+  failedCount: mappingErrors.length,
+  errorRate: ((mappingErrors.length / records.length) * 100).toFixed(2) + '%',
+
+  // File Generation
+  fileSizeMB: (csvContent.length / (1024 * 1024)).toFixed(2),
+
+  // Upload Performance
+  uploadDurationMs: uploadEnd - uploadStart,
+  uploadSpeedMBps: (fileSizeMB / (uploadDurationMs / 1000)).toFixed(2),
+
+  // State Management
+  timeSinceLastRun: Date.now() - new Date(lastTimestamp).getTime(),
+  recordsPerMinute: (records.length / (extractionDurationMs / 60000)).toFixed(0),
+};
+
+log.info('Extraction metrics', metrics);
+```
+
+### Alert Thresholds
+
+```typescript
+const ALERT_THRESHOLDS = {
+  // Duration Alerts
+  EXTRACTION_DURATION_MS: 5 * 60 * 1000, // 5 minutes
+  UPLOAD_DURATION_MS: 2 * 60 * 1000,      // 2 minutes
+  TOTAL_DURATION_MS: 10 * 60 * 1000,      // 10 minutes
+
+  // Error Rate Alerts
+  MAX_ERROR_RATE: 0.05, // 5% mapping errors
+  MAX_VALIDATION_FAILURES: 0.02, // 2% validation failures
+
+  // Volume Alerts
+  MAX_RECORDS_PER_RUN: 100000,
+  MIN_RECORDS_WARNING: 0, // Alert if no records found
+  MAX_FILE_SIZE_MB: 150, // 150MB
+
+  // State Alerts
+  MAX_TIME_SINCE_LAST_RUN_HOURS: 25, // Alert if >25 hours (should run hourly)
+  MAX_OVERLAP_BUFFER_SECONDS: 300,   // Alert if buffer >5 minutes
+};
+
+// Check thresholds
+if (metrics.extractionDurationMs > ALERT_THRESHOLDS.EXTRACTION_DURATION_MS) {
+  log.warn('Extraction duration exceeded threshold', {
+    duration: metrics.extractionDurationMs,
+    threshold: ALERT_THRESHOLDS.EXTRACTION_DURATION_MS,
+    recommendation: 'Consider reducing maxRecords or increasing extraction frequency'
+  });
+}
+```
+
+### Monitoring Dashboard Queries
+
+**Versori Platform Logs Query:**
+
+```
+# Successful extractions
+log_level:info AND message:"Extraction complete" AND jobId:*
+
+# Failed extractions
+log_level:error AND message:"Extraction workflow failed" AND jobId:*
+
+# Performance issues
+extractionDurationMs:>300000 OR uploadDurationMs:>120000
+
+# High error rates
+errorRate:>5
+
+# State management issues
+stateUpdated:false AND success:true
+```
+
+### Common Issues and Solutions
+
+**Issue**: "Extraction timeout after 10 minutes"
+
+- **Cause**: Too many records in single extraction
+- **Fix**: Reduce maxRecords, increase extraction frequency, or optimize query filters
+- **Prevention**: Monitor recordCount trends, set appropriate maxRecords
+
+**Issue**: "Mapping errors for 50% of records"
+
+- **Cause**: Schema mismatch between GraphQL response and mapping config
+- **Fix**: Run schema validation, update mapping config paths
+- **Prevention**: Use `npx fc-connect validate-schema` before deployment
+
+**Issue**: "S3 connection timeout"
+
+- **Cause**: Network issues, firewall, or connection pool exhaustion
+- **Fix**: Check S3 credentials, verify network connectivity
+- **Prevention**: Implement connection health checks, monitor connection status
+
+**Issue**: "State not updating after successful extraction"
+
+- **Cause**: KV write failure or intentional retry logic
+- **Fix**: Check KV logs, verify state update code executed
+- **Prevention**: Add KV write verification, log state updates explicitly
+
+**Issue**: "First run exceeds record limits"
+
+- **Cause**: No previous timestamp, fetches all historical records
+- **Fix**: Set fallbackStartDate close to current date, apply additional filters
+- **Prevention**: Use appropriate fallbackStartDate for initial runs
+
+**Issue**: "Excessive duplicate records in output"
+
+- **Cause**: Overlap buffer (expected) or timestamp not saved correctly
+- **Fix**: Verify newTimestamp saved WITHOUT buffer, check state persistence
+- **Prevention**: Monitor duplicate rates, verify state update logic
+
+---
+
+## Troubleshooting Quick Reference
+
+| Error Message | Likely Cause | Solution |
+|--------------|--------------|----------|
+| "Failed to create Fluent Commerce client" | Authentication failure | Check OAuth2 credentials, verify connection config |
+| "GraphQL query validation error" | Invalid query syntax | Validate query against schema with introspection tool |
+| "Pagination cursor invalid" | Stale cursor or query change | Reset extraction, verify cursor handling in query |
+| "Mapping failed: field not found" | Schema mismatch | Run schema validation, update mapping paths |
+| "S3 authentication failed" | Invalid credentials | Verify S3 credentials in activation variables |
+| "Connection pool exhausted" | Too many concurrent requests | Reduce concurrency, increase pool size |
+| "KV operation failed" | Versori KV issue | Check Versori platform status, retry operation |
+| "Job status not found" | Invalid jobId or expired | Verify jobId format, check KV retention policy |
+| "Memory limit exceeded" | Dataset too large | Reduce maxRecords, enable streaming mode |
+| "CSV generation failed" | Format-specific error | Check CSV generation logic, validate output |
+
+---
+
+---
+
+### Pattern 7: State Management & Date Overrides
+
+**Use Case**: Understand how state management works with scheduled and ad-hoc extractions.
+
+**How it works**:
+
+VersoriKV stores the last successful extraction timestamp to enable incremental sync:
+
+```typescript
+interface ExtractionState {
+  timestamp: string; // Last run timestamp (WITHOUT overlap buffer)
+  recordCount: number; // Number of records extracted
+  extractedAt: string; // When extraction completed
+  fileName?: string; // Generated filename
+  s3Key?: string; // S3 upload path
+  overlapBufferSeconds?: number; // Buffer configuration
+}
+```
+
+**State Priority Chain** (highest to lowest):
+
+1. **`fromDate` override** (manual date in webhook payload) - Highest priority
+2. **Stored state** (`await kv.get(stateKey)`) - Normal incremental mode
+3. **`fallbackStartDate`** (activation variable) - First run fallback
+
+**Three Scenarios**:
+
+#### Scenario 1: Normal Scheduled Runs (Incremental)
+
+```typescript
+// Payload: {} (empty - no overrides)
+
+// Behavior:
+// 1. Load last timestamp from KV: "2025-01-22T10:00:00Z"
+// 2. Apply overlap buffer: "2025-01-22T09:59:00Z" (query WITH buffer)
+// 3. Extract records updated since buffered time
+// 4. Calculate MAX(updatedOn) from results: "2025-01-22T14:30:00Z"
+// 5. Save new timestamp WITHOUT buffer: "2025-01-22T14:30:00Z"
+// 6. Next run starts from "2025-01-22T14:29:00Z" (with buffer)
+```
+
+**Test**:
+
+```bash
+# Trigger scheduled run (no payload needed)
+# State advances automatically
+curl -X POST https://workspace.versori.run/orders-extract-hourly
+```
+
+#### Scenario 2: Ad-hoc Extraction WITH State Update
+
+```typescript
+// Payload: { "fromDate": "2025-01-01T00:00:00Z", "updateState": true }
+
+// Behavior:
+// 1. Ignore stored state
+// 2. Use fromDate: "2025-01-01T00:00:00Z" (no buffer applied to manual dates)
+// 3. Extract all records since 2025-01-01
+// 4. Calculate MAX(updatedOn): "2025-01-22T14:30:00Z"
+// 5. Save new timestamp: "2025-01-22T14:30:00Z" (updates state!)
+// 6. Next scheduled run starts from this new timestamp
+```
+
+**Use Case**: One-time catch-up extraction that advances the state pointer.
+
+**Test**:
+
+```bash
+curl -X POST https://workspace.versori.run/orders-extract-webhook \
+  -H "x-api-key: your-api-key" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "fromDate": "2025-01-01T00:00:00Z",
+    "updateState": true
+  }'
+```
+
+#### Scenario 3: Ad-hoc Extraction WITHOUT State Update
+
+```typescript
+// Payload: { "fromDate": "2025-01-01T00:00:00Z", "updateState": false }
+
+// Behavior:
+// 1. Ignore stored state
+// 2. Use fromDate: "2025-01-01T00:00:00Z"
+// 3. Extract all records since 2025-01-01
+// 4. DO NOT update state
+// 5. Next scheduled run uses previous timestamp (unaffected)
+```
+
+**Use Case**: Historical backfill or testing without affecting incremental sync.
+
+**Test**:
+
+```bash
+curl -X POST https://workspace.versori.run/orders-extract-webhook \
+  -H "x-api-key: your-api-key" \
+  -H "Content-Type: application/json" \
+  -d '{
+    "fromDate": "2025-01-01T00:00:00Z",
+    "toDate": "2025-01-31T23:59:59Z",
+    "updateState": false
+  }'
+```
+
+**Why this matters**:
+
+- **Incremental sync** relies on state continuity
+- **Manual overrides** allow catch-up without breaking incremental flow
+- **Overlap buffer** prevents missed records at time boundaries
+- **State isolation** lets you test/backfill without affecting production sync
+
+---
+
+### Pattern 8: Optional GraphQL Query Logging
+
+**Use Case**: Debug extraction issues by logging the exact GraphQL query sent to Fluent Commerce API.
+
+**When to use**:
+
+- ✅ Debugging pagination issues
+- ✅ Verifying query variables (dates, filters, limits)
+- ✅ Development and testing
+- ❌ Production (verbose logs, potential secrets in variables)
+
+**How to enable**:
+
+Set `DEBUG_GRAPHQL=true` environment variable in Versori activation settings.
+
+**Implementation**:
+
+```typescript
+// In your extraction workflow
+const DEBUG_GRAPHQL = activation?.getVariable('DEBUG_GRAPHQL') === 'true';
+
+if (DEBUG_GRAPHQL) {
+  log.info('GraphQL Query Debug', {
+    query: ORDERS_QUERY,
+    variables: {
+      retailerId,
+      updatedAfter: bufferedLastRunTime,
+      first: pageSize,
+      after: null, // First page
+    },
+    pagination: {
+      pageSize,
+      maxRecords,
+      currentPage: 1,
+    },
+  });
+}
+
+const extractionResult = await orchestrator.extract({
+  query: ORDERS_QUERY,
+  resultPath: 'orders.edges.node',
+  variables: {
+    retailerId,
+    updatedAfter: bufferedLastRunTime,
+  },
+  pageSize,
+  maxRecords,
+});
+
+if (DEBUG_GRAPHQL) {
+  log.info('GraphQL Response Debug', {
+    totalRecords: extractionResult.stats.totalRecords,
+    totalPages: extractionResult.stats.totalPages,
+    validRecords: extractionResult.stats.validRecords ?? extractionResult.data.length,
+    firstRecordId: extractionResult.data[0]?.id,
+    lastRecordId: extractionResult.data[extractionResult.data.length - 1]?.id,
+  });
+}
+```
+
+**What gets logged**:
+
+```json
+{
+  "level": "info",
+  "message": "GraphQL Query Debug",
+  "query": "query GetOrders($retailerId: ID!, $updatedAfter: DateTime!, ...)",
+  "variables": {
+    "retailerId": "ACME",
+    "updatedAfter": "2025-01-22T09:59:00Z",
+    "first": 200,
+    "after": null
+  },
+  "pagination": {
+    "pageSize": 200,
+    "maxRecords": 10000,
+    "currentPage": 1
+  }
+}
+```
+
+**Versori Environment Variables**:
+
+Add to activation settings:
+
+```json
+{
+  "DEBUG_GRAPHQL": "true"
+}
+```
+
+**Testing**:
+
+```bash
+# Enable debug logging
+curl -X POST https://workspace.versori.run/orders-extract-hourly
+
+# Check Versori logs for "GraphQL Query Debug" entries
+# Verify query structure and variables are correct
+```
+
+**Sample Debug Output**:
+
+```
+[INFO] GraphQL Query Debug
+  query: "query GetOrders($retailerId: ID!, $updatedAfter: DateTime!, $first: Int!, $after: String) { ... }"
+  variables: { retailerId: "ACME", updatedAfter: "2025-01-22T09:59:00Z", first: 200, after: null }
+  pagination: { pageSize: 200, maxRecords: 10000, currentPage: 1 }
+
+[INFO] Extraction complete
+  totalRecords: 150
+  totalPages: 1
+  validRecords: 150
+  failedValidations: 0
+
+[INFO] GraphQL Response Debug
+  totalRecords: 150
+  totalPages: 1
+  validRecords: 150
+  firstRecordId: "order_123"
+  lastRecordId: "order_272"
+```
+
+**Key Benefits**:
+
+- Quickly identify pagination configuration issues
+- Verify date filters are applied correctly
+- Debug "no records found" scenarios
+- Validate ExtractionOrchestrator variable injection
+
+**Production Best Practice**: Disable `DEBUG_GRAPHQL` in production to reduce log volume and avoid logging sensitive data.
+
+---
+
+## Troubleshooting
+
+**Issue**: "All records failed mapping"
+
+**Cause**: Schema mismatch or incorrect field paths
+
+**Solutions**:
+
+1. Run `npx fc-connect introspect-schema` to get current schema
+2. Run `npx fc-connect validate-schema` to check mapping
+3. Review error details: `errors.map(e => e.errors)`
+
+---
+
+**Issue**: "State not updating, same orders exported every run"
+
+**Cause**: KV storage write failure or timestamp calculation error
+
+**Solutions**:
+
+1. Check KV logs for write errors
+2. Verify state update code is reached: add logging
+3. Check `newTimestamp` calculation logic
+4. Manually inspect KV value: `await kv.get(['extraction', 'orders', 'lastRunTime'])`
+
+---
+
+**Issue**: "Orders with no line items cause mapping errors"
+
+**Cause**: `order.items` is empty array or null
+
+**Solutions**:
+
+```typescript
+if (lineItems.length === 0) {
+  log.warn('Order has no line items, skipping', { orderId: order.id, ref: order.ref });
+  continue;
+}
+```
+
+---
+
+## See Also
+
+**Related Templates:**
+
+- [Orders to SFTP XML](./template-extraction-orders-to-sftp-xml.md) - Same entity, different format/destination
+- [Fulfillments to S3 CSV](./template-extraction-fulfillments-to-sftp-csv.md) - Fulfillment lifecycle tracking
+- [Products to S3 JSON](./template-extraction-products-to-s3-json.md) - Product catalog extraction
+
+**SDK Documentation:**
+
+- [ExtractionOrchestrator Guide](../../../../../02-CORE-GUIDES/extraction/modules/02-core-guides-extraction-08-extraction-orchestrator.md) - Auto-pagination and path-based extraction
+- [JobTracker Reference](../../../../../02-CORE-GUIDES/api-reference/modules/api-reference-05-services.md#jobtracker-new) - Job lifecycle management
+- [Universal Mapping Guide](../../../../../02-CORE-GUIDES/ingestion/modules/02-core-guides-ingestion-01-introduction.md) - Field transformation
+- [CSVParserService API](../../../../../02-CORE-GUIDES/parsers/modules/02-core-guides-parsers-02-csv-parser.md) - CSV generation
+
+---
+
+### Pattern 8: Backward Pagination (Optional - Advanced)
+
+**Use Case**: Extract data in reverse chronological order (newest to oldest) instead of oldest to newest.
+
+**When to Use**:
+
+- ✅ Need most recent records first (e.g., latest orders, recent inventory updates)
+- ✅ Time-bounded reverse traversal for auditing
+- ✅ Display newest-first in UI/reports
+- ❌ **Don't use for standard incremental sync** - use forward pagination (default)
+
+**GraphQL Query Requirements**:
+
+Your query must support backward pagination by including `$last` and `$before`:
+
+```graphql
+query GetData(
+  $retailerId: ID!
+  $first: Int # For forward pagination
+  $after: String # For forward pagination
+  $last: Int # For backward pagination
+  $before: String # For backward pagination
+) {
+  data(retailerId: $retailerId, first: $first, after: $after, last: $last, before: $before) {
+    edges {
+      cursor # ✅ REQUIRED
+      node {
+        id
+        createdAt
+        # ... other fields
+      }
+    }
+    pageInfo {
+      hasNextPage # For forward
+      hasPreviousPage # ✅ REQUIRED for backward
+    }
+  }
+}
+```
+
+**Implementation**:
+
+```typescript
+// Backward pagination - newest records first
+const result = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  variables: {
+    retailerId,
+    dateRangeFilter: { from: bufferedLastRunTime, to: effectiveEndTime },
+    // ❌ Don't include last/before - orchestrator injects them
+  },
+  pageSize: 200,
+  direction: 'backward', // ✅ Enable reverse pagination
+  maxRecords: 10000,
+});
+
+// Records are returned in reverse chronological order
+console.log(result.data[0].createdAt); // Newest
+console.log(result.data[result.data.length - 1].createdAt); // Oldest (within range)
+```
+
+**Key Differences from Forward Pagination**:
+
+| Aspect                 | Forward (Default)                | Backward                |
+| ---------------------- | -------------------------------- | ----------------------- |
+| **Direction**          | `direction: 'forward'` (default) | `direction: 'backward'` |
+| **Variables Injected** | `first`, `after`                 | `last`, `before`        |
+| **PageInfo Field**     | `hasNextPage`                    | `hasPreviousPage`       |
+| **Cursor Source**      | Last edge of page                | First edge of page      |
+| **Record Order**       | Oldest → Newest                  | Newest → Oldest         |
+
+**Important Notes**:
+
+1. **Orchestrator injects variables**: Don't pass `last` or `before` in your variables object - the orchestrator injects them based on `pageSize` and cursor tracking.
+
+2. **Query signature**: Your GraphQL query must declare `$last` and `$before` parameters even if you don't pass them explicitly.
+
+3. **PageInfo requirement**: Response must include `pageInfo.hasPreviousPage` or the orchestrator will throw an error.
+
+4. **Cursor requirement**: Each edge must include `cursor` field for pagination to work.
+
+**Example: Extract Latest 1000 Orders**
+
+```typescript
+const latestOrders = await orchestrator.extract({
+  query: ORDERS_QUERY,
+  resultPath: 'orders.edges.node',
+  variables: {
+    retailerId,
+    statuses: ['BOOKED', 'ALLOCATED'],
+  },
+  direction: 'backward', // Start from newest
+  maxRecords: 1000, // Stop after 1000 records
+  pageSize: 100, // 100 per page = 10 pages
+});
+
+// latestOrders.data[0] is the newest order
+// latestOrders.data[999] is the 1000th newest order
+```
+
+**When to Use Forward vs Backward**:
+
+```typescript
+// ✅ Forward (default) - For incremental sync
+const incrementalData = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  variables: {
+    dateRangeFilter: { from: lastSyncTime, to: now },
+  },
+  // direction defaults to 'forward'
+  // Processes oldest → newest for proper sequencing
+});
+
+// ✅ Backward - For "latest N records" use cases
+const latestData = await orchestrator.extract({
+  query: YOUR_QUERY,
+  resultPath: 'data.edges.node',
+  direction: 'backward',
+  maxRecords: 100, // Just get latest 100
+  // Gets newest → oldest
+});
+```
+
+**Pagination Variables Reference**:
+
+| Variable | Forward     | Backward    | Injected By  | Notes                    |
+| -------- | ----------- | ----------- | ------------ | ------------------------ |
+| `first`  | ✅ Used     | ❌ Not used | Orchestrator | From `pageSize`          |
+| `after`  | ✅ Used     | ❌ Not used | Orchestrator | From cursor (last edge)  |
+| `last`   | ❌ Not used | ✅ Used     | Orchestrator | From `pageSize`          |
+| `before` | ❌ Not used | ✅ Used     | Orchestrator | From cursor (first edge) |
+
+**Common Mistakes to Avoid**:
+
+```typescript
+// ❌ WRONG - Don't pass pagination variables
+const result = await orchestrator.extract({
+  variables: {
+    last: 200, // ❌ Orchestrator will override this
+    before: cursor, // ❌ Orchestrator manages cursor
+  },
+  direction: 'backward',
+});
+
+// ✅ CORRECT - Let orchestrator inject pagination
+const result = await orchestrator.extract({
+  variables: {
+    retailerId, // ✅ Your business variables only
+  },
+  pageSize: 200, // ✅ Orchestrator uses this for last/before
+  direction: 'backward',
+});
+```
+
+#### Optional: Reverse Pagination
+
+- Default: forward pagination ($first/$after) with pageInfo.hasNextPage.
+- Reverse: declare $last/$before in the query and include pageInfo.hasPreviousPage; set direction='backward' in the orchestrator call.
+
+GraphQL:
+
+```graphql
+query GetOrdersBackward(
+  $retailerId: ID!
+  $dateRangeFilter: DateRange
+  $last: Int!
+  $before: String
+) {
+  orders(retailerId: $retailerId, updatedOn: $dateRangeFilter, last: $last, before: $before) {
+    edges {
+      cursor
+      node {
+        id
+        ref
+        updatedOn
+      }
+    }
+    pageInfo {
+      hasPreviousPage
+    }
+  }
+}
+```
+
+SDK:
+
+```typescript
+await orchestrator.extract({
+  query: ORDERS_BACKWARD_QUERY,
+  resultPath: 'orders.edges.node',
+  variables: { retailerId, dateRangeFilter },
+  pageSize,
+  direction: 'backward',
+});
+```